back to funding overview.

ContextualEyes: A Context-Aware Surveillance System

Overview

Today, essentially all images come with GPS data and a time-stamp, unfortunately most automated image analysis algorithms ignore this information. We propose to develop algorithms that exploit this metadata and, in addition, we propose algorithms for extracting geospatial information directly from the imagery. Explicitly considering the geospatial context will advance the state of the art in automated imagery analysis and lead to tools for improved situational awareness. This proposal is part of a broad research agenda aimed at building ContextualEyes, a surveillance platform that deeply integrates geospatial awareness. This proposal takes important steps toward this goal. We focus on three main research objectives: (1) using known weather conditions for scene understanding, (2) estimating unknown weather conditions and (3) estimating the shape and motion of clouds. We also dedicate time for continued DoD engagement with pilot projects and support for technology transition.

Related Publication(s)

  1. PDF Jacobs N., Workman S., Souvenir R. 2016. Cloudmaps from Static Ground-View Video. Image and Vision Computing (IVC) 52:154–166. website bibtex
  2. PDF Zhai M., Workman S., Jacobs N. 2016. Detecting Vanishing Points using Global Image Context in a Non-Manhattan World. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). website code bibtex
  3. PDF Salem T., Workman S., Zhai M., Jacobs N. 2016. Analyzing Human Appearance as a Cue for Dating Images. In: IEEE Winter Conference on Applications of Computer Vision (WACV). 1–8. website bibtex
  4. PDF Mihail RP., Workman S., Bessinger Z., Jacobs N. 2016. Sky Segmentation in the Wild: An Empirical Study. In: IEEE Winter Conference on Applications of Computer Vision (WACV). 1–6. website bibtex
  5. PDF Baltenberger R., Zhai M., Greenwell C., Workman S., Jacobs N. 2016. A Fast Method for Estimating Transient Scene Properties. In: IEEE Winter Conference on Applications of Computer Vision (WACV). 1–8. website code bibtex
  6. PDF Murdock C., Jacobs N., Pless R. 2015. Building Dynamic Cloud Maps from the Ground Up. In: IEEE International Conference on Computer Vision (ICCV). 1–9. bibtex
  7. PDF Islam MT., Greenwell C., Souvenir R., Jacobs N. 2015. Large-Scale Geo-Facial Image Analysis. EURASIP Journal on Image and Video Processing (JIVP) 2015:1–14. website bibtex
  8. Islam MT., Workman S., Jacobs N. 2015. Face2GPS: Estimating Geographic Location from Facial Features. In: IEEE International Conference on Image Processing (ICIP). website bibtex
  9. Workman S., Greenwell C., Zhai M., Baltenberger R., Jacobs N. 2015. DeepFocal: A Method for Direct Focal Length Estimation. In: IEEE International Conference on Image Processing (ICIP). website bibtex
  10. Workman S., Souvenir R., Jacobs N. 2015. Scene Shape Estimation from Multiple Partly Cloudy Days. Computer Vision and Image Understanding (CVIU):116–129. website bibtex
  11. PDF Greenwell C., Spurlock S., Souvenir R., Jacobs N. 2014. GeoFaceExplorer: Exploring the Geo-Dependence of Facial Attributes. In: ACM SIGSPATIAL International Workshop on Crowdsourced and Volunteered Geographic Information (GEOCROWD). 32–37. website bibtex
  12. Workman S., Mihail RP., Jacobs N. 2014. A Pot of Gold: Rainbows as a Calibration Cue. In: European Conference on Computer Vision (ECCV). 820–835. website bibtex
  13. Shi F., Zhai M., Duncan D., Jacobs N. 2014. MPCA: EM-Based PCA For Mixed-Size Image Datasets. In: IEEE International Conference on Image Processing (ICIP). 1807–1811. website bibtex
  14. Zhai M., Shi F., Duncan D., Jacobs N. 2014. Covariance-Based PCA for Multi-Size Data. In: International Conference on Pattern Recognition (ICPR). 1603–1608. website bibtex
  15. PDF Islam MT., Workman S., Wu H., Souvenir R., Jacobs N. 2014. Exploring the Geo-Dependence of Human Face Appearance. In: IEEE Winter Conference on Applications of Computer Vision (WACV). 1042–1049. bibtex
  16. PDF Mihail RP., Blomquist G., Jacobs N. 2014. A CRF Approach to Fitting a Generalized Hand Skeleton Model. In: IEEE Winter Conference on Applications of Computer Vision (WACV). 409–416. bibtex
  17. PDF Jacobs N., King J., Bowers D., Souvenir R. 2014. Estimating Cloud Maps from Outdoor Image Sequences. In: IEEE Winter Conference on Applications of Computer Vision (WACV). 961–968. website bibtex
  18. PDF Jacobs N., Workman S., Souvenir R. 2013. Scene Geometry from Several Partly Cloudy Days. In: ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC). 1–6. website bibtex
  19. PDF Islam MT., Jacobs N., Wu H., Souvenir R. 2013. Images+Weather: Collection, Validation, and Refinement. In: IEEE CVPR Workshop on Ground Truth. 1–7. bibtex
  20. PDF Jacobs N., Islam MT., Workman S. 2013. Cloud Motion as a Calibration Cue. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1344–1351. website bibtex
  21. PDF Jacobs N., Abrams A., Pless R. 2013. Two Cloud-Based Cues for Estimating Scene Structure and Camera Calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 35:2526–2538. website bibtex
  22. PDF Murdock C., Jacobs N., Pless R. 2013. Webcam2Satellite: Estimating Cloud Maps from Webcam Imagery. In: IEEE Workshop on Applications of Computer Vision (WACV). 214–221. bibtex
  23. PDF Mihail RP., Jacobs N., Goldsmith J. 2012. Real Time Gesture Recognition With 2 Kinect Sensors. In: International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV). 1–7. bibtex
  24. PDF Abrams A., Tucek J., Jacobs N., Pless R. 2012. LOST: Longterm Observation of Scenes (with Tracks). In: IEEE Workshop on Applications of Computer Vision (WACV). 297–304. bibtex

Acknowledgements

This work was fully, or partially, supported by the Defense Advanced Research Project Agency (D11AP00255). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor