CAREER: Learning and Using Models of Geo-Temporal Appearance

  • Jacobs, Nathan (PI)

Grants and Contracts Details


Millions of static cameras exist around the world, capturing and sharing images of a wide variety of outdoor scenes. These cameras are deployed for many reasons, including environmental monitoring, video surveillance, traffic assessment, or sharing the natural beauty of a park. Together, these cameras represent an unprecedented resource for measuring how the world changes. However, extracting information from this imagery is challenging because many factors conspire to change the appearance of the scene in ways that corrupt simple methods for making the desired measurements. For example, when trying to measure atmospheric haze, it would be useful to know how far the buildings are from the camera. When trying to measure cloud height, it would be useful to know if the scene even views the sky. When trying to measure plant health, it would be useful to be able to control for other sources of color change, such as local weather conditions. This proposal presents a comprehensive research program aimed at developing computer vision algorithms for addressing these and other similar challenges. The long-term goal is to enable scientists to harness this global imaging resource and to provide tools for them to better utilize the cameras they deploy for their research. The core hypothesis is that by using long-term observations, coupled with the known location and time that the image was captured, it will be possible to automatically build rich scene models that describe the geometry, semantic layout, and weather conditions of an outdoor scene. These models will enable more precise reasoning about scientifically relevant changes in a scene. The proposed methods, which build on recent advances in machine learning and computer vision and the availability of massive image datasets, will be more accurate and robust than existing approaches for estimating such models, which typically rely on single images and ignore geo-temporal context.
Effective start/end date7/1/166/30/21


Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.