image placeholder

Professors Matthew Turk and Tobias Höllerer of the Four Eyes Lab were awarded a $477,428 National Science Foundation grant for a project titled "Crowd-Sourcing the World: Scalable Methods for Dynamic Structure from Motion.

The abundance of publicly available imagery from a variety of sources (consumer, industry, and government) and the proliferation of networked mobile devices equipped with cameras provide an opportunity to build large-scale 3D models that cover the entire world. Such global 3D models will be invaluable to a wide range of applications that require real-time access to environment structure, such as providing assistance to the visually impaired, exploring dangerous areas for search-and rescue operations, urban planning, self-driving cars, and virtual tourism. While current techniques are able to build city-scale reconstructions in less than a day, such static models can quickly become outdated due to seasonal variations, the construction, demolition, or modification of buildings, or other structural changes, thus limiting the usefulness of the model for real-world applications. The goal of this project is to enable global-scale dynamic reconstructions that can scale to eventually encompass all of the world’s 3D data, to which any user may contribute new visual data, thereby ensuring a more complete, up-to-date model and a better experience for users of applications that rely on such data.

The key research question driving this work is how to efficiently and accurately update a global-scale 3D reconstruction in order to support a dynamic global model. The research will use structure from motion (SfM) techniques to acquire global context for 3D modeling and then propagate local 3D reconstructions and other visual data back to the global model via a new technique called “globalization.” The proposed project addresses a major gap in large-scale SfM: rapidly extending and updating reconstructions to promote real-time use. This will be accomplished by building on preliminary work in large-scale image matching, localization, and 3D reconstruction. Extensive data collection and experimentation will be carried out to benchmark the performance of the proposed techniques and to assess the progress of the project and the utility of the methods, and a prototype mobile system for contributing imagery and browsing and updating models will be made available for public use.