Detailed real-time urban 3D reconstruction from video

M. Pollefeys; D. Nistér; J. M. Frahm; A. Akbarzadeh; P. Mordohai; B. Clipp; C. Engels; D. Gallup; S. J. Kim; P. Merrell; C. Salmi; S. Sinha; B. Talton; L. Wang; Q. Yang; H. Stewénius; R. Yang; G. Welch; H. Towles

doi:10.1007/s11263-007-0086-4

Detailed real-time urban 3D reconstruction from video

M. Pollefeys, D. Nistér, J. M. Frahm, A. Akbarzadeh, P. Mordohai, B. Clipp, C. Engels, D. Gallup, S. J. Kim, P. Merrell, C. Salmi, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stewénius, R. Yang, G. Welch, H. Towles

Computer Science

Research output: Contribution to journal › Article › peer-review

626 Scopus citations

Abstract

The paper presents a system for automatic, geo-registered, real-time 3D reconstruction from video of urban scenes. The system collects video streams, as well as GPS and inertia measurements in order to place the reconstructed models in geo-registered coordinates. It is designed using current state of the art real-time modules for all processing steps. It employs commodity graphics hardware and standard CPU's to achieve real-time performance. We present the main considerations in designing the system and the steps of the processing pipeline. Our system extends existing algorithms to meet the robustness and variability necessary to operate out of the lab. To account for the large dynamic range of outdoor videos the processing pipeline estimates global camera gain changes in the feature tracking stage and efficiently compensates for these in stereo estimation without impacting the real-time performance. The required accuracy for many applications is achieved with a two-step stereo reconstruction process exploiting the redundancy across frames. We show results on real video sequences comprising hundreds of thousands of frames.

Original language	English
Pages (from-to)	143-167
Number of pages	25
Journal	International Journal of Computer Vision
Volume	78
Issue number	2-3
DOIs	https://doi.org/10.1007/s11263-007-0086-4
State	Published - Jul 2008

Bibliographical note

Funding Information:
Acknowledgements We gratefully acknowledge the support of the DARPA UrbanScape project as well as the support of the DTO VACE project “3D Content Extraction from Video Streams”.

Funding

Acknowledgements We gratefully acknowledge the support of the DARPA UrbanScape project as well as the support of the DTO VACE project “3D Content Extraction from Video Streams”.

Funders	Funder number
Defense Advanced Research Projects Agency

Keywords

3D reconstruction
Depth map fusion
Large scale modeling
Plane sweeping
Stereo vision
Structure from motion
Urban reconstruction

ASJC Scopus subject areas

Software
Computer Vision and Pattern Recognition
Artificial Intelligence

Access to Document

10.1007/s11263-007-0086-4

Cite this

Pollefeys, M., Nistér, D., Frahm, J. M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S. J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., & Towles, H. (2008). Detailed real-time urban 3D reconstruction from video. International Journal of Computer Vision, 78(2-3), 143-167. https://doi.org/10.1007/s11263-007-0086-4

@article{d839ace955f74d84af9e592731443e9e,

title = "Detailed real-time urban 3D reconstruction from video",

abstract = "The paper presents a system for automatic, geo-registered, real-time 3D reconstruction from video of urban scenes. The system collects video streams, as well as GPS and inertia measurements in order to place the reconstructed models in geo-registered coordinates. It is designed using current state of the art real-time modules for all processing steps. It employs commodity graphics hardware and standard CPU's to achieve real-time performance. We present the main considerations in designing the system and the steps of the processing pipeline. Our system extends existing algorithms to meet the robustness and variability necessary to operate out of the lab. To account for the large dynamic range of outdoor videos the processing pipeline estimates global camera gain changes in the feature tracking stage and efficiently compensates for these in stereo estimation without impacting the real-time performance. The required accuracy for many applications is achieved with a two-step stereo reconstruction process exploiting the redundancy across frames. We show results on real video sequences comprising hundreds of thousands of frames.",

keywords = "3D reconstruction, Depth map fusion, Large scale modeling, Plane sweeping, Stereo vision, Structure from motion, Urban reconstruction",

author = "M. Pollefeys and D. Nist{\'e}r and Frahm, {J. M.} and A. Akbarzadeh and P. Mordohai and B. Clipp and C. Engels and D. Gallup and Kim, {S. J.} and P. Merrell and C. Salmi and S. Sinha and B. Talton and L. Wang and Q. Yang and H. Stew{\'e}nius and R. Yang and G. Welch and H. Towles",

note = "Funding Information: Acknowledgements We gratefully acknowledge the support of the DARPA UrbanScape project as well as the support of the DTO VACE project “3D Content Extraction from Video Streams”.",

year = "2008",

month = jul,

doi = "10.1007/s11263-007-0086-4",

language = "English",

volume = "78",

pages = "143--167",

number = "2-3",

}

Pollefeys, M, Nistér, D, Frahm, JM, Akbarzadeh, A, Mordohai, P, Clipp, B, Engels, C, Gallup, D, Kim, SJ, Merrell, P, Salmi, C, Sinha, S, Talton, B, Wang, L, Yang, Q, Stewénius, H, Yang, R, Welch, G & Towles, H 2008, 'Detailed real-time urban 3D reconstruction from video', International Journal of Computer Vision, vol. 78, no. 2-3, pp. 143-167. https://doi.org/10.1007/s11263-007-0086-4

TY - JOUR

T1 - Detailed real-time urban 3D reconstruction from video

AU - Pollefeys, M.

AU - Nistér, D.

AU - Frahm, J. M.

AU - Akbarzadeh, A.

AU - Mordohai, P.

AU - Clipp, B.

AU - Engels, C.

AU - Gallup, D.

AU - Kim, S. J.

AU - Merrell, P.

AU - Salmi, C.

AU - Sinha, S.

AU - Talton, B.

AU - Wang, L.

AU - Yang, Q.

AU - Stewénius, H.

AU - Yang, R.

AU - Welch, G.

AU - Towles, H.

N1 - Funding Information: Acknowledgements We gratefully acknowledge the support of the DARPA UrbanScape project as well as the support of the DTO VACE project “3D Content Extraction from Video Streams”.

PY - 2008/7

Y1 - 2008/7

N2 - The paper presents a system for automatic, geo-registered, real-time 3D reconstruction from video of urban scenes. The system collects video streams, as well as GPS and inertia measurements in order to place the reconstructed models in geo-registered coordinates. It is designed using current state of the art real-time modules for all processing steps. It employs commodity graphics hardware and standard CPU's to achieve real-time performance. We present the main considerations in designing the system and the steps of the processing pipeline. Our system extends existing algorithms to meet the robustness and variability necessary to operate out of the lab. To account for the large dynamic range of outdoor videos the processing pipeline estimates global camera gain changes in the feature tracking stage and efficiently compensates for these in stereo estimation without impacting the real-time performance. The required accuracy for many applications is achieved with a two-step stereo reconstruction process exploiting the redundancy across frames. We show results on real video sequences comprising hundreds of thousands of frames.

AB - The paper presents a system for automatic, geo-registered, real-time 3D reconstruction from video of urban scenes. The system collects video streams, as well as GPS and inertia measurements in order to place the reconstructed models in geo-registered coordinates. It is designed using current state of the art real-time modules for all processing steps. It employs commodity graphics hardware and standard CPU's to achieve real-time performance. We present the main considerations in designing the system and the steps of the processing pipeline. Our system extends existing algorithms to meet the robustness and variability necessary to operate out of the lab. To account for the large dynamic range of outdoor videos the processing pipeline estimates global camera gain changes in the feature tracking stage and efficiently compensates for these in stereo estimation without impacting the real-time performance. The required accuracy for many applications is achieved with a two-step stereo reconstruction process exploiting the redundancy across frames. We show results on real video sequences comprising hundreds of thousands of frames.

KW - 3D reconstruction

KW - Depth map fusion

KW - Large scale modeling

KW - Plane sweeping

KW - Stereo vision

KW - Structure from motion

KW - Urban reconstruction

UR - http://www.scopus.com/inward/record.url?scp=40949155911&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=40949155911&partnerID=8YFLogxK

U2 - 10.1007/s11263-007-0086-4

DO - 10.1007/s11263-007-0086-4

M3 - Article

AN - SCOPUS:40949155911

SN - 0920-5691

VL - 78

SP - 143

EP - 167

JO - International Journal of Computer Vision

JF - International Journal of Computer Vision

IS - 2-3

ER -

Detailed real-time urban 3D reconstruction from video

Abstract

Bibliographical note

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this