Real-time stereo using approximated joint bilateral filtering and dynamic programming

Liang Wang, Ruigang Yang, Minglun Gong, Miao Liao

Research output: Contribution to journalArticlepeer-review

35 Scopus citations

Abstract

We present a stereo algorithm that is capable of estimating scene depth information with high accuracy and in real time. The key idea is to employ an adaptive cost-volume filtering stage in a dynamic programming optimization framework. The per-pixel matching costs are aggregated via a separable implementation of the bilateral filtering technique. Our separable approximation offers comparable edge-preserving filtering capability and leads to a significant reduction in computational complexity compared to the traditional 2D filter. This cost aggregation step resolves the disparity inconsistency between scanlines, which are the typical problem for conventional dynamic programming based stereo approaches. Our algorithm is driven by two design goals: real-time performance and high accuracy depth estimation. For computational efficiency, we utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this aggregation process over two orders of magnitude. Over 90 million disparity evaluations per second [the number of disparity evaluations per seconds (MDE/s) corresponds to the product of the number of pixels and the disparity range and the obtained frame rate and, therefore, captures the performance of a stereo algorithm in a single number] are achieved in our current implementation. In terms of quality, quantitative evaluation using data sets with ground truth disparities shows that our approach is one of the state-of-the-art real-time stereo algorithms.

Original languageEnglish
Pages (from-to)447-461
Number of pages15
JournalJournal of Real-Time Image Processing
Volume9
Issue number3
DOIs
StatePublished - Sep 2014

Bibliographical note

Funding Information:
Ruigang Yang received the MS degree in computer science from Columbia University in 1998 and the PhD degree in computer science from the University of North Carolina, Chapel Hill, in 2003. He is an associate professor in the Computer Science in the Department at the University of Kentucky. His research interests include computer graphics, computer vision, and multimedia. He is a recipient of the US National Science Foundation CAREER award in 2004 and a member of the IEEE, the IEEE Computer Society and the ACM.

Funding Information:
Acknowledgments The authors would like to thank the editor and all the reviewers for their constructive comments. This work is supported in part by University of Kentucky Research Foundation, US National Science Foundation award IIS-0448185, CPA-0811647, MRI-0923131, National Science Foundation of China grant No. 60872069, and Zhejiang Provincial Natural Science Foundation of China grant 2011C11053.

Keywords

  • Bilateral filtering
  • Cost aggregation
  • Disparity map
  • Dynamic programming
  • Real-time stereo
  • Stereo video

ASJC Scopus subject areas

  • Information Systems

Fingerprint

Dive into the research topics of 'Real-time stereo using approximated joint bilateral filtering and dynamic programming'. Together they form a unique fingerprint.

Cite this