High-speed depth stream generation from a hybrid camera

Xinxin Zuo, Sen Wang, Jiangbin Zheng, Ruigang Yang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations


High-speed video has been commonly adopted in consumergrade cameras, augmenting these videos with a corresponding depth stream will enable new multimedia applications, such as 3D slow-motion video. In this paper, we present a hybrid camera system that combines a high-speed color camera with a depth sensor, e.g. Kinect depth sensor, to generate a depth stream that can produce both high-speed and high-resolution RGB+depth stream. Simply interpolating the low-speed depth frames is not satisfactory, where interpolation artifacts and lose in surface details are often visible. We have developed a novel framework that utilizes both shading constraints within each frame and optical ow constraints between neighboring frames. More specifically we present (a) an effective method to find the intrinsics images to allow more accurate normal estimation; and (b) an optimization-based framework to estimate the high-resolution/high-speed depth stream, taking into consideration temporal smoothness and shading/depth consistency. We evaluated our holistic framework with both synthetic and real sequences, it showed superior performance than previous state-of-the-art.

Original languageEnglish
Title of host publicationMM 2016 - Proceedings of the 2016 ACM Multimedia Conference
Number of pages10
ISBN (Electronic)9781450336031
StatePublished - Oct 1 2016
Event24th ACM Multimedia Conference, MM 2016 - Amsterdam, United Kingdom
Duration: Oct 15 2016Oct 19 2016

Publication series

NameMM 2016 - Proceedings of the 2016 ACM Multimedia Conference


Conference24th ACM Multimedia Conference, MM 2016
Country/TerritoryUnited Kingdom

Bibliographical note

Publisher Copyright:
© 2016 ACM.


  • Depth stream
  • High-speed imaging
  • Intrinisic decomposition
  • Shape from shading

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Software


Dive into the research topics of 'High-speed depth stream generation from a hybrid camera'. Together they form a unique fingerprint.

Cite this