Abstract
High-speed video has been commonly adopted in consumergrade cameras, augmenting these videos with a corresponding depth stream will enable new multimedia applications, such as 3D slow-motion video. In this paper, we present a hybrid camera system that combines a high-speed color camera with a depth sensor, e.g. Kinect depth sensor, to generate a depth stream that can produce both high-speed and high-resolution RGB+depth stream. Simply interpolating the low-speed depth frames is not satisfactory, where interpolation artifacts and lose in surface details are often visible. We have developed a novel framework that utilizes both shading constraints within each frame and optical ow constraints between neighboring frames. More specifically we present (a) an effective method to find the intrinsics images to allow more accurate normal estimation; and (b) an optimization-based framework to estimate the high-resolution/high-speed depth stream, taking into consideration temporal smoothness and shading/depth consistency. We evaluated our holistic framework with both synthetic and real sequences, it showed superior performance than previous state-of-the-art.
Original language | English |
---|---|
Title of host publication | MM 2016 - Proceedings of the 2016 ACM Multimedia Conference |
Pages | 878-887 |
Number of pages | 10 |
ISBN (Electronic) | 9781450336031 |
DOIs | |
State | Published - Oct 1 2016 |
Event | 24th ACM Multimedia Conference, MM 2016 - Amsterdam, United Kingdom Duration: Oct 15 2016 → Oct 19 2016 |
Publication series
Name | MM 2016 - Proceedings of the 2016 ACM Multimedia Conference |
---|
Conference
Conference | 24th ACM Multimedia Conference, MM 2016 |
---|---|
Country/Territory | United Kingdom |
City | Amsterdam |
Period | 10/15/16 → 10/19/16 |
Bibliographical note
Publisher Copyright:© 2016 ACM.
Keywords
- Depth stream
- High-speed imaging
- Intrinisic decomposition
- Shape from shading
ASJC Scopus subject areas
- Computer Graphics and Computer-Aided Design
- Human-Computer Interaction
- Computer Vision and Pattern Recognition
- Software