Automatic real-time video matting using time-of-flight camera and multichannel poisson equations

Liang Wang, Minglun Gong, Chenxi Zhang, Ruigang Yang, Cha Zhang, Yee Hong Yang

Research output: Contribution to journalArticlepeer-review

32 Scopus citations

Abstract

This paper presents an automatic real-time video matting system. The proposed system consists of two novel components. In order to automatically generate trimaps for live videos, we advocate a Time-of-Flight (TOF) camerabased approach to video bilayer segmentation. Our algorithm combines color and depth cues in a probabilistic fusion framework. The scene depth information returned by the TOF camera is less sensitive to environment changes, which makes our method robust to illumination variation, dynamic background and camera motion. For the second step, we perform alpha matting based on the segmentation result. Our matting algorithm uses a set of novel Poisson equations that are derived for handling multichannel color vectors, as well as the depth information captured. Realtime processing speed is achieved through optimizing the algorithm for parallel processing on graphics hardware. We demonstrate the effectiveness of our matting system on an extensive set of experimental results.

Original languageEnglish
Pages (from-to)104-121
Number of pages18
JournalInternational Journal of Computer Vision
Volume97
Issue number1
DOIs
StatePublished - Mar 2012

Bibliographical note

Funding Information:
Acknowledgements The authors would like to thank Mr. Mao Ye, Dr. Matt Steel and Dr. Melody Carswell for their help in data capture. This work is supported in part by University of Kentucky Research Foundation and US National Science Foundation award IIS-0448185, CPA-0811647, and MRI-0923131. Finally, the authors would like to thank anonymous reviewers for their constructive comments and suggestions.

Copyright:
Copyright 2012 Elsevier B.V., All rights reserved.

Keywords

  • Bilayer segmentation
  • Time-of-flight camera
  • Video matting

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Automatic real-time video matting using time-of-flight camera and multichannel poisson equations'. Together they form a unique fingerprint.

Cite this