Snapshot Compressive ToF+Spectral Imaging via Optimized Color-Coded Apertures

Hoover Rueda-Chacon, Juan F. Florez-Ospina, Daniel L. Lau, Gonzalo R. Arce

Research output: Contribution to journalArticlepeer-review

20 Scopus citations


Compressive multispectral imaging systems comprise a new generation of spectral imagers that capture coded projections of a scene where spectral data cubes are reconstructed computationally. Separately, time-of-flight (ToF) cameras obtain 2D range images where each pixel records the distance from the camera sensor to the target surface. The demand for these imaging modalities is rapidly increasing, and thus, there is strong interest in developing new image sensors that can simultaneously acquire multispectral-color-and-depth imagery (MS+D) using a single aperture. Work in this path has been mainly developed via RGB+D imaging. However, in RGB+D, the multispectral image is limited to three spectral channels, and the imaging system often relies on two image sensors. We recently proposed a compressive MS+D imaging device that used a digital-micromirror-device, requiring a bulky double imaging-and-relay path. To overcome the bulkiness and other difficulties of our previous imaging system, this work presents a more-compact MS+D imaging device with snapshot capabilities. It provides better spectral sensing, relying on a static color-coded-aperture (CCA) and a ToF sensor. To guarantee good quality in the recovery, we develop an optimization method for CCA based-on blue-noise-multitoning, solved via the direct-binary-search algorithm. A testbed-setup is reported along with simulated and real experiments that demonstrate the MS+D capabilities of the proposed system over static and dynamic scenes.

Original languageEnglish
Article number8695846
Pages (from-to)2346-2360
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number10
StatePublished - Oct 1 2020

Bibliographical note

Funding Information:
The authors thank Intel Corporation and the National Science Foundation for the grant No. 1538950 under the Visual and Experiential Computing initiative. Hoover Rueda-Chacon and Juan F. Florez-Ospina acknowledge Colciencias and Fulbright for their doctoral scholarships.

Publisher Copyright:
© 1979-2012 IEEE.


  • Compressive spectral imaging
  • MS+D
  • RGB+D
  • color-coded apertures
  • multitoning
  • time-of-flight imaging

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Snapshot Compressive ToF+Spectral Imaging via Optimized Color-Coded Apertures'. Together they form a unique fingerprint.

Cite this