Single Aperture Spectral+ToF Compressive Camera: Toward Hyperspectral+Depth Imagery

Spectral imaging involves the sensing of a large amount of spatial information across a multitude of wavelengths. Conventional approaches rely on scanning techniques to construct a spectral data cube. Recently, compressive spectral imaging (CSI) has allowed to estimate spectral images with as few as a single coded snapshot. On a different front, 3-D ranging imaging often involves scanning along one of the spatial dimensions to estimate the depth of an scene using structured light, or the use of two cameras as required by stereo-imaging techniques. Recently, Time-of-Flight (ToF) snapshot imaging has gained considerable attention, due to its accuracy and speed. To date, however, these imaging modalities (CSI and ToF) have been realized and implemented by separate independent imaging sensors. This paper presents the development of a single aperture compressive spectral + depth imaging camera that employs a commodity 3-D range ToF sensor as the sensing device of a coded-aperture-based compressive spectral imager. The proposed system uses a single aperture/single sensor; thus, representing a significant improvement over existing RGB+D cameras that integrate two separate image sensors, one for RGB and another for depth. In addition, the observable wavelength range of the CSI device is expanded from the visible to the near-infrared, resolving up to as many as 16 independent channels. The proposed system allows the addition of side-information by placing a grayscale or RGB camera in the same path of the single-aperture system, such that the quality of the spectral estimation is improved, while maintaining high-frame rates. We demonstrate the proposed ideas through real experimentation conducted on an assembled CSI+ToF testbed camera system.