Three Dimensional Light Display for Advanced Simulation & Training

This proposal aims to advance the state of the art for training and simulation tasks related to homeland security. America's first line of defense in any terrorist attack is the "first responder" community. First responders require special training. While computer-based simulators are already in use for such purposes, they are usually seen as marginally effective. One of the reasons is the lack of proper depth perception in a computer-simulated environment. Existing stereo techniques are limited by their narrow field of view, low resolution, and the long latency (head-tracking, display etc). And in particular, they do not scale beyond two or three persons. To effectively communicate and collaborate with others in a simulator, stereoscopic viewing is often sacrificed. To substantially improve the effectiveness of computer-based simulation and training systems for first responders, in particular to increase the visual fidelity by incorporating unrestricted stereoscopic viewing in an immersive environment, we propose to develop ajiexible and scalable three-dimensional display that will allow any number of trainees to simultaneously perceive from many different viewpoints full color, full parallax, full motion solid stereoscopic imagery, without resorting to headsets or user-tracking. Such a unique display can be the enabling technology for building more realistic and effective training and simulation systems for emergency and disaster scenarios. More specifically, we propose to use a dense array ofprojectors to electronically simulate an appropriate lightfield for a given scene (physical objects emit or reflect light in all directions-creating a light field). Users then look into the light field to perceive a three-dimensional scene. To accomplish this, our system must be capable of controlling the color and intensity of light rays passing through the viewing volume. In addition, there must be a greater number of rays (hence, projector pixels) that are significantly denser than can be found in traditional displays that display only one view at a time. We have explored potential configurations and are convinced that using a combination of commodity light projectors and PCs, specialized optics, and a novel camera-based calibration mechanism, such a display is feasible. Deliverables. We will pursue both the fundamental research and system integration issues towards the realization of the light-field display. The expected deliverables out of this project are .a desktop light field display constructed from four commodity video projectors, it should provide full parallax stereoscopic imagery for dynamic contents at video rate (i.e., over 30 fps); .a wall-size light field display constructed from at least 16 commodity video projectors. It should provide unrestricted immersive stereoscopic viewing for at least four people simultaneously. .a software package that performs display registration automatically. It can be used by technicians to set up and maintain light field displays; . a software package that can render in real-time special images for our light field display utilizing a cluster of commodity PCs; The input can be either a geometric model or a discreet light field captured by many cameras. While given the tight schedule and limited financial resources, it is beyond the scope of this project to deliver a full-fledged simulation and training system, our display has the potential to contribute to new systems that allow first responders in training to walk freely, examine objects at will, and collaborate with others as if they were in a "real" disaster scenario. Such a system would be a natural extension of this project. Our long term goal is to work with DHS experts to design and deliver a complete simulator that can better prepare the three million first responders for even unlikely emergency scenarios.