## Abstract

We present a novel use of commodity graphics hardware that effectively combines a plane-sweeping algorithm [Collins 1996] and view synthesis in a single step for real-time, on-line 3D view synthesis. Unlike typical stereo algorithms that use image-based metrics to estimate depths, we focus on using image-based metrics to directly estimate images. Using real-time imagery from a few calibrated cameras, our method can generate new images from nearby viewpoints, without any prior geometric information or requiring any user interaction, in real time and on line. For a desired new view C_{n}, we discretize the 3D space into a number of candidate focal planes {D_{i}} parallel to the image plane Cn of the desired view. These candidate planes discretize each desired view ray into a finite set of sample points. We step through the set of candidate focal planes, looking for the sample point along each view ray that offers the maximum color consistency among input images. Each final image pixel is determined by the color at this point of maximum consistency. To accomplish this, for each candidate plane Di we project (texture) the input images onto that plane. We then render the resulting textured plane onto the image plane of C_{n} to get an image (I_{i}) of D_{i}. We combine these two operations into a single homography (planar-to-planar) transformation. In the first row of Figure 1, we show a number of images from different planes. Note that each of these images contains the projections from all input images, and the area corresponding to the intersection of objects and the correct candidate focal plane remains sharp. For each pixel location (u,v) in Ii, we compute the mean and Sum of Squared Difference (SSD) score. The final color of (u,v) is the color with minimum SSD score in {I_{i}}. We have discovered a novel use of graphics hardware to carry out the entire computation on the graphics board. Modern graphic cards, such as NVIDIA's GeForce series, provide a programable means for per-pixel fragment coloring through the use of register combiners [Kilgard 2000]. We exploit this programmability, together with the texture mapping functions, to carry out the entire computation on the graphics board.

Original language | English |
---|---|

Title of host publication | ACM SIGGRAPH 2002 Conference Abstracts and Applications, SIGGRAPH 2002 |

Pages | 240 |

Number of pages | 1 |

ISBN (Electronic) | 1581135254, 9781581135251 |

DOIs | |

State | Published - Jul 21 2002 |

Event | International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2002 - San Antonio, United States Duration: Jul 21 2002 → Jul 26 2002 |

### Publication series

Name | ACM SIGGRAPH 2002 Conference Abstracts and Applications, SIGGRAPH 2002 |
---|

### Conference

Conference | International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2002 |
---|---|

Country/Territory | United States |

City | San Antonio |

Period | 7/21/02 → 7/26/02 |

## ASJC Scopus subject areas

- Computer Graphics and Computer-Aided Design
- Software