Interest points CSE P 576 Ali Farhadi Many slides from Steve Seitz, Larry Zitnick.
Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond...
-
date post
20-Jan-2016 -
Category
Documents
-
view
232 -
download
0
Transcript of Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond...
![Page 1: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/1.jpg)
Steve SeitzDept. Computer Science & Eng.
University of Washington
3D Photography:Beyond Perspective
![Page 2: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/2.jpg)
Perspective Projection
Humans evolved with perspective eyes• Capture light along rays that converge at a single point
![Page 3: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/3.jpg)
Cameras also evolved with perspective optics
• Optimized for humans, not computers!
![Page 4: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/4.jpg)
A non-perspective image
![Page 5: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/5.jpg)
![Page 6: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/6.jpg)
The Blue Marble, NASA satellite image composite
![Page 7: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/7.jpg)
The Blue Marble, NASA satellite image composite
![Page 8: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/8.jpg)
The Blue Marble, NASA satellite image composite
![Page 9: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/9.jpg)
Cyberware Scanner
![Page 10: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/10.jpg)
Print Gallery, by M.C. Escher, 1956
![Page 11: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/11.jpg)
Panorama from Disney’s 1940 film Pinocchio (from Wood et al., SIGGRAPH 1997)
![Page 12: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/12.jpg)
![Page 13: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/13.jpg)
What’s an Image?
An image is any 2D subset of rays in space• Actually, the light energy flowing along these rays• Any 2D “slice” of the plenoptic function
perspective image general image
![Page 14: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/14.jpg)
Non-perspective Imaging
Issues:• What other types of images are possible?• Which images are useful?• How can we capture these images?
![Page 15: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/15.jpg)
Path Images
![Page 16: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/16.jpg)
Path Images
![Page 17: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/17.jpg)
Path Images
x
y
t
![Page 18: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/18.jpg)
Path Images
x
y
t
![Page 19: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/19.jpg)
Path Images
Pushbroom images• satellite
Bolles et al. [87]• EPI
Tsuji et al. [92]• omni-directional image
Peleg et al. [97]• manifold mosaic
Radamacher & Bishop [99]• MCOP
. . .
x
y
t
![Page 20: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/20.jpg)
Linear Path
input pushbroom images
Video Cube Demo• application by Michael Cohen et al., Microsoft
![Page 21: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/21.jpg)
Circular Path
panorama(“concentric mosaic”)
circular EPI
input image
![Page 23: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/23.jpg)
What are these images good for?• Applications to computer graphics,
computer vision?
![Page 24: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/24.jpg)
Circular Stereo• [Ishiguro, Yamamoto, Tsuji, 92]
• [Peleg and Ben-Ezra, 99]
• [Shum, Kalai, Seitz, 99]
• [Nayar and Karmarkar, 00]
I
y
x x0 +n
x0 -n x
0
![Page 25: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/25.jpg)
Stereo Panorama
Disparity map result
![Page 26: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/26.jpg)
Stereo Panorama
Dark--close, light--far
![Page 27: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/27.jpg)
Circular Stereo
Advantages• 360 degree scene reconstruction• Uniform accuracy, optimal
![Page 28: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/28.jpg)
Pushbroom Stereo
y
x tx0 +n
x0 -n x
0
![Page 29: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/29.jpg)
Stereo Cyclographs
y
x x0 +n
x0 -n x
0
![Page 30: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/30.jpg)
![Page 31: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/31.jpg)
![Page 32: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/32.jpg)
Stereo Cyclograph Reconstruction
Computed from two cyclograph images• Using unmodified stereo matcher [Zitnick & Kanade]
![Page 33: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/33.jpg)
Stereo Path Images
Do these all produce stereo pairs?• two images with horizontal parallax
Yes
![Page 34: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/34.jpg)
Stereo Path Images
How about this path? No• Yes if the camera path is a conic
– line, circle, ellipse, parabola, hyperbola
• Must capture rays lying on doubly-ruled quadrics– [Padja 2001], [Seitz 2001]
![Page 35: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/35.jpg)
Stereo Parabolic Panoramas
perspective image
parabolic panorama
![Page 36: Steve Seitz Dept. Computer Science & Eng. University of Washington 3D Photography: Beyond Perspective.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649d385503460f94a122a1/html5/thumbnails/36.jpg)
Beyond PerspectiveCameras for humans, not machines!
• Need to rethink cameras• Image should suit the task
Future: cameras will evolve like CPU’s• First: task-specific cameras• Then: programmable cameras
– FPGA programmable camera arrays
Thanks to Jiwon Kim, Michael Cohen