Finding depth
description
Transcript of Finding depth
![Page 1: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/1.jpg)
1
Finding depth
![Page 2: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/2.jpg)
2
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 3: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/3.jpg)
3
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 4: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/4.jpg)
4
Depth from stereo
C1
P
C2
u2
v2
u1
v1P1 P2
• two cameras with known parameters
• infer 3D location of point seen in both images
• sub problem: correspondences
• for a point seen in the left image, find its projection in the right image
![Page 5: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/5.jpg)
5
Depth from stereo: déjà-vu math
C1
P
C2
u2
v2
u1
v1
2222222
1111111
)(
)(
wbvaucCP
wbvaucCP
P1P2
• unknowns are w1 and w2
• overconstrained system
• the u2v2 coordinates of a point seen at u1v1 are constrained to an epipolar line
![Page 6: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/6.jpg)
6
Epipolar line
C1
P
C2
u2
v2
u1
v1
P1
P2
• C1, C2, P1 define a plane
• P2 will be on that plane
• P2 is also on the image plane 2
• So P2 will be on the line defined by the two planes’ intersection
![Page 7: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/7.jpg)
7
Search for correspondences on epipolar line
C1
P
C2
u2
v2
u1
v1
P1
P2
• Reduces the dimensionality of the search space
• Walk on epipolar segment rather than search in entire image
![Page 8: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/8.jpg)
8
Parallel views
C1
P
C2
u2
v1
u1
v1
• Preferred stereo configuration
• epipolar lines are horizontal, easy to search
![Page 9: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/9.jpg)
9
Parallel views
C1
P
C2
u2
v1
u1
v1
u1
• Limit search to epipolar segment
• from u2 = u1 (P is infinitely far away) to 0 (P is close)
![Page 10: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/10.jpg)
10
Depth precision analysis
C1
P
C2
u2
v1
u1
v1
• 1/z linear with disparity (u1 – u2)
• better depth resolution for nearby objects
• important to determine correspondences with subpixel accuracy
![Page 11: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/11.jpg)
11
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 12: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/12.jpg)
12
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 13: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/13.jpg)
13
Depth from stereo problem
• Correspondences are difficult to find
• Structured light approach– replace one camera with projector– project easily detectable patterns– establishing correspondences becomes a lot
easier
![Page 14: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/14.jpg)
14
Depth from structured light
C1
P
C2
u2
v2
u1
v1
2222222
1111111
)(
)(
wbvaucCP
wbvaucCP
P1 P2
• C1 is a projector
• Projects a pattern centered at u1v1
• Pattern center hits object scene at P
• Camera C2 sees pattern at u2v2, easy to find
• 3D location of P is determined
![Page 15: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/15.jpg)
![Page 16: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/16.jpg)
16
![Page 17: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/17.jpg)
17
Depth from structured light challenges
• Associated with using projectors– expensive, cannot be used outdoors, not
portable
• Difficult to identify pattern– I found a corner, which corner is it?
• Invasive, change the color of the scene– one could use invisible light, IR
![Page 18: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/18.jpg)
18
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 19: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/19.jpg)
19
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 20: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/20.jpg)
20
Depth of field
aperture
object
image
C F
F’
• Thin lenses
• rays through lens center (C) do not change direction
• rays parallel to optical axis go through focal point (F’)
![Page 21: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/21.jpg)
21
Depth of field
aperture
object
image plane
C F
F’
• For a given focal length, only objects that are at a certain depth are in focus
![Page 22: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/22.jpg)
22
Out of focus
aperture
object
image plane
C F
F’
• When object at different depth
• One point projects to several locations in the image
• Out of focus, blurred image
![Page 23: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/23.jpg)
23
Focusing
aperture
object
image plane
C F
F’
• Move lens to focus for new depth
• Relationship between focus and depth can be exploited to extract depth
![Page 24: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/24.jpg)
24
Determine z for points in focus
aperture
object
image plane
C F
F’
z
fa
h
h
f
a i
hi
h
a f
z
![Page 25: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/25.jpg)
25
Depth from defocus
• Take images of a scene with various camera parameters
• Measuring defocus variation, infer range to objects
• Does not need to find the best focusing planes for the various objects
• Examples by Shree Nayar, Columbia U
![Page 26: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/26.jpg)
26
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 27: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/27.jpg)
27
Overview
• Depth from stereo
• Depth from structured light
• Depth from focus / defocus
• Laser rangefinders
![Page 28: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/28.jpg)
28
Laser range finders
• Send a laser beam to measure the distance– like RADAR, measures time of flight
![Page 29: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/29.jpg)
29
DeltaSphere - depth&color acquisition device
• Lars Nyland et al.
courtesy 3rd Tech Inc.
![Page 30: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/30.jpg)
30
• 300o x 300o panorama
• this is the reflected light
![Page 31: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/31.jpg)
31
• 300o x 300o panorama
• this is the range light
![Page 32: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/32.jpg)
32
courtesy 3rd Tech Inc.
spherical range panoramas
planar re-projection
![Page 33: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/33.jpg)
33
courtesy 3rd Tech Inc.
Jeep – one scan
![Page 34: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/34.jpg)
34
courtesy 3rd Tech Inc.
Jeep – one scan
![Page 35: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/35.jpg)
35
courtesy 3rd Tech Inc.
Complete Jeep model
![Page 36: Finding depth](https://reader035.fdocuments.in/reader035/viewer/2022062422/56814096550346895dac2d22/html5/thumbnails/36.jpg)
36