Advanced Features Jana Kosecka CS223b Slides from: S. Thurn, D. Lowe, Forsyth and Ponce.
Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick...
-
date post
19-Dec-2015 -
Category
Documents
-
view
220 -
download
4
Transcript of Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick...
![Page 1: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/1.jpg)
Stanford CS223B Computer Vision, Winter 2005
Lecture 5: Stereo I
Sebastian Thrun, Stanford
Rick Szeliski, Microsoft
Hendrik Dahlkamp and Dan Morris, Stanford
StereoStereo
![Page 2: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/2.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Stereo Vision: Illustration
http://www.well.com/user/jimg/stereo/stereo_list.html
![Page 3: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/3.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Stereo Vision: Outline
Basic Equations Epipolar Geometry Image Rectification Reconstruction Correspondence Dense and Layered Stereo (Active Range Imaging Techniques)
![Page 4: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/4.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Pinhole Camera Model
Imageplane Focal length f
Center ofprojection
![Page 5: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/5.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Pinhole Camera Model
Imageplane
),,( ZYXP
),,( ZYXP
f
Oy
x
z
Z
Z
Y
Y
X
X
ZZ
YY
XX
OPPO
![Page 6: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/6.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Pinhole Camera Model
Imageplane
),,( ZYXP
),,( ZYXP
f
Oy
x
z
)1,,()1,,(),,(Z
Yf
Z
XfyxZYX
YyXxZ
YfY
Z
XfXfZ
![Page 7: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/7.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Basic Stereo Derivations
),,(1 ZYXP 1Oy
x
z
f
2Oy
x
z
B
BfxxZ ,,, offunction a as for expression Derive 21
1p
2p
![Page 8: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/8.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Basic Stereo Derivations
),,(1 ZYXP 1Oy
x
z
f
2Oy
x
z
B
211
11
1
12
1
11 ,
xx
BfZ
Z
Bfx
Z
BXfx
Z
Xfx
![Page 9: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/9.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
What If…?
),,(1 ZYXP 1Oy
x
z
f
2Oy
x
z
B
1p
2p
),,(1 ZYXP 1Oy
x
z
1p
f2O
y
x
z
2p
![Page 10: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/10.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Epipolar Geometry
pl pr
P
Ol Or
Xl
Xr
Pl Pr
fl fr
Zl
Yl
Zr
Yr
Rrotation Tontranslati
![Page 11: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/11.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Epipolar Geometry
plp
r
P
Ol Orel er
Pl Pr
Epipolar Plane
Epipolar Lines
Epipoles
![Page 12: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/12.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Epipolar Geometry
Epipolar plane: plane going through point P and the centers of projection (COPs) of the two cameras
Epipoles: The image in one camera of the COP of the other
Epipolar Constraint: Corresponding points must lie on epipolar lines
![Page 13: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/13.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Essential Matrix
pl pr
P
Ol Orel er
Pl Pr
Orthogonality T, Pl, PlT: 0)( lT
l PTTP
)( TPRP lr Coordinate Transformation:
0
0
0
xy
xz
yz
TT
TT
TT
S
ll SPPT
0)( lT
rT SPPR
0lT
r RSPP
0)( lT
rT PTPRResolves to
RSE Essential Matrix 0lT
r EPP
![Page 14: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/14.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Essential Matrix
pl pr
P
Ol Orel er
Pl Pr
0
0
0
xy
xz
yz
TT
TT
TT
SRSE Essential Matrix
0 lTr Epp0l
Tr EPP
Projective Line: lr Epu
![Page 15: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/15.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Fundamental Matrix
Same as Essential Matrix in Camera Pixel Coordinates
0lTr pFp
0lTr Epp
Pixel coordinates 1 lT
r EMMF
Intrinsic parameters
![Page 16: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/16.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Computing F: The Eight-Point Algorithm
Input: n point correspondences ( n >= 8)– Construct homogeneous system Ax= 0 from
• x = (f11,f12, ,f13, f21,f22,f23 f31,f32, f33) : entries in F• Each correspondence give one equation• A is a nx9 matrix
– Obtain estimate F^ by SVD of A:• x (up to a scale) is column of V corresponding to the least
singular value– Enforce singularity constraint: since Rank (F) = 2
• Compute SVD of F:• Set the smallest singular value to 0: D -> D’• Correct estimate of F :
Output: the estimate of the fundamental matrix F’ Similarly we can compute E given intrinsic
parameters
0lTr pFp
TUDVA
TUDVF ˆ
TVUDF' '
![Page 17: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/17.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Recitification
Idea: Align Epipolar Lines with Scan Lines.
Question: What type transformation?
![Page 18: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/18.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Locating the Epipoles
pl pr
P
Ol Orel er
Pl Pr
Input: Fundamental Matrix F– Find the SVD of F– The epipole el is the column of V corresponding to the
null singular value (as shown above)– The epipole er is the column of U corresponding to the
null singular value (similar treatment as for el) Output: Epipole el and er
TUDVF
el lies on all the epipolar lines of the left image
0lTr pFp
0lTr eFp
0leF
![Page 19: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/19.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Stereo Rectification (see Trucco)
Stereo System with Parallel Optical AxesEpipoles are at infinity
Horizontal epipolar lines
pl
pr
P
Ol Or
Xl
Xr
Pl Pr
Zl
Yl
Zr
Yr
T
![Page 20: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/20.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
pl
pr
P
Ol Or
Pl Pr
Reconstruction (3-D): Idealized
![Page 21: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/21.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
pl
pr
P
Ol Or
Pl Pr
Reconstruction (3-D): Real
See Trucco/Verri, pages 161-171
![Page 22: Stanford CS223B Computer Vision, Winter 2005 Lecture 5: Stereo I Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford.](https://reader036.fdocuments.in/reader036/viewer/2022062714/56649d385503460f94a11285/html5/thumbnails/22.jpg)
Sebastian Thrun Stanford University CS223B Computer Vision
Summary Stereo Vision (Class 1)
Epipolar Geometry: Corresponding points lie on epipolar
line
Essential/Fundamental matrix: Defines this line
Eight-Point Algorithm: Recovers Fundamental matrix
Rectification: Epipolar lines parallel to scanlines
Reconstruction: Minimize quadratic distance