Robert Collins CSE486, Penn State Generalized Stereortc12/CSE486/lecture18.pdfEpipolar Geometry...
5
1 CSE486, Penn State Robert Collins Lecture 18: Generalized Stereo: Epipolar Geometry CSE486, Penn State Robert Collins Generalized Stereo Key idea: Any two images showing an overlapping view of the world can be treated as a stereo pair... ... we just have to figure out how the two views are related. Some of the most “beautiful” math in vision concerns describing how multiple views are related, geometrically. CSE486, Penn State Robert Collins Recall: Epipolar Constraint Important Stereo Vision Concept: Given a point in the left image, we don’t have to search the whole right image for a corresponding point. The “epipolar constraint” reduces the search space to a one-dimensional line. CSE486, Penn State Robert Collins Review : Simple Stereo System Left camera Right camera z x y z x y T x Same Y Coord! Epipolar line P Equation relating depth and disparity baseline disparity depth CSE486, Penn State Robert Collins Review: Epipolar Constraint Corresponding features are constrained to lie along conjugate epipolar lines (on the same row in the case of our simple setup). CSE486, Penn State Robert Collins General Stereo O 2 O 1 z 1 z 2 x 1 y 1 y 2 x 2 C 1 C 2 In general, the cameras may be related by In general, the cameras may be related by an arbitrary transformation (R,T) an arbitrary transformation (R,T) In general, intrinsic camera parameters In general, intrinsic camera parameters may be different, and even unknown may be different, and even unknown Epipolar Epipolar Matrix Matrix Fundamental Matrix Fundamental Matrix
Transcript of Robert Collins CSE486, Penn State Generalized Stereortc12/CSE486/lecture18.pdfEpipolar Geometry...
1
CSE486, Penn StateRobert Collins
Lecture 18:
Generalized Stereo:
Epipolar Geometry
CSE486, Penn StateRobert Collins
Generalized Stereo
Key idea: Any two images showing an overlapping view of the world can be treated as a stereo pair...
... we just have to figure out how the two views are related.
Some of the most “beautiful” math in vision concernsdescribing how multiple views are related, geometrically.
CSE486, Penn StateRobert Collins
Recall: Epipolar Constraint
Important Stereo Vision Concept:
Given a point in the left image, we don’t have to search the whole right image for a corresponding point.
The “epipolar constraint” reduces the search spaceto a one-dimensional line.
CSE486, Penn StateRobert Collins
Review : Simple Stereo System
Left camera
Right camera
zz
xx
yyzz
xx
yy
TTxx
Same Y Coord!
Epipolar line
P
Equation relatingdepth and disparity
baseline
disparity
depth
CSE486, Penn StateRobert Collins
Review: Epipolar Constraint
Corresponding features are constrained tolie along conjugate epipolar lines (on thesame row in the case of our simple setup).
CSE486, Penn StateRobert Collins
General Stereo
OO22
OO11
zz11 zz22xx11
yy11
yy22
xx22
CC11
CC22
In general, the cameras may be related byIn general, the cameras may be related byan arbitrary transformation (R,T)an arbitrary transformation (R,T)
In general, intrinsic camera parametersIn general, intrinsic camera parametersmay be different, and even unknownmay be different, and even unknown
EpipolarEpipolar MatrixMatrix
Fundamental MatrixFundamental Matrix
2
CSE486, Penn StateRobert Collins
EPIPOLAR GEOMETRY
CSE486, Penn StateRobert Collins Epipolar Geometry
eerreell OOrr
pprr
PP
ppll
OOll
EpipolesEpipoles::•• eell: left image of O: left image of Orr•• eerr: right image of : right image of OOll
EpipolarEpipolar plane:plane:•• Three points: Three points: OOll,O,Orr, and P define an , and P define an epipolarepipolar planeplaneEpipolarEpipolar lines and lines and epipolarepipolar constraint:constraint:•• Intersections of Intersections of epipolarepipolar plane with the image planesplane with the image planes•• Corresponding points are on Corresponding points are on ““conjugateconjugate”” epipolarepipolar lineslines
The following slides are from Dr.Camps, PSU
CSE486, Penn StateRobert Collins
BORING!!!Let’s try again...
CSE486, Penn StateRobert Collins
EPIPOLAR GEOMETRY
CSE486, Penn StateRobert Collins Epipolar Geometry
A Visualization
Would would Pinhead’s eye look like close up?
CSE486, Penn StateRobert Collins
Rays to Points in Scene
Tie threads onto the pinsand connectfocal pointto scenepoints
Now what would this look like to a second observer?
3
CSE486, Penn StateRobert Collins
Rays Seen from Second Observer
Q u ic k T im e ™ a n d a
T I F F ( L Z W ) d e c o m p r e s s o r
a r e n e e d e d t o s e e t h is p ic t u r e .
Image 2
epipole
epipolar lines
CSE486, Penn StateRobert Collins
Rays Seen by the First Viewer
Image 1
epipole
epipolar lines
CSE486, Penn StateRobert Collins
Epipolar Geometry
image1 image 2
Epipole : location of cam2as seen by cam1.
Epipole : location of cam1as seen by cam2.
CSE486, Penn StateRobert Collins
Epipolar Geometry
image1 image 2
Corresponding pointslie on conjugate epipolar lines
CSE486, Penn StateRobert Collins
Epipolar Geometry
image1 image 2
Conjugate epipolar lines inducea generalized 1D “scan-line” orderingon the images (analogous to traditionalscan line ordering of rows in an image)
CSE486, Penn StateRobert Collins
Epipole not Necessarily in Image
Q u ic k T im e ™ a n d a
T I F F ( L Z W ) d e c o m p r e s s o r
a r e n e e d e d t o s e e t h is p ic t u r e .
Image 2
epipole
epipolar lines
4
CSE486, Penn StateRobert Collins Epipolar Geometry
eerreell OOrr
pprr
PP
ppll
OOll
EpipolesEpipoles::•• eell: left image of O: left image of Orr•• eerr: right image of : right image of OOll
EpipolarEpipolar plane:plane:•• Three points: Three points: OOll,O,Orr, and P define an , and P define an epipolarepipolar planeplaneEpipolarEpipolar lines and lines and epipolarepipolar constraint:constraint:•• Intersections of Intersections of epipolarepipolar plane with the image planesplane with the image planes•• Corresponding points are on Corresponding points are on ““conjugateconjugate”” epipolarepipolar lineslines
The following slides are from Dr.Camps, PSU
CSE486, Penn StateRobert Collins
Epipolar Constraint:
Given Given EpipolesEpipoles::•• eell: left image of O: left image of Orr•• eerr: right image of : right image of OOll
Given pGiven pll::••consider its consider its epipolarepipolar line: pline: pll eell••find find epipolarepipolar plane: plane: OOll,p,pll,e,ell••intersect the intersect the epipolarepipolar plane with the right image planeplane with the right image plane••search for psearch for prr on the right on the right epipolarepipolar lineline
eerreell OOrr
ppll
OOll
PP
pprr
CSE486, Penn StateRobert Collins
Essential Matrix
eerreell OOrr
pprr
PP
ppll
OOllTT
PPll PPrr
Does this look familiar? Recall world to cameratransformation by (R,T). Here, we are transformingfrom camera to camera.
CSE486, Penn StateRobert Collins
Essential Matrix
eerreell OOrr
pprr
PP
ppll
OOllTT
PPll PPrr
CSE486, Penn StateRobert Collins
Essential Matrix
eerreell OOrr
pprr
PP
ppll
OOllTT
EpipolarEpipolar constraint: Pconstraint: Pll, T and P, T and Pll -- T are coplanar:T are coplanar:
PPll PPrr
CSE486, Penn StateRobert Collins
Essential Matrix
eerreell OOrr
pprr
PP
ppll
OOllTT
EpipolarEpipolar constraint: Pconstraint: Pll, T and P, T and Pll -- T are coplanar:T are coplanar:
PPll PPrr
5
CSE486, Penn StateRobert Collins Vector Product as a Matrix
Multiplication
S has rank 2 ; it depends only on TS has rank 2 ; it depends only on T
CSE486, Penn StateRobert Collins
Essential Matrix
eerreell OOrr
pprr
PP
ppll
OOllTT
EpipolarEpipolar constraint: Pconstraint: Pll, T and P, T and Pll -- T are coplanar:T are coplanar:
PPll PPrr
CSE486, Penn StateRobert Collins
Essential Matrix
eerreell OOrr
pprr
PP
ppll
OOllTT
EpipolarEpipolar constraint: Pconstraint: Pll, T and P, T and Pll -- T are coplanar:T are coplanar:
PPll PPrr
Essential Matrix:Essential Matrix:
CSE486, Penn StateRobert Collins
Essential Matrix Properties
• has rank 2
• depends only on the EXTRINSIC Parameters (R & T)
We will discuss more of the wonderful properties of this matrix next time...
