More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry...
Transcript of More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry...
![Page 1: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/1.jpg)
More on single-view geometry class 10
Multiple View GeometryComp 290-089Marc Pollefeys
![Page 2: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/2.jpg)
Multiple View Geometry course schedule
(subject to change)
Jan. 7, 9 Intro & motivation Projective 2D Geometry
Jan. 14, 16 (no class) Projective 2D Geometry
Jan. 21, 23 Projective 3D Geometry (no class)
Jan. 28, 30 Parameter Estimation Parameter Estimation
Feb. 4, 6 Algorithm Evaluation Camera Models
Feb. 11, 13 Camera Calibration Single View Geometry
Feb. 18, 20 Epipolar Geometry 3D reconstruction
Feb. 25, 27 Fund. Matrix Comp. Structure Comp.
Mar. 4, 6 Planes & Homographies Trifocal Tensor
Mar. 18, 20 Three View Reconstruction Multiple View Geometry
Mar. 25, 27 MultipleView Reconstruction Bundle adjustment
Apr. 1, 3 Auto-Calibration Papers
Apr. 8, 10 Dynamic SfM Papers
Apr. 15, 17 Cheirality Papers
Apr. 22, 24 Duality Project Demos
![Page 3: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/3.jpg)
Single view geometry
Camera model
Camera calibration
Single view geom.
![Page 4: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/4.jpg)
Gold Standard algorithmObjective
Given n≥6 2D to 2D point correspondences {Xi
↔xi
’}, determine the Maximum Likelyhood Estimation of P
Algorithm(i)
Linear solution:(a)
Normalization: (b)
DLT
(ii)
Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:
(iii)
Denormalization:
ii UXX~ = ii Txx~ =
UP~TP -1=
~ ~~
![Page 5: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/5.jpg)
![Page 6: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/6.jpg)
![Page 7: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/7.jpg)
![Page 8: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/8.jpg)
More Single-View Geometry
•
Projective cameras and planes, lines, conics and quadrics.
•
Camera calibration and vanishing points, calibrating conic and the IAC
** CPPQ =T
coneQCPP =T
![Page 9: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/9.jpg)
Action of projective camera on planes
[ ] [ ]⎥⎥⎦
⎤
⎢⎢⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡==
1ppp
10ppppPXx 4214321 Y
XYX
The most general transformation that can occur between a scene plane and an image plane under perspective imaging is a plane projective transformation
(affine camera-affine transformation)
![Page 10: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/10.jpg)
Action of projective camera on lines
forward projection
( ) μbaμPBPAμB)P(AμX +=+=+=
back-projection
lPT=Π
PXlX TT =Π
![Page 11: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/11.jpg)
Action of projective camera on conics
back-projection to cone
CPPQ Tco =
[ ] ⎥⎦⎤
⎢⎣⎡=⎥⎦
⎤⎢⎣⎡= 00
0CKK0|KC0KQ
TT
T
co
example:
![Page 12: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/12.jpg)
Images of smooth surfaces
The contour generator Γ
is the set of points X on S at which rays are tangent to the surface. The corresponding apparent contour γ
is the set of points x which are the image of X, i.e. γ
is the image of Γ
The contour generator Γ
depends only on position of projection center, γ
depends also on rest of P
![Page 13: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/13.jpg)
Action of projective camera on quadrics
back-projection to cone
TPPQC ** = 0lPPQlQ T*T*T ==ΠΠ
The plane of Γ
for a quadric Q is camera center C is given by Π=QC (follows from pole-polar relation)
The cone with vertex V and tangent to the quadric Q is the degenerate Quadric: TT
CO (QV)(QV)-QV)QV(Q = 0VQCO =
![Page 14: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/14.jpg)
The importance of the camera center
]C~|[IR'K'P'],C~|KR[IP −=−=
( ) PKRR'K'P' -1=
( ) ( ) xKRR'K'PXKRR'K'XP'x' -1-1 ===
( )-1KRR'K'HHx with x' ==
![Page 15: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/15.jpg)
Moving the image plane (zooming)
( ) xKK'0]X|[IK'x'0]X|K[Ix
1-===
( ) ⎥⎦⎤
⎢⎣⎡ −
==10
x~k)(1kIKK'H T01-
⎥⎦⎤
⎢⎣⎡=⎥⎦
⎤⎢⎣⎡=
⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡ −
=⎥⎦⎤
⎢⎣⎡ −
=
100kIK
10x~kA
10x~A
10x~k)(1kIK
10x~k)(1kIK'
TT0
T0
T0
T0
'/ ffk =
![Page 16: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/16.jpg)
Camera rotation
xKRK0]X|K[Rx'0]X|K[Ix
1-===
-1KRKH =
conjugate rotation
{ }θθ ii ee −μ,μμ,
![Page 17: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/17.jpg)
Synthetic view
(i)
Compute the homography that warps some a rectangle to the correct aspect ratio
(ii)
warp the image
![Page 18: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/18.jpg)
Planar homography mosaicing
![Page 19: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/19.jpg)
close-up: interlacingcan be important problem!
![Page 20: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/20.jpg)
Planar homography mosaicingmore examples
![Page 21: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/21.jpg)
Projective (reduced) notation
T4
T3
T2
T1 )1,0,0,0(X,)0,1,0,0(X,)0,0,1,0(X,)0,0,0,1(X ====
T4
T3
T2
T1 )1,1,1(x,)1,0,0(x,)0,1,0(x,)0,0,1(x ====
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−−−
=dcdbda
000000
P
Tdcba ),,,(C 1111 −−−−=
![Page 22: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/22.jpg)
Moving the camera center
motion parallax
epipolar line
![Page 23: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/23.jpg)
What does calibration give?
xKd 1−=
⎥⎦⎤
⎢⎣⎡= 0d0]|K[Ix
( )( ) ( )( )21-T-T
211-T-T
1
2-1-TT
1
2T
21T
1
2T
1
)xK(Kx)xK(Kx
)xK(Kx
dddd
ddcos ==θ
An image l defines a plane through the camera center with normal n=KTl measured in the camera’s Euclidean frame
![Page 24: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/24.jpg)
The image of the absolute conic
KRd0d]C~|KR[IPXx =⎟⎠⎞⎜
⎝⎛−== ∞
mapping between π∞
to an image is given by the planar homogaphy x=Hd, with H=KR
image of the absolute conic (IAC)
( ) 1-T-1T KKKKω ==− ( )1TCHHC −−a
(i)
IAC depends only on intrinsics(ii)
angle between two rays(iii)
DIAC=ω*=KKT
(iv)
ω ⇔ K (cholesky factorisation)(v)
image of circular points
( )( )2T
21T
1
2T
1
ωxxωxx
ωxxcos =θ
![Page 25: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/25.jpg)
A simple calibration device
(i)
compute H for each square (corners (0,0),(1,0),(0,1),(1,1))
(ii)
compute the imaged circular points H(1,±i,0)T
(iii)
fit a conic to 6 circular points(iv)
compute K from ω
through cholesky factorization
(= Zhang’s calibration method)
![Page 26: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/26.jpg)
Orthogonality = pole-polar w.r.t. IAC
![Page 27: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/27.jpg)
The calibrating conic
1T K1
11
KC −−
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡
−=
![Page 28: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/28.jpg)
Vanishing points( ) ( ) λKdaλPDPAλPXλx +=+==
( ) ( ) KdλKda limλ xlimvλλ
=+==∞→∞→
KdPXv == ∞
![Page 29: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/29.jpg)
ML estimate of a vanishing point from imaged parallel scene lines
![Page 30: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/30.jpg)
Vanishing lines
![Page 31: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/31.jpg)
![Page 32: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/32.jpg)
![Page 33: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/33.jpg)
Orthogonality relation
( )( )2T
21T
1
2T
1
ωvvωvv
ωvvcos =θ
![Page 34: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/34.jpg)
![Page 35: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/35.jpg)
![Page 36: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/36.jpg)
![Page 37: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/37.jpg)
![Page 38: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/38.jpg)
Five constraints gives us five equations and can determine w
![Page 39: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/39.jpg)
Calibration from vanishing points and lines
Assumes zero skew, square pixels and 3 orthogonal
vanishing points
Principal point is the orthocenter of the trinagle made of 3 orthogonol vanishing lines
![Page 40: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/40.jpg)
Assume zero skew, square pixels, calibrating conic is a circle;How to find it, so that we can get K?
![Page 41: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/41.jpg)
Assume zero skew , square pixels, and principal point is at the image centerThen IAC is diagonal{1/f^2, 1/f^2,1) i.e. one degree of freedom need one more Constraint to determine f, the focal length two vanishing points corresponding To orthogonal directions.
![Page 42: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/42.jpg)
![Page 43: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/43.jpg)
![Page 44: More on single-view geometry - EECS Instructional Support ...€¦ · More on single-view geometry class 10 Multiple View Geometry Comp 290-089. Marc Pollefeys. Multiple View Geometry](https://reader030.fdocuments.in/reader030/viewer/2022041020/5ecfcae51530b816ba5ec8ec/html5/thumbnails/44.jpg)
Next class: Two-view geometry Epipolar geometry