1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo...
-
Upload
vernon-reeves -
Category
Documents
-
view
220 -
download
2
Transcript of 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo...
![Page 1: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/1.jpg)
1
Stereo vision and
two-view geometry
• The goal of a stereo system is to get 3D information• A stereo system consists at least of two ‘converging’ cameras rigidly attached
Chapter 7 of the textbook
![Page 2: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/2.jpg)
2
One real example:
• stereo geometry: epipolar geometrygeometric relation between two images
• correspondencepixels (pts) in different images from the same pt
• reconstruction (triangulation)3d coordinate of the pt
Three topics:
![Page 3: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/3.jpg)
3
Intuitive epipolar geomegtry
Entirely characterized by the so-called epipolar geometry
Geometric concepts: • epipole: the image of the other camera center• epipolar plane: plane defined by the two camera centers and the space pt• epipolar lines: intersection of the epipolar plane and image plane• pencil of epipolar lines and planes• baseline: distance between two camera centers
![Page 4: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/4.jpg)
4
u
O
u’
Oepipole
epipolar line
epipolar plane
![Page 5: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/5.jpg)
5
One real example of epipolar lines:
![Page 6: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/6.jpg)
6
Algebraic characterisation of the epipolar geometry: the
fundamental matrixGiven a correspondence pair u and u’,
where F is 3 by 30 Fu u
T
Proof sketch: follow the geometric construction. • compute the epipole in the second image• compute the pt at infinity (or ray direction), reproject it onto the second• define the epipolar line by these two pts• use anti-symmetric matrix for cross-product
Assume the camera projection matrices are P=(I 0) and P’=(A a), it can be shown that F = [a] A.The procedure is the same even if P is of general form.
![Page 7: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/7.jpg)
7
• rank of F• ker(F) • how many d.o.f.• l’=F u • l = … •
0T
Fu u
Properties of the fundamental matrix F:
![Page 8: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/8.jpg)
8
Stereo Vision by ‘traditional’ calibrated approach
• calibration of each camera w.r.t. the same object: P and P’• (optional) rectification• disparity map using F• 3D reconstruction
Traditional (calibrated) stereo approach:
Correspondence using F (computed from P and P’)
two (or more) cameras rigidly attached=stereo rig=stereo system
![Page 9: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/9.jpg)
9
Obtain F from the given P and P’:
b B P a A P3 3 3 3 ,
o P ea A
o
,1
1
u BA x P uu A
x1
1
,0
Fu u BA e u e l
1
] [
![Page 10: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/10.jpg)
10
• Correspondence (discussed later) • 3D reconstruction : trianglulation
),(),,(),( iii iiiiiii vuzyxvu uXu
34333231
24232221
34333231
14131211
czcycxc
czcycxcv
czcycxc
czcycxcu
iii
iiii
iii
iiii
34333231
24232221
34333231
14131211
czcycxc
czcycxcv
czcycxc
czcycxcu
iii
iiii
iii
iiii
Same equation as the calibration, but unknowns are nowxi, yi, zi instead of cij
4
4 41
0
i
i
i
i
t
z
y
x
![Page 11: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/11.jpg)
11
u
O
u’
O’
Triangulation:
![Page 12: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/12.jpg)
12
‘Modern’ uncalibrated approach:Epipolar geometry by point
correspondences – two-view geometry
0 Fu uT
Because of
NB: it is more powerful, ‘calibration’ needs 3d info, point-correspondence does not, but not 3d reconstruction
![Page 13: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/13.jpg)
13
• 8 pts algorithm• 7 pts algorithm (minimal data)• (optimal and robust sol.)
ii uu Given compute F
![Page 14: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/14.jpg)
14
8-point algorithm (unstable)
0 Fu uT
0 33 32 31 23 22 21 13 12 11 f vf uf f v vf v uf v f u vf u uf u
0 )1, , , , , , , , (9 f v u v v v u v u v u u u
9 9
9
1, , , , , , , ,0 f
n
i i i i i i i i i i i iv u v v v uv u v u u u
9 9 90 f A n
T Tv u v u)1, , ( )1, , ( u u
expand
rewrite
for N points
rewrite
![Page 15: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/15.jpg)
15
• linear sol by svd with ||f||=1: f=v9• F’, rank enforcement afterwards by svd!
33 32 31
23 22 21
13 12 11
3 3 33 32 31 23 22 21 13 12 11 9) , , , , , , , , (
f f f
f f f
f f f
f f f f f f f f fF f
F V U
V U
F
T recompose
T decompose SVD
f f f
f f f
f f f
0 0 0
0 0
0 0
0 0
0 0
0 0
3
2
1
3
2
1
33 32 31
23 22 21
13 12 11
3 3
![Page 16: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/16.jpg)
16
7-point algorithm• one parameter solution by svd• from the vanishing determinant, get a cubic equation
9 8 9v v fy x
0 0 ) det(3 2 2 3 dy cxy y bx ax F
* * *
* * *
* * 11 11y c x c
F
02 3
d cz bz az
y
xz
![Page 17: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/17.jpg)
17
Normalisation 8 pt algorithmTo make the average point as close as possible to (1,1,1)!
• normalisation by transformations
• linear solution for
• rank enforcement
• denormalisation
Warning: unnormalised 8 pt algorithm is unstable!!!
u T u Tu u ˆ , ˆu uˆ ˆ
0 ˆ ˆ ˆ uF uT
F F ˆ ˆ
T F T F ˆ T
![Page 18: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/18.jpg)
18
Data normalisation: each image data is normlised independently!
1 0 0
0
0
) ( ) ( ,2
2 )) ( ( )) ( (1
1,
1
2 2
2 2
vs s
us s
v v u u dd
ns
v v s u u sn
vn
v un
u
i i ii
i i
i i
T
Tu uˆ
![Page 19: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/19.jpg)
19
Summary (or a unified view) of all methods of computation of the fundamental matrix
![Page 20: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/20.jpg)
20
Stereo correspondence
![Page 21: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/21.jpg)
21
• disparity: difference in image position of the same space pt• disparity map: dense pixel-to-pixel corrrespondences• stereo rectification: make the epipolar lines horizontal
an option to speed up the computation of disparity map
The epipolar geometry gives only a constraint, but not yet a unique solution to the question: where is the corresponding point in the second image of a given point in the first image?
![Page 22: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/22.jpg)
22
Rectification of a stereo pair of images: two images are transformed (by a projective transformation in image plane or by a camera rotation around the center).
u T u Tu u ˆ , ˆ
![Page 23: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/23.jpg)
23
New rectified image plane
• equivalent to a plane parallel to the base line• T and T’ can be computed from F, but many possibilities• only an option, simplify the computation!
![Page 24: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/24.jpg)
24
N
2n n p p n pd )) ( I ) I( ( min
Nn n p p n pd) ( I) I( max
or
Very often ZNCC (Zero Normalized Cross Correlation), on normalised images instead of I and I’,
Matching by correlation:
| || |) ( , ) () 1 2( ) 1 2( ) 1 2( ) 1 2(
n n
n nI I n I I n
n n n n
Convert all (2n+1)(2n+1) elements from a matrix into a vector of dim (2n+1)(2n+1)
n
) 1 2( ) 1 2( ) ( n n I I
![Page 25: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/25.jpg)
25
Two points u and u’ are in correspondence if • ZNCC(u,u’) is big enoug (close to 1)• dist(u’, Fu) is small enough (a few pixels)
![Page 26: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/26.jpg)
26
Correspondence by correlation:
• For each point u, compute all correlations in a neighborhood u+d with a window size s
• Take the pixel having the highest correlation score as the correspondence
• Cross-validate the correspondence in the opposite direction from the second to the first image
Correlation window
neighborhood
Cross-validate
![Page 27: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/27.jpg)
27
When applied to ‘interest points’, sparse correspondenceWhen applied to every pixel, dense disparity map
![Page 28: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/28.jpg)
28
Using more cameras to remove match ambiguity: a system of 3 cameras
1
2
3
![Page 29: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/29.jpg)
29
What can we do more with F?
• Without calibration, what can we get?
• When calibrated, essential matrix, its decomposition
![Page 30: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/30.jpg)
30
0 Fu uT
x K u Kx u ,
0 ) ( ) ( Kx F x KT
0 ) ( x FK K xT T
FK K E Ex xT T , 0
From uncalibrated F
Calibrated E
![Page 31: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/31.jpg)
31
Essential matrix: fundamental matrix for the calibrated points
The extra algebraic constraint: the equal singular values (more complicated) for E
Decomposition of the essential matrix into R and t
Relationship between E and F
E = [t] R From F = [a] A
![Page 32: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/32.jpg)
32
Decomposition of E
T(T)0
T0
VURR
UUSS
100
001
010
0R
000
001
010
0S
Two factorisation
3)ker( uSt Two translation
Twisted pair
SR[t]REUSVE ,T
![Page 33: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/33.jpg)
33
![Page 34: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/34.jpg)
34
• Given internal calibration K and K’
(more advanced studies allow us to remove this step by self-calibration that we will not handle)
• Compute F from point correspondences• Compute E• Decompose E to obtain R and t• Obtain P and P’• Triangulation
Summary of modern two-view approach:
![Page 35: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/35.jpg)
35
O
When space points are planar, a homography relating u and u’
u
O
u’
x
It is therefore a ‘collineation’ for COPLANAR points!
Never forget the coplanar case!
![Page 36: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/36.jpg)
36
From P=(I 0) , P’=(A a) and a known plane p^Tx=0,to get H
Or, the homography can be computed from at least 4 correspondingPoints, do it!
The homography uniquely determines point correspondences Unlike the fundamental matrix! But only for coplanar points.
)1,(, npanAH T
0,),( unxpaAuuuxP TT
So that Huu
![Page 37: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/37.jpg)
37
Panoramic image or image mosaicing
• the 3D scene is planar• the camera is rotating around the center (similar to rectification)
Example at HK airport (virtual tour), QuicktimeVRRealviz, stitcher, step-by-stephttp://iris.usc.edu/home/iris/elkang/iris-04/reports/2/techreport2.html
This homography leads to one important application:
![Page 38: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/38.jpg)
38
The images are related by a homography if the 3d scene is planar:
![Page 39: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/39.jpg)
39
x
A pencil of lines cut by two lines
A star of lines cut by two planes
Rotating the camera around the center is equivalent to a homographical Transformation of the image plane:
![Page 40: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/40.jpg)
40
• Compute point correspondences
• Compute the projective transformation between the two views
• Warp the first image onto the second
• Color-blend the overlapping areas
Compositing algorithm or mosacing algorithm:
![Page 41: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/41.jpg)
41
Quick-time VR
Inward-looking small object
Outward-looking large-scale environnement
Example of the virtual tour of HK airport: http://www.hkairport.com/eng/index.jsp
![Page 42: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/42.jpg)
42
Automatic computation of the fundamental matrix
Chicken-egg problem: we need corresponding points to compute F, we need F to establish correspondences …
Simultaneous automatic computation of correspondences and F
![Page 43: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/43.jpg)
43
Illustrative example of fitting a line to a set of 2D points
• the least squares solution (orthogonal regression) is optimal when no outliers• but it is becoming very fragile to outliers
![Page 44: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/44.jpg)
44
Robust line fitting
Fit a line to 2D data containing ‘bad points’---outliers
Solving two pbs: 1. A line fit to the data; 2. A classification of the data into ‘inliers’ and ‘outliers’‘inliers’: valid or good data satisfying the ‘line’ model‘outliers’: bad data not satisfying the model
![Page 45: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/45.jpg)
45
• randomly draw 2 data points• compute a line Li from these 2 points• compute the distance to the line Li for each data point• determine inliers/outliers by a threshold t• compute the number of inliers Si
• select the Li having the largest Si• re-estimate the final line using all inliers
How to find the best line?
repeating
then
![Page 46: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/46.jpg)
46
• randomly draw a sample of s data• initiate the model Mi• compute the distance to the model for each data pt• determine inliers/outliers by the threshold t• compute the size of inliers Si
• select the Mi having the largest Si• re-estimate using all inliers
Fischler and Bolles 1981
repeating
then
RANSAC (random sample consensus)
![Page 47: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/47.jpg)
47
The complete algorithm of automatic computation of F:
• detect points of interest in each image• compute the correspondences using correlation based method• RANSAC using 7-pt algo.• (non-linear optimal estimation on the final inliers: this is unnecessary in many cases, so just an option)
![Page 48: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/48.jpg)
48
• randomly draw a sample of 7 corresponding points• compute Fi• compute the distance to Fi for each corresponding pt• determine inliers/outliers by the threshold t• compute the size of inliers Si
• select the Fi having the largest Si• re-estimate the final F using all inliers
repeating
then
RANSAC using 7-pt algo to compute F
![Page 49: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/49.jpg)
49
))1(1log(/)1log( spN
Outlier proportion
s Sample size
Probability of success (only inliers)p
Ex: 99.9% success rate, 50% outliers s=2, N=17 s=4, N=72 s=6, N=293 s=7, N=588 s=8, N=1177
How many times to repeat?
![Page 50: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/50.jpg)
50
Robust statistics
From least-squares method to robust statistics (Ransac,least median of squares (LMS))
Handle ‘big errors’---outliers!
This is useful not only for computing F, but also for automatically computing a mosaicing of images!
![Page 51: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/51.jpg)
51
What tells us the F, the epipolar geometry?
![Page 52: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/52.jpg)
52
• Given internal calibration K and K’
(more advanced studies allow us to remove this step by self-calibration that we will not handle)
• Compute F from point correspondences• Compute E• Decompose E to obtain R and t• Obtain P and P’• Triangulation
Summary of ‘semi-modern’ two-view approach:
What do we obtain?
![Page 53: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/53.jpg)
53
What is a projective reconstruction?
What is a Euclidean (similarity) reconstruction?
A 3D reconstruction up to a 3D projective transformation.
A 3D reconstruction up to a 3D Euclidean transformation.
![Page 54: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/54.jpg)
54
Projective reconstruction without calibration
),]([),,( eFeP0IPF
2. Or the algebraic approach by epipolar geometry (cf. Faugeras’92 ECCV,What can be seen from an uncalibrated stereo rig?)
1. This is not unique, as for any v and lambda, we have
),]([ T eveFeP
3. The bottome line of numerical schema is the true ‘optimal’ of F
![Page 55: 1 Stereo vision and two-view geometry The goal of a stereo system is to get 3D information A stereo system consists at least of two ‘converging’ cameras.](https://reader036.fdocuments.in/reader036/viewer/2022062422/56649f155503460f94c2a4c2/html5/thumbnails/55.jpg)
55
Self-calibration
from only point correspondences using geometric self-consistency constraints
The original idea of Maybank&Faugeras91 key components: absolute conic, fundamental matrix and Kruppa equation
Later on: absolute quadric …
Projective metric reconstruction (uncalibrated)
Generally at least 3 images for self-calibrationwith constraints such as K=K’=K’’