Assignment 2 Compute F automatically from image pair (putative matches, 8-point, 7-point, iterative,...
-
date post
15-Jan-2016 -
Category
Documents
-
view
222 -
download
1
Transcript of Assignment 2 Compute F automatically from image pair (putative matches, 8-point, 7-point, iterative,...
Assignment 2Compute F automatically from image
pair
(putative matches, 8-point, 7-point, iterative, RANSAC, guided matching)
(due by Wednesday 19/03/03)
Rectification and structure computation
class 15
Multiple View GeometryComp 290-089Marc Pollefeys
Multiple View Geometry course schedule(subject to change)
Jan. 7, 9 Intro & motivation Projective 2D Geometry
Jan. 14, 16
(no class) Projective 2D Geometry
Jan. 21, 23
Projective 3D Geometry (no class)
Jan. 28, 30
Parameter Estimation Parameter Estimation
Feb. 4, 6 Algorithm Evaluation Camera Models
Feb. 11, 13
Camera Calibration Single View Geometry
Feb. 18, 20
Epipolar Geometry 3D reconstruction
Feb. 25, 27
Fund. Matrix Comp. Fund. Matrix Comp.
Mar. 4, 6 Structure Comp. Planes & Homographies
Mar. 18, 20
Trifocal Tensor Three View Reconstruction
Mar. 25, 27
Multiple View Geometry
MultipleView Reconstruction
Apr. 1, 3 Bundle adjustment Papers
Apr. 8, 10
Auto-Calibration Papers
Apr. 15, 17
Dynamic SfM Papers
Apr. 22, 24
Cheirality Project Demos
Two-view geometry
Epipolar geometry
3D reconstruction
F-matrix comp.
Structure comp.
Automatic computation of F
(i) Interest points(ii) Putative correspondences(iii) RANSAC (iv) Non-linear re-estimation of F(v) Guided matching(repeat (iv) and (v) until stable)
Select strongest features (e.g. 1000/image)
Feature points
Evaluate ZNCC,SSD,SAD for all features with similar coordinates
Keep mutual best matchesKeep mutual best matches
Still many wrong matches!Still many wrong matches!
10101010 ,,´´, e.g. hhww yyxxyx
?
Feature matching
Step 1. Extract featuresStep 2. Compute a set of potential matchesStep 3. do
Step 3.1 select minimal sample (i.e. 7 matches)
Step 3.2 compute solution(s) for F
Step 3.3 determine inliers
until (#inliers,#samples)<95%
samples#7)1(1
matches#inliers#
#inliers 90%
80%
70% 60%
50%
#samples
5 13 35 106 382
Step 4. Compute F based on all inliersStep 5. Look for additional matchesStep 6. Refine F based on all correct matches
(generate hypothesis)
(verify hypothesis)
RANSAC
restrict search range to neighborhood of epipolar line (1.5 pixels)
relax disparity restriction (along epipolar line)
Finding more matches: guided matching
geometric relations between two views is fully
described by recovered 3x3 matrix F
two-view geometry
Image pair rectification
simplify stereo matching by warping the images
Apply projective transformation so that epipolar linescorrespond to horizontal scanlines
e
e
map epipole e to (1,0,0)
try to minimize image distortion
problem when epipole in (or close to) the image
He001
Planar rectification
Bring two views Bring two views to standard stereo setupto standard stereo setup
(moves epipole to )(not possible when in/close to image)
~ image size
(calibrated)(calibrated)
Distortion minimization(uncalibrated)
(standard approach)
Polar re-parameterization around epipoles
Requires only (oriented) epipolar geometry
Preserve length of epipolar linesChoose so that no pixels are
compressed
original image rectified image
Polar rectification(Pollefeys et al. ICCV’99)
Works for all relative motionsGuarantees minimal image size
polar rectification: example
polar rectification: example
Example: Béguinage of Leuven
Does not work with standard Homography-based approaches
Example: Béguinage of Leuven
Stereo matching
• attempt to match every pixel• use additional constraints
Exploiting motion and scene constraints
• Ordering constraint• Uniqueness constraint• Disparity limit• Disparity continuity constraint
• Epipolar constraint Epipolar constraint (through rectification)
Ordering constraint
11 22 33 4,54,5 66 11 2,32,3 44 55 66
2211 33 4,54,5 6611
2,32,3
44
55
66
surface slicesurface slice surface as a pathsurface as a path
occlusion right
occlusion left
Uniqueness constraint
• In an image pair each pixel has at most one corresponding pixel• In general one corresponding pixel• In case of occlusion there is none
Disparity constraint
surface slicesurface slice surface as a pathsurface as a path
bounding box
dispa
rity b
and
use reconsructed features to determine bounding box
constantdisparitysurfaces
Disparity continuity constraint
• Assume piecewise continuous surface
piecewise continuous disparity• In general disparity changes
continuously• discontinuities at occluding
boundaries
Stereo matching
Optimal path(dynamic programming )
Similarity measure(SSD or NCC)
Constraints• epipolar
• ordering
• uniqueness
• disparity limit
• disparity gradient limit
Trade-off
• Matching cost (data)
• Discontinuities (prior)
(Cox et al. CVGIP’96; Koch’96; Falkenhagen´97; Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)
Hierarchical stereo matching
Dow
nsam
plin
g
(Gau
ssia
n p
yra
mid
)
Dis
pari
ty p
rop
ag
ati
on
Allows faster computation
Deals with large disparity ranges
(Falkenhagen´97;Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)
Disparity map
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y),y)
Example: reconstruct image from neighboring
images
Multi-view depth fusion
• Compute depth for every pixel of reference image• Triangulation• Use multiple views• Up- and down
sequence• Use Kalman filter
(Koch, Pollefeys and Van Gool. ECCV‘98)
Allows to compute robust texture
Point reconstruction
PXx XP'x'
linear triangulation
XP'x'PXx
0XP'x
0XpXp0XpXp0XpXp
1T2T
2T3T
1T3T
yxyx
2T3T
1T3T
2T3T
1T3T
p'p''p'p''pppp
A
yxyx
0AX
homogeneous
1X
)1,,,( ZYX
inhomogeneous
invariance?
e)(HX)(AH-1
algebraic error yes, constraint no (except for affine)
geometric error
0x̂F'x̂ subject to )'x̂,(x')x̂(x, T22 dd
X̂P''x̂ and X̂Px̂ subject toly equivalentor
possibility to compute using LM (for 2 or more points)
or directly (for 2 points)
Geometric error
Reconstruct matches in projective frame by minimizing the reprojection error
(see Hartley&Sturm,CVIU´97)Non-iterative optimal solution
Optimal 3D point in epipolar plane
Given an epipolar plane, find best 3D point for (x1,x2)
x1
x2
l1 l2
l1x1
x2l2
x1´
x2´
Select closest points (x1´,x2´) on epipolar lines
Obtain 3D point through exact triangulationGuarantees minimal reprojection error (given this epipolar plane)
Optimal epipolar plane
• Reconstruct matches in projective frame by minimizing the reprojection error
• Non-iterative methodDetermine the epipolar plane for reconstruction
Reconstruct optimal point from selected epipolar plane
222
211 XP,xXP,x dd
(Hartley and Sturm, CVIU´97)
222
211 αl,xαl,x DD
(polynomial of degree 6check all minima, incl ∞)
m1
m2
l1 l2
3DOF
1DOF
Reconstruction uncertainty
consider angle between rays
Line reconstruction
P'l'Pl
T
T
L
doesn‘t work for epipolar plane
Next class: Scene and plane homographies