Structure and Motion from Discrete Viewsaz/lectures/cv/lecture6.pdf · Structure and Motion from...
Transcript of Structure and Motion from Discrete Viewsaz/lectures/cv/lecture6.pdf · Structure and Motion from...
Structure and Motion from Discrete Views
• Introduction
• Computing the fundamental matrix, F, from corner correspondences
• Feature matching
• RANSAC
• Estimation
• Determining ego-motion from F• Computing a homography, H, from corner correspondences
• SIFT for wide baseline matching
• More than two views
• Batch and sequential solutions
Cameras rotating about their centre
image plane 1
image plane 2
• The two image planes are related by a homography H
• H depends only on the relation between the image planes and camera centre, C, not on the 3D structure
P = K [ I | 0 ] P’ = K’ [ R | 0 ]
H = K’ R K^(-1)
Four points define a projective transformation
Automatic computation of a homography
• Given an image pair:
1. Compute interest points (feature detection)
2. Compute putative (hypothesised) correspondences
3. Compute the homography from the correspondences
Interest point detection
500 points in each image
Initial correspondences Final correspondences
Computing a panoramic mosaic
• camera rotates about its centre
• focal length can also change (zoom)
Choice of mosaic frame
• This produces the classic "bow-tie"mosaic.
Choose central image as reference
Wide Baseline Matching• Suppose we wish to compute F between any image pair (not restricted to small camera motion between views)
Wide Baseline Matching• Why does the short baseline computation fail?• Recall three steps:
1.1.Compute interest pointsCompute interest points2.2.Compute putative point correspondencesCompute putative point correspondences3.3.RANSAC estimation of F and consistent correspondence set.RANSAC estimation of F and consistent correspondence set.
Derivative of Gaussian
gdxdf ∗
f
gdxd
Source: S. Seitz
Edge
Derivativeof Gaussian
Edge = maximumof derivative
Second derivative of Gaussian
gdxdf 2
2
∗
f
gdxd
2
2
Edge
Second derivativeof Gaussian (Laplacian)
Edge = zero crossingof second derivative
Source: S. Seitz
Matched filter for blob detection
Scale-normalized Laplacian response
Unnormalized Laplacian responseOriginal signal
maximum
Blob detection in 2D• Laplacian of Gaussian: Circularly symmetric operator for
blob detection in 2D
⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+∂∂
=∇ 2
2
2
222
norm yg
xgg σScale-normalized:
Scale-invariant feature detection
• Approximating the Laplacian with a difference of Gaussians:
( )2 ( , , ) ( , , )xx yyL G x y G x yσ σ σ= +
( , , ) ( , , )DoG G x y k G x yσ σ= −
(Laplacian)
(Difference of Gaussians)
Efficient implementation
Select canonical orientation
• Create histogram of local gradient directions computed at selected scale
• Assign canonical orientation at peak of smoothed histogram
• Each key specifies stable 2D coordinates (x, y, scale, orientation)
0 2π
Example
Feature Descriptor
• distribution of the gradient over an image patchgradient 3D histogram
→ →
image patch
y
x
4x4 location grid and 8 orientations (128 dimensions)
Epipolar geometry example
512 x 768 pixel images
interest points and scales
Now, match SIFT descriptors between images
All matches consistent with epipolar geometry
156 matches consistent with epipolar geometry
Epipolar geometry
Ex: Panoramas from an unordered image set
Autostitch, Mat Brown & David Lowe
Application: Post-production special effectsoriginal sequence
Augmentation