robust feature point matching ohne.ppt [Kompatibilitätsmodus]
Lecture 3a: Feature detection and matching
description
Transcript of Lecture 3a: Feature detection and matching
![Page 1: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/1.jpg)
Lecture 3a: Feature detection and matching
CS6670: Computer VisionNoah Snavely
![Page 2: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/2.jpg)
Reading
• Szeliski: 4.1
![Page 3: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/3.jpg)
Feature extraction: Corners and blobs
![Page 4: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/4.jpg)
Motivation: Automatic panoramas
Credit: Matt Brown
![Page 5: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/5.jpg)
http://research.microsoft.com/en-us/um/redmond/groups/ivm/HDView/HDGigapixel.htmHD View
Also see GigaPan: http://gigapan.org/
Motivation: Automatic panoramas
![Page 6: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/6.jpg)
Why extract features?
• Motivation: panorama stitching– We have two images – how do we combine them?
![Page 7: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/7.jpg)
Why extract features?
• Motivation: panorama stitching– We have two images – how do we combine them?
Step 1: extract featuresStep 2: match features
![Page 8: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/8.jpg)
Why extract features?
• Motivation: panorama stitching– We have two images – how do we combine them?
Step 1: extract featuresStep 2: match features
Step 3: align images
![Page 9: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/9.jpg)
Image matching
by Diva Sian
by swashford
![Page 10: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/10.jpg)
Harder case
by Diva Sian by scgbt
![Page 11: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/11.jpg)
Harder still?
NASA Mars Rover images
![Page 12: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/12.jpg)
NASA Mars Rover imageswith SIFT feature matches
Answer below (look for tiny colored squares…)
![Page 13: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/13.jpg)
Feature Matching
![Page 14: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/14.jpg)
Feature Matching
![Page 15: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/15.jpg)
Invariant local featuresFind features that are invariant to transformations
– geometric invariance: translation, rotation, scale– photometric invariance: brightness, exposure, …
Feature Descriptors
![Page 16: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/16.jpg)
Advantages of local features
Locality – features are local, so robust to occlusion and clutter
Quantity– hundreds or thousands in a single image
Distinctiveness: – can differentiate a large database of objects
Efficiency– real-time performance achievable
![Page 17: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/17.jpg)
More motivation…
Feature points are used for:– Image alignment (e.g., mosaics)– 3D reconstruction– Motion tracking– Object recognition– Indexing and database retrieval– Robot navigation– … other
![Page 18: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/18.jpg)
Snoop demo
What makes a good feature?
![Page 19: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/19.jpg)
Want uniqueness
Look for image regions that are unusual– Lead to unambiguous matches in other images
How to define “unusual”?
![Page 20: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/20.jpg)
Local measures of uniqueness
Suppose we only consider a small window of pixels– What defines whether a feature is a good or bad
candidate?
Credit: S. Seitz, D. Frolova, D. Simakov
![Page 21: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/21.jpg)
Local measure of feature uniqueness
“flat” region:no change in all directions
“edge”: no change along the edge direction
“corner”:significant change in all directions
• How does the window change when you shift it?• Shifting the window in any direction causes a big
change
Credit: S. Seitz, D. Frolova, D. Simakov
![Page 22: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/22.jpg)
Consider shifting the window W by (u,v)• how do the pixels in W change?• compare each pixel before and after by
summing up the squared differences (SSD)
• this defines an SSD “error” E(u,v):
Harris corner detection: the math
W
![Page 23: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/23.jpg)
Taylor Series expansion of I:
If the motion (u,v) is small, then first order approximation is good
Plugging this into the formula on the previous slide…
Small motion assumption
![Page 24: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/24.jpg)
Corner detection: the math
Consider shifting the window W by (u,v)• define an SSD “error” E(u,v):
W
![Page 25: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/25.jpg)
Corner detection: the math
Consider shifting the window W by (u,v)• define an SSD “error” E(u,v):
W
• Thus, E(u,v) is locally approximated as a quadratic error function
![Page 26: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/26.jpg)
The surface E(u,v) is locally approximated by a quadratic form.
The second moment matrix
Let’s try to understand its shape.
![Page 27: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/27.jpg)
Horizontal edge:
uv
E(u,v)
![Page 28: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/28.jpg)
Vertical edge:
uv
E(u,v)
![Page 29: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/29.jpg)
General caseWe can visualize H as an ellipse with axis lengths determined by the eigenvalues of H and orientation determined by the eigenvectors of H
direction of the slowest change
direction of the fastest change
(max)-1/2
(min)-1/2
const][
v
uHvu
Ellipse equation:max, min : eigenvalues of H
![Page 30: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/30.jpg)
Corner detection: the math
Eigenvalues and eigenvectors of H• Define shift directions with the smallest and largest change in error• xmax = direction of largest increase in E
max = amount of increase in direction xmax
• xmin = direction of smallest increase in E
min = amount of increase in direction xmin
xmin
xmax
![Page 31: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/31.jpg)
Corner detection: the mathHow are max, xmax, min, and xmin relevant for feature detection?
• What’s our feature scoring function?
![Page 32: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/32.jpg)
Corner detection: the mathHow are max, xmax, min, and xmin relevant for feature detection?
• What’s our feature scoring function?Want E(u,v) to be large for small shifts in all directions
• the minimum of E(u,v) should be large, over all unit vectors [u v]• this minimum is given by the smaller eigenvalue (min) of H
![Page 33: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/33.jpg)
Interpreting the eigenvalues
1
2
“Corner”1 and 2 are large,
1 ~ 2;
E increases in all directions
1 and 2 are small;
E is almost constant in all directions
“Edge” 1 >> 2
“Edge” 2 >> 1
“Flat” region
Classification of image points using eigenvalues of M:
![Page 34: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/34.jpg)
Corner detection summaryHere’s what you do
• Compute the gradient at each point in the image• Create the H matrix from the entries in the gradient• Compute the eigenvalues. • Find points with large response (min > threshold)
• Choose those points where min is a local maximum as features
![Page 35: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/35.jpg)
Corner detection summaryHere’s what you do
• Compute the gradient at each point in the image• Create the H matrix from the entries in the gradient• Compute the eigenvalues. • Find points with large response (min > threshold)
• Choose those points where min is a local maximum as features
![Page 36: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/36.jpg)
The Harris operator
min is a variant of the “Harris operator” for feature detection
• The trace is the sum of the diagonals, i.e., trace(H) = h11 + h22
• Very similar to min but less expensive (no square root)• Called the “Harris Corner Detector” or “Harris Operator”• Lots of other detectors, this is one of the most popular
![Page 37: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/37.jpg)
The Harris operator
Harris operator
![Page 38: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/38.jpg)
Harris detector example
![Page 39: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/39.jpg)
f value (red high, blue low)
![Page 40: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/40.jpg)
Threshold (f > value)
![Page 41: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/41.jpg)
Find local maxima of f
![Page 42: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/42.jpg)
Harris features (in red)
![Page 43: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/43.jpg)
Weighting the derivatives
• In practice, using a simple window W doesn’t work too well
• Instead, we’ll weight each derivative value based on its distance from the center pixel
![Page 44: Lecture 3a: Feature detection and matching](https://reader035.fdocuments.in/reader035/viewer/2022062217/56813c8b550346895da62ee7/html5/thumbnails/44.jpg)
Questions?