Distinctive Image Features from Scale-Invariant Keypoints
description
Transcript of Distinctive Image Features from Scale-Invariant Keypoints
![Page 1: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/1.jpg)
Distinctive Image Featuresfrom Scale-Invariant Keypoints
David Lowe
![Page 2: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/2.jpg)
object instance recognition (matching)
![Page 3: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/3.jpg)
Photosynth
![Page 4: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/4.jpg)
Challenges
• Scale change• Rotation• Occlusion• Illumination ……
![Page 5: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/5.jpg)
Strategy
• Matching by stable, robust and distinctive local features.
• SIFT: Scale Invariant Feature Transform; transform image data into scale-invariant coordinates relative to local features
![Page 6: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/6.jpg)
SIFT
• Scale-space extrema detection• Keypoint localization• Orientation assignment• Keypoint descriptor
![Page 7: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/7.jpg)
Scale-space extrema detection
• Find the points, whose surrounding patches (with some scale) are distinctive
• An approximation to the scale-normalized Laplacian of Gaussian
![Page 8: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/8.jpg)
Maxima and minima in a 3*3*3 neighborhood
![Page 9: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/9.jpg)
Keypoint localization
• There are still a lot of points, some of them are not good enough.
• The locations of keypoints may be not accurate.• Eliminating edge points.
![Page 10: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/10.jpg)
(1)
(2)
(3)
![Page 11: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/11.jpg)
Eliminating edge points
• Such a point has large principal curvature across the edge but a small one in the perpendicular direction
• The principal curvatures can be calculated from a Hessian function
• The eigenvalues of H are proportional to the principal curvatures, so two eigenvalues shouldn’t diff too much
![Page 12: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/12.jpg)
![Page 13: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/13.jpg)
Orientation assignment
• Assign an orientation to each keypoint, the keypoint descriptor can be represented relative to this orientation and therefore achieve invariance to image rotation
• Compute magnitude and orientation on the Gaussian smoothed images
![Page 14: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/14.jpg)
Orientation assignment
• A histogram is formed by quantizing the orientations into 36 bins;
• Peaks in the histogram correspond to the orientations of the patch;
• For the same scale and location, there could be multiple keypoints with different orientations;
![Page 15: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/15.jpg)
Feature descriptor
![Page 16: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/16.jpg)
Feature descriptor
• Based on 16*16 patches• 4*4 subregions• 8 bins in each subregion• 4*4*8=128 dimensions in total
![Page 17: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/17.jpg)
![Page 18: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/18.jpg)
![Page 19: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/19.jpg)
Application: object recognition
• The SIFT features of training images are extracted and stored
• For a query image1. Extract SIFT feature2. Efficient nearest neighbor indexing3. 3 keypoints, Geometry verification
![Page 20: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/20.jpg)
![Page 21: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/21.jpg)
![Page 22: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/22.jpg)
![Page 23: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/23.jpg)
Extensions
• PCA-SIFT1. Working on 41*41 patches2. 2*39*39 dimensions3. Using PCA to project it to 20 dimensions
![Page 24: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/24.jpg)
Surf
• Approximate SIFT• Works almost equally well• Very fast
![Page 25: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/25.jpg)
Conclusions
• The most successful feature (probably the most successful paper in computer vision)
• A lot of heuristics, the parameters are optimized based on a small and specific dataset. Different tasks should have different parameter settings.
• Learning local image descriptors (Winder et al 2007): tuning parameters given their dataset.
• We need a universal objective function.
![Page 26: Distinctive Image Features from Scale-Invariant Keypoints](https://reader036.fdocuments.in/reader036/viewer/2022081604/56814bce550346895db8a4b2/html5/thumbnails/26.jpg)
comments
• Ian: “For object detection, the keypoint localization process can indicate which locations and scales to consider when searching for objects”.
• Mert: “uniform regions may be quite informative when detecting
some types of ojbects , but SIFT ignore them”• Mani: “region detectors comparison”• Eamon:” whether one could go directly to a surface
representation of a scene based on SIFT features “