Semi-Local Affine Parts for Object Recognition
description
Transcript of Semi-Local Affine Parts for Object Recognition
![Page 1: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/1.jpg)
Semi-Local Affine PartsSemi-Local Affine Partsfor Object Recognitionfor Object Recognition
Svetlana Lazebnik, Jean PonceSvetlana Lazebnik, Jean PonceUniversity of Illinois at Urbana-ChampaignUniversity of Illinois at Urbana-Champaign
Cordelia Schmid Cordelia Schmid INRIA RhINRIA Rhôône-Alpesne-Alpes
BMVC 2004BMVC 2004
![Page 2: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/2.jpg)
OverviewOverview• Goal:
– Learning models for recognition of 3D object classes• Challenges:
– Geometric invariance– Robustness to clutter, occlusion– Weakly supervised learning
• Proposed approach: – An object representation using semi-local affine parts
![Page 3: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/3.jpg)
Low-Level Features: Local Affine RegionsLow-Level Features: Local Affine Regions
• This work: Laplacian detector (Gårding & Lindeberg, 1996)• Other detectors: Kadir et al. (2004), Matas et al. (2002),
Mikolajczyk & Schmid (2002), Tuytelaars & Van Gool (2004), etc.
![Page 4: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/4.jpg)
• In practice: two-image matching followed by validation
Learning PartsLearning Parts• Ideal approach: simultaneous correspondence search
across entire training set
validation setinitial pair
candidate part
![Page 5: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/5.jpg)
Two-Image MatchingTwo-Image Matching• Goal: to find collections of local affine regions that can be
mapped onto each other using a single affine transformation
• Implementation: greedy search based on geometric and photometric consistency constraints– Returns multiple correspondence hypotheses
– Automatically determines number of regions in correspondence
– Works on unsegmented, cluttered images (weakly supervised learning)
A
![Page 6: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/6.jpg)
Matching: DetailsMatching: Details• Initialization:
– Identify triples of neighboring regions (i, j, k) in first image– Find all triples (i', j', k') in the second image such that i'
(resp. j', k') is a potential match of i (resp. j, k), and j', k' are neighbors of i'
i
j
k
i'
j'
k'
![Page 7: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/7.jpg)
Matching (cont.)Matching (cont.)• Beginning with each seed triple, iterate:
– Estimate the affine transformation between centers of corresponding regions in current group of matches
A
![Page 8: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/8.jpg)
Matching (cont.)Matching (cont.)• Beginning with each seed triple, iterate:
– Estimate the affine transformation between centers of corresponding regions in current group of matches
– Determine geometric consistency of current group of matches
• Geometric consistency criteria:– Distance between ellipse centers
(residual)– Difference of major and minor axis
lengths– Difference of ellipse orientations
![Page 9: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/9.jpg)
Matching (cont.)Matching (cont.)• Beginning with each seed triple, iterate:
– Estimate the affine transformation between centers of corresponding regions in current group of matches
– Determine geometric consistency of current group of matches
– Search for additional matches in the neighborhood of the current group
![Page 10: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/10.jpg)
Matching: 3D ObjectsMatching: 3D Objects
![Page 11: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/11.jpg)
Matching: 3D ObjectsMatching: 3D Objects
closeup closeup
![Page 12: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/12.jpg)
Matching: FacesMatching: Faces
spurious match ???
![Page 13: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/13.jpg)
Finding Repeated Patterns and Finding Repeated Patterns and SymmetriesSymmetries
![Page 14: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/14.jpg)
Learning Object Models for RecognitionLearning Object Models for Recognition• Match multiple pairs of training images to produce a
set of candidate parts• Use additional validation images to evaluate
repeatability of parts and individual regions • Retain a fixed number of parts having the best
repeatability score
![Page 15: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/15.jpg)
Recognition Experiment: ButterfliesRecognition Experiment: Butterflies
• 26 training images per class– 8 initial pairs– 10 validation images
• 437 test images• 619 images total
Admiral Swallowtail Machaon Monarch 1 Monarch 2 Peacock Zebra
![Page 16: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/16.jpg)
Butterfly PartsButterfly Parts
![Page 17: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/17.jpg)
RecognitionRecognition
• Top 10 parts per class used for recognition• Relative repeatability score:• Classification results:
total number of regions detectedtotal part size
Total part size (smallest/largest)
![Page 18: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/18.jpg)
Classification Rate vs. Classification Rate vs. Number of PartsNumber of Parts
![Page 19: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/19.jpg)
Detection Results (ROC Curves)Detection Results (ROC Curves)
Circles: reference relative repeatability rates. Red square: ROC equal error rate (in parentheses)
![Page 20: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/20.jpg)
Successful Detection ExamplesSuccessful Detection ExamplesTraining images
Test images (blue: occluded regions)
All regions found in the test images
![Page 21: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/21.jpg)
Unsuccessful Detection ExamplesUnsuccessful Detection ExamplesTraining images
Test images (blue: occluded regions)
All regions found in the test images
![Page 22: Semi-Local Affine Parts for Object Recognition](https://reader035.fdocuments.in/reader035/viewer/2022070503/56816393550346895dd48c0f/html5/thumbnails/22.jpg)
Future WorkFuture Work• Goal:
– Recognize highly variable, non-rigid object categories
• Proposed approach: – Treat semi-local affine parts as “black boxes”– Model spatial relations between parts– Learn these relations from training data in a weakly
supervised fashion