Efficient Algorithms for Matching
description
Transcript of Efficient Algorithms for Matching
![Page 1: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/1.jpg)
Efficient Algorithms for Matching
Pedro FelzenszwalbTrevor DarrellYann LeCunAlex Berg
![Page 2: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/2.jpg)
Efficient Algorithms for Matching
Pedro FelzenszwalbTrevor DarrellYann LeCunAlex Berg
Polynomial & exact
![Page 3: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/3.jpg)
Efficient Algorithms for Matching
Pedro FelzenszwalbTrevor DarrellYann LeCunAlex Berg
Polynomial & exactMultilinear & approximate
![Page 4: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/4.jpg)
Efficient Algorithms for Matching
Pedro FelzenszwalbTrevor DarrellYann LeCunAlex Berg
Polynomial & exactMultilinear & approximate
“Fast?” but very good
![Page 5: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/5.jpg)
Efficient Algorithms for Matching
Pedro FelzenszwalbTrevor DarrellYann LeCunAlex Berg
Polynomial & exactMultilinear & approximate
“Fast?” but very good
Happy when things work
![Page 6: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/6.jpg)
First Criticism
• Efficiently computing the wrong solution is not so useful…
First Response
• Even if say, an algorithm does not solve object recognition, it can still be a useful tool…
![Page 7: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/7.jpg)
Why Matching?• Ideas hatched before me
– Statistical Pattern Theory (Ulf Grenander)– Deformable Templates– Fischler & Elshlager– Etc. at least by the early 1970’s
• “transform” and “appearance” parameters
• Matching to estimate transform
),( ATM Used to be continuous, now often discreteVery general, Translation / Diffeomorphism / AssignmentImage / Features / “Parts” / etc.
![Page 8: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/8.jpg)
Why Matching?• Ideas hatched before me
– Statistical Pattern Theory (Ulf Grenander)– Deformable Templates– Fischler & Elschlager– Etc. at least by the early 1970’s
• “transform” and “appearance” parameters
• Matching to estimate transform
),( ATM Used to be continuous, now often discreteVery general, Translation / Diffeomorphism / AssignmentImage / Features / “Parts” / etc.
MODEL
TRANSFORM
IMAGE
![Page 9: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/9.jpg)
Why Matching?• Ideas hatched before me
– Statistical Pattern Theory (Ulf Grenander)– Deformable Templates– Fischler & Elschlager– Etc. at least by the early 1970’s
• “transform” and “appearance” parameters
• Matching to estimate transform
),( ATM Used to be continuous, now often discreteVery general, Translation / Diffeomorphism / AssignmentImage / Features / “Parts” / etc.
MODEL
TRANSFORM
IMAGE
![Page 10: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/10.jpg)
Why Matching?• Ideas hatched before me
– Statistical Pattern Theory (Ulf Grenander)– Deformable Templates– Fischler & Elschlager– Etc. at least by the early 1970’s
• “transform” and “appearance” parameters
• Matching to estimate transform – Searching over diffeomorphisms difficult– Searching over discrete assignments easier?
),( ATM Used to be continuous, now often discreteVery general, Translation / Diffeomorphism / AssignmentImage / Features / “Parts” / etc.
MODEL
TRANSFORM
IMAGE
![Page 11: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/11.jpg)
Search for a Transformation
Model of Car
Image
?
![Page 12: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/12.jpg)
Find Transformation Using Correspondence
Model of Car
Image
•Search through a discrete set of possible point correspondences•Objective function should be close to cost of the original model•Use the discrete correspondences to obtain a continuous transformation if needed
Sometimes…
![Page 13: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/13.jpg)
Find Transformation Using Correspondence
Model of Car
Image
Why it works…Sometimes we can measure consistency of model appearance locallyInspired by branch and bound: “If local appearance is inconsistent, any alignment with that appearance is bad.” My preferred way of motivating local features…
![Page 14: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/14.jpg)
Find Transformation Using Correspondence
Model of Car
Image
Sometimes local appearance is notenough, so we model some versionof spatial constraints.
Do not make the problem harder than it was…
![Page 15: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/15.jpg)
Linear Assignmente.g. Hungarian Algorithm
Just Features, no Geometry
Individual feature matches provide most of the solution.
![Page 16: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/16.jpg)
Quadratic Assignment(Adding Geometric Constraints)
Individual feature matches provide most of the solution.Geometric consistency only has to clean things up a little.
In this case we formulate the matching as an Integer Quadratic Programming problem and look for an approximate solution…
![Page 17: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/17.jpg)
Second Meta-Comment
• Even if a problem can have very difficult instances, the effective complexity of certain instances might be quite low. This can be quite difficult to verify formally.
![Page 18: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/18.jpg)
Use Alignment to Compare
Model of Car
•Given alignment evaluate the model•Note: we might have been done already Grauman et al, Zhang et al•Actually do some alignment and check the quality of the fit.
Back to the alignment…
![Page 19: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/19.jpg)
Humans can be very efficient
• Simon Thorpe: animal or not in <=150 ms– “Feed forward” process– Difficult to retrain (familiarization does not make a
difference)– Salient parts of images are actually processed more
rapidly.– Support for some styles of current algorithms– Neurophysiological evidence for some mid-level
vision (illusory contours, figure ground, etc.) von der Heydt et al
That’s all fine, but
![Page 20: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/20.jpg)
What is an Object?• Apple yes, mist no• A rule of thumb is that objects have some
definite spatial support…• Image/Scene? Some context models treat
scenes or images as objects• Face, eyes, nose, eye-lashes • We can build SPT models for all of these…
![Page 21: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/21.jpg)
Heuristics
• Take advantage of the data– Sometimes a single feature is enough– For efficiency need to weigh this against how often
that feature is found– Many (?) object recognition datasets allow easy
discrimination between categories with only very simple features extracted from the whole image, eg Pascal and Caltech 101.
– Segmentation or Figure/Ground -- might as well see if there is an object there before trying to recognize it…
![Page 22: Efficient Algorithms for Matching](https://reader036.fdocuments.in/reader036/viewer/2022062411/56816792550346895ddcc7d6/html5/thumbnails/22.jpg)
An Approach1. Extract features from an image2. Look features up in a large database
• Approximate Nearest Neighbor algorithms can make this sub-linear.
• Each entry tells us about which hypotheses [Object,position,pose,...] might be present.
3. Use a “short list” of these to check in more detail using a matching framework
• Simple matching can actually be indexed. Indyk et al, Grauman et al.
4. Finally use a matching to align models and apply more expensive processing