Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli &...
-
date post
23-Jan-2016 -
Category
Documents
-
view
228 -
download
0
Transcript of Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli &...
![Page 1: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/1.jpg)
Model: Parts and Structure
![Page 2: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/2.jpg)
History of Idea
• Fischler & Elschlager 1973
• Yuille ‘91• Brunelli & Poggio ‘93• Lades, v.d. Malsburg et al. ‘93• Cootes, Lanitis, Taylor et al. ‘95• Amit & Geman ‘95, ‘99 • Perona et al. ‘95, ‘96, ’98, ’00,
Huttenlocher et al. ’00
• Many papers since 2000
![Page 3: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/3.jpg)
Representation• Object as set of parts
– Generative representation
• Model:– Relative locations between parts– Appearance of part
• Issues:– How to model location– How to represent appearance– Sparse or dense (pixels or regions)– How to handle occlusion/clutter
Figure from [Fischler73]
![Page 4: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/4.jpg)
Example scheme
• Model shape using Gaussian distribution on location between parts
• Model appearance as pixel templates• Represent image as collection of
regions– Extracted by template matching:
normalized-cross correlation
• Manually trained model– Click on training images
![Page 5: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/5.jpg)
Sparse representation+ Computationally tractable (105 pixels 101 -- 102 parts)
+ Generative representation of class
+ Avoid modeling global variability
+ Success in specific object recognition
- Throw away most image information
- Parts need to be distinctive to separate from other classes
![Page 6: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/6.jpg)
The correspondence problem• Model with P parts• Image with N possible locations for each part
• NP combinations!!!
![Page 7: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/7.jpg)
Connectivity of parts• Complexity is given by size of maximal clique in graph• Consider a 3 part model
– Each part has set of N possible locations in image– Location of parts 2 & 3 is independent, given location of L– Each part has an appearance term, independent between parts.
L
32
Shape Model
![Page 8: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/8.jpg)
Different graph structures
1
3
4 5
6
2
1
3
4 5
6
2
Fully connected Star structure
1
3
4
5
6
2
Tree structure
O(N6) O(N2) O(N2)
• Sparser graphs cannot capture all interactions between parts
![Page 9: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/9.jpg)
Some class-specific graphs• Articulated motion
– People– Animals
• Special parameterisations– Limb angles
Images from [Kumar05, Feltzenswalb05]
![Page 10: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/10.jpg)
Regions or pixels• # Regions << # Pixels• Regions increase tractability
but lose information• Generally use regions:
– Local maxima of interest operators
– Can give scale/orientation invariance
Figures from [Kadir04]
![Page 11: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/11.jpg)
How to model location?
• Explicit: Probability density functions
• Implicit: Voting scheme
• Invariance– Translation– Scaling– Similarity/affine– Viewpoint
Similarity transformationTranslation and ScalingTranslationAffine transformation
![Page 12: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/12.jpg)
• Probability densities– Continuous (Gaussians)– Analogy with springs
• Parameters of model, and – Independence corresponds to zeros in
Explicit shape model
![Page 13: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/13.jpg)
Shape• Shape is “what remains after differences due to translation,
rotation, and scale have been factored out”. [Kendall84]
• Statistical theory of shape [Kendall, Bookstein, Mardia & Dryden]
X
Y
Figure SpaceU
V
Shape Space
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=
N
N
y
yx
x
X
M
M
1
1
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=
N
N
v
vx
u
U
M
M
3
3
Figures from [Leung98]
![Page 14: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/14.jpg)
Representation of appearance
• Dependencies between parts– Common to assume independence– Need not be – Symmetry
•Needs to handle intra-class variation
–Task is no longer matching of descriptors–Implicit variation (VQ appearance)–Explicit probabilistic model of appearance (e.g. Gaussians in SIFT space or PCA space)
![Page 15: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/15.jpg)
Representation of appearance• Invariance needs to match that of
shape model
• Insensitive to small shifts in translation/scale– Compensate for jitter of features– e.g. SIFT
• Illumination invariance– Normalize out– Condition on illumination of
landmark part
![Page 16: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/16.jpg)
Parts and Structure demo
• Gaussian location model – star configuration• Translation invariant only
– Use 1st part as landmark
• Appearance model is template matching• Manual training
– User identifies correspondence on training images
• Recognition– Run template for each part over image– Get local maxima set of possible locations for each part– Impose shape model - O(N2P) cost– Score of each match is combination of shape model and
template responses.
![Page 17: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/17.jpg)
Demo images• Sub-set of Caltech face dataset• Caltech background images
![Page 18: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/18.jpg)
![Page 19: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/19.jpg)
• Task: Estimation of model parameters
Learning using EM
• Let the assignments be a hidden variable and use EM algorithm to learn them and the model parameters
• Chicken and Egg type problem, since we initially know neither:
- Model parameters
- Assignment of regions to parts
![Page 20: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/20.jpg)
Example scheme, using EM for maximum likelihood learning
1. Current estimate of
...
Image 1 Image 2 Image i
2. Assign probabilities to constellations
Large P
Small P
3. Use probabilities as weights to re-estimate parameters. Example:
Large P x + Small P x
new estimate of
+ … =
![Page 21: Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,](https://reader033.fdocuments.in/reader033/viewer/2022051418/56649d265503460f949fcc9e/html5/thumbnails/21.jpg)
Learning Shape & Appearance simultaneously Fergus et al. ‘03