Lecture 15: Object recognition: Part based generative...
Transcript of Lecture 15: Object recognition: Part based generative...
![Page 1: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/1.jpg)
Lecture 15 -Fei-Fei Li
Lecture 15: Object recognition:
Part‐based generative models
Professor Fei‐Fei LiStanford Vision Lab
8‐Nov‐111
![Page 2: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/2.jpg)
Lecture 15 -Fei-Fei Li
What we will learn today?
• Introduction• Constellation model
– Weakly supervised training– One‐shot learning
• (Problem Set 4 (Q1))
8‐Nov‐112
![Page 3: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/3.jpg)
Lecture 15 -Fei-Fei Li
Challenges: intra‐class variation
8‐Nov‐113
![Page 4: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/4.jpg)
Lecture 15 -Fei-Fei Li
Usual Challenges:
Variability due to:
• View point
• Illumination
• Occlusions
8‐Nov‐114
![Page 5: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/5.jpg)
Lecture 15 -Fei-Fei Li
Basic issues• Representation
– 2D Bag of Words (BoW) models; – Part‐based models; – Multi‐view models;
• Learning– Generative & Discriminative BoWmodels– Generative models– Probabilistic Hough voting
• Recognition– Classification with BoW– Classification with Part‐based models
8‐Nov‐115
![Page 6: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/6.jpg)
Lecture 15 -Fei-Fei Li
Basic issues• Representation
– 2D Bag of Words (BoW) models; – Part‐based models; – Multi‐view models;
• Learning– Generative & Discriminative BoWmodels– Generative models– Probabilistic Hough voting
• Recognition– Classification with BoW– Classification with Part‐based models
8‐Nov‐116
![Page 7: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/7.jpg)
Lecture 15 -Fei-Fei Li
Basic issues• Representation
– 2D Bag of Words (BoW) models; – Part‐based models; – Multi‐view models (Lecture #19);
• Learning– Generative & Discriminative BoWmodels– Generative models– Probabilistic Hough voting
• Recognition– Classification with BoW– Classification with Part‐based models
8‐Nov‐117
![Page 8: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/8.jpg)
Lecture 15 -Fei-Fei Li
Problem with bag‐of‐words
• All have equal probability for bag‐of‐words methods
• Location information is important
8‐Nov‐118
![Page 9: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/9.jpg)
Lecture 15 -Fei-Fei Li
Model: Parts and Structure
8‐Nov‐119
![Page 10: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/10.jpg)
Lecture 15 -Fei-Fei Li
• Fischler & Elschlager 1973
• Yuille ‘91• Brunelli & Poggio ‘93• Lades, v.d. Malsburg et al. ‘93• Cootes, Lanitis, Taylor et al. ‘95• Amit & Geman ‘95, ‘99 • et al. Perona ‘95, ‘96, ’98, ’00, ‘03• Huttenlocher et al. ’00• Agarwal & Roth ’02
etc…
Parts and Structure Literature
8‐Nov‐1110
![Page 11: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/11.jpg)
Lecture 15 -Fei-Fei Li
The Constellation ModelT. Leung
M. Burl
Representation
Detection
Shape statistics – F&G ’95Affine invariant shape – CVPR ‘98
CVPR ‘96ECCV ‘98
M. WeberM. Welling Unsupervised Learning
ECCV ‘00Multiple views - F&G ’00 Discovering categories - CVPR ’00
R. Fergus
L. Fei-Fei
Joint shape & appearance learningGeneric feature detectors
One-Shot LearningIncremental learning
CVPR ’03Polluted datasets - ECCV ‘04
ICCV ’03CVPR ‘04
8‐Nov‐1111
![Page 12: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/12.jpg)
Lecture 15 -Fei-Fei Li
A B
DC
Deformations
8‐Nov‐1112
![Page 13: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/13.jpg)
Lecture 15 -Fei-Fei Li
Presence / Absence of Features
occlusion
8‐Nov‐1113
![Page 14: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/14.jpg)
Lecture 15 -Fei-Fei Li
Background clutter
8‐Nov‐1114
![Page 15: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/15.jpg)
Lecture 15 -Fei-Fei Li
Foreground modelGenerative probabilistic model
Gaussian shape pdfClutter modelUniform shape pdfProb. of detection
0.8 0.75
0.9
# detections
pPoisson(N2|2)
pPoisson(N1|1)
pPoisson(N3|3)
Assumptions: (a) Clutter independent of foreground detections(b) Clutter detections independent of each other
Example1. Object Part Positions
3a. N false detect2. Part Absence
N1
N2
3b. Position f. detect
N3
8‐Nov‐1115
![Page 16: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/16.jpg)
Lecture 15 -Fei-Fei Li
Learning Models `Manually’
• Obtain set of training images
• Label parts by hand, train detectors
• Learn model from labeled parts
• Choose parts
8‐Nov‐1116
![Page 17: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/17.jpg)
Lecture 15 -Fei-Fei Li
Recognition1. Run part detectors exhaustively over image
2032
e.g.
0000
4
3
2
1
h
NNNN
h
1
2
3
3
2
41
1
2 3
1
2
2. Try different combinations of detections in model- Allow detections to be missing (occlusion)
3. Pick hypothesis which maximizes:
4. If ratio is above threshold then, instance detected
),|(),|(
HypClutterDatapHypObjectDatap
8‐Nov‐1117
![Page 18: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/18.jpg)
Lecture 15 -Fei-Fei Li
So far…..• Representation
– Joint model of part locations– Ability to deal with background clutter and occlusions
• Learning– Manual construction of part detectors– Estimate parameters of shape density
• Recognition– Run part detectors over image– Try combinations of features in model– Use efficient search techniques to make fast
8‐Nov‐1118
![Page 19: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/19.jpg)
Lecture 15 -Fei-Fei Li
Unsupervised LearningWeber & Welling et. al.
8‐Nov‐1119
![Page 20: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/20.jpg)
Lecture 15 -Fei-Fei Li
(Semi) Unsupervised learning
•Know if image contains object or not•But no segmentation of object or manual selection of features
8‐Nov‐1120
![Page 21: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/21.jpg)
Lecture 15 -Fei-Fei Li
Unsupervised detector training - 1
• Highly textured neighborhoods are selected automatically• produces 100-1000 patterns per image
10
10
8‐Nov‐1121
![Page 22: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/22.jpg)
Lecture 15 -Fei-Fei Li
Unsupervised detector training - 2
“Pattern Space” (100+ dimensions)
8‐Nov‐1122
![Page 23: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/23.jpg)
Lecture 15 -Fei-Fei Li
Unsupervised detector training - 3
100-1000 images ~100 detectors
8‐Nov‐1123
![Page 24: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/24.jpg)
Lecture 15 -Fei-Fei Li
• Task: Estimation of model parameters
Learning
• Let the assignments be a hidden variable and use EM algorithm to learn them and the model parameters
• Chicken and Egg type problem, since we initially know neither:
- Model parameters
- Assignment of regions to foreground / background
• Take training images. Pick set of detectors. Apply detectors.
8‐Nov‐1124
![Page 25: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/25.jpg)
Lecture 15 -Fei-Fei Li
ML using EM1. Current estimate
...
Image 1 Image 2 Image i
2. Assign probabilities to constellations
Large P
Small P
3. Use probabilities as weights to re-estimate parameters. Example:
Large P x + Small P x
new estimate of
+ … =
8‐Nov‐1125
![Page 26: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/26.jpg)
Lecture 15 -Fei-Fei Li
Detector Selection
ParameterEstimation
Choice 1
Choice 2 ParameterEstimation
Model 1
Model 2
Predict / measure model performance(validation set or directly from model)
Detectors (100)
•Try out different combinations of detectors (Greedy search)
8‐Nov‐1126
![Page 27: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/27.jpg)
Lecture 15 -Fei-Fei Li
Frontal Views of Faces
• 200 Images (100 training, 100 testing)
• 30 people, different for training and testing
8‐Nov‐1127
![Page 28: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/28.jpg)
Lecture 15 -Fei-Fei Li
Learned face modelPre-selected Parts
Model Foreground pdf
Sample Detection
Parts in Model
Test Error: 6% (4 Parts)
8‐Nov‐1128
![Page 29: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/29.jpg)
Lecture 15 -Fei-Fei Li
Face images
8‐Nov‐1129
![Page 30: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/30.jpg)
Lecture 15 -Fei-Fei Li
Background images
8‐Nov‐1130
![Page 31: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/31.jpg)
Lecture 15 -Fei-Fei Li
Preselected Parts
Model Foreground pdf
Sample Detection
Parts in Model
Car from RearTest Error: 13% (5 Parts)
8‐Nov‐1131
![Page 32: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/32.jpg)
Lecture 15 -Fei-Fei Li
Detections of Cars
8‐Nov‐1132
![Page 33: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/33.jpg)
Lecture 15 -Fei-Fei Li
Background Images
8‐Nov‐1133
![Page 34: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/34.jpg)
Lecture 15 -Fei-Fei Li
3D Object recognition – Multiple mixture components
8‐Nov‐1134
![Page 35: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/35.jpg)
Lecture 15 -Fei-Fei Li
3D Orientation Tuning
Frontal Profile
0 20 40 60 80 10050
55
60
65
70
75
80
85
90
95
100Orientation Tuning
angle in degrees
% Correct
% Correct
8‐Nov‐1135
![Page 36: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/36.jpg)
Lecture 15 -Fei-Fei Li
So far (2)…..• Representation
– Multiple mixture components for different viewpoints
• Learning– Now semi-unsupervised– Automatic construction and selection of part detectors– Estimation of parameters using EM
• Recognition– As before
• Issues:-Learning is slow (many combinations of detectors)-Appearance learnt first, then shape
8‐Nov‐1136
![Page 37: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/37.jpg)
Lecture 15 -Fei-Fei Li
Issues• Speed of learning
– Slow (many combinations of detectors)• Appearance learnt first, then shape
– Difficult to learn part that has stable location but variable appearance
– Each detector is used as a cross-correlation filter, giving a hard definition of the part’s appearance
• Would like a fully probabilistic representation of the object
8‐Nov‐1137
![Page 38: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/38.jpg)
Lecture 15 -Fei-Fei Li
Object categorization
Fergus et. al.
CVPR ’03, IJCV ‘068‐Nov‐1138
![Page 39: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/39.jpg)
Lecture 15 -Fei-Fei Li
Detection & Representation of regions
Appearance
Location
Scale
(x,y) coords. of region centre
Radius of region (pixels)
11x11 patchNormalizeProjection onto
PCA basis
c1
c2
c15
……
…..
Gives representation of appearance in low-dimensional vector space
• Find regions within image
• Use salient region operator(Kadir & Brady 01)
8‐Nov‐1139
![Page 40: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/40.jpg)
Lecture 15 -Fei-Fei Li
Motorbikes example•Kadir & Brady saliency region detector
8‐Nov‐1140
![Page 41: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/41.jpg)
Lecture 15 -Fei-Fei Li
Foreground modelGaussian shape pdf
Poission pdf on # detections
Uniform shape pdf
Generative probabilistic model (2)
Clutter model
Gaussian part appearance pdf
Gaussian background appearance pdf
Prob. of detection
0.8 0.75 0.9
Gaussian relative scale pdf
log(scale)
Uniformrelative scale pdf
log(scale)
based on Burl, Weber et al. [ECCV ’98, ’00]
8‐Nov‐1141
![Page 42: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/42.jpg)
Lecture 15 -Fei-Fei Li
MotorbikesSamples from appearance model
8‐Nov‐1142
![Page 43: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/43.jpg)
Lecture 15 -Fei-Fei Li
Recognized Motorbikes
8‐Nov‐1143
![Page 44: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/44.jpg)
Lecture 15 -Fei-Fei Li
Background images evaluated with motorbike model
8‐Nov‐1144
![Page 45: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/45.jpg)
Lecture 15 -Fei-Fei Li
Frontal faces
8‐Nov‐1145
![Page 46: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/46.jpg)
Lecture 15 -Fei-Fei Li
Airplanes
8‐Nov‐1146
![Page 47: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/47.jpg)
Lecture 15 -Fei-Fei Li
Spotted cats
8‐Nov‐1147
![Page 48: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/48.jpg)
Lecture 15 -Fei-Fei Li
Summary of results
Dataset Fixed scale experiment
Scale invariant experiment
Motorbikes 7.5 6.7
Faces 4.6 4.6
Airplanes 9.8 7.0
Cars (Rear) 15.2 9.7
Spotted cats 10.0 10.0
% equal error rateNote: Within each series, same settings used for all datasets
8‐Nov‐1148
![Page 49: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/49.jpg)
Lecture 15 -Fei-Fei Li
Comparison to other methods
Dataset Ours Others
Motorbikes 7.5 16.0 Weber et al. [ECCV ‘00]
Faces 4.6 6.0 Weber
Airplanes 9.8 32.0 Weber
Cars (Side) 11.5 21.0Agarwal
Roth [ECCV ’02]
% equal error rate
8‐Nov‐1149
![Page 50: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/50.jpg)
Lecture 15 -Fei-Fei Li
Why this design?• Generic features seem to well in finding consistent parts
of the object
• Some categories perform badly – different feature types needed
• Why PCA representation?– Tried ICA, FLD, Oriented filter responses etc.– But PCA worked best
• Fully probabilistic representation lets us use tools from machine learning community
8‐Nov‐1150
![Page 51: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/51.jpg)
Lecture 15 -Fei-Fei Li
S. Savarese, 2003
8‐Nov‐1151
![Page 52: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/52.jpg)
Lecture 15 -Fei-Fei Li P. Buegel, 15628‐Nov‐1152
![Page 53: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/53.jpg)
Lecture 15 -Fei-Fei Li
One-Shot learningFei-Fei et. al.
ICCV ’03, PAMI ‘068‐Nov‐1153
![Page 54: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/54.jpg)
Lecture 15 -Fei-Fei Li
Algorithm Training Examples Categories
Burl, et al. Weber, et al. Fergus, et al. 200 ~ 400
Faces, Motorbikes, Spotted cats, Airplanes,
Cars
Viola et al. ~10,000 Faces
Schneiderman, et al. ~2,000 Faces, Cars
Rowley et al.
~500 Faces
8‐Nov‐1154
![Page 55: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/55.jpg)
Lecture 15 -Fei-Fei Li
1 2 3 4 5 6 7 8 90
10
20
30
40
50
60
log2 (Training images)
Cla
ssifi
catio
n er
ror
(%)
Generalisation performance
TestTrain
Number of training examples
Previously
6 part Motorbike model
8‐Nov‐1155
![Page 56: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/56.jpg)
Lecture 15 -Fei-Fei Li
How do we do better than what statisticians have told us?
• Intuition 1: use Prior information
• Intuition 2: make best use of training information
8‐Nov‐1156
![Page 57: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/57.jpg)
Lecture 15 -Fei-Fei Li
Prior knowledge: meansShapeAppearance
8‐Nov‐1157
![Page 58: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/58.jpg)
Lecture 15 -Fei-Fei Li
Bayesian frameworkP(object | test, train) vs. P(clutter | test, train)
)object()trainobject,|test( pp
Bayes Rule
dpp )trainobject,|()object,|test(
Expansion by parametrization
8‐Nov‐1158
![Page 59: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/59.jpg)
Lecture 15 -Fei-Fei Li
MLPrevious Work:
Bayesian frameworkP(object | test, train) vs. P(clutter | test, train)
)object()trainobject,|test( pp
Bayes Rule
dpp )trainobject,|()object,|test(
Expansion by parametrization
8‐Nov‐1159
![Page 60: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/60.jpg)
Lecture 15 -Fei-Fei Li
One-Shot learning: pp object,train
Bayesian frameworkP(object | test, train) vs. P(clutter | test, train)
)object()trainobject,|test( pp
Bayes Rule
dpp )trainobject,|()object,|test(
Expansion by parametrization
8‐Nov‐1160
![Page 61: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/61.jpg)
Lecture 15 -Fei-Fei Li
1
2n
model () space
Each object model
Gaussian shape pdfGaussian part
appearance pdf
Model Structure
8‐Nov‐1161
![Page 62: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/62.jpg)
Lecture 15 -Fei-Fei Li
2n
model distribution: p()• conjugate distribution of p(train|,object)
1
model () space
Each object model
Gaussian shape pdfGaussian part
appearance pdf
Model Structure
8‐Nov‐1162
![Page 63: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/63.jpg)
Lecture 15 -Fei-Fei Li
Learning Model Distribution
• use Prior information
• Bayesian learning
• marginalize over theta
Variational EM (Attias, Hinton, Minka, etc.)
ppp object ,train trainobject,
8‐Nov‐1163
![Page 64: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/64.jpg)
Lecture 15 -Fei-Fei Li
E-Step
Random initializationVariational EM
prior knowledge of p()
new estimate of p(|train)
M-Step
new ’s
8‐Nov‐1164
![Page 65: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/65.jpg)
Lecture 15 -Fei-Fei Li
ExperimentsTraining:
1- 6 randomly
drawn images
Testing:
50 fg/ 50 bg images
object present/absent
Datasets
spotted catsairplanes motorbikesfaces
8‐Nov‐1165
![Page 66: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/66.jpg)
Lecture 15 -Fei-Fei Li
Faces
Airplanes
Motorbikes
Spotted cats
8‐Nov‐1166
![Page 67: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/67.jpg)
Lecture 15 -Fei-Fei Li
Experiments: obtaining priors
spotted cats
airplanes
motorbikes
faces
model () space
8‐Nov‐1167
![Page 68: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/68.jpg)
Lecture 15 -Fei-Fei Li
Experiments: obtaining priors
spotted cats
faces
airplanes
motorbikes
model () space
8‐Nov‐1168
![Page 69: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/69.jpg)
Lecture 15 -Fei-Fei Li
Number of training examples
8‐Nov‐1169
![Page 70: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/70.jpg)
Lecture 15 -Fei-Fei Li
Number of training examples
8‐Nov‐1170
![Page 71: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/71.jpg)
Lecture 15 -Fei-Fei Li
Number of training examples
8‐Nov‐1171
![Page 72: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/72.jpg)
Lecture 15 -Fei-Fei Li
Number of training examples
8‐Nov‐1172
![Page 73: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/73.jpg)
Lecture 15 -Fei-Fei Li
Algorithm Training Examples Categories Results(e
rror)
Burl, et al. Weber, et al. Fergus, et al. 200 ~ 400
Faces, Motorbikes, Spotted cats, Airplanes,
Cars
5.6 - 10 %
Viola et al. ~10,000 Faces 7-21%
Schneiderman, et al. ~2,000 Faces, Cars 5.6 – 17%
Rowley et al.
~500 Faces 7.5 –24.1%
BayesianOne-Shot 1 ~ 5 Faces, Motorbikes,
Spotted cats, Airplanes8 –
15 %
8‐Nov‐1173
![Page 74: Lecture 15: Object recognition: Part based generative modelsvision.stanford.edu/teaching/cs231a_autumn1213... · Fei-Fei Li Lecture 15 - • Task: Estimation of model parameters Learning](https://reader033.fdocuments.in/reader033/viewer/2022042310/5ed81dcf0fa3e705ec0dddc3/html5/thumbnails/74.jpg)
Lecture 15 -Fei-Fei Li
What we have learned today?
8‐Nov‐1174
• Introduction• Constellation model
– Weakly supervised training– One‐shot learning
• (Problem Set 4 (Q1))