Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object...
Transcript of Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object...
![Page 1: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/1.jpg)
Li Fei-Fei, Princeton Rob Fergus, MIT
Antonio Torralba, MIT
Recognizing and Learning Object Categories: Year 2007
CVPR 2007 Minneapolis, Short Course, June 17
![Page 2: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/2.jpg)
Plato said… • Ordinary objects are classified together if they
`participate' in the same abstract Form, such as the Form of a Human or the Form of Quartz.
• Forms are proper subjects of philosophical investigation, for they have the highest degree of reality.
• Ordinary objects, such as humans, trees, and stones, have a lower degree of reality than the Forms.
• Fictions, shadows, and the like have a still lower degree of reality than ordinary objects and so are not proper subjects of philosophical enquiry.
![Page 3: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/3.jpg)
How many object categories are there?
Biederman 1987
![Page 4: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/4.jpg)
So what does object recognition involve?
![Page 5: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/5.jpg)
Verification: is that a lamp?
![Page 6: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/6.jpg)
Detection: are there people?
![Page 7: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/7.jpg)
Identification: is that Potala Palace?
![Page 8: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/8.jpg)
Object categorization
mountain
building tree
banner
vendor people
street lamp
![Page 9: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/9.jpg)
Scene and context categorization
• outdoor • city • …
![Page 10: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/10.jpg)
Computational photography
![Page 11: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/11.jpg)
Assisted driving
meters
met
ers
Ped
Ped
Car
Lane detection
Pedestrian and car detection
• Collision warning systems with adaptive cruise control, • Lane departure warning systems, • Rear object detection systems,
![Page 12: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/12.jpg)
Improving online search
Query: STREET
Organizing photo collections
![Page 13: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/13.jpg)
Challenges 1: view point variation
Michelangelo 1475-1564
![Page 14: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/14.jpg)
Challenges 2: illumination
slide credit: S. Ullman
![Page 15: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/15.jpg)
Challenges 3: occlusion
Magritte, 1957
![Page 16: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/16.jpg)
Challenges 4: scale
![Page 17: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/17.jpg)
Challenges 5: deformation
Xu, Beihong 1943
![Page 18: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/18.jpg)
Challenges 6: background clutter
Klimt, 1913
![Page 19: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/19.jpg)
Challenges 7: intra-class variation
![Page 20: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/20.jpg)
History: early object categorization
![Page 21: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/21.jpg)
• Turk and Pentland, 1991 • Belhumeur, Hespanha, &
Kriegman, 1997 • Schneiderman & Kanade 2004 • Viola and Jones, 2000
• Amit and Geman, 1999 • LeCun et al. 1998 • Belongie and Malik, 2002
• Schneiderman & Kanade, 2004 • Argawal and Roth, 2002 • Poggio et al. 1993
![Page 22: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/22.jpg)
![Page 23: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/23.jpg)
Object categorization: the statistical viewpoint
)|( imagezebrap
)( ezebra|imagnopvs.
• Bayes rule:
)()(
)|()|(
)|()|(
zebranopzebrap
zebranoimagepzebraimagep
imagezebranopimagezebrap
⋅=
posterior ratio likelihood ratio prior ratio
![Page 24: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/24.jpg)
Object categorization: the statistical viewpoint
)()(
)|()|(
)|()|(
zebranopzebrap
zebranoimagepzebraimagep
imagezebranopimagezebrap
⋅=
posterior ratio likelihood ratio prior ratio
• Discriminative methods model posterior
• Generative methods model likelihood and prior
![Page 25: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/25.jpg)
Discriminative
• Direct modeling of
Zebra
Non-zebra
Decision boundary
)|()|(
imagezebranopimagezebrap
![Page 26: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/26.jpg)
• Model and
Generative )|( zebraimagep ) |( zebranoimagep
Low Middle
High MiddleLow
)|( zebranoimagep)|( zebraimagep
![Page 27: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/27.jpg)
Three main issues
• Representation – How to represent an object category
• Learning – How to form the classifier, given training data
• Recognition – How the classifier is to be used on novel data
![Page 28: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/28.jpg)
Representation – Generative /
discriminative / hybrid
![Page 29: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/29.jpg)
Representation – Generative /
discriminative / hybrid – Appearance only or
location and appearance
![Page 30: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/30.jpg)
Representation – Generative /
discriminative / hybrid – Appearance only or
location and appearance
– Invariances • View point • Illumination • Occlusion • Scale • Deformation • Clutter • etc.
![Page 31: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/31.jpg)
Representation – Generative /
discriminative / hybrid – Appearance only or
location and appearance
– invariances – Part-based or global
w/sub-window
![Page 32: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/32.jpg)
Representation – Generative /
discriminative / hybrid – Appearance only or
location and appearance
– invariances – Parts or global w/sub-
window – Use set of features or
each pixel in image
![Page 33: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/33.jpg)
– Unclear how to model categories, so we learn what distinguishes them rather than manually specify the difference -- hence current interest in machine learning
Learning
![Page 34: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/34.jpg)
– Unclear how to model categories, so we learn what distinguishes them rather than manually specify the difference -- hence current interest in machine learning)
– Methods of training: generative vs. discriminative
Learning
![Page 35: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/35.jpg)
– Unclear how to model categories, so we learn what distinguishes them rather than manually specify the difference -- hence current interest in machine learning)
– What are you maximizing? Likelihood (Gen.) or performances on train/validation set (Disc.)
– Level of supervision • Manual segmentation; bounding box; image
labels; noisy labels
Learning
Contains a motorbike
![Page 36: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/36.jpg)
– Unclear how to model categories, so we learn what distinguishes them rather than manually specify the difference -- hence current interest in machine learning)
– What are you maximizing? Likelihood (Gen.) or performances on train/validation set (Disc.)
– Level of supervision • Manual segmentation; bounding box; image
labels; noisy labels – Batch/incremental (on category and image
level; user-feedback )
Learning
![Page 37: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/37.jpg)
– Unclear how to model categories, so we learn what distinguishes them rather than manually specify the difference -- hence current interest in machine learning)
– What are you maximizing? Likelihood (Gen.) or performances on train/validation set (Disc.)
– Level of supervision • Manual segmentation; bounding box; image
labels; noisy labels – Batch/incremental (on category and image
level; user-feedback ) – Training images:
• Issue of overfitting • Negative images for discriminative methods
Priors
Learning
![Page 38: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/38.jpg)
– Unclear how to model categories, so we learn what distinguishes them rather than manually specify the difference -- hence current interest in machine learning)
– What are you maximizing? Likelihood (Gen.) or performances on train/validation set (Disc.)
– Level of supervision • Manual segmentation; bounding box; image
labels; noisy labels – Batch/incremental (on category and image
level; user-feedback ) – Training images:
• Issue of overfitting • Negative images for discriminative methods
– Priors
Learning
![Page 39: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/39.jpg)
– Scale / orientation range to search over – Speed – Context
Recognition
![Page 40: Recognizing and Learning Object Categories: Year 2007€¦ · Recognizing and Learning Object Categories: Year 2007 CVPR 2007 Minneapolis, Short Course, June 17 . Plato said… •](https://reader036.fdocuments.in/reader036/viewer/2022071000/5fbd0d715bae2b43175acdb7/html5/thumbnails/40.jpg)
Hoi
em, E
fros,
Her
bert,
200
6