MSRI workshop, January 2005

Object Recognition • Collected databases of objects on uniform background

(no occlusions, no clutter)• Mostly focus on viewpoint variations (at fixed scale)

• PCA based techniques, Eigenimages [Turk 91], Leonardis & Bishoph 98]• Not robust with respect to • occlusions, clutter changes of viewpoint• Requires segmentation

MSRI workshop, January 2005

History - Recognition

• Alternative descriptors - Color histogram [Swain 91] (not discriminative enough)

• Geometric invariants [Rothwell 92] - Function with a value independent of the transformation - Invariant for image rotation : distance of two points - Invariant for planar homography : cross-ratio

MSRI workshop, January 2005

Figure from “Efficient model library access by projectively invariant indexing functions,” by C.A. Rothwell et al., Proc. Computer Vision and Pattern Recognition, 1992, copyright 1992, IEEE - (courtesy Forsythe, Ponce CV, Prentice Hall)

MSRI workshop, January 2005

Recognition with local photometric invariants

[ Local greyvalue invariants for image retrieval, C. Schmid and R. Mohr, PAMI 1997 ]

> 5000images

Semi-local constraints, neighboring points should match, angles, lengthratios should be similar

MSRI workshop, January 2005

Object Recognition - Present

• Can representations of objects be learned automatically • Given datasets of images containing unsegmented objects

• Which objects can be recognized in images ? • Which object parts are distinctive ? • What are the parameters of global shape or geometry ?

MSRI workshop, January 2005

Figure from, “A general framework for object detection,” by C. Papageorgiou, M. Oren and T. Poggio, Proc. ICCV, copyright 1998, IEEE

Recognition by finding patternsGeneral strategy:

search image windows at a range of scales correct for illuminationPresent corrected window to classifier: - face/no face classifier

Figure from A Statistical Method for 3D Object Detection Applied to Faces and Cars, H. Schneiderman and T. Kanade, CVPR, 2000.

MSRI workshop, January 2005

Constellation of Parts Model

Fischler & Elschlagar, 1973

• [Coots, Taylor et al’95] Univ. of Manchester• [G.Csurka et al’04] Xerox Research Europe• [Webber et al’00, Fei-Fei Li et al 03] Caltech• [Fergus et al’03,04] Oxford

• Constellation of parts models• Object detection part, category recognition

MSRI workshop, January 2005

Constellation of Parts Model• Strategy for learning models for recognition

Idea: Learn generative probabilistic model of objects

1. Run part detectors, obtain parts (location, appearance, scale)

2. Form likely object hypothesis, update the probability model and validate - hypothesis is particular configuration of parts

Recognition (computing likelihood ratio)

MSRI workshop, January 2005

Foreground model

Gaussian shape pdf

Prob. of detection

Gaussian part appearance pdf

Generative probabilistic model courtesy of R. Fergus (presentation)

Uniform shape pdf

Clutter model

Gaussian appearance pdf

Gaussian relative scale pdf

Log(scale)

0.8 0.75 0.9

Poission pdf on # detections

Uniformrelative scale pdf

Log(scale)

MSRI workshop, January 2005

Constellation model and Bayesian Framework

• P parts: location X, Scale S and Appearance A.

• Distribution is modeled by hidden parameters .( e.g. Mean, Covariance of Gaussian)

• Maximum Likelihood (ML) with a single value (Fergus et al) . • Approximation to make the integral tractable (Li Fei-Fei et al)

Appearance, shape, scale, hypothesis

MSRI workshop, January 2005

Motorbikes (Fergus’s results)

Samples from appearance model

MSRI workshop, January 2005

References

• L. Fei-Fei, R. Fergus, and P. Perona. A Bayesian approach to unsupervised One-Shot learning of Object categories. Proc. ICCV. 2003

• Fergus, R. , Perona, P. and Zisserman, A., A Visual Category Filter for Google Images ,Proc. Of of the 8th European Conf. on Computer Vision, ECCV 2004.

• Y. Amit and D. Geman, “A computational model for visual selection”, Neural Computation, vol. 11, no. 7, pp1691-1715, 1999.

• M. Weber, M. Welling and P. Perona, “Unsupervised learning of models for recognition”, Proc. 6th ECCV, vol. 2, pp. 101-108, 2000.

MSRI workshop, January 2005

Recognition …

• Integration of multiple view models (Complex 3D objects)• Generative vs Discriminative Models• Scaling issues > 10000 object • Recognition of object categories

• Alternative models of context• intra-object-within-class variations (chairs)• Different feature types • Enable models with large number of parts

• Image based retrieval – annotating by semantic context• Associating words with pictures