Active Appearance Models

Dhruv BatraECE CMU

Active Appearance Models

1. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 (H.Burkhardt & B. Neumann Ed.s). Vol. 2, pp. 484-498, Springer, 1998

2. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", IEEE PAMI, Vol.23, No.6, pp.681-685, 2001

3. G.J. Edwards, A. Lanitis, C.J. Taylor, T. F. Cootes. “Statistical Models of Face Images Improving Specificity”, BMVC (1996)

Essence of the Idea “Interpretation through synthesis”

Form a model of the object/image (Learnt from the training dataset)

I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp. 135 - 164.

Essence of the Idea (cont.) Explain a new example in terms of the model parameters

So what’s a model

Model

“Shape” “texture”

Active Shape Modelstraining set

Texture Models

warp to mean shape

Random Aside Shape Vector provides alignment

=

43Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

Random Aside Alignment is the key

1. Warp to mean shape

2. Average pixels

Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

Random Aside Enhancing Gender

more same original androgynous more opposite

D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70-76

Random Aside (can’t escape structure!)

Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

Antonio Torralba & Aude Oliva (2002)

Averages: Hundreds of images containing a person are

averaged to reveal regularities in the intensity patterns across

all the images.

Random Aside (can’t escape structure!)

Tomasz Malisiewicz, http://www.cs.cmu.edu/~tmalisie/pascal/trainval_mean_large.png

Random Aside (can’t escape structure!)“100 Special Moments” by Jason Salavon

Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml

Random Aside (can’t escape structure!)“Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon

1960s 1970s 1980sJason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml

Back (sadly) to Texture Models

raster scan

Normalizations

PCA Galore

Reduce Dimensions of shape vector

Reduce Dimension of “texture” vector

They are still correlated; repeat..

Object/Image to Parameters

modeling

~80

Playing with the Parameters

First two modes of shape variation First two modes of gray-level variation

First four modes of appearance variation

Active Appearance Model Search Given: Full training model set, new image to be interpreted,

“reasonable” starting approximation

Goal: Find model with least approximation error

High Dimensional Search: Curse of the dimensions strikes again

Active Appearance Model Search Trick: Each optimization is a similar problem, can be learnt

Assumption: Linearity

Perturb model parameters with known amount

Generate perturbed image and sample error

Learn multivariate regression for many such perterbuations

Active Appearance Model Search Algorithm: current estimate of model parameters: normalized image sample at current estimate

Active Appearance Model Search Slightly different modeling:

Error term:

Taylor expansion (with linear assumption)

Min (RMS sense) error:

Systematically perturb and estimate by numerical differentiation

Active Appearance Model Search (Results)