Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards...
-
Upload
cornelia-imogene-wheeler -
Category
Documents
-
view
221 -
download
0
description
Transcript of Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards...
![Page 1: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/1.jpg)
Active Appearance Models
Dhruv BatraECE CMU
![Page 2: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/2.jpg)
Active Appearance Models
1. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 (H.Burkhardt & B. Neumann Ed.s). Vol. 2, pp. 484-498, Springer, 1998
2. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", IEEE PAMI, Vol.23, No.6, pp.681-685, 2001
3. G.J. Edwards, A. Lanitis, C.J. Taylor, T. F. Cootes. “Statistical Models of Face Images Improving Specificity”, BMVC (1996)
![Page 3: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/3.jpg)
Essence of the Idea “Interpretation through synthesis”
Form a model of the object/image (Learnt from the training dataset)
I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp. 135 - 164.
![Page 4: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/4.jpg)
Essence of the Idea (cont.) Explain a new example in terms of the model parameters
![Page 5: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/5.jpg)
So what’s a model
Model
“Shape” “texture”
![Page 6: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/6.jpg)
Active Shape Modelstraining set
![Page 7: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/7.jpg)
Texture Models
warp to mean shape
![Page 8: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/8.jpg)
Random Aside Shape Vector provides alignment
=
43Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
![Page 9: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/9.jpg)
Random Aside Alignment is the key
1. Warp to mean shape
2. Average pixels
Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
![Page 10: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/10.jpg)
Random Aside Enhancing Gender
more same original androgynous more opposite
D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70-76
![Page 11: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/11.jpg)
Random Aside (can’t escape structure!)
Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Antonio Torralba & Aude Oliva (2002)
Averages: Hundreds of images containing a person are
averaged to reveal regularities in the intensity patterns across
all the images.
![Page 12: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/12.jpg)
Random Aside (can’t escape structure!)
Tomasz Malisiewicz, http://www.cs.cmu.edu/~tmalisie/pascal/trainval_mean_large.png
![Page 13: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/13.jpg)
Random Aside (can’t escape structure!)“100 Special Moments” by Jason Salavon
Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml
![Page 14: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/14.jpg)
Random Aside (can’t escape structure!)“Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon
1960s 1970s 1980sJason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml
![Page 15: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/15.jpg)
Back (sadly) to Texture Models
raster scan
Normalizations
![Page 16: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/16.jpg)
PCA Galore
Reduce Dimensions of shape vector
Reduce Dimension of “texture” vector
They are still correlated; repeat..
![Page 17: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/17.jpg)
Object/Image to Parameters
modeling
~80
![Page 18: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/18.jpg)
Playing with the Parameters
First two modes of shape variation First two modes of gray-level variation
First four modes of appearance variation
![Page 19: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/19.jpg)
Active Appearance Model Search Given: Full training model set, new image to be interpreted,
“reasonable” starting approximation
Goal: Find model with least approximation error
High Dimensional Search: Curse of the dimensions strikes again
![Page 20: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/20.jpg)
Active Appearance Model Search Trick: Each optimization is a similar problem, can be learnt
Assumption: Linearity
Perturb model parameters with known amount
Generate perturbed image and sample error
Learn multivariate regression for many such perterbuations
![Page 21: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/21.jpg)
Active Appearance Model Search Algorithm: current estimate of model parameters: normalized image sample at current estimate
![Page 22: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/22.jpg)
Active Appearance Model Search Slightly different modeling:
Error term:
Taylor expansion (with linear assumption)
Min (RMS sense) error:
Systematically perturb and estimate by numerical differentiation
![Page 23: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/23.jpg)
Active Appearance Model Search (Results)
![Page 24: Active Appearance Models Dhruv Batra ECE CMU. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc.](https://reader036.fdocuments.in/reader036/viewer/2022062401/5a4d1b367f8b9ab05999d07e/html5/thumbnails/24.jpg)