03.11.2008, Tim Landgraf

Active Appearance Models

AG KI, Journal Club 03 Nov 2008

03.11.2008, Tim Landgraf

03.11.2008, Tim Landgraf

03.11.2008, Tim Landgraf

The Idea

• Objects are modelled in shape and grey-level appearance (training necessary)

• New model instances are synthesized and matched onto the new image

• Model parameters are altered according to the quality of the fit

03.11.2008, Tim Landgraf

The Idea

• Generate new model x

x = μ + P*b

from mean model μ and some (b) linear combination of principal components P

• Fit Ix to image region Ii, by altering b according to (Ix – Ii ) = ΔI

offline

online

03.11.2008, Tim Landgraf

creating the model: step by step

1. annotate landmark points2. align the shapes3. PCA (find modes of shape

variation)4. make data shape-free 5. normalize grey values6. PCA (find modes of grey value variation)7. PCA (on the combined model)

Example: 122 landmarks for the face image

03.11.2008, Tim Landgraf

What the … PCA?

• Principal Component Analysis

(aka: Karhunen-Loeve Transform)

BASICS

03.11.2008, Tim Landgraf

PCA, cont.

• used for decorrelation, dimension reduction, generalization

• Data is assumed to be:– Linear– Gaussian (unimodal)

• Principal components: eigenvectors of the Covariance matrix

BASICS

03.11.2008, Tim Landgraf

AAMexplorer

03.11.2008, Tim Landgraf

Fitting the model onto the image

• x = μ + P*b

• simplest approach: Δb = A*ΔI

• „learn“ A:– perturbate known model b‘ = b + Δb and store

the change of image ΔI – Find A by multi-variate linear regression– (note:) A connects grey-value appearance

with all model params

/* reminder */

03.11.2008, Tim Landgraf

Optimization vs. Learning

Initial position

optimum

optimum

Small perturbations

03.11.2008, Tim Landgraf

Extras

• Iterative Approach:– b1 = b0 + kΔb with k \in {0.25, …, 2.0}

– evaluate error and accept new estimate b1, if better fit, otherwise change k

• Multi-resolution: use pyramids to extend the prediction to greater ranges

03.11.2008, Tim Landgraf

AAMs: Properties

• good results if initial guess within 20 pixels and 10% scale

• depends on training image background appearance, too