Face Model Fitting with Generic, Group-specific, and Person-specific Objective Functions

Chair for Image Understanding and Knowledge-based Systems

Institute for Informatics

Technische Universität München

Sylvia Pietzsch

sylvia.pietzsch@cs.tum.edu

2008, January 22nd 2/15Technische Universität MünchenSylvia Pietzsch

Overview Model-based Image Interpretation

Generic Objective Functions

Specific Objective Functions

Experimental Results

Conclusion and Outlook

2008, January 22nd 3/15Technische Universität MünchenSylvia Pietzsch

Model-based image interpretation The model

The model contains a parameter vector that represents the model’s configuration.

The objective function Calculates a value that indicates how accurately a parameterized model matches an image.

The fitting algorithm Searches for the model parameters that describe the image best, i.e. it minimizes the objective function.

2008, January 22nd 4/15Technische Universität MünchenSylvia Pietzsch

Local Objective Functions

2008, January 22nd 5/15Technische Universität MünchenSylvia Pietzsch

Ideal Objective FunctionsP1: Correctness property:

Global minimum corresponds to the best fit.P2: Uni-modality property:

The objective function has no local extrema. ¬ P1 P1

¬P2

P2

Don’t exist for real-world images

Only for annotated images: fn( I , x ) = | cn – x |

2008, January 22nd 6/15Technische Universität MünchenSylvia Pietzsch

Learning the Objective Function

x x x xx

xxx x xxx x x x

x x xx x

x xx x x x x

x xxx x

Ideal objective function generates training data Machine Learning technique generates calculation rules

2008, January 22nd 7/15Technische Universität MünchenSylvia Pietzsch

Learning the Objective Function (1)

2008, January 22nd 8/15Technische Universität MünchenSylvia Pietzsch

Learning the Objective Function (2)

2008, January 22nd 9/15Technische Universität MünchenSylvia Pietzsch

Learning the Objective Function (3)

6 styles · 3 sizes · (5 · 5) locations = 450 features

2008, January 22nd 10/15Technische Universität MünchenSylvia Pietzsch

Specific Objective Functions Single Images

The objective function has to take any appearance of a human face into consideration.

➱ moderate accuracy

Image SequenceThe appearance of a person‘s face only changes slightly. Consider particular characteristics of the visible person,

e.g. beard, glasses, bald head,...

➱ increase of accuracy

Challenges: Learn specific objective functions for groups of persons offline. Detect the correct group online.

2008, January 22nd 11/15Technische Universität MünchenSylvia Pietzsch

Evaluation: Fitting Results

45 persons from news broadcasts on TV

2008, January 22nd 12/15Technische Universität MünchenSylvia Pietzsch

Group-specific Objective Functions automatically subdivide persons into partitions

learn objective function for each partition

best possible partitioning: minimize the mean fitting error when applying the group-specific objective functions to the partitions

number of partitions influences specificity

2008, January 22nd 13/15Technische Universität MünchenSylvia Pietzsch

Experimental Results

P26

P44

P42

λ = 3.9

2008, January 22nd 14/15Technische Universität MünchenSylvia Pietzsch

Conclusion and Outlook large variation in facial appearance challenges

model fitting trade-off between generality and accuracy group-specific objective functions perform

substantially better for persons within the group of avail for tracking face models through image

sequences Further work: automatic classification to

determine on-line, which partition a person belongs to

2008, January 22nd 15/15Technische Universität MünchenSylvia Pietzsch

Thank you!