1

Selective Transfer Machine for Personalized Facial Action Unit

Detection

Wen-Sheng Chu, Fernando De la Torre and Jeffery F. CohnRobotics Institute, Carnegie Mellon University

July 9, 2013

2

AU 6+12

Facial Action Units (AU)

3

Main Idea

4

Related Work: Features

5

Related Work: Classifiers

6

Feature Bias

Person specific!

7

Occurrence Bias

8

Selective Transfer Machine (STM) Formulation

Maximizes margin of penalized SVM

Minimize distribution mismatch

9

Goal (1): Maximize penalized SVM margin

marginpenalized loss

10

Goal (2): Minimize Distribution Mismatch

• Kernel Mean Matching (KMM)*

* “Covariate shift by kernel mean matching”, Dataset shift in machine learning, 2009.

11

Goal (2): Minimize Distribution Mismatch

Groundtruth

Bad estimatorfor testing data!

12

Better fitting!

Groundtruth

Selection by reweighting training data

Goal (2): Minimize Distribution Mismatch

13

14

Optimization: Alternate Convex Search

15

Optimization: Alternative Convex Search

16

Compare with Relevant Work

[1] "Covariate shift by kernel mean matching," Dataset shift in machine learning, 2009.

[2] "Transductive inference for text classification using support vector machines," In ICML 1999.

[3] "Domain adaptation problems: A DASVM classification technique and a circular validation strategy," PAMI 2010.

17

Experiments

• Features– SIFT descriptors on 49 facial landmarks– Preserve 98% energy using PCA

Datasets #Subjects #Videos #Frm/vid ContentCK+ 123 593 ~20 NeutralPeakGEMEP-FERA 7 87 20~60 ActingRU-FACS 29 29 5000~7500 Interview

18

Experiment (1): Synthetic Data

19

• Two protocols– PS1: train/test are separate data of the same subject

– PS2: training subjects include test subject (same protocol in [2])

• GEMEP-FERA

Experiment (2): Comparison with Person-specific (PS) Classifiers

20

Experiment (2): Selection Ability of STM

21

• 123 subjects, 597 videos, ~20 frames/video

Experiment (3): CK+

22

Experiment (4): GEMEP-FERA

• 7 subjects, 87 videos, 20~60 frames/video

23

• 29 subjects, 29 videos, 5000~7000 frames/vid

Experiment (5): RU-FACS

24

Summary

• Person-specific biases exist among face-related problems, esp. facial expression

• We propose to alleviate the biases by personalizing classifiers using STM

• Next– Joint optimization in terms of – Reduce the memory cost using SMO– Explore more potential biases in face problems,

e.g., occurrence bias

25

Questions?

[1] "Covariate shift by kernel mean matching," Dataset shift in machine learning, 2009.

[2] "Transductive inference for text classification using support vector machines," In ICML 1999.

[3] "Domain adaptation problems: A DASVM classification technique and a circular validation strategy," PAMI 2010.

[4] “Integrating structured biological data by kernel maximum mean discrepancy”, Bioinformatics 2006.

[5] “Meta-analysis of the first facial expression recognition challenge,” IEEE Trans. on Systems, Man, and Cybernetics, Part B, 2012.

http://humansensing.cs.cmu.edu/wschu/