Principal Component Analysis of Face Properties

Samarasena Buchala1, Neil Davey1, Tim Gale1,2, Ray Frank1

1 School of Computer Science, University of Hertfordshire, College Lane, Hatfield, AL10 9AB, UK

2 Department of Psychiatry, QEII Hospital, Welwyn Garden City, AL7 4HQ, UK

Overview

• Principal Component Analysis – Face Recognition

• Principal Component Analysis

• Analysis of components of PCA

– Linear Discriminant Analysis of PCA components

– Does PCA efficiently encodes information in face images

– Analysis of gender, ethnicity, age, and identity

– Do components encode information related to multiple properties

Face Properties

• Human face is considered to be special in terms of biological and social roles

• Has multiple properties from which they can be categorised at different levels of specificity – gender, ethnicity, age, identity, expression, degree of attractiveness, typicality, attractiveness, so on.

• Widely researched in the fields of Computer Science and Psychology

Principal Component Analysis (PCA) on Face Images

• Dimensionality reduction – Sirovich and Kirby (1987)

• Face recognition – Turk and Pentland (1991)

• Benchmark for face recognition algorithms - (Moon & Phillips, 2001).

• Distinctiveness effects of faces - (Hancock, 1996)

• “other-race effect” - (O'Toole et al., 1991b)

• Dimensional-based model of facial expression - (Calder et al., 2001)

Research Questions

• Does PCA encode information related to gender, ethnicity, age, and identity efficiently?

• What information do PCA encode?

• Are there components (features) of PCA that encode multiples properties?

PCA

• The aim of the PCA is a linear reduction of D dimensional data to d dimensional data (d<D), while preserving as much information, in the data, as possible.

• Linear functions y1= w1 Xy2= w2 X***yd= wd X

Y= W X• X – inputs; Y – outputs, components; W – eigenvectors, eigenfaces, basis

vectors

x1

w1

w2

x2

How many components?

• Usual choice consider the first d PC’s which account for some percentage, usually above 90 %, of the cumulative variance of the data.

• This is disadvantageous if the last components are interesting

W2

W1

x1

x2

Dataset

• A subset of FERET dataset

• 2670 grey scale frontal face images

• Rich in variety: face images vary in pose, background lighting, presence or absence of glasses, slight change in expression

Property No.Categorie

s

Categories No. Face

s

Gender 2Male 1603

Female 1067

Ethnicity 3

Caucasian 1758

African 320

East Asian 363

Age 5

20 – 29 665

30 – 39 1264

40 – 49 429

50 – 59 206

60+ 106

Identity 358Individuals with 3

or more examples

1161

Dataset

• Each image is pre-processed to a 65 X 75 resolution.

• Aligned based on eye locations

• Cropped such that little or no hair information is available

• Histogram equalisation is applied to reduce lighting effects

Does PCA efficiently represents information in face images?

• Images of 65 × 75 resolution leads to a dimensionality of 4875. • The first 350 components accounted for 90% variance of the

data.• Each face is thus represented using 350 components instead of

4875 dimensions

• Classification employing 5-fold cross validation, with 80% of faces in each category for training and 20% of faces in each category for testing

• for identity recognition leave-one-out method is used.• LDA is performed on the PCA data• Euclidean measure is used for classification

Property Classification

Gender 86.4%

Ethnicity 81.6%

Age 91.5%

Identity 90%

What information does PCA encode? – Gender

• Gender encoding power estimated using the LDA• 3rd component carries highest gender encoding power followed by the 4 th components• All important components are among the first 50 components

0 10 20 30 40 500

1

2

3

4

5

6

7

8

9

Components

Gen

der

En

cod

ing

Po

wer

What information does PCA encode? – Gender

Reconstructed images from the altered components (a) third and (b) fourth components. The components are progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D.

• Third component encodes information related to the complexion, length of the nose, presence or absence of hair on the forehead, and texture around the mouth region.

• Fourth component encodes information related to the eyebrow thickness, presence or absence of smiling expression

-6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD

Gender

(a) Face examples with the first two being female and the next two being male faces. (b) Reconstructed faces of (a) using the top 20 gender important components. (c) Reconstructed faces of (a) using all components, except the top 20 gender important components.

What information does PCA encode? – Ethnicity

• 6th component carries highest ethnicity encoding power followed by the 15th components

• All ethnicity important components are among the first 50 components

0 10 20 30 40 500

1

2

3

4

5

6

7

8

9

10

Components

Eth

nic

ity

En

cod

ing

Po

wer

Ethnicity

Reconstructed images from the altered components (a) 6th and (b) 4th components. The components are progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D.

• 6th component encodes information related to complexion, broadness and length of the nose

• 15th component encodes information related to length of the nose, complexion, and presence or absence of smiling expression

-6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD

What information does PCA encode? – Age

0 10 20 30 40 500

1

2

3

4

5

6

7

8

9

10

Components

Ag

e E

nco

din

g P

ow

er• Age – 20-39 and 50-60+ age

groups termed as young and old)

• 10th component is found to be the most important for age

Reconstructed images from the altered tenth component. The component is progressively added by quantities of -6 S.D (extreme left) to +6 S.D (extreme right) in steps of 2 S.D

-6 SD -4 SD -2 SD Mean +2 SD +4 SD +6 SD

What information does PCA encode? – Identity

• Many components are found to be important for identity. However, their importance magnitude is small.

• These components are widely distributed and not restricted to the first 50 components

0 20 40 60 80 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Components

Iden

tity

En

cod

ing

Po

wer

Can a single component encode multiple properties?

• A grey beard informs that the person is a male and also, most probably, old.

• As all important components of gender, ethnicity, and age are among the first 50 components there are overlapping components.

• One example is the 3rd component which is found to be the most important for gender and second most important for age

0 10 20 30 40 500

1

2

3

4

5

6

7

8

9

Components

Gen

der

En

cod

ing

Po

wer

0 10 20 30 40 500

1

2

3

4

5

6

7

8

9

10

Components

Ag

e E

nco

din

g P

ow

er

Can a single component encode multiple properties?

Normal distribution plots of the (a) third (b) and fourth components for male and female classes of young and old age groups.

-10 -5 0 5 10 15 200

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

Component 3

Young maleYoung femaleOld maleOld female

-10 -5 0 5 10 150

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Component 4

Young maleYoung femaleOld maleOld female

Summary

• PCA encodes face image properties such as gender, ethnicity, age, and identity efficiently.

• Very few components are required to encode properties such as gender, ethnicity and age and these components are amongst the first few components which capture large part of the variance of the data. Large number of components are required to encode identity and these components are widely distributed.

• There may be components which encode multiple properties.