Facial Expression Recognition

By: Stephanie Tsai

Nazia Hashmi

Michelle Aleong

What is Facial Expression Recognition?

Facial Expression Recognition has been defined as the biometric identification by scanning a person’s face and matching it against a library of faces

Process by which the brain and mind understand and interpret the human face

Why Facial Expression?

Behavioral assessment of emotion and paralinguistic displays

Facial nerve disorders  Computer systems that understand human behavior Speech recognition. Security systems. Lie detection. Video compression in telecommunications. Emotion for animation.

FACS

Facial Action Coding System Most widely used method for measuring and

describing facial behaviors Explains how to categorize facial behaviors

based on the muscles that produce them Goal is to create a reliable means for skilled

human scorers to determine the category in which to fit each facial behavior.

Once upon a time…

Developed by Paul Ekman (UCSF) & William Friesen of Langley Porter Neuropsychiatric Institute in San Francisco in 1978

Current computer programs being developed at the University of Pittsburgh and Carnegie Mellon University, the other by a team at the Salk Institute in La Jolla, California

Action Units DiagramAction Unit Description Facial Muscle Example Image

1 Inner Brow Raiser Frontalis, pars medialis

2 Outer Brow Raiser Frontalis, pars lateralis

3 Brow Lowerer Corrugator supercilii

5 Upper Lid Raiser Levator palpebrae superioris

10 Upper Lip Raiser Levator labii superioris

25 Lips Part Depressor labii inferioris

How it Works

Action Units (AUs) are the measurement units of FACS

44 AUs FAC coder “dissects” the expression and

decomposes it into the specific AUs that produce the movement

Scoring

The scores consist of the list of AUs that produce it

Descriptive only AU 1+5+25

Problems with FACS

Human-observer based methods for measuring facial expression are labor intensive, qualitative, and difficult to standardize.

Less than 100% inter observer reliability

Superman (aka. the computer) to the rescue!! Goal is to make feasible more rigorous,

quantitative measurement of facial expression in diverse applications

Computers can recognize specific action units

Unbiased based on person's gender, race or age.

Automated Face Analysis

Automated Face Analysis Training data on group of more than 200 people

of different racial and ethnic backgrounds.“The hardest and most time-consuming part of all

this work is collecting a database of images that is diverse enough and big enough to train the computer," says Sejnowski.

3-generation system developed at CMU

Generation I

Generation 2

Generation 3

Current Research

Competitions to explore the different methods to analyze the expressions from the same set of videos

Research unit ongoing at CMU Department of Computer Science

Are you ready to have a computer know what you’re

feeling?