Detecting Faces in Images: A Survey

By: Ming-Hsuan Yang, David J. Kriegman, and Narendra Ahuja

Presented By: Neal Audenaert

Agenda

IntroductionApproaches

Knowledge-basedFeature invariantTemplate matchingAppearnce-based

Databases and EvaluationDiscussion

Agenda

IntroductionApproaches

Knowledge-basedFeature invariantTemplate matchingAppearnce-based

Databases and EvaluationDiscussion

Introduction

DomainFace detection (not recognition)Still images

ObjectivesComprehensive survey of techniquesDiscussion of performance measures

LimitationsMethods are not directly comparable

Challenges

PoseStructural componentsFacial expressionOcclusionImage orientationImaging conditions

General Tasks

Face localizationFacial feature detectionFace recognitionFace authenticationFace trackingFacial expression recognition

Agenda

IntroductionApproaches

Knowledge-basedFeature invariantTemplate matchingAppearnce-based

Databases and EvaluationDiscussion

Survey of Techniques

Knowledge BasedTop-downBottom-up

Template BasedDefined templatesLearned templates

Knowledge-basedFeature invariant

Template matchingAppearance-based

Survey of Techniques

XAppearance-based

XXTemplate Matching

XFeature Invariant

XKnowledge-basedDet.Loc.Approach

Knowledge-based Feature Invariant Template Matching Appearance-based

Knowledge-Based Top-Down Methods

Main Idea: Use knowledge about what constitutes a face faces to define rules

Strengths: Frontal faces in uncluttered scenes

Weaknesses:Translating knowledge into rulesEnumeration of cases

Knowledge-based Feature Invariant Template Matching Appearance-based

Knowledge-Based Top-Down Methods

Bottom-Up Feature-Based Methods

Main Idea: Describe relationships between invariant features using statistical models

Strengths: Improved invariance for different poses and lighting conditions

Weaknesses: Corruption of individual due to illumination, noise, or occlusion

Knowledge-based Feature Invariant Template Matching Appearance-based

Bottom-Up Feature-Based Methods

Facial FeaturesTextureSkin ColorMultiple Features

Knowledge-based Feature Invariant Template Matching Appearance-based

Template Matching

Main Idea: Find correlation values with a standard face pattern for face contour, eyes, nose, and mouth

Strengths: Simple to implement

Weaknesses: Cannot deal with variation in scale, pose, and shape

Alternatives: Multiresolution, multiscale, subtemplates, and deformable templates

Knowledge-based Feature Invariant Template Matching Appearance-based

Appearance-Based Methods

Main Idea: Use statistical analysis and machine learning techniques to learn “template” characteristics

Strengths: Most successful approach

Weaknesses: Relatively complex to implement, high-dimensionality requires many training examples

Knowledge-based Feature Invariant Template Matching Appearance-based

Overview of Techniques

EigenfacesDistribution-Based MethodsNeural Networks (ANN)Support Vector Machines (SVM)Sparse Network of Winnows (SNoW)Naïve Bayes ClassifierHidden Markov Models (HMM)Information Theoretic ApproachesInductive Learning

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Eigenfaces

Pedro?

Main Idea: Calculate distance between an instance and exemplary data in a reduced dimensional space

Build a face map

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Distribution-Based Methods

Fit a distribution model to examples

Project example into reduced dimensional space

Build classifier to decide face/non-face

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Sung and Poggio

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Sung and Poggio

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Sung and Poggio

MahalanobisPCA for each cluster

Representative sample of non-face images?

Bootstrap approach

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Yang, Ahuja, Kriegman

Method 1:Factor Analysis

Instead of PCADoes not define a mixture model

Estimate mixture model using EMMethod 2:

Fisher’s Linear DiscriminantClass decomposistion using Kohonen’s Self Organizing MapsML decision rule to detect faces

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Neural Networks

Two class pattern recognition Advantage: capture complex class conditional desnsityDrawback: Requires extensively tuning

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Support Vector Machines

Estimating hyperplane is expensiveEvaluation is fast

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Sparse Network of Winnows

Detect images with:Different features and expressionsDifferent posesDifferent lighting conditions

Primitive and multiscale featuresTailored for domains where

Number of features is largeFeatures unknown a priori

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Naïve Bayes Classifier

Estimate joint probability of local appearance

c.f. bottom-up methods

Emphasize local appearanceSome patterns are more unique

Detects some ratated and profile faces

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Hidden Markov Model

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Hidden Markov Model

AlternativesKarhunen Loeve Traformcoefficients as input to HMMUse HMM to learn face to non-face transition

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Information Theoretic Approaches

Markov Random Fields (MRF)Model context-dependent entities

Kullback relative inforamtionMaximize information-based discriminant between the two classes

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. TheoryInductive

Inductive Learning

C4.5 AlgorithmBuilds a decision tree8x8 pixel window

Represented as 30 value vectorEntropy, mean, std dev. of pixel value

Find-SGaussian clusters to aproximatedistribution of face patternsFind-S to learn thresholding

Knowledge-based Feature Invariant Template Matching Appearance-based

EigenfacesDistributionANNSVMSNoWBaysianHMMInfo. Theory Inductive

Agenda

IntroductionApproaches

Knowledge-basedFeature invariantTemplate matchingAppearnce-based

Databases and EvaluationDiscussion

AT&T Cambridge Laboratories Face Database

Face Image Databases

Features of Databases

Designed for single studySmallHighly constrainedOriented to face recognition

Benchmark Test Sets

Benchmark Test Sets

Performance Evaluation

Agenda

IntroductionApproaches

Knowledge-basedFeature invariantTemplate matchingAppearnce-based

Databases and EvaluationDiscussion