Using LBP Histogram For Face

Recognition On Android Platform

Elizabeth Yesudasan

Ashish Mathew

A.Prakruti Prasad

Divya Racha

Introduction & Motivation

• Why Face Recognition?

• Problems in existing techniques.

• Motivation for using LBP

Literature Review

Biometrics

Physiological

Face Recognition

Iris Recognition

DNA Matching

Fingerprint Recognition

Behavioral

Signature Recognition

Key Stroke Recognition

Voice Recognition

Bio = Life, Metrics = To Measure

Two Modes of Face Recognition

IDENTIFICATION

-figure out “Who is X?”

-accomplished by system performing a

“one-to-many” search

VERIFICATION

-answer the question “Is this X?”

-accomplished by the system performing a

“one-to-one” search

First automatic face recognition system was developed by Kanade in

the year 1973

The Five Step Process

Capture

Find Face in an

image

Extract Features

Compare Templates

Declare Matches

Categorization of Face Recognition

Techniques

Face Recognition

Global Features (Holistic

Features)

PCA LDA

Local Features

HMM LBP

Hybrid Features

Hybrid LFA

Categorization of Face Recognition

Techniques

Facial Representation using LBP

• Facial Image is divided into regions or blocks.

• Description of each region computed using LBP.

• Descriptors are combined into a spatially enhanced

histogram or feature vector.

Local Binary Patterns

• The local binary pattern (LBP) operator is a non-

parametric 3x3 kernel which summarizes the local

spatial structure of an image.

• Features

• Faces can be seen as a composition of micro-patterns which

are well described by LBP operator

• Highly Discriminative

• Invariant to monotonic gray scale transformations.

• Computational Simplicity

LBP – The Original Operator

𝑳𝑩𝑷 𝒙𝒄, 𝒚𝒄 = 𝒔(𝒊𝒏 − 𝒊𝒄)𝟐𝒏

𝟕

𝒏=𝟎

𝑖𝑐 – Grey value of center pixel (𝑥𝑐 , 𝑦𝑐)

𝑖𝑛 – Grey value of the 8 corresponding pixels

s(x) = 1 if x≥0

s(x) = 2 if x<0

LBP – The Original Operator Used For

Feature Extraction (Example)

LBP – The Extended Operator

• The Original Operator was extended by Ojala to a circular

neighborhood of different radius size.

• Known as the multi-scale operator - LBPP,R

• The LBPP,R notation refers to P equally spaced pixels on

a circle of radius R

a) P=8, R=1 b) P=8, R=2 c) P=16, R=2

Uniform Patterns

• Ojala in his work reveals that it is possible to use only a subset

of the 2P LBP patterns to describe textured images.

• This subset is called uniform patterns or fundamental

patterns.

• A LBP is called uniform if it contains maximal 2 transitions

from 0 to 1 and vice versa.

• Can be used to reduce dimensions of feature vector.

Uniform Patterns

• Each of these patterns has its own bin in the histogram.

• The rest of the patterns with more than 2 transitions will be

accumulated into a single bin. (In our project, bin 0)

• This type of LBP histogram is denoted 𝑳𝑩𝑷𝒖𝟐(𝑷,𝑹), containing less

than 2P bins.

• 59 bins are required to represent 𝐿𝐵𝑃𝑢2(8,𝑅) histogram.

• This reduction is possible because, uniform patterns are enough to

describe a textured image, as shown in the table below.

Project Design

• Software Model – Waterfall Model

REQUIREMENTS

DESIGN

IMPLEMENTATION

TESTING

MAINTENANCE

• OS – Windows XP or 7

• Eclipse IDE

• Android SDK

• ADT Plugin for Eclipse

Software

• Android Smartphone

Hardware

Activity Diagram - Add Records

Activity Diagram - Request Match

Activity Diagram – Update Records

Deployment View

Implementation

• Development Tools

• Android SDK

• AVD Manager

• Dalvik Debug Monitor Server (ddms):

• Android Emulator

• SQLite3

• Eclipse IDE

• ADT Plugin for Eclipse

• Face Processing Tool

• Graphical Layout Editor

• User Interface was built with the custom XML editor and a variety of

widgets provided with the ADT Plugin for Eclipse

Training Stage

Test Stage

Face Detection

• Integral Part of the Face Recognition Process

• Challenges –

• Pose variance, Feature occlusion, Facial expression, Imaging

conditions

• Some common approaches -

• Haar-Cascades

• Skin Segmentation

• Template Matching

• LBP Features can also be used for face Detection.

Face Detection

• Approach used for this Project –

• Android.media.FaceDetector class from the Android SDK

• Face is considered detected if confidence ratio > 0.3

• Details obtained with each face detected

• Distance between the eyes

• Midpoint between the eyes

• Face’s pose (rotation around X, Y, Z axis)

Image Preprocessing

Crop and Scale Image

• The face region is extracted by the following assumptions.

• Width = 2 x eyeDistance

• Height = 2.5 x eyeDistance

• Each extracted face is scaled to 100x100 pixels.

Histogram Equalization

Face Mask

• Application of face mask is to avoid non-face parts in

image.

• Example of a face mask -

• This step can be avoided, by using weighted regions.

Feature Vector Extraction

• Feature Vector – Good face descriptor

• In this project, feature vector extraction is done in 3 levels.

• 1st Level – The entire face region is seen as a block

• 2nd Level – Face region is divided into 2x2 blocks

• 3rd Level – Face region is divided into 4x4 blocks

Feature Vector Extraction

LBP8,1

LBP8,1u2

Histogram =

H(8,1)

1-Level Image

2-Level &

3-Level Image

LBP8,2

LBP8,2u2

Histogram =

H(8,2)

1-Level Image

2-Level &

3-Level Image

P=8, R=1 P=8, R=2

Ultimate Feature Vector

Size = 2872 bins

Face Identification

• In this project, dissimilarity metrics is used to match faces.

• Common Dissimilarity Metrics –

• I – Input Image, M – Match Image(from Database)

• Histogram Intersection – 𝑫 𝑰,𝑴 = 𝒎𝒊𝒏 𝐈𝐢,𝑴𝒊 𝒏=𝟏𝒊=𝟎

• Log Likelihood Classification – 𝑳 𝑰,𝑴 = 𝑰𝒊 𝒍𝒐𝒈𝑴𝒊 𝒏=𝟏𝒊=𝟎

• Chi Square Classification – 𝝌𝟐 𝑰,𝑴 = 𝑰𝒊−𝑴𝒊 𝟐

𝑰𝒊+𝑴𝒊

𝒏=𝟏

𝒊=𝟎

Weighted Chi square

• Each block is assigned a weight according to importance.

• Example - Block weights for 7x7 region

Weighted Chi-square Classifier considers these weights wj.

• 𝝌𝟐 𝑰,𝑴 = 𝒘𝒋𝑰𝒊−𝑴𝒊 𝟐

𝑰𝒊+𝑴𝒊

𝒊,𝒋

Cropped Yale Face Database

Testing

Input Expected Output Actual Output

Images with faces in it are given

to the FaceDetector Class

Faces to be detected in the image. Faces detected in the image.

Detected Face Image with face

feature co-ordinates

Face image to be cropped and

scaled to 100x100 pixels.

Face image cropped and scaled to

100x100 pixels

100x100 pixel cropped face

image is given to the

toGrayscale method

Image is to be converted to

grayscale

Image converted to grayscale

Gray scale Histogram is input to

the Histogram Equalization

method

Grayscale histogram should be

globally equalized.

Grayscale histogram has been

globally equalized.

Equalized face image provided

to the LBPOperator function.

2872 bin enhanced feature vector

histogram is to be generated.

2872 bin enhanced feature vector

histogram has been generated

Conclusion and Future Work

• LBP Histogram Feature is suitable for face recognition.

• Because of its computational simplicity, can work on most

smart phones.

• Accuracy Rate of 96.67% is achieved.

• Reasons for misclassification –

• Non-linear illumination transforms.

• Nose and Image alignment not taken under consideration.

• There may be better algorithms we are not sure of.

Future Work

• Non linear illumination transforms and nose alignment

should be considered.

• Promising method to detect faces should be found.

• Should find a method to reduce the dimensions of the

ultimate feature vector.

References • Kuikui Lu and Lanfang Dong, “Using LBP histogram for face recognition on Android platform”

Computer Research and Development (ICCRD) 3rd International Conference, 2011.

•

• G. Heusch, Y. Rodriguez and S. Marcel, “Local Binary Patterns as an Image Preprocessing for Face

Authentication,” IEEE International Conference on Automatic Face and Gesture Recognition, 2006.

•

• T. Ahomen, A. Hadid, and M. Pietikainen, “Face description with local binary patterns: Application to

face recognition,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 28, 2006.

•

• Yann Rodriguez and S´ebastien Marcel, “Face Authentication Using Adapted Local Binary Pattern

Histograms”, European Conference on Computer Vision (ECCV), 2006.

•

• Ojala, T., Pietik¨ainen, M., M¨aenp¨a¨a, T.: Multiresolution gray-scale and rotation invariant texture

classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine

intelligence 24 (2002) 971–987

•

• W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face recognition: A literature survey,” ACM

Computing Surveys, vol. 35, no. 4, pp. 399–458, Dec 2003

•

• P. Viola and M.J. Jones, “Robust Real-Time Face Detection”, International Journal of Computer Vision

57, 137-154, 2004

• http://www.cse.oulu.fi/MVG/Research/LBP

THANK YOU