47

Chapter 3

SPATIAL DOMAIN FACE RECOGNITION BASED ON MOLE

AND EDGE DETECTION

3.1 Introduction

In this chapter, the spatial domain techniques for face recognition

such as mole detection and edge detection are discussed. In mole

detection based face recognition, the face images are preprocessed for

illumination normalization using homomorphic filtering. Normalized

Cross Correlation (NCC) coefficients [45] are computed using complement

of Laplacian of Gaussian (LOG) template and skin segmentation using

Grab-cut method [39] is used to identify and validate mole by fixing

predefined NCC threshold values.

In face recognition using edge detection, the face images of different

persons with pose variations are considered. The faces images are of

different size are converted into some standard size by using resize

technique. The noise elimination and illumination compensations are

carried out by preprocessing techniques. The Canny edge operator is

applied on face images. The pose variations of a same person face images

are rotated to a required angle to extract edge distance features.

48

3.2 Mole based Face Recognition Model

In this section Face Recognition using Mole Detection (FRMD) is

discussed with block diagram shown in Figure 3.1.

Fig 3.1: Block diagram of the FRMD.

3.2.1 Raw Image

Color or gray scale raw image with at least one mole is considered for

the analysis.

3.2.2 Illumination Compensation

The illumination compensation is used to remove the illumination

variation in the image so that moles and birth marks are clearly visible

using homomorphic filtering. Generally image is represented as a two-

Illumination

Compensation

Mole Candidate

Detection

Validation of Mole

Candidates

Facial Skin

Segmentation

Skin Regions Non-Skin Regions

Raw Image

49

dimensional function of the form I(x, y), whose value at spatial coordinate

(x, y) is a positive scalar quantity and is determined by the source of an

image. The digital image is a product of the amount of source

illumination incident on the scene being viewed and the amount of

illumination reflected by the objects in the scene as given in Equation

(3.1)

( )1.3................................................)y,x(L*)y,x(R)y,x(I =

where R (x, y) = Amount of illumination reflected from an object

L (x, y) = Amount of source illumination incident on an object

I (x, y) = Intensity of an image and is the illumination-reflectance model

Figure 3.2 shows the block diagram of homomorphic filter. The image I(x,

y) in the spatial domain is the product of R(x, y) and L(x, y) is converted

into summations by applying natural log, which intern converted into

Fourier Transform and is passed through low pass filter. The reverse

procedure is adopted to get an illumination compensated image in the

spatial domain. Figure 3.3 shows the original image and illumination

compensated image after passing through homomarphic filter.

Fig 3.2: Block Diagram of Homomorphic Filtering.

I(x, y)

EXP

I' (x, y)

ln

L (u, v)

F

F-1

50

(a) (b)

Fig. 3.3: (a) Original Image and (b) Illumination Compensated Image.

3.2.3 Mole Candidate Detection

The Laplacian operator is a template which implements second-order

differencing (zero-crossing edge detector) as given in Equation (3.2)

( )3.2............................................2)f(x1)f(x2f(x)(x)11f

1)(x1f(x)1f(x)11f

+−++=

+−=

-

First smoothing is done using Gaussian filter and then Laplacian

operation. The Gaussian smoothing filter is convolved with the Laplacian

filter to obtain hybrid filter and then convolve this hybrid filter with an

image to achieve the required result as given in Equation (3.3).

( )33.....................22σ

2y2x

e2σ2

2y2x -1

4σΠ

1y)(x,LOG

+

−+

= -

where LOG = Laplacian of Gaussian filter

51

The template complement of LOG is used which is close resemblance

to the blob-like appearance of moles. NCC is computed for a small subset

of scales distributed across the desired search range. The output image

of each scale (sk), all the local maxima (xi, yi: sk) to pinpoint candidate

positions in 2D is determined and only these points are further

considered. The correlation coefficients for the remaining points are

computed using templates that corresponds to mole sizes 0.5sk to 2sk.

The point is discarded if the maximum response across these scales is

below a fixed threshold, otherwise the points are considered for

subsequent processing. Considering scale and space independently leads

to drawback of causing duplicate point detections, i.e., candidates

located at different scales and/or coordinates are actually responding to

the same feature in the image and hence remove all duplicates except for

the one with largest scale. The number of scales (range and sample

steps) and the NCC threshold is chosen such that all the marked points

could be located. Template detection typically reduces the number of

candidates for further processing to 1-2% of the pixels representing a

face. NCC matching with complement of laplacian of Gaussian filter as

template is used for valid mole detection and the same procedure is

repeated for more than one mole to get the maximum correlation

coefficients for each mole candidate.

52

3.2.4 Facial Skin Segmentation

The mole present on the facial skin is used for the identification

process. Grab-Cut segmentation of an image is used for the separation of

skin and nonskin regions to identify mole candidates on a skin region

and is used for image synthesis. Figure 3.4 gives the test and segmented

images to bifurcate skin and nonskin regions.

(a) Original Image (b) Segmented Image

(c) Original Image (d) Segmented Image

Fig. 3.4: Test Images with Corresponding Segmented Images

53

3.2.5 Validation of Mole/birthmark Candidates

The coordinates of mole is checked with segmented image. If the mole

lies well within the skin region, is considered for further processing and

is rejected if it lies in the nonskin region.

The Figure 3.5 shows the validation process to separate the prominent

mole required for face detection using NCC and segmented image. The

mole candidate is detected by computing NCC coefficient and comparing

with pre-defined NCC threshold value. The mole is validated after NCC

coordinates are checked with the segmented skin region of an image.

(a) (b)

Fig 3.5: (a) NCC of Test Image and (b) Segmented Test Image

3.3 Proposed FRMD Algorithm

Problem Definition:

Face image with minimum of one mole is given as the input, face

recognition is the output.

The objectives are

(i) To detect the mole candidate.

54

(ii) Validation of detected mole candidate using skin segmentation.

(iii) Mole based face recognition.

Assumptions:

(i) Pose variation is less than 10°.

(ii) Face image should consist of at least one prominent mole.

Table 3.1 gives the algorithm of FRMD to detect and validate the mole

present on face for personal identification.

Table 3.1: Algorithm of FRMD

• Input: Face image with minimum of one mole.

• Output: Valid mole present on the face.

(i) Raw color or gray scale image is considered and enhanced by

Morphological processing to improve the contrast.

(ii) Illumination compensation is used to remove illumination

variant of an image by the use of homomorphic Filter.

(iii) Mole candidate detection by Normalized Cross Correlation

matching with the help of complement of the Laplacian of

Gaussian filter mask as template.

(iv) Facial skin segmentation by Grab-Cut method to separate

skin regions from non-skin regions.

(v) Validation of detected mole candidate by comparing its NCC

coefficient with pre-defined threshold value and its coordinates

with the segmentation results.

55

3.4 Results and Performance Analysis

The face images of variable light and pose with at least one mole on a

skin region are considered for performance analysis as shown in Figure

3.6. The NCC matching technique with complement of Laplacian of

Gaussian template gives highest NCC value for a particular mole. The

Figure 3.7 gives the images with the prominent mole shown by

rectangular box on images and their corresponding NCC images.

NCC threshold value accepts or rejects a particular NCC value of a

mole to classify as valid or invalid mole. The NCC Value depends on the

mole size, darkness and uniqueness with respect to its surrounding

region.

Tables 3.2 and 3.3 gives NCC values for first and second mole of 5 test

images for different template sizes viz., 9-15, 16-21, 22-27, and 28-33. It

is observed that as template size increases, the NCC values decreases in

general. The template size 9-15 gives the better NCC values compared to

other template sizes since normal mole size lies in this range.

Table 3.4 gives the different threshold values ranging from 0.3 to 0.85

for 6 test images consists of prominent moles with corresponding NCC

values. No ranges of threshold values are neglected since there is an

equal probability of detection and failure in each range. If a face image

contains more than two or three moles which are prominent enough,

56

then threshold values are adjusted manually so that all prominent moles

are recorded without rejection.

(a) (b)

(c) (d)

(e) (f)

Fig 3.6: Samples of Test Images

57

(a) (b)

(c) (d)

Fig 3.7: Test Images with Corresponding NCC Images.

Figure 3.8 shows LOG template and its histogram. The texture

variation of the mole is centrally dark and decreases gradually towards

the end. The disadvantages of LOG template is a sudden variation from

center to the outer area as shown in the Figure 3.8(a) and the histogram

of LOG template gives random variation in intensity as shown in the

Figure 3.8(b). Figure 3.9 shows complement of Laplacian of Gaussian

58

template and its corresponding histogram. The complement of Laplacian

of Gaussian template has smooth variation from center to the outer area

and the histogram gives gradual variation in intensity, which is an

advantage compared to LOG template.

Table 3.2: The Mole Detection based on Various Ranges of Template Sizes from 9 to 15 and 16 to 21

Table 3.3: The Mole Detection based on Various Ranges of Template Sizes from 22 to 27 and 28 to 33

Test Image No.

Template sizes used for NCC matching

9 to 15 16 to 21

1 Mole1 Mole2 Mole1 Mole2

2 0.4277 0.5256 0.3103 0.3951

3 0.3344 - 0.1341 -

4 0.8032 0.3668 Failed 0.4478

5 0.5434 - 0.3431 -

Test Image No.

Template sizes used for NCC matching

22 to 27 28 to 33

1 Mole1 Mole2 Mole1 Mole2

2 0.2271 0.288 0.1917 0.1927

3 Failed - Failed -

4 Failed Failed Failed Failed

5 Failed - Failed -

59

Table 3.4: The Detection/failure of Mole based on NCC Values

The NCC value of test images using LOG template is shown in the

Figures 3.10(b) and 3.10(e) has less intensity values. The NCC values of

test images with complement of Laplacian of Gaussian template is as

shown in the Figures 3.10(c) and 3.10(f) has improved intensity values.

Table 3.5 gives the comparison of NCC values for existing algorithm

Skin Detail Analysis for Face Recognition (SDAFR) using LOG template

[45] and the proposed algorithm FRMD using complement of LOG

template for 6 images with percentage of increase in NCC values. NCC

values of FRMD are better compared to SDAFR, which indicates that the

identifying the valid mole is better, hence face recognition of the

proposed algorithm is improved compared to the existing algorithm.

Test Image Number

Threshold Values (NCC Coefficients)

NCC values of the

prominent mole

0.3 to 0.49

0.5 to 0.64

0.65 to 0.85

1 Detected Failed Failed 0.4107

2 Detected Failed Failed 0.3990

3 Failed Failed Detected 0.9132

4 Failed Detected Failed 0.6114

5 Failed Failed Detected 0.8379

6 Detected Failed Failed 0.5748

60

(a) (b)

Fig 3.8: (a) LOG Template and (b) Histogram of the LOG Template

(a) (b)

Fig 3.9: (a) Complement of Laplacian of Gaussian Template and (b)

Histogram of the Complement of LOG Template.

61

(a) Girl Image (b) NCC Image using (c) NCC Image using LOG Template Complement of LOG Template

(d) Boy Image (e) NCC Image using (f) NCC Image using LOG Template Complement of LOG Template

Fig 3.10: Original Images and their NCC Images using LOG and

Complement of LOG Template.

Table 3.5: NCC Values of SDAFR and FRMD

Test Image No.

Existing SDAFR [45]

Proposed FRMD

% Increase in NCC Values

1 0.1926 0.4107 113.23

2 0.3634 0.3990 9.7963

3 0.4328 0.9132 110.99

4 0.1714 0.6114 256.70

5 0.3993 0.8379 109.84

6 0.1274 0.5748 351.17

62

3.5 Proposed FREI Model

The Face Recognition using Edge Information (FREI) is presented in

this section. The block diagram of FREI is shown in the Figure 3.11.

3.11: Block Diagram of FREI

Match/ Nonmatch

Classification

Features

Edge Detection

Noise Reduction

Color

Conversion

Illumination Compensation

Image Resize

Test Image Database

age

Image Resize

Illumination Compensation

Color Conversion

Noise Reduction

Edge Detection

Features

63

3.5.1 The Face Image Database

The face images of different persons are captured using still cameras

and mobile phones with different illumination and pose variations. The

face image samples of twenty persons as local face database are shown

in Figure 3.12. Yale-B [126], CMU-PIE [128] and Indian face database

[129] are given in Figures 3.13, 3.14 and 3.15 respectively.

The Indian face database contains a set of face images taken in

February, 2002 in the IIT, Kanpur campus. There are eleven different

images of each of 40 distinct subjects. The size of each image is 640x480

pixels, with 256 grey levels per pixel. The images are organized in two

main directories - males and females. The pose orientations of the face

are looking front, looking left, looking right, looking up, looking up

towards left, and looking up towards right, looking down. The emotions

on faces are neutral, smile, laughter, sad/disgust.

When a person needs access to any security system then his

identification has to be approved depending on the already existing his

face in the database. But the main issue here is that one cannot expect

the person to stand in front of the camera in the same position every

time he needs an access. There are chances of variations in the position

of their face either towards the left or right by some degree. This face

image captured will not be matching with the data base. This leads to

pose variation problem resulting in false identification. Hence the

64

database should have face images with variations in pose. The pose

variations of person’s face images are created by rotating left and right

through one degree of variation in an angle as shown in Figure 3.16.

Fig. 3.12: Samples of Local Face Database

Fig. 3.13: Samples of Yale-B Face Database

65

Fig.3.14: Samples of CMU-PIE Face Database

Fig. 3.15: Samples of Indian Face Database

66

Fig. 3.16: Images being Rotated Left and Right through One Degree of

Variation (Pose Variation)

3.5.2 Preprocessing

The face image is resized, illumination variation compensation, color to

gray conversion, noise removal and edge detection process are carried

out in preprocessing stage.

67

(i) Image Resize:

The images collected for the database is captured by cameras with

different resolutions will result in images with different sizes. For

example in VGA cameras the photos are of kilobytes in size, whereas in

mega pixel cameras they are of several mega bytes in size, therefore

images need to be resized into a single required standard size say 114 *

114. Figure 3.17(a) shows the original image and the Figure 3.17(b)

shows the resized image of required size.

Fig. 3.17: (a) Original Image and (b) Resized Image

(ii) Illumination Compensation:

Whenever an image is captured, the uniform illumination of an image

can’t be assured. Variation in illumination can lead to pixel variations

which in turn has an effect on the final results. The illumination

compensation is required to eliminate the problem of illumination

variations. The images are set with a common illumination value which

68

leads to uniformity and can yield better results. With this all the images

captured with varying lighting conditions will be made uniform by using

histogram matching process. This ensures same light intensity and

increases the accuracy of the image being matched. The image shown in

Figure 3.18(a) is the original image with poor illumination is converted

into an image of good illumination as in Figure 3.18(b).

(a) (b)

Fig. 3.18: (a) Original Image and (b) Illumination Compensated Image

(iii) Color Conversion:

The color image is M*N*3 array of color pixels, that corresponds to the

red, green and blue components at a specific spatial location. Data class

of the component images determines the range of values. If an RGB

image is of class double then the pixel value ranges from 0 to 1. Similarly

the range of values for RGB images of class unit8 are [0, 255] and for

unit16 are [0, 65535]. The number of bits used to represent pixel values

of image components determines the bit depth of an RGB image. Number

69

of possible colors in an RGB image is in the order of 32 )( b , where b is the

number of bits in each image component. For an 8-bit color image, the

number of colors is 16,777,216. The number corresponding to RGB

model is very high, but in gray scale image value varies from 0 to 255,

where 0 represents black and 255 represents white. Three dimensional

RGB images are converted into two dimensional gray scale images for

easy processing. The Figure 3.19 shows conversion from color image to

gray scale image.

(a) (b)

Fig. 3.19: (a) Color Image and (b) Gray Scale Image

(iv) Noise Reduction:

Dirt on cameras or scanner lens, imperfection in the scanner lighting

etc., introduces the noise in the image. A filtering function is used to

remove the noise in the image and the noise reduction filter is applied to

the binary image for eliminating single black pixel on white background.

8-neighbors of chosen pixels are examined if the number of black pixels

are greater than white pixels then it is considered as black otherwise

70

white. The Figure 3.20(a) shows image with pepper and salt noise and

Figure 3.20(b) shows noiseless image.

(a) (b)

Fig. 3.20: Images with and without Noise

(v) Edge Detection Method:

The edge detection using segmentation process considers slowly

varying gray levels, abrupt changes in gray levels, random changes in

gray values and any discontinuities in intensity values. The edge

detection operators [133] such as Prewitt method, Sobel method, Zero

Crossing method, Laplacian of Guassian method, Robert method and the

Canny edge detection method are used to obtain edges in an image. In

the proposed method Canny edge detection technique is used. The

Canny edge detector computes edges by looking for local maxima of the

gradient of f(x, y). The gradient output is based on the derivatives of

Gaussian filter. The canny edge operator uses two thresholds to detect

strong and weak edges and considers weak edges in the output only if

they are connected to strong edges. The local gradient is given by

Equations (3.4) and (3.5)

71

( )4.3..................................................2/1]yGxG[)y,x(G +=

where Gx and Gy

are the first derivatives of the function f(x, y).

( )5.3.......................................................................G

Gtan)y,x(α

Y

X1

=

−

where α(x, y) is edge direction.

The Canny edge detection operator is applied on original gray scale

image shown in Figure 3.21. The edges obtained by canny edge detection

operator is show in Figure 3.22 with different threshold values of Canny

edge operator. It is observed that as the threshold value increases the

number of edges decreases.

Fig. 3.21: Original Gray Scale Image

(a)Threshold Value 1 (b)Threshold Value 1.75 (c)Threshold Value 3

Fig. 3.22: Canny Edge Detected Images with Different Thresholds.

72

3.5.3 Pose Variation Compensation and Features

In many real situations, however, it is difficult to collect multiple

gallery images in different poses and therefore the ability of face

recognition algorithm to tolerate pose variations is desirable. If face

recognition does not have a good pose tolerance, given a frontal face

image, the system appears to require cooperative subjects who look

directly at the camera and face recognition is no longer passive and non-

intrusive. Therefore, pose invariance is a key ability for face recognition

to achieve its advantages of being non-intrusive over other biometric

traits.

The images with different pose variations are converted into a required

standard angled image for verification. The Canny edge detection is

applied on face image to obtain edges of an image. The pose variations

are computed by rotating an image left or right based on the required

pose of an image. The image is rotated by calculating angle using

Equations (3.6), (3.7) and (3.8). The angle calculations are shown in the

Figure 3.23.

)6.3.....(..........................................................................................574E

573Etanα 1

−

−=

−

)7.3.(..........................................................................................574E

572

3E1E

tanβ 1

−

−

+

= −

73

and )8.3.(....................................................................................................βαγ −=

Rotate the image either clockwise or anticlockwise by an angle ‘γ’.

Fig. 3.23: Angle Calculation

Figure 3.24: Distances used for Face Recognition

74

The edge distance features such as distance between eye tips, distance

between midpoint of eyes-to-nose, cross distance from eye-to-nose tip,

distance between nose-to-lip and cross distance between eye tip midpoint

of lip are measured is shown in Figure 3.24. The final feature vector is

obtained by concatenating all edge distances.

3.5.4 Verification

The extracted features of test face image are compared with the face

database features by using distance formula ED. If the extracted features

of the test face image are well within the threshold value, then the test

face is accepted with database as matching, else it is rejected.

3.6 Algorithm of FREI

Table 3.6 gives FREI algorithm to identify face based on edge distance

features of face image.

Problem definition: Given test face image and large face image database.

The objectives:

• To recognize a person using edge information

• To increase the recognition rate

The features of test face image are compared with the features of the

database images for authentication of a person.

75

Table 3.6: Algorithm of FREI

3.7 Results and Performance Analysis

For performance analysis, the face images of one hundred persons are

considered with twenty samples each persons of different pose angles per

person i.e., total number of images available are 2000. The recognition

rate is computed by creating a database with 100 persons and 19

samples per person i.e., totally 1900 face images in the database. For

test face image one sample per person is considered.

The PRR for face images with and without pose compensations for

various face databases are tabulated in the Table 3.7. It is observed that

the PRR with pose variation compensation is better compared to the

• Input: Test face image and face image database.

• Output: Face recognition

1. Preprocessing using resize, illumination compensation,

conversion of color image into gray level image, noise reduction

and edge detection.

2. Feature extraction based on Canny edge detection for face

recognition.

3. Verification of test face image with the face database using ED.

4. If the test image ED is less than the threshold value then accepted

else rejected.

76

techniques without pose variation compensation. The PRR of the

proposed algorithm is compared with existing algorithm presented by

Kailash and Sanjay [123] is tabulated in Table 3.8. The PRR is high in

the case of proposed algorithm since pose variation compensation is

used.

Table 3.7: PRR on Different Face Databases of Proposed Algorithm

Face

Databases

PRR

Without Pose

Compensation

With Pose

Compensation

Local 69.4 94.6

Yale-B 62.7 89.2

CMU-PIE 64.8 91.5

Indian 66.3 93.6

Table 3.8: The PRR Comparison of Proposed Algorithm with Existing

Algorithm on Indian Face Database

3.8 Summary

The Face is a physiological trait used to authenticate a person. The

face recognition using mole detection as well as edge information are

Authors Techniques PRR

Kailash and

sanjay[123]

PCA 90.5

ICA 92.5

Proposed Method Edge Detection with

Pose Compensation 93.2

77

proposed. In mole detection technique the illumination compensation

using homomorphic filtering is performed for clear visibility of the mole.

NCC matching with complement of Laplacian of Gaussian template is

used to detect the mole using intensity value and position with

predefined NCC threshold values. Validation of the mole is determined by

comparing the co-ordinates of the detected moles with the Grab-cut

segmented image and the mole present in the skin region is accepted as

a valid mole.

In edge information technique, the face images are preprocessed and

then Canny edge operator is applied to obtain the edges of face images.

Pose variations are compensated by rotating an image to an angle of

required value. The test image features are compared with database

images features using ED. It is observed that the PRR is better in the

case of proposed algorithm compared to existing algorithm.