IEEE ICARCV 2009

Massimo Tistarelli; E-Mail: [email protected] 1

Graph Application on Face for

Personal Authentication and

Recognition

Authors: D. R. Kisku, A. Rattani, *M. Tistarelli, P. Gupta

*Contact author: [email protected] Vision Laboratory, DAP,University of Sassari, Italy.


Why face recognition problem is a

complex problem in computer vision?

� Due to variations in illumination� Variations in nearby clutter� Variability in translation, rotation, scale and pose� Due to facial expression� Due to occlusion� Due to lighting conditions


Face recognition algorithms:

� Appearance-based method (uses holistic texture features) [1-3]

� Feature-based approach (uses geometrical facial features) [4]

� Model-based approach (uses active appearance models, shape models, fitting morphable models)


Agenda:

� Introduction to SIFT-based face recognition model� Processes in the proposed model � SIFT features extraction� Discussion of the related works� EBGM face model� Proposed face recognition model� Results and discussion� Concluding remarks� Bibliography


Introduction to SIFT-based face

recognition model

� Face recognition models are built with graph topology drawn on Scale Invariant Feature Transform (SIFT) features [6-7], [10].

� SIFT features are extracted from face images, which are invariant to rotation, scaling and partly illumination. Also invariant to 3D projective transform (see Figure 1).

� Face projections on images, represented by a graph, can be matched onto new images by maximizing a similarity function taking into account spatial distortions and the similarities of the local features.


Contd…

50 100 150 200

50

100

150

200

Face image SIFT features are extracted from Face image

Figure 1. Left figure shows the face image and right figure shows the face image with SIFT features.


Processes in the proposed model:

� First the face image is photometrically normalized by using histogram equalization [11].

� The rotation and scale invariant SIFT features are then extracted from the face image.

� 2D feature space is divided into sub-regions with extracted SIFT features.

� Finally the graph-based topology is used for matching the pair of sub-regions of the corresponding two face images.


SIFT feature extraction:

The scale invariant feature transform, called SIFT descriptor, was proposed by Lowe [10] and proved to be invariant to image rotation, scaling, and partly illumination changes. The basic idea of the SIFT descriptor is detecting feature points efficiently through a staged filtering approach that identifies stable points in the scale-space. This is achieved by the following steps:

� Select candidates for feature points by searching peaks in the scale-space from a difference of Gaussian (DoG) function.

� Localization of feature points by using the measurement of their stability.


Contd…

� Assign orientations based on local image properties.� Calculate the feature descriptors which represent local

shape distortions and illumination changes.


Discussion of related works: The SIFT-

based face recognition models

� Gallery image based match constraint (GIBMC) [11]

� Reduced point based match constraint (RPBMC) [11]


Gallery image based match constraint

(GIBMC) [11]:� An assumption has been made the fiducial points would be

available around identical positions in the face image.� SIFT feature points are extracted from both the probe face

and gallery face images. � Euclidean distance metric is used to eliminate the false pair

matches and obtained a set of correspondence matched pairs.

� Identical number of interest points are not found on both faces. Due to that many feature points might be discarded either from the second face or from the first face, or many repetitions might be available for a single point either on the second face or on the first face (see Figure 2 and Figure 3).


Gallery image based match constraint

Contd…….� After computing distances between a pair of points, only the

minimum pair distance would be taken.

X1

X2

X3

Xi X

Y1

Y9

Yi

First Face Image Second Face Image

7.6

7.2

9.7

6.9

10.9

50 100 150 200 250 300 350 400

50

100

150

200

Figure 2. The corresponding points of the First Face Image mapped into the second Face Image.

Figure 3. Feature points and their matches are shown for a pair of faces, computed from Euclidean distance metric


Reduced point based match constraint

(RPBMC) [11]:

� Limitations due to multiple assignments estimated in gallery image based match constraint is removed and the technique is furthermore extended by reduced point based match constraint.

� The false matches due to multiple assignments are eliminated by pairing the points with the minimum distance.

� The false matches due to one way assignments are eliminated by removing the correspondence links that do not have any corresponding assignment from the other face (see Figure 4 and Figure 5).


Reduced point based match constraint…..

Contd…

X2

X3

XN

Y9

First Face Image Second Face Image

9.7

7.2

7.6

50 100 150 200 250 300 350 400

50

100

150

200

50 100 150 200 250 300 350 400

50

100

150

200

Figure 4. Elimination of false matches.

Figure 5. Example of reduced point based match constraint.


Elastic Bunch Graph Matching (EBGM)

face recognition model:

� In EBGM face model [4], a single labeled graph is matched onto an image.

� A labeled graph has a set of jets arranged in a particular spatial order. A corresponding set of jets can be selected from the Gabor-wavelet transform of the image.

� The image jets initially have the same relative spatial arrangement as the graph jets, and each image jet corresponds to one graph jet.

� The similarity of the graph with the image is simply the average jet similarity between image and graph jets.


Proposed face recognition model: Regular

grid based match constraint (RGBMC)

� The graph matching technique has been developed with the concept of matching of the corresponding sub-graphs for a pair of face images.

� First the face image is divided into sub-images, using a regular grid with overlapping regions.

� The matching between a pair of face images is performed by comparing sub-images and computing distances between all pairs of corresponding sub-image graphs in a pair of face images.

� Finally, averaging the dissimilarity scores for a pair of sub-images.


Results and discussion

� The proposed graph matching strategy, namely, RGBMC and the other two techniques, namely, GIBMC and RPBMC have been evaluated and tested on the IITK face database and on the BANCA database [11].

� EBGM face recognition model [4] has tested on IITK face database only.

� From the receiver operating characteristics (ROC) curve, it is clearly seen that the regular grid based match constraint outperform other two SIFT-based methods along with the EBGM face model while test is performed on IITK face database (for illustration, see Figure 6 and the Table 1).


Contd…

� When the test is performed on BANCA database with the three SIFT-based graph matching techniques including the proposed RGBMC, the RGBMC outperformed other two methods with G1 group face images.

� But, with G2 group face images the proposed RGBMC shows increased in errors than the RPBMC method.

� The overall performance of the proposed model is much more satisfactory while is compared with other methods on the two datasets.


Contd…

0 10 20 30 40 50 60 70 80 9078

80

82

84

86

88

90

92

94

96

98

100

False Positive Error

Fal

se N

egat

ive

Err

or

ROC Curve

GIBMC

RPBMCRGBMC

EBGM

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100


Fal

se N

egat

ive

Err

or

ROC Curve for G1 Group

RPBMCRGBMCGIBMC

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100


Fal

se N

egat

ive

Err

or

ROC Curve for G2 Group

RPBMCRGBMCGIBMC

Figure 6. ROC curves for different techniques are tested on large size IITK face database are shown.

Figure 7. On the right, ROC curves for different SIFT-based techniques are tested on small size BANCA face database are shown.


Contd…

5.984.895.987.064.657.31Proposed RGBMC

9.978.259.9711.687.8712.07EBGM

8.486.838.4810.136.4610.5RPBMC

12.9711.3112.9714.6310.9415GIBMC

AvgWER(%)

WER(R=10)

WER(R=1.0)

WER(R=0.1)

FAR(%)

FRR(%)

Table 1: FRR, FAR and WER are shown for different techniques computed from IITK face database.


Contd…

4.122.02

6.241.61

10.026.09

WER (R = 10) on G1WER (R = 10) on G2

4.62.52

6.661.92

10.136.46

WER (R = 1) on G1WER (R = 1) on G2

4.073.01

7.092.24

10.246.83

WER (R = 0.1) on G1WER (R = 0.1) on G2

RGBMC (%)RPBMC (%)GIBMC (%)

Table 2: Weighted Error Rates are shown for three SIFT-based techniques computed from BANCA face database.


Concluding remarks:

� The work shows a remarkable increase in the performance of the system with respect to the previous two works based on the SIFT features.

� The obtained results show the capability of the system in respect of illumination changes and occlusions occurring in the database or the query face.

� The proposed method is found to be superior while it is compared with the EBGM technique.

� Due to invariant nature of SIFT features it is realized that even comparison made with the some well known SIFT-based techniques along with the EBGM model on the different standard databases, the proposed model drawn on SIFT features is shown to be robust and efficient technique in terms of facial expressions, illumination changes, lighting conditions, occlusions, pose changes, etc.


References:

[1] M. Turk, and A. Pentland, “Eigenfaces for Recognition,” J. Cognitive Neuroscience, vol. 3, pp. 71-86, 1991.

[2] W. Zhao, R. Chellappa, and P. Phillips, “Subspace Linear Discriminant Analysis for Face Recognition,” Technical Report CAR-TR-914, Center for Automation Research, Univ. of Maryland, College Park, 1999.

[3] T.W. Lee, Independent Component Analysis: Theory and Applications. Boston: Kluwer Academic Publishers, 1998.

[4] L. Wiskott, J.-M. Fellous, N. Kruger, and C. von der Malsburg, “Face recognition by elastic bunch graph matching”, IEEE Trans. Pattern Anal. Mach. Intell. 19 (7), pp. 775–779, 1997.

[5] A.M. Martinez, “Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class,” IEEE Trans. Pattern Analysis andMachineIntelligence, vol. 24, no. 6, pp. 748-763, June 2002.


Contd…

[6] M. Bicego, A. Lagorio, E. Grosso, and M. Tistarelli, “On the use of SIFT features for face authentication”, Proc. of IEEE Int Workshop on Biometrics, in association with CVPR, pp. 35, 2006.

[7] D. G. Lowe , “Object recognition from local scale invariant features,” International Conference on Computer Vision, Corfu, Greece, pp. 1150–1157, September 1999.

[8] K. Mikolajczyk, C. Schmid: An affine invariant interest point detector. Proc. of European Conference on Computer Vision, 2002, pp. 128-142.

[9] T. Tuytelaars, L.V. Gool: Wide baseline stereo matching based on local affinely invariant regions. Proc. of British Machine Vision Conference, pp. 412-422, 2000.

[10] D. G. Lowe. Distinctive image features from scale invariant keypoints. Int. Journal of Computer Vision, 60(2), 2004.

[11] D. R. Kisku, A. Rattani, E. Grosso, and M. Tistarelli, “Face identification by SIFT-based complete graph topology”, IEEE workshop on Automatic Identification Advanced Technologies, pp. 63-68, 2007.


Thank You !


Questions ???

IEEE ICARCV 2009

Technology

Transcript of IEEE ICARCV 2009