CHAPTER 4 STUDY OF MULTIMODAL BIOMETRIC TECHNIQUES...

05

CHAPTER – 4

STUDY OF MULTIMODAL BIOMETRIC TECHNIQUES AND DESIGN AND

DEVELOPMENT OF MULTIMODAL BIOMETRIC AUTHENTICATION

MODEL WITH FINGERPRINT AND FACE RECOGNITION

4.1 Multimodal biometric techniques

Human has capability to identify the person based on physiological or behavioral

characteristics. But some time, more multiple modality or multiple evidences are required

to identify the person. In this section, an introduction to multibiometric system has been

given to have clear idea of such type of systems.

4.1.1 Introduction to multi biometrics

Biometric systems can be designed to recognize a person based on information acquired

from multiple biometric sources. Such systems, known as multibiometric systems, can be

expected to be more accurate due to the presence of multiple pieces of evidence.

Multibiometric systems offer several advantages over traditional unibiometric systems

listed below [1]:

1. Multibiometric systems can offer substantial improvement in the matching

accuracy of a biometric system depending upon the information being combined

and the fusion methodology adopted.

2. Multibiometric systems address the issue of non-universality or insufficient

population coverage.

3. It becomes increasingly difficult for an impostor to spoof multiple biometric traits

of a legitimately enrolled individual.

4. Multibiometric systems effectively address the problem of noisy data.

5. Multibiometric systems help in the continuous monitoring or tracking of an

individual in situations when a single trait is not sufficient.

6. A multibiometric system may be viewed as a fault tolerant system which

continues to operate even when certain biometric sources become unreliable due

to sensor or software malfunction, or deliberate user manipulation.

4.1.2 Types of multibiometric systems

Multibiometric systems can be categorized by considering the way you are taking

multiple sources of evidence. There can be different options to have sources of evidence.

Consideration can be made for multiple instances, multiple algorithms, multiple sensors

or sometime multiple traits. Based on this, we can categorize multibiometric systems in

following categories:

1. Multi-sensor systems

2. Multi-algorithm systems

3. Multi-instance systems

4. Multi-sample systems

5. Multimodal systems

6. Hybrid systems

Let have a brief look at these systems.

1. Multi-sensor systems

With this kind of systems, multiple sensors are used to capture different images to have

different kind of evidences from single trait. For example, it is possible to use CCD

camera and range sensor to get multiple kind of information about the face. Though, the

cost will increase, but rich information set will be available. Marcialis and Roli, 2004a

discuss a scheme to fuse the fingerprint information of a user obtained using an optical

and a capacitive fingerprint sensor [2]. The authors, in their work, indicate that the two

sensors provide complementary information thereby resulting in better matching

accuracy.

2. Multi-algorithm systems

With this kind of systems, same information captured from evidence can be processed

with multiple algorithms. Fingerprint image can be processed with minutiae based and

correlation based method simultaneously.

Rose et al. made experiment with fingerprint recognition by using texture based and

minutiae based algorithm for diverse feature extraction [3].

As there is no need to have extra sensor or equipment, so cost effectiveness is there. As

well as user convenience is more.

3. Multi-instance systems

These kinds of systems use multiple instances of same biometric trait. They are also

called multi-unit systems. For example, for fingerprint recognition, we can take two

fingers from both the hands, i.e. one from each. Neither they are using multiple sensors

not they are using multiple algorithms. So, complexity can be reduced as well as cost

effectiveness will be more. Another case can be considered for iris, where we can take

both the iris.

4. Multi-sample systems

Here, one sensor can be used to get multiple samples of the same trait of the same finger

or iris, so variations can be captured as well as completeness of the features can be

captured. For example, Front face image along with left and right face images are

captured to create a complete 3-D ace image for recognition. Hill et al. prepared similar

kind of system [4].

5. Multi-modal systems

With these systems, multiple evidences from different biometric traits to recognition the

person. For example, earlier multimodal biometric systems used face and voice trait to

identify person. The experiment was carried out by Brunelli and Falvigna [5]. The

identification accuracy can be significantly improved by utilizing an increasing number

of traits although the curse-of-dimensionality phenomenon would impose a bound on this

number. The curse-of-dimensionality limits the number of attributes (or features) used in

a pattern classification system when only a small number of training samples is available

[6]. The restrictions will be imposed on practical implementation by cost of deployment,

time, throughput time etc.

6. Hybrid systems

Hybrid refers to the systems that integrate a subset of the five type of systems discussed

above. Brunelli and Falavigna described an arrangement in which two speaker

recognition algorithms are combined with three face recognition algorithms at the match

score and rank levels via a HyperBF network [5]. This system can be considered as multi-

algorithmic as well as multimodal. Hybrid systems try to extract as much information as

possible.

Figure 4.1: Various multibiometric system examples

4.1.2.1 Processing sequence

The processing sequence adopted by multibiometric system refers to the order in which

the acquired information is used in order to take a decision. The concentration is made

not in the order of acquisition, but the order in which they are processed. The samples can

be acquired sequentially, but can be processed simultaneously. In the serial or cascade

mode, the processing of information takes place sequentially. The cascading scheme can

improve user convenience as well as allow fast and efficient searches in large scale

identification tasks. A multibiometric system designed to operate in the parallel mode

generally has a higher accuracy because it utilizes more evidence about the user for

recognition. Most multibiometric systems have a parallel architecture because the

primary goal of system designers has been to reduce the error rates of biometric systems

and not necessarily the throughput and/or processing time.

In addition to these two modes of operation, it is possible to have a hierarchical (tree-like)

architecture to combine the advantages of both cascade and parallel architectures [7]. In

this method, a subset of the acquired modalities may be combined in parallel, while the

remaining modalities may be combined in a serial fashion. This architecture can be

dynamically determined based on the quality of the individual biometric samples as well

as the possibility of encountering missing biometric data.

4.1.3 Introduction to multimodal biometrics

Multimodal biometrics refers to the use of a combination of two or more biometric

modalities in a verification / identification system. Identification based on multiple

biometrics represents an emerging trend. The most compelling reason to combine

different modalities is to improve the recognition rate. This can be done when biometric

features of different biometrics are statistically independent. There are other reasons to

combine two or more biometrics. One is that different biometric modalities might be

more appropriate for the different applications. Another reason is simply customer

preference [9].

4.1.4 Example multimodal biometric systems

Work in the area of multimodal biometric systems started in mid 1990s. Few example

multimodal biometric systems are shown here.

Modalities fused Authors Level of

fusion Fusion methodology

Face and voice Brunelli and

Falavigana, 1995

Match

score and

rank

Geometric weighted average;

HyperBF

Kittler et al.,1998 Match

score

Sum, product, min, max and

median rules

Ben-yacoub et al.,

1999

Match

score

SVM, Multilayer perceptron, C4.5

decision tree, Fisher‘s linear

discrimination, Bayesian classifier

Bigaun et al., 1997 Match

score

Statistical model based on

Bayesian theory

Face, voice and

lip movement

Frischholz and

Dieckmann, 2000

Match

score,

decision

Weighted sum rule, majority

voting

Face and

fingerprint

Hong and Jain,

1998

Match

score

Product rule

Snelick et al., 2005 Match

score

Sum rule, weighted sum rule

Face, fingerprint

and hand

geometry

Ross and Jain,

2003

Match

score

Sum rule, decision trees,

discriminant function

Face, fingerprint

and voice

Jain et al., 199b Match

score

Likelihood ratio

Face and iris Wand et al., 2003 Match

score

Sum rule, weighted sum rule,

fisher‘s linear discrimination ,

neural network

Face and gait Shakhnarovich et

al., 2001

Match

score

Sum rule

Kale et al., 2004 Match

score

Sum and product rules

Face and ear Chang et al., 2003 Sensor Concatenation of raw images

Face and

palmprint

Feng et al., 2004 Feature Feature concatenation

Fingerprint, hand

geometry and

voice

Toh et al., 2004 Match

score

Weighted sum rule

Fingerprint and

hand geometry

Toh et al., 2003 Match

score

Reduced multivariate polynomial

model

Fingerprint and

voice

Toh and Yau,2005 Match

score

Functional link network

Fingerprint and

signature

Fierrez-aguilar et.

al.,2005c

Match

score

SVM in which quality measures

are incorporated

Voice and

signature

Krawczyak and

Jain,2005

Match

score

Weighted sum rule

Table 4.1: Various multimodal biometric systems

4.1.5 Advantages of multimodal biometric systems

Multimodal biometric systems are expected to be more reliable due to the presence of

multiple pieces of evidence. These systems are also able to meet the stringent

performance requirements imposed by various applications [10].

Multimodal systems address the problem of non-universality: it is possible for a subset of

users to not possess a particular biometric. For example, the feature extraction module of

a fingerprint authentication system may be unable to extract features from fingerprints

associated with specific individuals, due to the poor quality of the ridges. In such

instances, it is useful to acquire multiple biometric traits for verifying the identity.

Multimodal systems also provide anti-spoofing measures by making it difficult for an

intruder to spoof multiple biometric traits simultaneously. By asking the user to present a

random subset of biometric traits, the system ensures that a live user is indeed present at

the point of acquisition. However, an integration scheme is required to fuse the

information presented by the individual modalities.

4.2 Introduction to different fusion techniques in multimodal biometric systems

The fusion can be achieved in two different ways. The first is information fusion prior to

matching and the second method is fusion after matching [8].

1. Fusion prior to matching

Fusion prior to matching can be achieved in two different ways:

1. Sensor level fusion

2. Feature level fusion

Sensor level fusion is applicable only if the multiple sources represent samples of the

single biometric trait obtained either using a single sensor or different compatible

sensors.

Feature level fusion is achieved by combining different feature sets extracted from

multiple biometric sources. Feature sets could be either homogeneous or heterogeneous.

The consolidation of feature set creates problems as the feature sets originate from

different algorithm and modalities.

2. Fusion after matching

Fusion after matching can be achieved in three different ways:

1. Matching score level fusion

2. Rank level fusion

3. Decision level fusion

Matching score level fusion provides richest set of information.

Rank level fusion consolidates the ranks output by the individual subsystems in order to

derive a consensus rank of each identity. Rank level fusion provides less information with

compare to match score level fusion.

Decision level fusion is carried out at decision level when the decisions output by the

individual matcher are available. COTS (Commercial off The Shelf) matchers provide

only the final decision and those decisions are evaluated with the help of some rules like

―AND‖ or ―OR‖, majority voting, Bayesian decision fusion etc. Here there will be a

problem of least information about the features or scores of different modalities.

Here the researcher has selected two biometric traits, so instead of fusion before

matching, which mostly deals with single modality, concentration has been given to

fusion after matching. So, here is an introduction to fusion methods after matching.

1. Match score level fusion

This fusion technique is also known as measurement level or confidence level fusion. It is

comparatively easy to consolidate the scores generated by different biometric matchers.

This method is the most commonly used method for fusion.

Here we have to identify the pattern only in two classes: genuine (Truly what something

is said to be; authentic) or impostor (A person who pretends to be someone else in order

to deceive others, esp. for fraudulent gain). In general there are three different methods to

achieve match score level fusion. They are:

1. Density based score fusion

2. Transformation based score fusion

3. Classifier based score fusion

As the match score level fusion use scores from different modalities based on different

scaling methods, the scores cannot be combined or used directly. It is required to perform

score normalization, thereby converting the scores into common domain or scale.

Score normalization can be carried out with different methods. Here are some methods of

normalization worked well with different modalities. Slobodan Ribaric and Ivan Fratric

carried out experiments for bimodal biometrics with palmprint and facial features [11].

They adopted match score level fusion for fusion of information. They discovered new

normalization technique – piecewise linear normalization. They calculated EER (Equal

Error Rate) and minimum TER (Total Error Rate) with different normalization technique.

They achieved EER of 2.79 % and min. TER of 5.15 % with this normalization. The

chart of comparison is given below:

Normalization

Technique

Piecewise

linear

median-

MAD Tanh Minmax

EER 2.79 2.79 3.05 3.12

Min. TER 5.15 5.42 5.74 6.39

Table 4.2: EER and Min. TER under different normalization techniques

Mingxing He et al. Proposed a new method of normalization for scores in score level

fusion, Reduction of High-scores Effect normalization (RHE) [12]. They experimented

on four different databases with multi modality of fingerprint, face and fingervein. They

revealed that RHE performs better with compare to other techniques of score

normalization in score level fusion.

FAR (%) GAR (%)

Minmax Z-score Tanh RHE

0.01 97.9 98.2 97.7 99.4

0.001 96.9 97.0 95.8 98.2

Table 4.3: Performance of sum rule-based fusion on NIST-multimodal database [12]

After performing normalization, the next step is to perform fusion of the scores. Here are

some examples of different fusion models for score level fusion. Gian Luca Marcialis and

Fabio Roli suggested the following model for score level fusion of fingerprint and face

traits [13].

Figure 4.2: Score level fusion [13]

They carried out experiments on multimodal data set made up of 100 subjects with two

independent face and fingerprint data sets. With the above given scheme, they achieve

improvement in the error rate. Their results showed that fusion has improved the

reliability of the system by reducing the gap between expected and real performance.

Feifei Cui and Gongping Yang performed biometric fusion with fingerprint and finger

vein recognition [14]. They did this with score level fusion. They collected 2880

fingerprint and finger vein images from 80 fingers. With score level fusion they achieved

the following performance:

Biometrics method Recognition rate

Fingerprint 95.3 %

Finger vein 93.72 %

Score level fusion 98.74%

Table 4.4: Recognition rate for fingerprint and finger vein fusion

These results shows that score level fusion works well with compare to unimodal

biometric traits.

Fawaz Alsaade experimented score level fusion with face and voice biometrics. He

investigated the results under three data conditions and with min-max normalization. He

used Adaptive Neuro – Fuzzy Inference System (ANFIS) for decision making [15]. He

was able to achieve 0 % EER with ANFIS approach with clean data of both face and

voice biometrics. Table 4.5 shows experimental outcomes carried out by Fawaz Alsaade.

Modality EER%

Voice (TIMIT Database) 2.55

Face (XM2VTS Database) 3.57

Fused: voice and face by BFS (Brute force search) 0.05

Fused: voice and face by SVM 0.68

Fused: voice and face by ANFIS 0

Table 4.5: Multimodal biometric verification based on clean biometric data

Sarat C. Dass et al. proposed a framework to combine the match score from multiple

modalities with the use of likelihood ratio statistic computed using generalized densities

which were estimated from genuine and impostor match scores[16]. They conducted

experiments on two different databases with different number of users. The details of

databases are shown in Table 4.6:

Database Modalities No. of Users

MSU-Multimodal Fingerprint, Face, Hand-geometry 100

NIST-Multimodal Fingerprint (Two Fingers), Face (Two

matchers)

517

Table 4.6: Details of databases used by Sarat Dass et al. [16]

5

They proposed two different approaches to combine evidences based on generalized

densities:

1. Product rule

2. Copula model

With the above given database and these two score fusion methods, they achieved

consistently high performance.

Romaissaa Mazouni and Abdellatif Rahmoun fused face and speech modalities with five

different methods of score level fusion: Particle Swarm Optimization (PSO), Adaptive

Neuro Fuzzy Interface Systems (ANFIS), Genetic Algorithm (GA), Brute Force Search

(BFS), and Support Vector Machine (SVM) [17]. They did their experiments with three

kinds of datasets: Clean data, varied data, and degraded data. They derived the conclusion

that Genetic algorithm (GA) and Particle Swarm Optimization performed best among all

five method even in worst conditions.

2. Decision level fusion

Now a day, if you are using commercial off the shelf tools for biometric verification, then

decision level fusion is the only option for fusion, as they don‘t provide the data about the

scores or feature neither they provide details about the ranking of different users after

comparison. Decision level fusion is also referred as abstract level fusion. They only

provide the result of matching in the form of whether the user is genuine or imposter.

With decision level fusion, there are different rules that can be used to authenticate the

user. Lam and Suen proposed majority voting rule [18].They also proposed behavioral

knowledge space method. Xu et al. proposed weighted voting based on Dempster -

Shafer theory [19]. Daugman proposed AND/ OR rules for deciding the decision [20].

The general and mostly used approach for decision level fusion is majority voting. Here

the input sample is given the identity for which the majority of the matchers are agreed.

AND and OR rules are used rarely, because as they combine two different matchers, so

sometimes degradation of performance could be there with this method [20]. The main

benefit of the majority voting method is that neither you require prior knowledge about

the matcher nor the training is required for final decision making [8]. Domingos and

Pazzani suggested that naïve Bayesian decision fusion works very well even if the

matchers are dependent to each other [21].

3. Rank level fusion

The rank level fusion is generally adopted for the identification of a person rather than

verification. In verification, as we have to compare the template only with one template

in the database, here we have to generate rank of identities in sorted order with all

modalities. Then after with the help of one method of fusion, we have to fuse the ranking

for each person available for different modalities. Then the identity with lowest score is

identified as the correct person. This method provide more accuracy with compare to just

a identifying best match with one modality. But the only thing is that, it provides less

information for fusion purpose. With compare to match score level fusion, here you can

easily compare the ranking from different modalities. So the decision making is easy.

Md. Maruf Monwar and Marina L. Gavrilova carried out rank level fusion with face,

signature and ear biometric traits [22]. They performed experiments with PCA and

fisher‘s LDA. The rank of individual matchers was combined with highest rank, Borda

count, and logistic regression approaches. With this approach, the performance was

improved performance even with low quality of data. Table 6 shows performance of the

experiment.

Systems Biometric identifiers Fusion level and approach EER

Md. Maruf Monwar et al. Face, Ear, Signature Rank ; logical regression 1.12%

Garcia – Salicetti et al. Signature, Voice Match score; 1.88%

Nandkumar et al. Fingerprint Match score 3.39%

Table 4.7: Comparison of different multibiometric systems [22]

Ajay kumar and Sumit Shekhar suggested combination of multiple palmprint

representations to achieve improvement in the performance with compare to individual

performance [23]. They performed various rank level combinations like, Borda Count,

Logistic Regression, Highest rank method and Bucklin majority voting approach. With

this approach, they performed experiments with NIST BSSR database. The nonlinear

fusion approach gave best results for first – rank recognition rates. Average rank one

recognition rate was of 99%.

With rank level fusion, three most common approaches are Borda count method, Logistic

Regression method and Highest rank method. Out these three methods, Borda count and

highest rank method do not use statistical information of the classifier performance. But

with Logistic regression method, statistical information is required and weights are

assigned to classifiers. These weights depend on the data.

Abaza and Ross performed experiments with two modalities: Fingerprint and Face [24].

They evaluated results with two databases: WVU and NIST. With Modified Highest

Rank method, they achieved rank - 1accuracy of ~ 99 % on WVU dataset. They proposed

Q-based rank algorithms for rank level fusion. They were able to improve the

performance by ~4 %.

4.2.1 Score normalization in biometric systems

Score normalization refers to changing the location and scale parameters of the match

score distributions at the outputs of the individual matchers, so that the match scores of

different matchers are transformed into a common domain. When the parameters used for

normalization are determined using a fixed training set, it is referred as fixed score

normalization. Here, the set of match scores available for training the fusion module of a

multibiometric system is examined and a suitable statistical model is chosen to fit to the

data. Based on the model, the score normalization parameters are determined. In adaptive

score normalization, the normalization parameters are estimated based on the match score

of the current test sample. This approach has the ability to adapt to variations in the input

data such as the changes in the duration of the speech signals in speaker recognition

systems [1].

The comparison of various normalization techniques can be done on the basis of

robustness and efficiency. Here is the comparison of features of different score

normalization techniques:

Normalization technique Robustness Efficiency

Min-max No High

Decimal scaling No High

Z-score No High

Median and MAD Yes Moderate

Double sigmoid Yes High

Tanh estimators Yes High

Table 4.8: Characteristics of various normalization techniques

Here, in the case of face recognition, as the Euclidean distances have large values which

requires to be normalized, while in case of fingerprint recognition, the scores are

generated in the range of 0.0 to 1.0, which don‘t required to be normalized. The

researcher has defined his own normalization technique to normalize face scores. The

steps of normalization technique are shown below.

1. Calculate sum of all vector values (Euclidian distance).

2. Calculate average Euclidian distance

3. Calculate difference between average Euclidian distance and all other distances.

4. Divide each difference with average Euclidian distance and get value in the range

of [-1,1].

4.3 Deciding method of fusion

Based on the discussion in 4.2, it is clear that match score level fusion provides rich

information compared to rank level fusion or decision level fusion. Specially, when it is

required to normalize scored available with different modalities, there are many

mechanisms through which it is easier to normalize the score. At the same time

performance of the match score level fusion is also acceptable. Decision level fusion

provides very less information i.e. only the results of modalities in terms of final

decision. Rank level fusion provided ranks of different matches. It is possible to assign

some weights to these ranks. But considering the richness of information, the researcher

has adopted match score level fusion for multimodal biometric authentication system

[25].

4.4 Building multimodal biometric system using face and fingerprint recognition

The model of proposed basic multimodal biometric system is shown in figure 4.3

.

Figure 4.3: Basic multimodal biometric system model

As shown in basic model, initially system will capture the face and fingerprint sample of

the person. The face sample will be compared with master face DB and calculate

Euclidean distance. The fingerprint sample will be compared with master fingerprint DB

and minutiae features will be matched. Both the scores will be combined to generate

match score. Then system will identify maximum score and will be displayed along with

respective person index.

The elaborated view of the system is shown in figure 4.4. The system captures face and

fingerprint weights. These weights will be multiplied with respective face and fingerprint

matching scores. Then both will be added to get final score. In the next step, scores of all

the persons are compared and maximum match score will be identified. Person index of

maximum match score will be find out and both values i.e. maximum score and person

index will be displayed on GUI.

Master

face DB

Master FP

DB

Calculate Euclidean

distance

Calculate minutiae

score

Display maximum

matching score and person

index

Figure 4.4: Multimodal biometric system model

The detail about the creation of master databases is mentioned in section 3.2. The

important aspect is the fusion method which has been used here. The researcher has used

match score level fusion. With this fusion method, the scores of both modalities for

respective person will be combined. One more important aspect is the use of

normalization technique used for score normalization. The researcher has designed novel

normalization technique to normalize face score. Normalization is not required for

Applying normalization on 30

face scores

Capture weight for

face score

Generate combined score multiplying

weights with match score and

Euclidean distance and summation

Capture weight for

fingerprint score

Identify person index with maximum

score and display with text control

Master face

database

Master fingerprint

database db.mat

Comparison engine to calculate

Euclidean distances from all database

samples and generate file with 30

Euclidean distances

Comparison engine calculates match

score from each database sample

template and generate file with 30

match scores

fingerprint score as it is generated in the range of 0.0 to 1.0. Another important aspect is

to assign weights for both modalities. The sum total of weights of both modalities must

be 1.0. The used has to decide the weights of both modalities. The flow chart

representation is shown in figure 4.5.

4.4.1 Flowchart representation of multimodal biometric system

Figure 4.5: Flow chart representation of Multimodal biometric system

Master face

database

Execute Multimodal IDE

and select Multimodal face

and fingerprint option

Calculate Euclidean

distance from test

sample and compare

with database samples‘

Euclidean distance

Load test face

sample from

test database

End process

Start

Load test

fingerprint

sample from

test database

Master fingerprint

database db.mat

Identify minutiae

features from test

sample and compare

with db.mat templates Apply normalization

process with 30

comparison scores

Enter weight for

fingerprint score

Enter weight for

face score

Calculate combined score by

multiplying weights with

respective face and fingerprint

scores

Display person index with

maximum matching score

4.4.2 GUI representation of multimodal biometric system

The steps for operating with multimodal biometric system are shown here.

Step 1: Run multimodal_ide.m to load GUI of multimodal biometric system.

Execute multimodal_ide.m file to load GUI of multimodal biometric system. The GUI

contains Unimodal and Multimodal menu options.

Step 2: Select „Multimodal‟ menu option and „Multimodal with face and fingerprint‟

submenu option

Initially, it is required to run multimodal_ide.m file to load GUI of the system. After

loading GUI, next step is select ‗Multimodal‘ menu option. On selection of this menu

option, system will show submenu containing two options. Out of these two options,

select ‗Multimodal with face and fingerprint‘ option.

Step 3: System hide unnecessary controls from GUI on selection of submenu option

After selection of ‗Multimodal with face and fingerprint‘ option, system will hide

unnecessary components from GUI. The system will show four buttons, 2 axes controls

and 4 edit controls required for this system.

Figure 4.6: Multimodal IDE with all components

Menu selection

options

Axes control 1 &

Axes control 2

Edit controls

Figure 4.7: Selection of ‗Multimodal‘ menu option

Figure 4.8: Load necessary components of multimodal system

Step 4: Click on „Load face image‟ button to select face test sample

Figure 4.9: Selection of test face sample

The next step is to load test samples. First, to load face test sample, click on ‗Load face

image‘ button. The system will show file dialog box to select face test image. Select face

test sample image. The system will load that image in axes control 1.

Step 5: Click on „Load fingerprint image‟ to load fingerprint test sample

After loading face test sample, it is required to load fingerprint test image. So, click

‗Load fingerprint image‘ button in the next step as shown in figure 4.10.

Step 6: Select fingerprint test image from test database

On clicking ‗Load fingerprint image‘, system will show file open dialog to select

fingerprint test sample. Select fingerprint test sample from test database. The GUI is

shown in figure 4.11.

Figure 4.10: Click ‗Load fingerprint image‘ button to load fingerprint test image

Step 7: Enter the weights of face and fingerprint traits in edit controls

After loading face and fingerprint test samples, next step is to enter face and fingerprint

weights. Face and fingerprint scores should be multiplied with appropriate weights. The

sum of both weights should be 1.0. So, weights can be there in the range of 0.0 to 1.0.

Step 8: Press „Match face and fingerprint‟ button to start process of identification

After entering face and fingerprint weights, next step is to press ‗Match face and

fingerprint‘ button. On clicking this button, test face and fingerprint samples will be

compared with master face and fingerprint database and match scores are generated.

After generating match scores, both the scores will be first normalized by user defined

method of normalization. After normalization, face and fingerprint scores are multiplied

with respective weights and summation of this multiplied scores will be done. After this

summation, the score with maximum will be identified and respective person index will

also be identified. Both the highest score and person identity are loaded in edit control

box.

Figure 4.11: Selection of fingerprint test sample

Step 9: On completion of identification process, click on „Clear all‟ to clear the content

of axis controls and edit controls

After completion of identification process, next step is to click ‗Clear all‘ button. On

pressing the button, the content of axis controls and edit controls will be cleared. Now,

the user can start for another identification process.

Figure 4.12: Displaying test fingerprint image in axis control 2

Figure 4.13: Enter face and fingerprint weights in edit controls

Figure 4.14: Press ‗Match face and fingerprint‘ button to start process of identification

Figure 4.15: Click ‗Clear all‘ button to clear content of all controls

4.5 Experimentation results

The researcher has performed experiments with above mentioned multimodal biometric

system model using face and fingerprint recognition. Total 30 samples have been selected

from each modality i.e. face and fingerprint. For each person, one sample has been

selected. The experiment was carried with different weight combinations. Range of

weights starting from 0.1 and 0.9, respective for face and fingerprint modality. It goes up

to 0.9 and 0.1, respective for face and fingerprint modality. Score normalization was

applied to have normalized face scores. The results are shown in the table 4.9. The

performance representation is shown in Figure 4.16.

Here is the evaluation of performance using 30 test samples.

faceweight fpweight Success Failure Success rate

(GAR)%

Failure rate

(FRR)%

0.1 0.9 28 2 93.33 6.67

0.2 0.8 27 3 90.00 10.00

0.3 0.7 27 3 90.00 10.00

0.4 0.6 28 2 93.33 6.67

0.5 0.5 26 4 86.67 13.33

0.6 0.4 25 5 83.33 16.67

0.7 0.3 25 5 83.33 16.67

0.8 0.2 25 5 83.33 16.67

0.9 0.1 24 6 80.00 20.00

Table 4.9: Results of experiments of multimodal biometric system with face and

fingerprint recognition

Matlab code representation for evaluation of performance of the system is shown here:

-----------------------------------------------------------------------

%% multimodal_evaluation.m

%% Setting database for face recognition TrainDatabasePath='D:\phd_dtm_practical\Multimodal

Biometrics\TrainDatabase_Face';

load('db.mat');

Path='D:\phd_dtm_practical\Multimodal Biometrics\Testsamples2\'; imagecounter=100; successrate=0; failurerate=0; personidentity=0;

for l=1:30 imagecount=imagecounter+l; testface=[num2str(imagecount) '.pgm'];

testface= strcat(Path ,testface); S1 = imread(testface); % imshow(S1); imwrite(S1,'D:\phd_dtm_practical\Multimodal

Biometrics\temp_face.pgm','pgm');

TestImage='D:\phd_dtm_practical\Multimodal

Biometrics\temp_face.pgm';

T = CreateDatabase(TrainDatabasePath); [m, A, Eigenfaces] = EigenfaceCore(T); [OutputName] = Recognition(TestImage, m, A, Eigenfaces);

%% Identifying normalized scores for face recognition facescore = []; facescore1 = []; fid1=fopen('face_score.txt','r'); j=1; facescore=fscanf(fid1,'%f',60);

fclose(fid1);

for i=1:30 j=(i*1)+(i-1); k=j+1; if(facescore(j)<facescore(k)) facescore1 = [facescore1 facescore(j)]; else facescore1 = [facescore1 facescore(k)]; end end

totalscore=0; for j=1:30 totalscore=totalscore+facescore(j); end

%average score avgscore=totalscore/30; disp(avgscore);

%minimum score minscore=min(facescore); disp(minscore);

deviation=avgscore-minscore;

facescore2 = []; for i=1:30 deviation=avgscore-facescore1(i); temp=deviation/avgscore; disp(temp); facescore2 = [facescore2 temp];

end

fid2=fopen('finalfacescore.txt','wt');

for i=1:30 fprintf(fid2,'%f \n',facescore2(i)); end

fclose(fid2);

%% Fingerprint matching from FP database testfinger=[num2str(imagecount) '.jpg']; testfinger= strcat(Path ,testfinger); S2 = imread(testfinger); imwrite(S2,'D:\phd_dtm_practical\Multimodal

Biometrics\temp_fp.jpg','jpg');

filename='D:\phd_dtm_practical\Multimodal Biometrics\temp_fp.jpg'; img = imread(filename); if ndims(img) == 3; img = rgb2gray(img); end % Color Images disp(['Extracting features from ' filename ' ...']); ffnew=ext_finger(img,1);

%% FOR EACH FINGERPRINT TEMPLATE, CALCULATE MATCHING SCORE IN

COMPARISION WITH FIRST ONE S=zeros(30,1); hscore=0.0; Identity=''; count=100;

%Open file to write fp matching score fid2=fopen('fp_score.txt','wt');

for i=1:30 second = count+i; S(i)=match(ffnew,ff{i}); %Write fp match score in file fprintf(fid2,'%f \n',S(i)); if S(i)> hscore hscore=S(i); Identity=second; end fprintf([num2str(S(i)) '\n']); end

%close file fclose(fid2); %% Find final scores and indetify person % Assign weights for face and fingerprint

faceweight= 0.5; fpweight= 0.5;

finalscore = [];

%Calculate final score by addition of face score and fingerprint score

for %each sample

fid3=fopen('totalscore.txt','wt');

for i=1:30 totalscore=(faceweight*facescore2(i))+(fpweight*S(i)); fprintf(fid3,'%d -- %f \n',imagecount,totalscore); finalscore=[finalscore totalscore]; display(totalscore); end

fclose(fid3);

%Identify maximum score and decide person index for i=1:30 if(i==1) maxscore=finalscore(i); index=1; else if(maxscore<finalscore(i)) maxscore=finalscore(i); index=i; end end end

personidentity =100+index;

if(personidentity == imagecount) successrate=successrate+1; else failurerate=failurerate+1; end

fprintf(['Imagecount is ' num2str(imagecount) '\n']); fprintf(['Personidentity ' num2str(personidentity) '\n']);

fprintf(['successrate is ' num2str(successrate) '\n']); fprintf(['failurerate is ' num2str(failurerate) '\n']);

pause(5); end

-----------------------------------------------------------------------

%%multimodal_weight.m

fid1=fopen('finalfacescore.txt','r'); fid2=fopen('finalfpscore.txt','r'); fid3=fopen('multimodalweightresults.txt','a'); facescore = []; fpscore = []; fcount=1;

count=100; tscore=0.0; success=0; failure=0;

faceweight=0.9; fpweight=0.1;

tempface = []; tempfp = [];

facescore=fscanf(fid1,'%f',900); fpscore=fscanf(fid2,'%f',900);

for i=1:30 maxscore=0; person=count+i; totalscore = []; index=0;

for j=1:30 tempface = [tempface facescore(fcount)]; tempfp = [tempfp fpscore(fcount)]; tscore=(facescore(fcount)*faceweight) +

(fpscore(fcount)*fpweight); totalscore = [totalscore tscore]; fcount=fcount+1; if (j==1) maxscore=totalscore(j); index=1; else if(maxscore<totalscore(j)) maxscore=totalscore(j); index=j; end end end index=index+100; if(person==index) success=success+1; else failure=failure+1; end % pause(2); end successrate=(success/30)*100; fprintf(['Successful identification :' num2str(success) '\n']); fprintf(['Failed identification :' num2str(failure) '\n']); fprintf(fid3,'%f \t %f \t %d \t %d \t

%f\n',faceweight,fpweight,success,failure,successrate);

fclose(fid1); fclose(fid2); fclose(fid3); -----------------------------------------------------------------------

Figure 4.16: Performance representation of multimodal biometric system

4.6 Conclusion

The results have been shown in section 4.5. Result shows that with more weightage of

face scores, success rate will be lees. But with more weight assignment with fingerprint

scores, success rate can be achieved more. With fingerprint weights of 0.9 and 0.6,

success rate can be achieved up to 93.33 %, which is quite more compared to unimodal

biometric system performance.

References

[1] Arun Ross, Karthik Nandkumar, Anil Jain, ―Handbook of Multibiometrics‖, Springer,

2006

[2] Marcialis, G. L. and Roll, F. ―Fingerprint Verification by Fusion of Optical and

Capacitive Sensors‖, Pattern Recognition Letters, 25(11):1315-1322, 2004

[3] Ross, A., Jain, A. K., and Reisman, J, ―A Hybrid Fingerprint Matcher‖, Pattern

Recognition, 36(7): 1661-1673, 2003

[4] Hill, H., Schyns, P. G., and Akamatsu, S., ― Information and Viewpoint Dependence

in Face Recognition‖, Cognition, 62(2):201-222, 1997

0

10

20

30

40

50

60

70

80

90

100

Successrate

Successrate

[5] Brunelli, R. and Falavigna, D., ― Person Identification Using Multiple Cues‖, IEEE

Transactions on Pattern Analysis and Machine Intelligence, 17(10):955-966, 1995

[6] Jain, A. K. and Chandrasekaran, B., ―Dimensionality and Sample Size Considerations

in Pattern Recognition Practice‖, In Krishnaiah, PR. and Kanal, L. N., editors. Handbook

of Statistics, volume 2, pages 835-855, 1982

[7] Maltoni, D., Maio, D., Jain, A. K., and Prabhakar, S., ―Handbook of Fingerprint

Recognition‖, Springer-Verlag, 2003

[8] Anil Jain, Karthik Nandakumar, Arun Ross, ―Score Normalization in Multimodal

Biometric Systems‖, Pattern Recognition, 2005

[9] Ashish Mishra, ―Multimodal Biometrics it is: Need for Future Systems‖, IJCA,

Volume 3 – No.4, June 2010

[10] Hong, L., Jain, A.K., ―Integrating faces and fingerprints for personal identification‖,

IEEE Trans. PAMI 20 (12), 1295–1307, 1998

[11] Slobodan Ribaric, Ivan Fratric, ―A matching score normalization technique for

multimodal biometric systems‖.

[12] M. He, et al., ―Performance evaluation of score level fusion in multimodal biometric

systems‖, Pattern Recognition, 2009

[13] Gian Luca Marcialis, Fabio Roli, ―Score-level fusion of fingerprint and face

matchers for personal verification under ‗stress‘ conditions‖, 14th IEEE International

Conference on Image Analysis and Processing ICIAP 2007, IEEE, pp. 259- 264, 2007

[14] F. Cui, et al., Score Level Fusion of Fingerprint and Finger Vein Recognition,

Journal of Computational Information Systems 7: 16, 5723-5731, 2011

[15] Fawaz Alsaade, ―Neuro-Fuzzy Logic Decision in a Multimodal Biometrics Fusion

System‖, Scientific Journal of King Faisal University (Basic and Applied Sciences)

Vol.11 No.2 1431, 2010

[16] Sarat C. Dass, et al., A Principled Approach to Score Level Fusion in Multimodal

Biometric Systems, AVBPA 2005: 1049-1058

[17] Romaissaa Mazouni, et al., On Comparing Verification Performances of Multimodal

Biometrics Fusion Techniques, International Journal of Computer Applications, Volume

33– No.7:24-29

[18] Lam, L. and Suen, C. Y., Optimal Combination of Pattern Classifiers, Pattern

Recognition Letters, 16:945-954.

[19] Xu, L., Krzyzak, A., and Suen, C. Y., Methods for Combining Multiple Classifiers

and their Applications to Handwriting Recognition, IEEE Transactions on Systems, Man,

and Cybernetics, 22(3):418-435.

[20] Daugman, J., Combining Multiple Biometrics,

http://www.cl.cam.ac.uk/users/jgdlOOO/combine/combine.html.

[21] Domingos, P. and Pazzani, M., On the Optimality of the Simple Bayesian Classifier

under Zero-One Loss, Machine Learning, 29(2-3): 103-130.

[22] Md. Maruf Monwar, Marina L. Gavrilova, Multimodal Biometric System Using

Rank-Level Fusion Approach, IEEE Transactions on Systems, Man and Cybernatics –

Part B: Cybernatics, Vol. 39, No. 4, August 2009, pp. 867-878

[23] Ajay Kumar, Sumit Shekhar, Personal Identification Using Multibiometrics Rank-

Level Fusion, IEEE Transactions on Systems, Man and Cybernatics- Part C: Applications

and reviews

[24] Ayman Abaza, Arun Ross, Quality Based Rank-Level Fusion in Multibiometric

Systems, Proc. of 3rd IEEE International Conference on Biometrics: Theory,

Applications and Systems (BTAS), (Washington DC, USA), September 2009.

[25] Divyakant T Meva, C K Kumbharana, ―Comparative study of different fusion

techniques in multimodal biometric authentication‖, International Journal of Computer

Applications, Volume 66, No. 19, March 2013, pp. 16-19

CHAPTER 4 STUDY OF MULTIMODAL BIOMETRIC TECHNIQUES...

Documents

Transcript of CHAPTER 4 STUDY OF MULTIMODAL BIOMETRIC TECHNIQUES...