Methods of Deriving Biometric ROC Curves from the k -NN Classifier
description
Transcript of Methods of Deriving Biometric ROC Curves from the k -NN Classifier
![Page 1: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/1.jpg)
Robert S. Zack May 8, 2010
METHODS OF DERIVING BIOMETRIC ROC CURVES FROM THE k-NN CLASSIFIER
![Page 2: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/2.jpg)
Agenda
Introduction to ROC Curves Classification Multi-Class Issues and Solutions New Derivation Methods Weak and Strong System Training Use Cases Search for a Topic Publications Dissertation Status Questions
![Page 3: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/3.jpg)
Introduction to ROC Curves
Used for binary decisions Signal detection – signal / no signal Medical diagnosis – disease / no disease Biometric authentication – you are the person you claim to be /
you are not In biometrics the ROC curve varies from FAR=1 & FRR=0 at one
end to FAR=0 & FRR=1 at other FAR = False Accept Rate – the rate an imposter is falsely accepted FRR = False Reject Rate – the rate the correct person is falsely
rejected ROC Charts are expressed in terms of percentages (0-100%) or
probabilities (0-1). These are used interchangeably.
![Page 4: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/4.jpg)
Authentication Analogy
Supreme Court – nine judges Usual procedure – majority required to make decision Like 9NN needing majority to authenticate a user
ROC Curve – creates many potential procedures Need 9 votes to make decision (very conservative) Need 8, 7, 6 votes to make decision (conservative) Need 5 votes to make decision (majority) Need 4, 3, 2 votes to make decision (liberal) Need 1 or even 0 votes to make decision (very liberal)
![Page 5: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/5.jpg)
Anatomy of a Biometric ROC Curve
Conservative is too restrictive.
Positive classification requires strong evidence.
Liberal is too open.
Requires weak evidence.
![Page 6: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/6.jpg)
Parametric Procedures
Parametric techniques are well studied.
Data follows a normal or Gaussian distribution.
Vary a threshold to obtain the tradeoff between FAR/FRR.
Probability density functions can be calculated without estimation.
![Page 7: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/7.jpg)
Parametric ROC Derivation
![Page 8: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/8.jpg)
Classification
1. The k-NN classifier is well studied.2. Biometrics classification problems can have many
classes. 3. It is easier to work with a large or unknown
population if the data is converted from a multi-class to a two-class decision.
4. Cha Dichotomy Model.
![Page 9: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/9.jpg)
K-NN Nonparametric Classifier
k-NN is nonparametric.
A vector-difference model is used to covert a many class problem into a two class, binary problem.
Uses Euclidean distance
k-NN Classification Procedure for k=5, Adapted from Pattern Classification, Duda, et al.
![Page 10: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/10.jpg)
Cha Dichotomy Model
Simplifies complexity
Transforms a feature space into a distance vector space.
Uses distance measures.
Multi-class to two Class Transformation Process, Adapted from Yoon et al (2005)
![Page 11: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/11.jpg)
m-kNN Method
Pure Rank Method. Evaluate the top 7
NN. Q is authenticated
if # within-class matches is >= decision threshold of 4NN.
Unweighted. All W’s are equal in weight.
![Page 12: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/12.jpg)
wm-kNN Method
Rank method weighted by rank order.
Authenticate if W choices are > weighted match (m)
Score varies from 0 to =k(k+1)/2 or 5+4+3+2+1
For every m, FAR/FRR pair or ROC point.
If m=0, FAR=1, FAR=0 …All users accepted.
If m=15, FAR=small, FRR=large, few Q’s accepted.
![Page 13: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/13.jpg)
m-kNN and wm-kNN ROC’s
LapFree – Weak Training
![Page 14: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/14.jpg)
m-kNN and wm-kNN ROC’s
DeskFree – Weak Training
![Page 15: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/15.jpg)
t-kNN Method
A distance threshold method.
A positive vote is within a distance threshold from the user’s sample.
Uses feature vector space distances only.
At 0, no distance vectors are authenticated. FAR=0, FRR=100%. At t=100, all distance vectors are authenticated. FAR=100, FRR=0.
![Page 16: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/16.jpg)
t-kNN Method
DeskFree (left) and LapFree (right) Data
![Page 17: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/17.jpg)
ht-kNN Method
Weighted vote based on distances to the kNN.
Hybrid of rank method and vector space distances.
For each test sample, the within-class weight (WCW) is calculated based on the distance vectors. DeskFree (left) and LapFree (right) Data
![Page 18: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/18.jpg)
New Nonparametric ROC Methods
1. Need m votes out of k for decision• Pure rank method
2. Need wm votes for decision, but some judges get more than one vote (weighted method)• Rank method weighted by rank order
3. A positive vote is within a distance threshold from the user’s sample• Uses feature vector space distances only
4. Weighted vote based on distances to the kNN• Hybrid of rank method and vector space distances
![Page 19: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/19.jpg)
Weak & Strong Training
Weak Training• People used in testing not used in training
Independent sets of users for testing and training Strong Training
• People used in testing also used in training Usually to augment the different training people
• But new difference-vectors used to authenticate• For example, users provide 8 samples – 5 for training
and 3 to match against for authentication
![Page 20: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/20.jpg)
Weak & Strong Training
1 3 5 7 9 11 13 15 17 19 2190.00%
91.00%
92.00%
93.00%
94.00%
95.00%
96.00%
97.00%
98.00%
99.00%
100.00%
kNN Performance
DeskFree (WT) LapFree (WT)
DeskFree (ST18) DeskFree (ST36)
DeskFree (ST54)
Nearest Neighbor
Per
cent
Acc
urac
y
![Page 21: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/21.jpg)
Use Cases
On-line test taking – Authentication Application Enroll students at the start of a class. Collect biometric
samples. Authenticate users are who they should be using off-
line batch processing. Corporate Compliance Training/Test Administration
Enroll employees at some point prior to the training or test administration. Collect biometric samples. Refresh them at designated intervals.
Authenticate users are who they should be.
![Page 22: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/22.jpg)
Future Work
Real-time authentication. Accuracy Improvements. Error Cost Analysis. Measurement Error.
![Page 23: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/23.jpg)
Initial Search for a Topic
Started program in Fall 2008. Entered DPS with an idea to research a topic in the
area of mobile computing. Quickly discarded the idea.
Continued to search for ideas by participating as a Customer for IT691/CS691Projects. Became exposed to Facial and Keystroke Biometrics.
Continued working with Keystroke Biometrics and eventually found a topic with the help of Dr. Tappert.
![Page 24: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/24.jpg)
Idea Vetting
The first few presentations of the topic met with a lot of resistance. It took some time to develop the “so what”.
Every Research Seminar was recorded so that I could go back and listen to criticisms.
Participated as co-author to several papers on the subject. Some papers were peer-reviewed and submitted for publication.
![Page 25: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/25.jpg)
Publications [1] J. Abbazio, S. Perez, D. Silva, R. Tesoriero, F. Penna, and R. S. Zack, "Face Biometric Systems," in
Student-Faculty Research Day, CSIS, Pace University, White Plains, 2009, pp. C1.1-C1.8. [2] A. Amatya, J. Aliperti, T. Mariutto, A. Shah, M. Warren, R. S. Zack, and C. C. Tappert, "Keystroke
Biometric Authentication System Experimentation," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2009, pp. C4.1-C4.8.
[3] A. C. Caicedo, K. Chan, D. A. Germosen, S. Indukuri, M. N. Malik, D. Tulasi, M. C. Wagner, R. S. Zack, and C. C. Tappert, "Keystroke Biometric: Data/Feature Experiments," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010.
[4] K. Doller, S. Chebiyam, S. Ranjan, E. Little-Tores, and R. S. Zack, "Keystroke Biometric System Test Taker Setup and Data Collection," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010.
[5] S. Janapala, S. Roy, J. John, L. Columbu, J. Carrozza, R. S. Zack, and C. C. Tappert, "Refactoring a Keystroke Biometric System," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010, pp. B1.1-B1.8.
[6] M. Lam, U. Patel, M. Schepp, T. Taylor, and R. S. Zack, "Keystroke Biometric: Data Capture Resolution Accuracy," in Student-Faculty Research Day, CSIS, Pace University, White Plains, 2010.
[7] C. C. Tappert, S.-H. Cha, M. Villani, and R. S. Zack, "A Keystroke Biometric System for Long-Text Input," International Journal of Information Security and Privacy, Pending Publication, 2010.
[8] R. S. Zack, C. C. Tappert, S.-H. Cha, J. Aliperti, A. Amatya, T. Mariutto, A. Shah, and M. Warren, "Obtaining Biometric ROC Curves from a Non-Parametric Classifier in a Long-Text-Input Keystroke Authentication Study," vol. 268, Pace University, 2009.
![Page 26: Methods of Deriving Biometric ROC Curves from the k -NN Classifier](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816565550346895dd7ee3d/html5/thumbnails/26.jpg)
Questions