Interoperability and the Stability Score Index

37
INTEROPERABILITY AND THE STABILITY SCORE INDEX Zach Moore, Stephen Elliott, Kevin O’Connor, Shimon Modi

description

The stability score index, conceptualized in 2013, was designed to address the weaknesses of the zoo menagerie and other performance metrics by quantifying the relative stability of a user from on condition to another. In this paper, the measure of interoperability is the stability score from enrolling on one sensor and verifying on multiple sensors. The results showed that like performance, individual performance were not stable across these sensors. When examining stability by sensor family (capacitance, optical and thermal) we find that capacitive as the enrollment sensor were the least stable. Both enrolling and verifying on a thermal sensor, individuals were the most stable of the three family types. With respect to interaction type, enrolling on touch and verifying on swipe was more stable than enrolling on swipe and verifying on swipe, which was an interesting finding. Individuals using the thermal sensor generated the most stable stability scores.

Transcript of Interoperability and the Stability Score Index

Page 1: Interoperability and the Stability Score Index

INTEROPERABILITY AND THE

STABILITY SCORE INDEXZach Moore, Stephen Elliott, Kevin O’Connor, Shimon Modi

Page 2: Interoperability and the Stability Score Index

INTRODUCTION

Page 3: Interoperability and the Stability Score Index

•Wanted to look at interoperability of fingerprint

images across sensors in the context of

stability

•Quality changes across sensors, but does this

affect stability?

INTRODUCTION

Page 4: Interoperability and the Stability Score Index

• Shimon, 2008

• Analyzed interoperability of fingerprint sensors

• How this affected system performance

• Minutiae based matching

• O’Connor, 2013

• Looked at the instability of the zoo animals across different force levels

• Created the stability score index (SSI)

RELATED WORK

Page 5: Interoperability and the Stability Score Index

STABILITY SCORE INDEX

Page 6: Interoperability and the Stability Score Index

METHODOLOGY

Page 7: Interoperability and the Stability Score Index

• Cleaned the data

• Only used subjects who had three enrollment captures and three testing captures on all sensors

• Created dataruns

• Ran the data through Megamatcher to get genuine and impostor scores

• Ran the scores through Oxford Wave to get zoo analysis

• Used the zoo analysis to calculate stability scores

METHODOLOGY

Page 8: Interoperability and the Stability Score Index

•Divided datasets

METHODOLOGY

Page 9: Interoperability and the Stability Score Index

Sensor Enrollment

Samples

Testing

Samples

Total

Samples

Atmel 483 483 966

Authentec 483 483 966

Crossmatch 483 483 966

Digital Persona 483 483 966

Fujitsu 483 483 966

Futronic 483 483 966

Identix 483 483 966

UPEK S 483 483 966

UPEK T 483 483 966

SAMPLES

• 161 subjects

• 6 captures each

• 3 enrollment

• 3 testing

Page 10: Interoperability and the Stability Score Index

RESULTS

Page 11: Interoperability and the Stability Score Index

AVERAGE SSI GROUPING MATRIX

Page 12: Interoperability and the Stability Score Index

AVERAGE SSI GROUPING MATRIX

Page 13: Interoperability and the Stability Score Index

SUBJECT 43 STABILITY

Page 14: Interoperability and the Stability Score Index

SENSOR MATRIX

Page 15: Interoperability and the Stability Score Index

SENSOR MATRIX SUBJECT 273

Page 16: Interoperability and the Stability Score Index

SENSOR MATRIX VALUES

Page 17: Interoperability and the Stability Score Index

SENSOR ENROLL BOXPLOT

Page 18: Interoperability and the Stability Score Index

SENSOR TEST BOXPLOT

Page 19: Interoperability and the Stability Score Index

ACTION TYPE MATRIX

Page 20: Interoperability and the Stability Score Index

ACTION TYPE MATRIX VALUES

Page 21: Interoperability and the Stability Score Index

ACTION TYPE ENROLL BOXPLOT

Page 22: Interoperability and the Stability Score Index

ACTION TYPE TEST BOXPLOT

Page 23: Interoperability and the Stability Score Index

SENSOR TYPE MATRIX

Page 24: Interoperability and the Stability Score Index

SENSOR TYPE MATRIX VALUES

Page 25: Interoperability and the Stability Score Index

SENSOR TYPE ENROLL BOXPLOT

Page 26: Interoperability and the Stability Score Index

SENSOR TYPE TEST BOXPLOT

Page 27: Interoperability and the Stability Score Index

INTERACTION TYPE MATRIX

Page 28: Interoperability and the Stability Score Index

INTERACTION TYPE MATRIX VALUES

Page 29: Interoperability and the Stability Score Index

INTERACTION TYPE ENROLL BOXPLOT

Page 30: Interoperability and the Stability Score Index

INTERACTION TYPE TEST BOXPLOT

Page 31: Interoperability and the Stability Score Index

HISTOGRAM OF SSI BY ENROLLMENT

SENSOR

• Data is not normal

• Ran Kruskal-Wallis test

00

1

2

3

4

5

6

41.0- 00.0 41.0 82.0 24.0 65.0 07.0 48.

0.1760 0.1303 1440

0.2782 0.1788 1440

0.1691 0.1361 1440

0.1685 0.1320 1440

0.1896 0.1430 1440

0.1741 0.1410 1440

0.1802 0.1406 1440

0.2023 0.1499 1440

0.1634 0.1313 1440

Mean StDev N

S

ytisn

eD

IS

A

rosneS llornE

T KEPU

S KEPU

xitnedI

cinortuF

ustijuF

anosreP latigiD

hctaMssorC

cetnehtuA

lemt

N lamro

Page 32: Interoperability and the Stability Score Index

•H0= the median SSI scores are equal

•Ha= the median SSI scores are not equal

KRUSKAL-WALLIS TEST

Page 33: Interoperability and the Stability Score Index

Sensor H DF P

Atmel 58.80 8 0

Authentec 221.45 8 0

Crossmatch 63.75 8 0

Digital Persona 56.33 8 0

Fujitsu 121.45 8 0

Futronic 81.66 8 0

Identix 102.72 8 0

UPEK S 109.80 8 0

UPEK T 82.62 8 0

KRUSKAL-WALLIS RESULTS

Page 34: Interoperability and the Stability Score Index

•All p-values resulted in p=0

•Reject H0

• Meaning the medians of the SSIs across the

sensors are significantly different

KRUSKAL-WALLIS RESULTS

Page 35: Interoperability and the Stability Score Index

CONCLUSION

Page 36: Interoperability and the Stability Score Index

• Subjects are not stable across different sensors

using SSI

• Enrolling on Authentec produced the worst SSIs

overall, but testing on it did not show the same

pattern

• Predicting how unstable a user will be from

enrollment to testing would increase performance

CONCLUSION

Page 37: Interoperability and the Stability Score Index

•Look at stability across force levels

•See if type of sensor plays a role (thermal,

swipe, touch, etc.)

•Analyze the image quality of the images and

look for a relationship

FUTURE WORK