Testing and Certification of Biometric Components and System in Europe

a report on the intermediate findings of the BioTesting Europe Project

Maria Margarida Castro NevesFraunhofer IGD, Germany

Margarida.Castro-Neves@igd.fraunhofer.de

2

Agenda

1. About the “BioTesting Europe” project

2. Identified EU needs for testing in biometrics

3. Issues & Gaps in testing capabilities

4. Improving EU capabilities for assuring performance

3

BioTesting Europe

Project details– 9 month project: finishing by Dec 2007– Supporting Activity under “Preparatory Actions for Security Research” Partners: – European Biometrics Forum (coordinator)– National Physical Laboratory (UK)– Fraunhofer IGD (Germany)– EC/JRC Ispra (Italy)Objectives– Consult to determine EU’s needs for testing of biometrics (Inventory)– Identify where improved testing capabilities required (Gap Analysis)– Prepare work plan/roadmap of coordinated actions to further develop

biometrics testing and certification capabilities– Define the ‘business case’ for testing

4

European Approach

This is why national governments / authorities should support a European approach for testing certificates:– The vendors would not survive to pay

for 27 national tests/certificates– Not-testing (before installing) would

undermine the EU-widesecurity policy for the border control process

– We need to provide a comparable security at all border control points along the EU perimeter

Vice-Versa recognition works (well) forCC-certification. It should work alsofor Biometric Performance certification!

5

Project scope

Stakeholders consulted– Suppliers

• Vendors• System Integrators

– Operators• end customer

– Test organisations• Independent 3rd party labs• In-house test labs• Certification authorities

– Academics

Applications considered(Criteria: relevance and urgency)

• Passports• AFIS• Visas (VIS BMS)• Identity documents• Registered traveller

Potential Scope:• Systems• Sub-systems• Devices • Processes• Personnel (training & education)

6

Questions to be answered

What testing is needed?

Which components should be certified?

Who should perform these tests?

What standards are applicable?

What do we already have & what needs to be developed ?

What R&D is needed?

What are the costs and who will pay/invest?

Inventory based on 38 Questionnaires

7

Example: e-borders

What needs testing for e-Passports and border control e-Gates?– Qualities of enrolment

• Procedures• Operating environment

– Interoperability– Efficiency at the border

• Throughput• Accuracy• Accessibility

– Usability• Consistency of processes

8

Testing needed / Tests conductedNeeds to be tested Who tests Comments

Operators Suppliers Test-labs

Performance S T Component level tests

O (Sub-)System level tests

Accuracy 1:1 O S T

Accuracy 1:N (with large N) O S Need v.large databases

Failure to Enrol/Acquire O T Need representative population & environment

Throughput O S T

Interoperability T E.g. MINEX, MTIT

Conformance

Data format (levels1&2) S T Some test tools

Data format (level 3 – semantic level)

S? No methodologies / reference data

API O S

9

To be tested Who tests Comments

Biometric data quality S T Standards being developed

Software kit to assess data quality

? Validation / Calibration needed?

Sensor testing

Quality & Conformance S E.g. Appendix F

Sensor ruggedness S T traditional type of test

Production quality S? Are all sensors the same quality as the tested/certified one

Usability / Accessibility O? Not tested to any standard

Security

Anti-spoofing S T Few products tested under CC

Data protection T? Similar to security audit

Safety S T CE plus?

Personnel O

10

Observations

Testing is carried out by Suppliers, Operators, and Test Organisations– Mostly by suppliers & operators– Most current test needs are being addressed

• By ad-hoc means rather than using standard schema / references

3rd party tests & certification will be complementaryto suppliers’ and operators’ tests– Suppliers will test during development & production– Operators need to test on their own data

• “Helps us understand our system”

Standard tests & certification must meet real needs – Certify against applicable levels of performance, test scenario, etc.– Must be a return on investment in carrying out the tests

11

Observations / Gaps

Fragmented approach to testing– Few common requirements identified– Disconnect between component-level tests & system-level tests

• Component-level performance not predictive of system-level performance

No methodologies / standards for some key areas of testing– Usability/Accessibility (of particular EU interest)– Level-3 conformance to data format standards

• i.e. is the record an accurate representation of the characteristic– …

Biometrics not a mature technology – still many unknowns about performance– E.g. long-term performance of face, fingerprint, iris

• Ageing of face compared to photo image over lifetime of passport• Performance expectations fingerprinting children (age limits)

12

Observations / Gaps

Usability and Accessibility– Diverse concepts for Human-Computer-Interface (HCI) among vendors, creating

confusion for data subjects– Standardization of usability related issues is not progressed far: ISO 24779 (Icons

& Symbols) is in early Working Draft status– R&D: How can we separate out usability impacts on biometric performance?

Need for test data– Determining high accuracy requires a lot of data – Data protection legislation often prevents sharing/saving data– Release of any data may compromise its use in testing– Possible Technical Solutions:

• Possibility to consider synthetic data?• If the test data can not travel to the System-Under-Test could the system

travel to the data?

14

Organisational structures (under consideration)

Do we need a network of test organisations?– European – International?– Which existing institution can take the role of an accreditation body?– Criteria for including a test laboratory in such a network?

• Which type of labs are accepted: – Governmental lab / Independent lab– Consultant / integrators lab– Industry lab

• No closed group - transparent conditions needed• What are the criteria that a lab drops out of the network

15

Organisational structures (under consideration)

As resources are limited - where should the focus of testing be?– Biometric Performance testing– Protocol testing (according to SC17.3 work)– Security testing along Common Criteria …

What role for “Qualified product lists” / “certification”?– Some performance aspects better suited to certification than others

• Conformance to standard • Interoperability• FAR/FRR – too dependent on target population/environment

– Scope of certificate • Application specific?• Duration?

16

Conclusions

BioTesting project underway– Project finishes soon, but comments/opinions welcomed

Testing of usability issues is becoming urgent to achieve desired levels of performance & interoperability

Focus of test and certification seems certain to change as industry matures

17

Further information

Contact points– max.snijder@eubiometricsforum.com

• +31 624 603809 (direct)• +353 1 488 5810 (secretariat)

– tony.mansfield@npl.co.uk• +44 20 8943 7029

– christoph.busch@igd.fraunhofer.de• +49 6151 155 536

– margarida.castro-neves@igd.fraunhofer.de• +49 6151 155 535

Website– www.biotestingeurope.eu