LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

20
IT’S JUST A PRINTER… LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS Lachlan Turner, CSC Labs Technical Director Alan Sukert, Xerox Product Security Specialist International Common Criteria Conference September 2013

Transcript of LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

Page 1: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

IT’S JUST A PRINTER… LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

Lachlan Turner, CSC Labs Technical Director Alan Sukert, Xerox Product Security Specialist

International Common Criteria Conference September 2013

Page 2: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 2

MFDs are complex, embedded network devices

• MFDs have: – One or more operating systems

– Network controller and firmware

– One or more hard disk drives

– Web server

– Hardware ports

– Page Description Language interpreters (PS & PCL)

– Fax

– Network Interfaces

…multiple points of vulnerability

Page 3: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 3

Which have been exploited….

Page 4: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 4

Even Xerox have reported vulnerabilities…

Page 5: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 5

No vulnerabilities reported for the 9700…

Page 6: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 6

The Xerox Security Model

Page 7: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 7

The Xerox Security Model – evaluated functionality

Page 8: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 8

Xerox Common Criteria Evaluations

ColorQube 9200 series (undergoing evaluation)

WorkCentre 7120 WorkCentre 4250/4260

WorkCentre 7700 series (undergoing evaluation)

WorkCentre 5700 series

WorkCentre 5135/5150 WorkCentre 5225/5230

Xerox Color 550/560 Printer Xerox 4112/4127 Copier/Printer

WorkCentre 7500 series WorkCentre 5300 series

Page 9: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 9

Xerox Evaluation History (subset)

0

5

10

15

20

25

EAL3 (DE)

EAL3 (CA)

EAL3 (CA)

EAL3 (AUS)

EAL2 (CA)

EAL2 (USA)

EAL2 (USA)

Time to Evaluate (months)

Page 10: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 10

Xerox wanted a smarter approach

• Certification within 3 months of product launch • Achieved by:

– Strategic plan / schedule for products to be certified – Starting CC process before product launch – Better communication between Development and Security teams – Leverage strong lab relationship

Page 11: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 11

Xerox Evaluation History – Process Improvements

0

5

10

15

20

25

EAL3 (DE)

EAL3 (CA)

EAL3 (CA)

EAL3 (AUS)

EAL2 (CA)

EAL2 (USA)

EAL2 (USA)

Time to Evaluate (months)

Product Launch

Page 12: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 12

Reducing Cost of Evaluation by 40%

• Xerox and CSC have reduced evaluation/certification cost by 40% • Common platform across machines / strategic ST development

– Enable more machines per evaluation

Page 13: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 13

Reducing Cost of Evaluation by 40%

• Simplification of CC evidence – Development (ADV). Re-align to CEM and remove unnecessary detail.

• Functional Specification (FSP) 500 pages to 200 pages • TOE Design (TDS) 100 pages to 30 pages

– Life Cycle (ALC). Replace source process documents (30+) with CC specific documents (3).

– Testing (ATE). Reduce test traceability from 30 pages to 8 pages (due to simplified FSP).

• Re-use across evaluations

Page 14: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 14

Reducing Cost of Evaluation by 40%

• Close engagement with CC consultant / advocate • Continuity of evaluation team / lab

– Xerox and CSC have been working together for over 10 years – Test team product familiarity – POC familiarity (Xerox / CSC working relationships)

• Continuity of scheme (NIAP) – Request same certifiers / validators (not always possible) – Process awareness

Page 15: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 15

Protection Profiles?

• Xerox heavily involved in MFD PP development (IEEE 2600) – Multi-vendor collaboration – Adopted by NIAP in 2010 – EAL2 + ALC_FLR.2 – “…great but lengthy experience” – “Takes a lot of work and discussions..”

Page 16: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 16

Protection Profiles?

• Xerox involved in development a new MFD PP – Collaboration between IPA (Japan) and NIAP (USA) – Multi-vendor collaboration (same as IEEE 2600) – EAL0? (similar to NDPP) – “…Original schedule was way optimistic; ….it will take a couple of years to do

this”

• Motivation for involvement? – Drive what will be in the PP – Align product features to what will be in the PP

Page 17: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 17

What is next for Xerox?

• Continue working with CSC • Certify as many MFDs as business case support • Expand into product lines that have not been evaluated • Hints on new features?

– WebDAV – Tablet / smartphone support

Page 18: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 18

Predictions for the future of CC?

• Relevance will depend on the policy drivers that require CC • Realization that it is very hard to achieve and maintain a set of

Collaborative PPs • Gap between vision and reality • Splintering of evaluation markets (e.g. Europe, Asia stay with EALs /

others chase cPP) – Vendors back to performing multiple evaluations?

• Reconsideration of PP only policies

Page 19: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 19

THANK YOU

Lachlan Turner, CSC Labs Technical Director, [email protected]

Alan Sukert, Xerox Product Security Specialist, [email protected]

mailto:[email protected]�
mailto:[email protected]�
Page 20: LESSONS LEARNED OVER TEN YEARS OF CC EVALUATIONS

CSC Security Testing and Certification Labs 20


Designing Road Diet Evaluations: Lessons Learned from San Jose’s Lincoln Avenue Road ... · 2019-12-14 · Designing Road Diet Evaluations: Lessons Learned from San Jose’s Lincoln

Designing Road Diet Evaluations: Lessons Learned from San Jose’s Lincoln Avenue Road ... · 2019-12-14 · Designing Road Diet Evaluations: Lessons Learned from San Jose’s Lincoln

Evaluation Review 1: Lessons learned from agriculture and food security programme evaluations in Asia

Evaluation Review 1: Lessons learned from agriculture and food security programme evaluations in Asia

Study on Legal Instruments and Lessons Learned from the Evaluations ...ec.europa.eu/europeaid/how/evaluation/evaluation_reports/reports/... · Study on Legal Instruments and Lessons

Study on Legal Instruments and Lessons Learned from the Evaluations ...ec.europa.eu/europeaid/how/evaluation/evaluation_reports/reports/... · Study on Legal Instruments and Lessons

Minnesota Mississippi cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc w wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc ... Wisconsin

Minnesota Mississippi cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc w wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc wc ... Wisconsin

Lessons Learned About the Implementation of Large-Scale ... · Lessons Learned About the Implementation of Large-Scale Evaluations: - Outcomes - 1 •Especially given their expense,

Lessons Learned About the Implementation of Large-Scale ... · Lessons Learned About the Implementation of Large-Scale Evaluations: - Outcomes - 1 •Especially given their expense,

Individual Evaluations - Harvard Universityscholar.harvard.edu/.../files/gov30individualevaluations.pdfGovernment 30 FAS Course Evaluations, Individual Evaluations Fall 2008 - 2009

Individual Evaluations - Harvard Universityscholar.harvard.edu/.../files/gov30individualevaluations.pdfGovernment 30 FAS Course Evaluations, Individual Evaluations Fall 2008 - 2009

Lecture 5 Heuristic evaluations & Early prototype Evaluations

Lecture 5 Heuristic evaluations & Early prototype Evaluations

Working with The Public Sector in Using CC Licenses: Lessons Learned

Working with The Public Sector in Using CC Licenses: Lessons Learned

The Primary Question Clinical Mental Health Counseling Program Evaluation How will lessons learned from our program evaluations be used to make counselor.

The Primary Question Clinical Mental Health Counseling Program Evaluation How will lessons learned from our program evaluations be used to make counselor.

Countries from Experience with Technical Education and · literature review takes into account the lessons learned, previous reviews and evaluations of programmes within this field,

Countries from Experience with Technical Education and · literature review takes into account the lessons learned, previous reviews and evaluations of programmes within this field,

ENQA seminar:First external evaluations of quality assurance agencies – lessons learned Panel discussion: Practicalities and challenges of self and external.

ENQA seminar:First external evaluations of quality assurance agencies – lessons learned Panel discussion: Practicalities and challenges of self and external.

D14.3 Extended Virtual Factory Lessons Learned and Evaluations … · 2017-04-25 · 20/04/2015 Deliverable D14.3 Extended Virtual Factory Lessons Learned and Evaluation – M24 report

D14.3 Extended Virtual Factory Lessons Learned and Evaluations … · 2017-04-25 · 20/04/2015 Deliverable D14.3 Extended Virtual Factory Lessons Learned and Evaluation – M24 report

DFID COUNTRYPROGRAMME EVALUATIONS SYNTHESISOF2006/2007 EVALUATIONS · 2016-03-29 · DFID COUNTRYPROGRAMME EVALUATIONS SYNTHESISOF2006/2007 EVALUATIONS Julian Barr and CharlotteVaillant

DFID COUNTRYPROGRAMME EVALUATIONS SYNTHESISOF2006/2007 EVALUATIONS · 2016-03-29 · DFID COUNTRYPROGRAMME EVALUATIONS SYNTHESISOF2006/2007 EVALUATIONS Julian Barr and CharlotteVaillant

ISO LESSONS LEARNED - Knowledge @ Urbanknowledge.urbanengineers.com/assets/LL0906b1.pdf12/13/2006 1 iso lessons learned date: 12/13/2006 to: urban engineers, inc. cc: iso management

ISO LESSONS LEARNED - Knowledge @ Urbanknowledge.urbanengineers.com/assets/LL0906b1.pdf12/13/2006 1 iso lessons learned date: 12/13/2006 to: urban engineers, inc. cc: iso management

LESSONS LEARNED, EVALUATIONS & AFTER ACTION REVIEW …

LESSONS LEARNED, EVALUATIONS & AFTER ACTION REVIEW …

Effective banking products CC evaluations.

Effective banking products CC evaluations.

2017-2018 Evaluative Exercises · 2017-2018 lessons learned studies and evaluations: A summary 2017-2018 Evaluative Exercises A SUMMARY OF LESSONS LEARNED STUDIES AND EVALUATIONS.

2017-2018 Evaluative Exercises · 2017-2018 lessons learned studies and evaluations: A summary 2017-2018 Evaluative Exercises A SUMMARY OF LESSONS LEARNED STUDIES AND EVALUATIONS.

D12.3 FITMAN Extended Smart Factory lessons learned … · learned and evaluations (first) Document Owner: ... first place it is important that data is available but also regarding

D12.3 FITMAN Extended Smart Factory lessons learned … · learned and evaluations (first) Document Owner: ... first place it is important that data is available but also regarding

What I learned from CC - Jay Yoon

What I learned from CC - Jay Yoon

Family Finding What Have We Learned? 1 |. Agenda Opening/Introduction to Family Finding.…Bob Friend What Do Recent Evaluations Tell Us?..........Karin.

Family Finding What Have We Learned? 1 |. Agenda Opening/Introduction to Family Finding.…Bob Friend What Do Recent Evaluations Tell Us?..........Karin.