CC in the Real World

1

© 2

007

a

tse

c in

form

atio

n s

ecu

rity

8th ICCC: CC in the Real World

NA

SA

are

ow

ners

of

this

pic

ture

pro

duce

d by

The

Vis

ible

Ear

th

When technologies based on the state of the art in a rapidly maturing discipline are put into practice some

“real world” issues surface for the practitioners to cope with.

Fiona Pattinson

CC in the Real World

2

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


In the U.S. GAO report:(CC consumers)

• March 2006– difficulty in matching agencies’ needs with the

availability of NIAP-evaluated products – vendors’ lack of awareness regarding the

evaluation process – a reduction in the number of validators to

certify products – a lack of performance measures and difficulty

in documenting the effectiveness of the NIAP process

http://www.gao.gov/new.items/d06392.pdf

3

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


Vendors Issues

• May 2007 GCN: “Symantec: Common Criteria is bad for you” – CC is a closed standard development– Does not address areas where vulnerabilities

occur – Starts late in the development process – It is difficult to produce a complete ST at the

beginning of a project– Assurance vs convenience balance

http://www.gcn.com/online/vol1_no1/44205-1.html

4

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


General Issues with CC…• Timeliness

– Takes too long

• Cost– Costs too much

• Validity/Scope of certification– Too narrow

• Mutual Recognition– limited

• Composition– Was not addressed

• Comprehensibility– Too hard to cope with

David Brewer: Is the Common Criteria the only way?”: https://www.ipa.go.jp/event/iccc2005/pdf/B2-10.pdf

5

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• Over the last few years:– greater emphasis in understanding the

risks presented by a greater number and variety of IT products

– Emphasis on certification as a solution to help assist risk based decisions

• Scalability–Can the CC schemes handle the

needed volume?

“The product I want to buy is not on the list.”

6

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


Rise in number of evaluations

Number of Evaluated Products reported on the CC Portal

0

20

40

60

80

100

120

140

160

180

200

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

yearJuly

ITSEC Evaluations*

(*From “Is the Common Criteria the only way?”

, David Brewer, 2005)

7

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• This *is* a FAQ from new vendors & teams– Most national schemes publish good beginner

info– So does the CC Portal– Many labs provide basic information (and

consultancy!)– “I’ve heard that CC is expensive, takes forever,

needs lots of specialised documents, is hard work.. etc”

“I don’t know where to start with CC.”

'Does Common Criteria need marketing?', ICCC 2006, Tokyo.

8

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• Effects– Some claim it is used as a trade control

measure– Prices smaller companies out of the market

• Compare– FIPS 140-2: Smaller scope, smaller cost

• Consultation and testing components

– ISMS: • Consultation and auditing costs

“It costs HOW much?! ”

9

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• This may be true for a first time evaluation

• There’s a learning curve for product teams using CC

• As a product and dev team’s “evaluation career” progress it is possible to have certification coincide to within a few weeks of the end of the development cycle

“Our products are released every 6 months, but we were told that

our CC evaluation would take 9-18 months.”

10

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


Example: Linux

200720062005200420032002

IBMSLES 8

HPRHEL3-U3

IBMRHEL3-U2 Unisys

RHEL3-U5RHEL4

HPRHEL3

IBMRHEL4-U1

HPRHEL4-U2

SGIRHEL4-U4

HPRHEL5

IBMRHEL5

IBMSLES8 SP3

HPSLES8 SP3

HPSLES9

SGISLES9 SP2

RHEL 5RHEL 4RHEL 3

SLES10 SP1SLES10 SLES9SP2

SLES9 SLES8 SP3 SLES9 SP3SLES8

SP4

RHEL 4U1

RHEL 4U3

Distribution version release

Certification date for platform evaluation under CC

OEL4 U4/U5

11

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• CC is not a direct process or product improvement standard

• If an organization has assurance provision as part of a product-line strategy then security process and product improvement are vital– Informal, or formal (SSE:CMM ISO/IEC 21827, CMMI,

TickIT, even ISO/IEC 9001 and 90003)– Processes may even address the needs of particular

standards

• Secure System Design– Design to provide assurance

How can I make CC easier next time?

Improving development process with assurance provision in mind

12

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• Unrealistic boundaries are specified in order to:– Reduce evaluation effort (time/cost)– Avoid problem areas

• CC is deliberately flexible, and does not prevent ST writers defining unrealistic boundaries

• The specification of inappropriate boundaries bring CC into disrepute as consumers are left with claims that are meaningless in their objective of assessing their risk

“But you didn’t evaluate anything I use or that’s

remotely useful.”

13

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• In practice this can be alleviated by:– Scheme policies– Specification and adoption of appropriate PPs– Ethical laboratories and consultants advise their

customers appropriately

• Compare with :– Specifications e.g. FIPS 140-2

• Boundary is more rigorous but still mis-used sometimes

– ISMS(& QMS) • (At least for QMS) the specifications for scope have been

mis-used and caused disrepute.

14

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


“It doesn’t evaluate a whole product”

• Security functionality only– No assurance given for non-security functionality

• Some product vulnerabilities appear from the TOE environment and are “dismissed” by broad assumptions– This is technically correct– Hard for real-world users to understand– Does it help assess risk more accurately in the real-

world?• Somewhat true also for FIPS 140-2, although

CMVP certificates and security policies tend to reference the whole product

15

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


• CC is very flexible – SFR extensions– PP– Interpretations– Ongoing standards development

• It is difficult to produce a complete ST at the beginning of a project– “Before and during the evaluation, the ST specifies “what is to be

evaluated”. In this role, the ST serves as a basis for agreement between the developer and the evaluator on the exact security properties of the TOE and the exact scope of the evaluation…. After the evaluation, the ST specifies “what was evaluated…” (CC part 1 A.3.1)

• Compare– FIPS 140-2 which is more restrictive, and actually means that there are

several products that it can’t be used for or that it is very difficult to be used for.

– ISMS can be used for any organization

“CC is inflexible; it can’t be used for my product!”

16

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


Yes it is!

• The more assurance end users need, the harder work it is

• Internal effort is related to evaluation strategy, and can be reduced– Process improvements

“It’s hard work”

17

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


“It assumes a waterfall model for development”

• No it doesn’t! CC paradigm is flexible and it fits all development models.

• Spiral methods, pair programming, iterative and incremental, agile, open source development, rapid prototyping etc can and have been used.

• The evaluation too can be built in to the processes of these models and use similar paradigms

• Examples: – The successful evaluation of Open Source development

paradigm for various Linux evals, the provision of evaluation artifacts and evidence back to the community.

– An iterative evaluation methodology to track and pace iterative product development.

19

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


It is a formalized methodology, and documentation that would not otherwise be produced is required in order to provide high assurance– E.g. ST, Correspondence analyses,

• Other documentation is often produced “just for CC” that the lab should be able to find in a regular mature product development – E.g. Design documentation, Process

documentation, Vulnerability assessment

20

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


“Documentation is never the cause of a security failure so

why emphasise that!”• Documentation is an evaluation tool. It shows

– The design is sound: A product can (and often is) designed poorly. Design flaws => vulnerabilities

– The design is implemented to the level of abstraction specified– It shows the system was designed to implement the security

requirements

• Specialised documentation is an important part of providing assurance: Especially boundaries– PPs and ST for CC – Security Policy, Finite State Model (FSM) for FIPS 140-2– Scope and Statement of Applicability (SOA) for ISMS

• Labs must be prepared to read and use existing documentation in any format.

21

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


“We want ONE evaluation that’s accepted everywhere”

• Mutual Recognition• CCRA /SOGIS are examples• ISO/IEC 15408

• Failed for Smartcards• Failing because of need for higher levels

of assurance (medium level robustness PPs) which are not covered under current MRAs

22

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


Addressing these issues using the CC paradigm

• Developing useful Protection Profiles• Using a “staged” evaluation strategy, so that initial

evaluation of a product at a lower evaluation level builds a good platform for later evaluation at a higher level

• Pursuing ongoing communication between consumers, vendors, evaluation labs, and schemes, so that emerging consumer requirements on mature product lines are clearly understood by all parties in the evaluation process

• Conducting development, Common Criteria consulting, and Common Criteria evaluation efforts in parallel with development efforts, so that certification of new products is timely (atsec example: Red Hat Linux evaluation project)

http://www.atsec.com/downloads/pdf/efficient_cc_evaluations.pdf

23

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


Conclusion• Real World Issues occur with every certification

frameworkBoth standard and the certification scheme have to

be sound.Some specialised documentation is necessary, but

this should be minimal.The definition of clear boundaries are vital.Balance “certification as a tax” vs “real-life

improvement and benefits of certification”Communication is vital

Mutual Recognition arrangementsIncluding training for all stakeholders (Overseers,

Labs, Vendors, Consumers, Sponsors)

24

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


Thank You

Fiona Pattinsonatsec Information Security Corporation

Austin, Texas, USA

Phone: +1 512 615 7382

Email: [email protected]

www.atsec.com

25

© 2

007

a

tse

c in

form

atio

n s

ecu

rity


References & Bibliography• Olthoff, K. G. 1999, 'A cursory examination of market forces driving the Common Criteria', in New Security

Paradigms Workshop, ACM Press, pp. 61-66.• Gordon, L. A. & Loeb, M. P. 2001, Economic Aspects of Information Security, Rainbow Technologies.• Mahon, G. 2002, 'Developers backward engineering their products to meet the Common Criteria', in International

Common Criteria Conference (ICCC), Ottawa.• Merkow, M. 2003, Security assurance through the Common Criteria, New Riders Pub., Indianapolis, IN.• Apted, A. J. , Carthigaser, M. & Lowe, C. 2002, 'Common problems with the Common Criteria', in International

Common Criteria Conference, Ottawa.• Aizuddin, A. , The Common Criteria ISO/IEC 15408– The insight, some thoughts, questions and issues.• Smith, R. E. , Trends in Government Endorsed Security Product Evaluations, Secure Computing Corporation.• The common criteria -- Good, bad or indifferent? , Information Security Technical Report, vol. 2, no. 1, pp. 5-6 PY

- 1997 AU - L.• Security evaluation and certification: The future of a national scheme. , Information Security Technical Report, vol.

2, no. 1, pp. 6-6 PY - 1997 AU - H.• Brewer, D. 2005, 'Is the Common Criteria the only way?', International Common Criteria Conference, Tokyo.• Oldegård, M. & Farnes, M.-L. 2005, 'Does Common Criteria need marketing?', International Common Criteria

Conference, Tokyo.• Mueller, S., 4/10/2006, ”Efficient Common Criteria Evaluations” :

http://www.atsec.com/downloads/pdf/efficient_cc_evaluations.pdf• Katzke, S. 2004, The Common Criteria (CC) Years (1993-2008): Looking Back and Ahead.• GAO. 2006, Information Assurance: National Partnership offers benefits, but Faces Considerable Challenges,

United States Government Accountability Office.• Government, Computer News, 2007/05/04, “Symantec: Common Criteria is bad for you” Jackson, J:

http://www.gcn.com/online/vol1_no1/44205-1.html• CCMB-2006-09-01 "Common Criteria for Information Technology Security Evaluation Version 3.1: Part 1:

Introduction and general model" The Common Criteria Management Board.

CC in the Real World

Documents

Transcript of CC in the Real World