Tracking the Progress of an SDL Program

Lessons from the GymCassio Goldschmidt

June 29th, 2009

2

Introduction

3

Who am I?

Cassio Goldschmidt– Sr. Manager, Product Security

– Chapter Leader, OWASP Los Angeles

• Education– MBA, USC

– MS Software Engineering, SCU

– BS Computer Science, PUCRS

– Certified Software Sec. Lifecycle Professional – CSSLP, (ISC)2

• When I’m not in the office…– Volleyball (Indoor, Beach)

– Coding

– Gym…

4

Typical Project Lifecycle

DESIGN CODE TEST SUPPORT

5

How your workout looks like

Exercise: Pile Squat

Repetitions: 35

Weight: 20 lbs

May 13th Workout

Exercise: Barbell Squat

Repetitions: 35

Weight: 150 lbs

Exercise: Rev. Curl

Repetitions: 20

Weight: 25 lbs

6

How your METRICS should look like

Exercise: Pile Squat

Repetitions: 35

Weight: 20 lbsMay 13

th Sec. Metrics

Exercise: Barbell Squat

Repetitions: 35

Weight: 150 lbs

Exercise: Rev. Curl

Repetitions: 20

Weight: 25 lbs

Exercise type:CWE

7

How your METRICS should look like

CWE: 79 - XSS

Repetitions: 35

Weight: 20 lbsMay 13

th Sec. Metrics

Exercise: Barbell Squat

Repetitions: 35

Weight: 150 lbs

Exercise: Rev. Curl

Repetitions: 20

Weight: 25 lbs

Number of Reps:Number of Findings

8

How your METRICS should look like

CWE: 79 - XSS

Findings: 10

Weight: 20 lbsMay 13

th Sec. Metrics

Exercise: Barbell Squat

Repetitions: 35

Weight: 150 lbs

Exercise: Rev. Curl

Repetitions: 20

Weight: 25 lbs

Exercise Intensity:CVSS

9

How your METRICS should look like

CWE: 20 – Input Val

Findings: 1

CVSS: 8.6

May 13th Sec. Metrics

CWE: 79 - XSS

Findings: 3

CVSS:

CWE: 314

Findings: 1

CVSS: 2.3

DESIGN

Threat Model

TEST

Pen Test

Support

Vul. Mgmt

Common Weakness Enumeration

11

Common Weakness EnumerationWhat is it?

• A common language for describing software security weaknesses

• Maintained by the MITRE Corporation with support from the National Cyber Security Division (DHS).

• Hierarchical– Each individual CWE represents a single vulnerability type

– Deeper levels of the tree provide a finer granularity

– Higher levels provide a broad overview of a vulnerability

12

Common Weakness EnumerationPortion of CWE structure

13

Common Weakness EnumerationWhat data is available for each CWE?

• Weakness description• Applicable platforms and programming languages• Common Consequences• Likelihood of Exploit• Coding Examples• Potential Mitigations• Related Attacks• Time of Introduction• Taxonomy Mapping

Link to CWE Page on XSS

14

Common Weakness Enumeration How useful is this information?

Pie Chart showing the frequency of CWEsfound in penetration tests

Common Vulnerability Scoring System

16

• Objective (and “perfect enough”) metric• A universal way to convey vulnerability severity

– Can be used for competitive analysis

• CVSS score ranges between 0.0 and 10.0– Can be expressed as high, medium, low as well

• Composed of 3 vectors– Base

• Represents general vulnerability severity: Intrinsic and immutable

– Temporal• Time-dependent qualities of a vulnerability

– Environmental• Qualities of a vulnerability specific to a particular IT environment

Common Vulnerability Scoring System (CVSS)What is it?

17

Common Vulnerability Scoring System (CVSS)BASE Vector

Access Vector

Access Complexity

Authenti…

Network High None

Adjacent Network

Medium Single Instance

Local Low Mult. Instances

Undefined Undefined Undefined

Confident… Integrity Availability

None None None

Partial Partial Partial

Complete Complete Complete

Undefined Undefined Undefined

Exploitability Impact

• Sample Score: 7.5• Sample Vector: (AV:N/AC:L/Au:N/C:P/I:P/A:P)• Every CVSS score should be accompanied by the

corresponding vector

18

Common Vulnerability Scoring System (CVSS)The Calculator

Training and Metrics.

20

Training and MetricsA special activity in the SDL

• Security training is what food is to a workout• Same workout metrics do not apply• Quality of your intake affects overall performance• Staff needs ongoing training

21

Training and Metrics Security Learning Process

22

Training and Metrics Security Learning ProcessUnderstand who is the audience

• Previous knowledge about secure coding and secure testing

• Programming languages in use• Supported platforms• Type of product

23

Training and Metrics Security Learning Process

Train everyone involved in the SDL• Developers: Secure Coding, Threat Model• QA: Security Testing, Tools• Managers: Secure Development Lifecycle (also known as Symmunize)

24

Training and Metrics Security Learning ProcessQuality Assurance - Capture the flag

• Use Beta software• Approximately 3 hours long• Top 3 finders receive prizes and are invited

to explain what techniques and tools they used to find the vulnerabilities to the rest of the group

25

Training and Metrics Security Learning Process Pos Class Survey

• Anonymous• Metrics

• Class content • Instructor knowledge • Exercises

26

Training and Metrics Security awareness is more than training

Knowledge Sharing Activities

Tech Exchanges

Cutting Edge

CTO Newsletter Articles

Conclusions and final thoughts

28

Why This Approach Makes Sense?

DESIGN CODE TEST SUPPORT

• Compare Apples to Apples• Quantify results in a meaningful way to “C” executives

– Past results can be used to explain impact of new findings

– Can be simplified to a number from 1-10 or semaphore (green, yellow and red).

– Can be used for competitive analysis• Harder to game CVSS• CWE can be easily mapped to different taxonomies

Copyright © 2007 Symantec Corporation. All rights reserved.  Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries.  Other names may be trademarks of their respective owners.This document is provided for informational purposes only and is not intended as advertising.  All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law.  The information in this document is subject to change without notice.

Thank You!

Cassio Goldschmidt

cassio_goldschmidt@symantec.com

cassio@owasp.org