Secure Programming Lecture 9: SecureDevelopment

David Aspinall, Informatics @ Edinburgh

15th October 2019

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

Recap

We’ve looked in detail at important vulnerabilityclasses:

É overflows, stack and heapÉ injections, command and SQLÉ race conditions

We’ve seen secure development processes from theoutside:

É vulnerability advisories, CVE classificationsÉ maturity model for secure software dev’t: BSIMM

It’s time to delve a bit more into secure developmentactivities included in BSIMM.

A Building Security In Process

We’ll look at a:

Secure Software Development Lifecycle (SSDLC)

due to Gary McGraw in his 2006 book SoftwareSecurity: Building Security In.

Work by McGraw and others has been combined in thebest practices called Building Security In used in BSIMM.This is promoted by the US-CERT.

To avoid debates over specific development processes,BSI indexes best practice activities. The activities relateto lifecycle stages.

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

McGraw’s Three Pillars

In Building Security In, Gary McGraw proposes three“pillars” to use throughout the lifecycle:

É I: Applied Risk ManagementÉ process: identify, rank then track risk

É II: Software Security TouchpointsÉ designing security ground up, not “spraying on”É seven security-related activities

É III: KnowledgeÉ knowledge as applied information about securityÉ e.g., guidelines or rules enforced by a toolÉ or known exploits and attack patterns

Security activities during development

How should secure development practices beincorporated into traditional software development?

0. treat security separately as a new activity (wrong)1. invent a new, security-aware process (another fad)2. run security activities alongside traditional

In business, “touchpoints” are places in a product/saleslifecycle where a business connects to its customers.

McGraw adapts this to suggest “touchpoints” in softwaredevelopment where security activities should interactwith regular development processes.

Security activities during lifecycleMcGraw identified 7 touchpoint activity areas,connecting to software development artefacts. Inlifecycle order:

É Abuse cases (in requirements)É Security requirements (in requirements)É Risk analysis (in design)É Risk-based security tests (in test planning)É Code review (in coding)É Risk analysis (in testing)É Penetration testing (in testing and deployment)É Security operations (during deployment)

His process modifies one adopted by Microsoft after theinfamous Trustworthy computing Gates Memo in 2002.

Exercise. For each touchpoint (detailed shortly),identify the development artefact(s) it concerns.

Touchpoints in the software developmentlifecycle

The numbers are a ranking in order of effectiveness.

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

1. Code review

Most effective touchpoint: eliminate problems at source.

Evidence since 1970s shows bugs are orders ofmagnitude cheaper to fix during coding than later in thelifecycle.

Industry is still learning this; code QA processes aren’t aswidely deployed as you might imagine.

Code review types

É Manual code reviewÉ can find subtle, unusual problemsÉ an onerous task, especially for large code basesÉ but adopted dev cycle in some agile processes (e.g.,

Google)

É Automatic static analysisÉ increasingly sophisticated tools automate scanningÉ very useful but can never understand code perfectlyÉ and may need human configuration, interpretation

Especially effective for simple bugs such as overflows.

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

2. Architectural risk analysis

Design flaws are not obvious from staring at code; theyneed to be identified in the design phase.

Architectural risk analysis considers security duringdesign:

É the security threats that attackers pose to assetsÉ vulnerabilities that allow threats to be realisedÉ the impact and probability of an attackÉ hence the risk, as risk = probability × impactÉ countermeasures that may be put into place

Example: poor protection of secret keys; risk is deemedhigh that attacker can read key stored on the filesystemand then steal encrypted document. A countermeasureis to keep encryption keys on dedicated USB tokens.

Risk analysis in general

É Several approaches:É financial loss oriented (cost versus damage)É mathematical (or pseudo-mathematical) risk ratingsÉ qualitative methods using previous knowledge

É If possible, should use specialist non-developersÉ requires understanding business impactÉ perhaps legal and regulatory frameworkÉ devs often strongly opinionated, fixed assumptions

É The (modern) Board-level viewÉ a Chief Risk Officer manages risk across

organisationÉ balance security with other types of risk

Common steps in risk analysis

1. Study system (specs, design docs, code if ready)2. Identify threats and attacker types/routes3. List possible vulnerabilities in the software4. Understand planned security controls (& risks. . . )5. Map attack scenarios (routes to exploit)6. Perform impact analysis7. Using likelihood estimates, rank risks8. Recommend countermeasures in priority/cost order

Particular risk analysis methods refine these.

In steps 2 and 3, may use checklists of threat types andpreviously known vulnerabilities; also general“goodness” guidelines.

Security design guidelines

Saltzer and Schroeder (1975)’s classic principles:

1. Economy of mechanism: keep it simple, stupid2. Fail-safe defaults: e.g., no single point of failure3. Complete mediation: check everything, every time4. Open design: assume attackers get the source & spec5. Separation of privilege: use multiple conditions6. Least privilege: no more privilege than needed7. Least common mechanism: beware shared resources8. Psychological acceptability: are security ops usable?

Exercise. If you haven’t studied these already, youshould review them in detail.

Microsoft STRIDE approach

STRIDE is mnemonic for categories of threats inMicrosoft’s method:

É Spoofing: attacker pretends to be someone elseÉ Tampering: attacker alters data or settingsÉ Repudiation: user can deny making attackÉ Information disclosure: loss of personal infoÉ Denial of service: preventing proper site operationÉ Elevation of privilege: user gains power of root user

Exercise. Recall the definitions of the classic CIAsecurity properties (confidentiality, integrity, availability).Explain which properties each threat type attacks.

The STRIDE approach

STRIDE uses Data Flow Diagrams to chase data througha system.

É Consider each data flow, manipulation, or storage:É Are there vulnerabilities of type S,T,R,I,D,E?É Are there routes to attack?

É Design mitigations (countermeasures)

STRIDE was designed as a developer-friendly mechanism

É devs may not know end user’s risk toleranceÉ so de-emphasises risk assessment, business impact

See MSDN magazine, Nov 2006.

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

3. Penetration testingCurrent dominant methodology (alongside bolt-onprotection measures, outside the lifecycle). Effectivebecause it considers a program in final environment.

É Finds real problemsÉ demonstrable exploits easily motivates repair costsÉ process “feels” good: something gets “better”

É Drawback: no accurate sense of coverageÉ ready made pen testing tools cover only easy bugsÉ system-specific architecture and controls ignored

Beware Dijkstra’s famous remark: Testing shows thepresence, not the absence of bugs. Just running somestandard pen-testing tools is a very minimal test.

Example: by feeding data to form elements, a browserplugin pen testing tool uncovers XSS vulnerabilities.

Bad use of pen testing

É Pen testing by external consultants is limitedÉ they may know tools but not system being testedÉ judgements about code limited (esp if black-box)

É Developers only patch to fix problems they’re toldaboutÉ Other patches may not be appliedÉ Patches can introduce new problemsÉ Patches often only fix symptom, not root cause

É Black box pen testing too limitedÉ Modern professional pen testing uses source

Good use of pen testing

McGraw advocates using pen testing:

É At the unit level, earlier in development:É automatic fault-injection with fuzzing tools

É Before deployment, as a last checkÉ not a first check for security, after deployment!É risk-based, focus on configuration and environment

É Metrics-driven: tracking problem reductionÉ not imagining zero=perfect securityÉ use exploits as regression tests

É For repairing software, not deploying work-arounds

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

4. Security testing

Security testing complements QA processes whichensure main functional requirements are error free.

É Test security functionalityÉ test security functions with standard methodsÉ consider them as part of main requirementsÉ write test cases for encryption key update

É Test security based on attack patterns oridentified abuse casesÉ apply risk analysis to prioritizeÉ consider attack patternsÉ test for injection vulns in environment vars

Traditional testing vs security testing

Traditional testing

Testers check a reasonably clear list of desiredbehaviours.

“The system shall. . . [do X, Y, and Z]”

É Explicit functional requirementsÉ check use cases, operate as expectedÉ customer can add/remove items from cart

É Sometimes explicit non-functional requirementsÉ check usability, performanceÉ user experience (UX) is pleasingÉ updating cart takes at most 5 seconds

Traditional testing vs security testing

Security testing

Testers check an unclear list of undesirable behavioursare absent.

“The system shall not. . . ”

É Rarely explicit non-functional non-requirementsÉ check undefined, unexpected behaviours impossibleÉ check safe recovery under abnormal conditionsÉ a negative size input doesn’t exhaust memoryÉ a web server crash doesn’t display debug info

A strategy for security testing

1. Understand the attack surface by enumerating:É program inputsÉ environment dependencies: hardware, software,

people, . . .

2. Use risk analysis outputs to prioritize componentsÉ often highest: code accessed by anonymous, remote

users

3. Work through attack patterns using fault-injection:É use manual input, fuzzers or proxies

4. Check for security design errorsÉ privacy of network trafficÉ controls on storage of data, ACLsÉ authenticationÉ random number generation

Automating security tests

Just as with functional testing, we can benefit frombuilding up suites of automated security tests.

1. Think like an attacker2. Design test suites to attempt malicious exploits3. Knowing system, try to violate specs/assumptions

This goes beyond random fuzz testing approaches.

Specially designed whitebox fuzz testing is successfulat finding security flaws (or, generating exploits).

One approach: use dynamic test generation, using symbolicexecution to generate inputs that reach error conditions (e.g., buffer

overflow).

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

5. Abuse cases

Idea: describe the desired behaviour of the systemunder different kinds of abuse/misuse.

É Work through attack patterns, e.g.É illegal/oversized input

É Examine assumptions made, e.g.É interface protects access to plain-text dataÉ cookies returned to server as they were sent

É Consider unexpected events, e.g.É out of memory error, disconnection of server

Specific detail should be filled out as for a use case.

Related idea: anti-requirements.

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

6. Security requirements

Security needs should be explicitly considered at therequirements stage.

É Functional security requirements, e.g.É use cryptography to protect sensitive stored dataÉ provide an audit trail for all financial transactionsÉ only gather essential data (privacy)

É Emergent security requirements, e.g.É recover on ill-formed input (avoid DoS)É minimise side channels (avoid confidentiality leaks)

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

7. Security operations

Security during operations means managing the securityof the deployed software.

Traditionally this has been the domain of informationsecurity professionals.

The idea of this touchpoint is to combine expertise ofinfosecs and devs.

This has become more critical recently with the rise ofdevops.

Information security professionals

Expert in:

É Incident handlingÉ Range and mechanisms of vulnerabilitiesÉ Understanding and deploying desirable patchesÉ Configuring firewalls, IDS, virus detectors, etc

But are rarely software experts.

Taking part in the development process can feed backknowledge from attacks, or join in security testing.

Infosec people understand pentesting from the outsideand less from inside. Network security scanners arecurrently more effective than application scanners.

Coders

Expert in:

É Software designÉ ProgrammingÉ Build systems, overnight testing

But rarely understand security in-the-wild.

Coders focus on the main product, easy to neglect thedeployment environment. E.g., VM host environmentmay be easiest attack vector.

OutlineOverview

Lifecycle security touchpoints

1. Code review and repair

2. Architectural risk analysis

3. Penetration testing

4. Risk-based security testing

5. Abuse cases

6. Security requirements

7. Security operations

Summary

Summary

This lecture outlined some SSDLC activities.

The descriptions were quite high-level. BSIMMdocuments over 100 activities used in real-worldSSDLCs.

Exercise. For each of the touchpoints, find specificdocumented examples of their use in a developmentprocess. (McGraw’s book has some, but there are plentyof other sources).

Exercise. Practice thinking about the touchpoints byconstructing scenarios. Consider the development of aparticular piece of software or a system. Imagine whatsome of the touchpoints might uncover or recommend.

Review questions

É Describe 5 secure development lifecycleactivities and the points in which they would beused in a compressed 4-stage agile developmentmethod (use case, design, code, test).

É What kinds of security problem is code review betterat finding compared with architectural risk analysis?

É Why is risk analysis difficult to do at the codinglevel?

É What is the main drawback of penetration testing,especially when it is applied as an absolute measureof security of a software system?

References and credits

Material in this lecture is adapted from

É Software Security: Building Security In, by GaryMcGraw. Addison-Wesley, 2006.

É The Art of Software Security Testing, by Wysopal,Nelson, Dai Zovi and Dustin. Addison-Wesley, 2007.

É Build Security In, the initiative of US-CERT athttps://buildsecurityin.us-cert.gov/.