Software Architectural Risk Analysis (SARA): SSAI Roadmap · – DRAFT Managing Risk from...
Transcript of Software Architectural Risk Analysis (SARA): SSAI Roadmap · – DRAFT Managing Risk from...
Frédéric PainchaudDRDC Valcartier / Systems of Systems
November 2010
Software Architectural Risk Analysis (SARA):SSAI Roadmap
1
Agenda
• Introduction
• Software Architectural Risk Analysis
– Linking to SSAI Activities
• Conclusion
2
Introduction
• Risk?
– “The net negative impact of the exercise of a vulnerability, considering both the probability and the impact of occurrence”.
– “A function of the likelihood of a given threat-source’s exercising a particular potential vulnerability, and the resulting impact of that adverse event on the organization”.
• Risk Management?
– “The process of identifying risk, assessing risk, and taking steps to reduce risk to an acceptable level”.
– Gary Stoneburner, Alice Goguen and Alexis Feringa, Risk Management Guide for Information Technology Systems, NIST, Special Publication 800-30.
3
Introduction
• The three processes of Risk Management:
– Risk assessment (aka Risk analysis)
– Risk mitigation
– Evaluation and assessment
• Why?
– To forecast potential problems;
– To develop and implement appropriate controls to avoid identified problems;
– To plan the actions to be taken if the controls go wrong, if uncontrolled identified problems arise (residual risk) and if unforeseen problems happen.
Risk assessmentR
isk mitigation
Evaluation and
assessment
4
Introduction
• Software Architectural Risk Analysis (SARA)?
– An adaptation of general Risk Assessment (Analysis), the first step of Risk Management.
• SARA and Risk Management in general are more an art than a science. Their processes are defined but practice and experiencetailor them to the particular organization.
5
Introduction
• Going back to Risk Management, for IT systems, it must be integrated into the Software Development Life Cycle (SDLC) to be effective.
• The risk management methodology is (roughly) the same regardless of the SDLC phase for which the assessment is being conducted.
6
Introduction
SDLC Phases Support from Risk Management activities
Phase 1: Initiation • System risk identification
• System requirements incl. security
• System CONOPSPhase 2: Development or acquisition • Architecture design
• Coding practices
– e.g., security (CERT C, ISO C)
• Testing
– e.g., security testingPhase 3: Implementation (in the environment)
• Validation w.r.t. the requirements and operational environment
Phase 4: Operation and maintenance • Continual evaluation and assessment
Phase 5: Disposal • Proper disposal and system migration
7
Introduction
• Participation in the risk management process is the business of many key players in organizations, including senior managers, business operations and IT procurement managers, IT security program managers and computer security officers, IT administrators and trainers.
8
Architectural Risk AnalysisSoftware Architectural Risk Analysis
Architecture documentation
Step 1. System characterization
One-page overview
Step 2. Threatidentification
Step 3. Vulnerabilityidentification
Step 4. Controlanalysis
History of system attacksSources of intelligence
Threat statement
Security testingresults
Vulnerability statement
Plannedcontrols
List of controls
Step 5. Attack likelihood determination
Attack likelihood rating
Step 6. Impactanalysis
Impact rating
Step 7. Risk determination
Step 8. Control recommendations
Step 9. Results documentation
Rated risks
Recommended controls or modifications
Architectural Risk Analysis Report
One-page overview
8
9
Dividing SARA Steps into Meetings
• Kick-Off Meeting (1/2 day):– Objective: First familiarization with the system to study (CSD
& CSD API)– Agenda:
• Overview of the SARA process• Overview of the meeting agenda• Introductory presentation of the CSD
– Meeting product: CD with introductory documentation and presentations on the CSD
– Post-meeting actions:• Prepare available documentation, source code and demo• Familiarize with the architecture and prepare questions
10
Step 1. System Characterization
Architecture documentation
Step 1. System characterization
One-page overview
• Mostly design diagrams• Questionnaires, on-site interviews, tools, …
• Update or create design diagrams (validate against operational system)• Merge the views and abstract the design levels to produce a one-page overview (ambiguity analysis can help)
The basis for the entire architectural risk analysis
11
Software Architecture Recovery Tools
• When your architectural documentation is incomplete, out-of-date or inexistent and when you have the source code, there are robust tools available to help.
• Refer to Philippe Charland’s presentation on the CD.
12
Software Architecture Recovery Tools
• These tools are useful when you have the source code. When you only have the binary, there are currently two choices:
1. you use decompilers to generate source code of varying quality from the binary and then you use these tools or
2. you manually analyze the binary with the help of specialized tools to accelerate the process.
• These tools are useful to accomplish step 1 of the Architectural Risk Analysis process. When it comes to managing the whole process, a solution like KDM Workbench is more appropriate.
13
Dividing SARA Steps into Meetings
• Meeting #1 (June 21-23):– Objective: Perform Step 1 – System characterization– Agenda:
• Complete the overview of the architecture and interfaces (complete global picture, CSD and CSD API)
• Demo of the system (CSD, CLIC, Webapps, TestClient)– Meeting products:
• 2 CDs with complete documentation and source code for the CSD and the CSD API
• Draft architecture– Post-meeting actions:
• Study available documentation, setup a network and deploy CSD• Refine the high-level architecture of the system• Study CAPEC in light of the understanding of the architecture and
select applicable attack patterns• Find and gather history of past attacks from operations
14
Step 2. Threat Identification
One-page overview
Step 2. Threatidentification
History of system attacksSources of intelligence
Threat statement
OPS, ASIC, Darknets/Blacknets, CAPEC,STRIDE, SANS Top Cyber Security Risks, …
• Misuse and abuse cases (add the time you take for use cases)• Attack resistance analysis
• Distributed architectures, dynamic code generation and interpretation, APIs across stateless protocols, rich internet applications, service-oriented architectures, …
Identifies threats, their level of motivation,their capacities and their likelihood
OPS: OperationsASIC: All-Source Intelligence CentreCAPEC: Common Attack Pattern Enumeration and Classification, http://capec.mitre.orgSTRIDE: Spoofing identity, Tampering with data, Repudiation, Information disclosure, Denial of service, Elevation of privilege,
http://msdn.microsoft.com/en-us/library/ee823878(CS.20).aspx
15PAGE 15
16
Dividing SARA Steps into Meetings
• Meeting #2 (November 9-10):
– Objective: Perform Step 2 – Threat identification
– Agenda:
• Presentation of past attacks
• Develop misuse and abuse cases, being inspired by high-level architecture and selected attack patterns
• Present misuse and abuse cases to lead architect and developer
– Meeting products:
• An Excel spreadsheet with the selected and prioritized attack patterns
• 9 broad misuse and abuse cases
– Post-meeting actions:
• Collect source code and prepare presentations for workshop with MARPAC/JTFP
17
Step 3. Vulnerability Identification
OPS, ASIC, Darknets/Blacknets, seven perniciouskingdoms, 19 deadly sins, OWASP Top 10, CWE,Open Source vulnerability database, CVE,questionnaires, on-site interviews, tools, …
Identifies vulnerabilities and their exploitability
One-page overview
Step 3. Vulnerabilityidentification
History of system attacksSources of intelligence
Security testingresults
Vulnerability statement
Assess evidences that controls are working properly
• Underlying framework weakness analysis• System’s dependencies
• Ambiguity analysis (in requirements and design)• Trust modeling (security zones)• Data sensitivity modeling (privacy and integrity)
Seven pernicious kingdoms: Brian Chess and Gary McGraw’s taxonomy, http://www.cigital.com/papers/download/bsi11-taxonomy.pdf19 deadly sins: Michael Howard’s list, http://blogs.msdn.com/michael_howard/archive/2005/07/11/437875.aspxOWASP: Open Web Application Security Project, http://www.owasp.org/index.php/Category:OWASP_Top_Ten_ProjectCWE: Common Weakness Enumeration, http://cwe.mitre.org/CVE: Common Vulnerabilities and Exposures, http://cve.mitre.org/
18PAGE 18
19
Step 4. Control Analysis
Must be considered for future assessments
Identifies current and future controls and their effectiveness
Assess evidences that controls are working properly
• Controls are technical or non-technical• Controls are preventive or detective• Ambiguity analysis (in requirements and design)
• Trust modeling (security zones)• Data sensitivity modeling (privacy and integrity)
One-page overview
Step 4. Controlanalysis
Security testingresults
Plannedcontrols
List of controls
20
Step 5. Attack Likelihood Determination
Identifies the likelihood of threats exercising vulnerabilities, that is, thelikelihood of attack scenarios
• No black magic: a function of threat and vulnerability likelihood, the latter being dependent on controls• Ambiguity analysis (in requirements and design)
• Threat modeling (attack surface)
Threat statement
Vulnerability statement
List of controls
Step 5. Attack likelihood determination
Attack likelihood rating
21
Step 5. Attack Likelihood Determination
Threat likelihoodHigh Medium Low
High High High Medium
Medium High Medium LowLow Medium Low Low
Vulnerability likelihood
22
Step 6. Impact Analysis
Identifies the magnitude of impacts
• Needs key players: senior management, business operations managers and IT security program managers• Interviews• Impacts can be measured quantitatively or qualitatively
• Examples: lost revenue, maintenance cost, loss of public confidence, …
One-page overview
Step 6. Impactanalysis
Impact rating
23
Step 7. Risk Determination
Identifies the risks with their associated levels
• Again, no black magic: a function of attack likelihood and impact• Associates attacks with impacts
Attack likelihood rating
Impact rating
Step 7. Risk determination
Rated risks
24
Step 7. Risk Determination
Attack likelihoodHigh Medium Low
High High High Medium
Medium High Medium LowLow Medium Low Low
Impact
25
Dividing SARA Steps into Meetings
• Meeting #3 (November 22-25, Technical SARA track: November 22-23):
– Objective (Technical SARA track): Perform Step 3 – Vulnerability identification
– Agenda:
• Test CSD with Fortify 360 SCA tool
• Analyze Fortify report with Audit Work Bench and Fortify 360 server
– Meeting product: A Fortify 360 report, tweaked and analyzed to eliminate false positives that can be eliminated automatically or quickly
– Post-meeting actions:
• Perform Step 4 – Control analysis, to find true and false positives from Fortify report
• Relate misuse and abuse cases to vulnerabilities (true positives)
• Perform Steps 5, 6 and 7 – Attack likelihood determination, impact analysis and risk determination
26
Step 8. Control Recommendations
• Risks prioritization (cost-benefit analysis)• For each risk, recommend one or more new controls or system modifications that will eliminate or mitigate that risk and are appropriate to the organization’s operations
Step 8. Control recommendations
Rated risks
Recommended controls or modifications
27
Step 9. Results Documentation
A report that describes the architecture, the identified threats, the risks with their associated levels, exploited vulnerabilities and impacts, and the final recommendations on controls and modifications to implement
Threat statement
Step 9. Results documentation
Rated risks
Recommended controls or modifications
Architectural Risk Analysis Report
One-page overview
28
Dividing SARA Steps into Meetings
• Meeting #4 (2 days):– Objectives:
• Perform Step 8 – Control recommendations• Prepare Step 9 – Results documentation• Overall debriefing
– Agenda:• Develop mitigation plans for priority misuse and abuse cases• Write final report’s table of contents• Debrief on the whole process
– Meeting products:• Mitigation plans• Final report’s table of contents
– Post-meeting actions:• Perform Step 9 – Results documentation
29
Software Architectural Risk Analysis and Risk Management in Practice
• Cigital’s SARA Methodology (mapped on the processes presented here)
• Microsoft’s
– STRIDE Threat Model (Spoofing identity, Tampering with data, Repudiation, Information disclosure, Denial of service, Elevation of privilege)
• http://msdn.microsoft.com/en-us/library/ee823878(CS.20).aspx
– Security Risk Management Guide
• http://technet.microsoft.com/en-us/library/cc163143.aspx
• Sun’s Adaptive Countermeasure Selection Mechanism/Security Adequacy Review (ACSM/SAR)
– Discussed in Secure Coding: Principles and Practices, Mark G. Graff & Kenneth R. van Wyk, 2003, ISBN 0-596-00242-4.
30
Software Architectural Risk Analysis and Risk Management in Practice• Department of Homeland Security “Build Security In” website,
https://buildsecurityin.us-cert.gov/
• NIST’s
– Recommended Security Controls for Federal Information Systems and Organizations, Special Publication 800-53 Rev. 3, August 2009.
– DRAFT Managing Risk from Information Systems: An Organizational Perspective, Special Publication 800-39, April 2008.
– DRAFT Guide for Applying the Risk Management Framework to Federal Information Systems: A Security Life Cycle Approach, Special Publication 800-37 Rev. 1, November 2009.
– Risk Management Guide for Information Technology Systems, Special Publication 800-30, July 2002.
– http://csrc.nist.gov/publications/PubsSPs.html
31
Software Architectural Risk Analysis and Risk Management in Practice
• SEI’s Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE)
– http://www.cert.org/octave/
• ISACA’s Control Objectives for Information and Related Technology (COBIT)
– http://www.isaca.org/Template.cfm?Section=COBIT6&Template=/TaggedPage/TaggedPageDisplay.cfm&TPLID=55&ContentID=7981
SEI: Software Engineering InstituteISACA: Information Systems Audit and Control Association
32
Conclusion
• Do not forget:– The threats and the system change over time: SARA and risk management
are ongoing and evolving– Plan for re-evaluation and re-assessment
• Keys for success in SARA:– Senior management’s commitment
• You need time and effort from the key people in the team (lead architect & developer)
• Since SARA should be ongoing, it needs to be integrated in the development cycle
– Competence required in the risk analysis team:• Software architectural risk analysis process, with the flexibility needed
to adapt• Threats and vulnerabilities (the attacker’s perspective)• System and controls (the defender’s perspective)
33