Title of Presentation - UCAIugosgug.ucaiug.org/utilisec/Shared Documents/Presentation… · PPT...
Transcript of Title of Presentation - UCAIugosgug.ucaiug.org/utilisec/Shared Documents/Presentation… · PPT...
© 2011 Carnegie Mellon University
CERT® Resilience Management Model
CERT-RMM Overview
David WhiteCERT Resilient Enterprise Management Team
2© 2011 Carnegie Mellon University
NoticesNO WARRANTY
THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
Use of any trademarks in this presentation is not intended in any way to infringe on the rights of the trademark holder.
This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected].
This work was created in the performance of Federal Government Contract Number FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The Government of the United States has a royalty-free government-purpose license to use, duplicate, or disclose the work, in whole or in part and in any manner, and to have or permit others to do so, for government purposes pursuant to the copyright license under the clause at 252.227-7013.
3© 2011 Carnegie Mellon University
CERT | Software Engineering Institute | Carnegie Mellon
Software Engineering Institute (SEI)• Federally funded research and development center
based at Carnegie Mellon University
• Basic and applied research in partnership with government and private organizations
• Helps organizations improve development, operation, and management of software-intensive and networked systems
CERT – Anticipating and solving our nation’s cybersecurity challenges• Largest technical program at SEI
• Focused on internet security, secure systems, operational resilience, and coordinated response to security issues
4© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
5© 2011 Carnegie Mellon University
Operational resilience and operational risk Setting context
6© 2011 Carnegie Mellon University
Operational resilience defined
Resilience: The physical property of a material when it can return to its original shape or position after deformation that does not exceed its elastic limit [wordnet.princeton.edu]
Operational resilience: The emergent property of an organization that can continue to carry out its mission in the presence of operational stress and disruption that does not exceed its limit[CERT-RMM]
Where does the stress and disruption come from? Risk.
7© 2011 Carnegie Mellon University
Operational resilience and operational risk
Operational resilience emerges from effective operational risk managementOperational risk categories:
Actions of people
Systems and
technology failures
Failed internal
processes
External events
8© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
9© 2011 Carnegie Mellon University
CERT® Resilience Management Model (CERT-RMM) A platform for improvement and measurement
10© 2011 Carnegie Mellon University
What is CERT®-RMM?
CERT-RMM is a capability model for managing and improving operational resilience.
“…an extensive super-set of the things an organization could do to be more resilient.”
- CERT-RMM adopter
• Guides implementation and management of operational resilience activities
• Converges key operational risk management activities: security, BC/DR, and IT operations
• Defines maturity through capability levels (like CMMI)
• Enables measurement
• Improves confidence in how an organization responds in times of operational stress
11© 2011 Carnegie Mellon University
Imperatives for building CERT-RMM
Increasingly complex operational environments; traditional approaches failing
Siloed nature of operational risk activities; a lack of convergence
Lack of common language or taxonomy
Overreliance on technical approaches
Lack of means to measure organizational capability
Inability to confidently predict outcomes, behaviors, and performance under times of stress
Tech reliance
Global economy
Open boundaries
Complexity
Cultural shifts
12© 2011 Carnegie Mellon University
CERT-RMM background
CERT-RMM
800+ practices for security, BC, & IT ops
Collaboration with high maturity organizations
20+ years of security mgmt knowledge at CERT
CMMI architecture and experience
Piloting in private and government organizations
13© 2011 Carnegie Mellon University
Organizational context
Four asset types:• People – the human capital of the organization• Information – data, records, knowledge in physical or digital form• Technology – software, systems, hardware, network• Facilities – offices, data centers, labs – the physical places
Service Mission
Service Mission
people information technology facilities
Productive ActivitiesService Mission
Organization Mission
Serv
ice
Assets in Production
14© 2011 Carnegie Mellon University
Organizational context - disruption
Service Mission
Service Mission
people info tech facilities
Service Mission
Organization Mission
Serv
ice
XX X
XX
Operational risk can disrupt an asset
And lead to organizational disruption
15© 2011 Carnegie Mellon University
Security Domain BC/DR Domain
tech
Building resilience at the asset level
SustainProtect
Protection strategies Keep assets from
exposure to disruptionTypically implemented as
“security” activities
Sustainment strategies Keep assets productive
during adversityTypically implemented as
“business continuity” activities
16© 2011 Carnegie Mellon University
Security Domain BC/DR Domain
tech
Manage Condition Manage Consequence
Building resilience at the asset
SustainProtect
Manage Risk
The optimal “mix” of these strategies depends on the value of the asset and the cost of deploying and maintaining the strategy.
17© 2011 Carnegie Mellon University
Organizational context
Service Mission
Service Mission
people info tech facilities
Service Mission
Organization Mission
Serv
ice
SustainProtectSustainProtectSustainProtect SustainProtect
Operational Resilience Management System
CERT-RMMfocuses here
18© 2011 Carnegie Mellon University
Asset in Production
Resilience management in the life cycle
Resilience management covers the life cycle of an asset.
Operational resilience management focuses on the deploy, operate, and decommission phases, but must reach back to address issues during development.
Plan Operate RetireDeploy
Acquire
Develop
Design
19© 2011 Carnegie Mellon University
Plan Operate RetireDeploy
Acquire
Develop
Design
CMMI-DEV
CMMI-ACQ
CERT-RMM
CMMI-SVC
DEVELOPMENT OPERATION
CERT-RMM position in life cycle
20© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
21© 2011 Carnegie Mellon University
CERT-RMM ArchitectureHow the model is put together
22© 2011 Carnegie Mellon University
CERT-RMM: 26 process areas in 4 categories
EngineeringADM Asset Definition and Management
CTRL Controls Management
RRD Resilience Requirements Development
RRM Resilience Requirements Management
RTSE Resilient Technical Solution Engineering
SC Service Continuity
Enterprise ManagementCOMM Communications
COMP Compliance
EF Enterprise Focus
FRM Financial Resource Management
HRM Human Resource Management
OTA Organizational Training & Awareness
RISK Risk Management
Operations ManagementAM Access Management
EC Environmental Control
EXD External Dependencies Management
ID Identity Management
IMC Incident Management & Control
KIM Knowledge & Information Management
PM People Management
TM Technology Management
VAR Vulnerability Analysis & Resolution
Process ManagementMA Measurement and Analysis
MON Monitoring
OPD Organizational Process Definition
OPF Organizational Process Focus
23© 2011 Carnegie Mellon University
Focused Activity
What to do to achievethe capability
How to accomplish the goal
How to implement the practicePoints of connection to other practice bodies
CERT-RMM process area architecture
ProcessArea
SpecificGoals
Specific Practices
Sub-practices
Maturity Elements
Three Generic Goals
GenericPractices
Sub-practices
24© 2011 Carnegie Mellon University
CERT-RMM links to codes of practice
ProcessArea
SpecificGoals
Specific Practices
Sub-practices
Codes of Practice:BS25999-1:2006CMMI v1.2CMMI for ServicesCobiT 4.1COSO ERMDRII GAPFFIEC Handbooks (Security, BCP) ISO 20000-2:2005(E) (ITIL-related)ISO 24762:2008(E)ISO 27002:2005NFPA 1600 (2007)PCI DSS v1.1Val-IT
25© 2011 Carnegie Mellon University
CERT-RMM numbers
4Categories
26Process Areas
251Specific
Practices
13Generic
Practices per process area
94Specific Goals
3Generic Goals
per process area
26© 2011 Carnegie Mellon University
Where to start
To use the model, start by selecting any number of process areas (or even parts of process areas) that align with your objectives.
Starting with 1 process area or a few specific goals is completely acceptable.
There is no requirement to use the entire model—use whatever parts of the model make sense for your situation.
27© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
28© 2011 Carnegie Mellon University
Measuring maturity — the CERT-RMM capability dimensionMeasuring process institutionalization to determine capability under stress
29© 2011 Carnegie Mellon University
Institutionalization
What does institutionalization look like?
It describes when something has become ingrained in the way an organization operates.
”institutionalize.” Dictionary.cambridge.org Advanced Learner's Dictionary. Cambridge University Press. 14 Sep. 2010. <http://dictionary.cambridge.org/dictionary/british/institutionalize_2>.
30© 2011 Carnegie Mellon University
Practices are performed
Process institutionalization in CERT-RMM
Processes are acculturated,
defined, measured,
and governed
Level 3
• Defined
Level 2
• Managed
Level 1
• Performed
Level 0
• Incomplete
Capability levels are used in CERT-RMM to measure process institutionalization
Practices are incomplete
Higher degrees of institutionalization translate to more stable processes that
• produce consistent results over time
• are retained during times of stress
31© 2011 Carnegie Mellon University
Capability Levels and Generic Goals
Capability levels apply independently to each process area• An organization could target level 1 in one process area and level 3
in another• Provides for very flexible application of the model
Generic goals define capability levels:
To achieve: An organization must satisfy:
Capability Level 1 Generic Goal 1
Capability Level 2 Generic Goals 1 and 2
Capability Level 3 Generic Goals 1, 2, and 3
32© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
33© 2011 Carnegie Mellon University
COMP: ComplianceOne process area in-depth
34© 2011 Carnegie Mellon University
COMP – Compliance process area
Purpose: ensure awareness of and compliance with an established set of relevant internal and external guidelines, standards, practices, policies, regulations, and legislation, and other obligations (such as contracts and service level agreements) related to managing operational resilience
Collect once — comply many times• Data collection is one of the most expensive activities for compliance• Understand intersecting requirements to leverage compliance data • Develop a compliance knowledgebase with strong data validation
35© 2011 Carnegie Mellon University
COMP:SG4Monitor compliance activities
SG4.SP1: Evaluate compliance activities
COMP:SG3Demonstrate satisfaction of compliance obligations
SG3.SP1: Collect and validate compliance data
SG3.SP2: Demonstrate the extent of compliance obligation satisfactionSG3.SP3: Remediate areas of non-compliance
Compliance: specific goals & practices
COMP:SG2Establish compliance obligations
SG2.SP1: Identify compliance obligations
SG2.SP2: Analyze obligations
SG2.SP3: Establish ownership for meeting obligations
Specific Goals Specific PracticesCOMP:SG1Prepare for compliance management
SG1.SP1: Establish a compliance plan
SG1.SP2: Establish a compliance program
SG1.SP3: Establish compliance guidelines and standards
36© 2011 Carnegie Mellon University
Achieving capability level 1 in COMPGeneric Goals Generic PracticesGG1 Achieve Specific Goals GG1.GP1 Perform Specific Practices
Achieve capability level 1 by satisfying generic goal 1, which means:
• Perform the COMP specific practices (all 10 of them) so that you • Satisfy the COMP specific goals (all 4 of them)
√ √
37© 2011 Carnegie Mellon University
GG2 Institutionalize a Managed Process
GG2.GP1 Establish Process Governance
GG2.GP2 Plan the Process
GG2.GP3 Provide Resources
GG2.GP4 Assign Responsibility
GG2.GP5 Train People
GG2.GP6 Manage Work Product Configurations
GG2.GP7 Identify and Involve Relevant Stakeholders
GG2.GP8 Monitor and Control the Process
GG2.GP9 Objectively Evaluate Adherence
GG2.GP10 Review Status with Higher Level Managers
Achieving capability level 2 in COMPGeneric Goals Generic PracticesGG1 Achieve Specific Goals GG1.GP1 Perform Specific Practices√ √
√ √
√
√
√
√
√
√
√
√
√
Achieve capability level 1 plus satisfy generic goal 2 by performing the associated 10 generic practices.
38© 2011 Carnegie Mellon University
GG3 Institutionalize a Defined Process
GG3.GP1 Establish a Defined Process
GG3.GP2 Collect Improvement Information
GG2 Institutionalize a Managed Process
GG2.GP1 Establish Process Governance
GG2.GP2 Plan the Process
GG2.GP3 Provide Resources
GG2.GP4 Assign Responsibility
GG2.GP5 Train People
GG2.GP6 Manage Work Product Configurations
GG2.GP7 Identify and Involve Relevant Stakeholders
GG2.GP8 Monitor and Control the Process
GG2.GP9 Objectively Evaluate Adherence
GG2.GP10 Review Status with Higher Level Managers
Achieving capability level 3 in COMPGeneric Goals Generic PracticesGG1 Achieve Specific Goals GG1.GP1 Perform Specific Practices√ √
√ √
√
√
√
√
√
√
√
√
√
√ √
√
39© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
40© 2011 Carnegie Mellon University
SC: Service ContinuityOne process area in-depth
41© 2011 Carnegie Mellon University
SC – Service Continuity
Purpose:To ensure the continuity of essential operations of services and related assets if a disruption occurs as a result of an incident, disaster, or other disruptive event.
Contains• 7 specific goals
• 20 specific practices
• ~40 pages
42© 2011 Carnegie Mellon University
SG3 Develop Service Continuity Plans
SG3.SP1 Identify Plans to be Developed
SG3.SP2 Develop and Document Service Continuity Plans
SG3.SP3 Assign Staff to Service Continuity Plans
SG3.SP4 Store and Secure Service Continuity Plans
SG3.SP5 Develop Service Continuity Plan Training
SG2 Identify and Prioritize High-Value Services
SG2.SP1 Identify the Organization’s High-Value Services
SG2.SP2 Identify Internal and External Dependencies and Interdependencies
SG2.SP3 Identify Vital Organizational Records and Databases
SC specific goals 1-3 and practicesSpecific Goals Specific PracticesSG1 Prepare for Service
ContinuitySG1.SP1 Plan for Service Continuity
SG1.SP2 Establish Standards and Guidelines for Service Continuity
43© 2011 Carnegie Mellon University
SG7 Maintain Service Continuity Plans
SG7.SP1 Establish Change Criteria
SG7.SP2 Maintain Changes to Plans
SG6 Execute Service Continuity Plans
SG6.SP1 Execute Plans
SG6.SP2 Measure the Effectiveness of the Plans in Operation
SG5 Exercise Service Continuity Plans
SG5.SP1 Develop Testing Program and Standards
SG5.SP2 Develop and Document Test Plans
SG5.SP3 Exercise Plans
SG5.SP4 Evaluate Plan Test Results
SC specific goals 4-7 and practicesSpecific Goals Specific PracticesSG4 Validate Service
Continuity PlansSG4.SP1 Validate Plans to Requirements and Standards
SG4.SP2 Identify and Resolve Plan Conflicts
44© 2011 Carnegie Mellon University
Achieving capability level 1 in SCGeneric Goals Generic PracticesGG1 Achieve Specific Goals GG1.GP1 Perform Specific Practices
Achieve capability level 1 by satisfying generic goal 1, which means:
• Perform the SC specific practices (all 20 of them) so that you • Satisfy the SC specific goals (all 7 of them)
√ √
45© 2011 Carnegie Mellon University
GG2 Institutionalize a Managed Process
GG2.GP1 Establish Process Governance
GG2.GP2 Plan the Process
GG2.GP3 Provide Resources
GG2.GP4 Assign Responsibility
GG2.GP5 Train People
GG2.GP6 Manage Work Product Configurations
GG2.GP7 Identify and Involve Relevant Stakeholders
GG2.GP8 Monitor and Control the Process
GG2.GP9 Objectively Evaluate Adherence
GG2.GP10 Review Status with Higher-Level Managers
Achieving capability level 2 in SCGeneric Goals Generic PracticesGG1 Achieve Specific Goals GG1.GP1 Perform Specific Practices√ √
√ √
√
√
√
√
√
√
√
√
√
Achieve capability level 1 plus satisfy generic goal 2 by performing the associated 10 generic practices.
46© 2011 Carnegie Mellon University
GG3 Institutionalize a Defined Process
GG3.GP1 Establish a Defined Process
GG3.GP2 Collect Improvement Information
GG2 Institutionalize a Managed Process
GG2.GP1 Establish Process Governance
GG2.GP2 Plan the Process
GG2.GP3 Provide Resources
GG2.GP4 Assign Responsibility
GG2.GP5 Train People
GG2.GP6 Manage Work Product Configurations
GG2.GP7 Identify and Involve Relevant Stakeholders
GG2.GP8 Monitor and Control the Process
GG2.GP9 Objectively Evaluate Adherence
GG2.GP10 Review Status with Higher-Level Managers
Achieving capability level 3 in SCGeneric Goals Generic PracticesGG1 Achieve Specific Goals GG1.GP1 Perform Specific Practices√ √
√ √
√
√
√
√
√
√
√
√
√
√ √
√
47© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
48© 2011 Carnegie Mellon University
Using CERT-RMMA process for improvement
49© 2011 Carnegie Mellon University
Using CERT-RMM for improvement
Recognize Objective
Determine Scope
Identify Gaps
Analyze Gaps
Implement Changes
Evaluate Results
50© 2011 Carnegie Mellon University
Recognizing objectives
Objectives frame and provide context
Answer the question: What are we trying to accomplish with the improvement effort?
Typical themes:• Are we doing all that we should to manage business continuity (or
security, IT ops, or a combination)?• How can we minimize the potential disruption from <some known
risk or category of risk>?• How can we improve the efficiency, effectiveness, or consistency of
our operational risk management activities (security, BC, & IT ops)?• Do our policies and guidelines produce the risk management
activities that we want them to? How can we improve policy?
Recognize Objective
Determine Scope
Identify Gaps
Analyze Gaps
Implement Changes
Evaluate Results
51© 2011 Carnegie Mellon University
Determining scope
Two elements:• Organizational scope:
On which part of the organization will we focus?
• Model scope:Which parts of the CERT-RMM will we use?— Whole process areas (1-6 typically)— Parts of process areas (a set of practices)
Both elements should align with objectives and sponsorship
Model scoping can be easily accomplished by walking the model outline in a small workshop or meeting
Recognize Objective
Determine Scope
Identify Gaps
Analyze Gaps
Implement Changes
Evaluate Results
52© 2011 Carnegie Mellon University
Organizational scope
Where, in the organization, process improvement will be focused
Must consider• Span of sponsorship
developed in Initiating phase• Span of authority of the
improvement team• Schedule feasibility for
desired improvements
Determine Scope
53© 2011 Carnegie Mellon University
1
1.1
1.1.1
1.1.1.1
1.1.2
1.1.2.1
1.1.2.2
1.2
1.2.1
1.2.1.1
1.3
1.3.1
1.3.1.1
1.3.2
1.3.2.1
1.3.2.2
1.3.3
1.3.3.1
Organizational scoping example -1
Organizational Unit
Suppose that we are performing process improvement on the part of the organization defined by 1.3 and its subunits
First, we have to understand where the CERT-RMM practices are performed or designate where they will be performed
Determine Scope
54© 2011 Carnegie Mellon University
Model scope
Determines which areas of the model will be selected for process improvement
When selecting, consider process areas that• May be causing “pain” or perceived weakness
• Align with regulatory or industry initiatives and objectives
• Align with organizational objectives or initiatives
• Support other organizational process improvement initiatives such as Six Sigma or ITIL
• Explore areas in which the organization needs to develop competency
Determine Scope
55© 2011 Carnegie Mellon University
CERT-RMM model scope in detail -1Model Scope
Process Areas
Capability Level Targets
Asset Scope
People
Information
Technology
Facilities
Resilience Scope
Business Continuity
Security
IT Operations
Determine Scope
56© 2011 Carnegie Mellon University
CERT-RMM model scope in detail -2Model Scope
Process Areas
Capability Level Targets
Asset Scope
People
Information
Technology
Facilities
Resilience Scope
Business Continuity
Security
IT Operations
Fine-grained model scoping options
Determine Scope
57© 2011 Carnegie Mellon University
PA-level scope example
None
None
Information security incidents only
Information security compliance only
Information and technology assets onlyADM
IMC
TM
KIM
COMP
0 1 2 3
Capability Profile Scoping Caveats
Determine Scope
58© 2011 Carnegie Mellon University
CERT-RMM model scope in detail -3Model Scope
Specific & Generic Practices
Capability Level Targets
Asset ScopePeople
Information
Technology
Facilities
Resilience ScopeBusiness Continuity
Security
IT Operations
Fine-grained model scoping options
Determine Scope
59© 2011 Carnegie Mellon University
Practice-level scope example
Example scope for IT Disaster Recovery activities.
Determine Scope
Note: PAs with no selected practices are hidden.
60© 2011 Carnegie Mellon University
Identifying gaps
Methods:
Rigorous: CERT-RMM Capability Appraisals• Three classes: A (most rigorous), B, and C (least)• Outputs include detailed practice-level characterizations and written
findings statements
Lightweight: CERT-RMM Compass• Questionnaire-based gap analysis instrument from CERT• In development now
Informal: gap analysis roundtable or workshop• Assemble a group of internal experts• Informally evaluate the organization’s implementation of the model
practices in a workshop setting
Recognize Objective
Determine Scope
Identify Gaps
Analyze Gaps
Implement Changes
Evaluate Results
61© 2011 Carnegie Mellon University
CERT-RMM appraisal comparison
ProcessArea
SpecificGoals
Specific Practices
Generic Goals
GenericPractices
Appraisal team:
Depth of investigation:
Resource requirements:
Class ACapability Level
Ratings(0, 1, 2, or 3)
Goal Ratings(Satisfied or Not Satisfied)
Characterization of implementation on
5-point scale (Fully, Largely, Partially,
Not, Not Yet Implemented)
Findings statements (strengths & weaknesses)
4 or more
High
High
Class B
--
--
Characterization of approach on 3-point scale
(High, medium, low)
Statements (strength/weakness)
2 or more
Medium
Medium
Class C
--
--
Characterization of intent on 3-point scale
(High, medium, low)
Statements (strength/weakness)
1 or more
Low
Low
Mod
el-R
elat
ed O
utpu
tsE
ffor
t
Identify Gaps
62© 2011 Carnegie Mellon University
CERT-RMM appraisal comparison
ProcessArea
SpecificGoals
Specific Practices
Generic Goals
GenericPractices
Appraisal team:
Depth of investigation:
Resource requirements:
Class ACapability Level
Ratings(0, 1, 2, or 3)
Goal Ratings(Satisfied or Not Satisfied)
Characterization of implementation on
5-point scale (Fully, Largely, Partially,
Not, Not Yet Implemented)
Findings statements (strengths & weaknesses)
4 or more
High
High
Class B
--
--
Characterization of approach on 3-point scale
(High, medium, low)
Statements (strength/weakness)
2 or more
Medium
Medium
Class C
--
--
Characterization of intent on 3-point scale
(High, medium, low)
Statements (strength/weakness)
1 or more
Low
Low
Mod
el-R
elat
ed O
utpu
tsE
ffor
t
Identify Gaps
May be scoped at the practice level
Scoped at the process area level
64© 2011 Carnegie Mellon University
Sample class B/C scope
Example scope for IT Disaster Recovery activities.
Note: PAs with no practices in scope are hidden.
Identify Gaps
65© 2011 Carnegie Mellon University
Sample class B/C appraisal output
For IT Disaster Recovery activities:
Note: PAs with no practices in scope are hidden.
Identify Gaps
66© 2011 Carnegie Mellon University
Sample class A appraisal output Identify Gaps
67© 2011 Carnegie Mellon University
Sample class A appraisal output Identify GapsClass A appraisals must be scoped to include full process areas
Class A appraisals include goal ratings
Class A appraisals include Capability Level ratings. These results would yield Capability Level 0 because at least one specific goal is not satisfied.
68© 2011 Carnegie Mellon University
Sample appraisal findings
Strengths• The service continuity testing program is complete, rigorous, well-
implemented, consistently-followed, and provides valuable feedback for the improvement of preparedness activities across the organization.
• …
Weaknesses• Internal dependencies are well-identified in support of service
continuity planning, but external dependencies are not.• While service continuity plans are being executed appropriately in
the organization, no evidence was provided to show that plans are being evaluated for their effectiveness in operation.
• …
Identify Gaps
69© 2011 Carnegie Mellon University
Sample appraisal findings
Strengths• The service continuity testing program is complete, rigorous, well-
implemented, consistently-followed, and provides valuable feedback for the improvement of preparedness activities across the organization.
• …
Weaknesses• Internal dependencies are well-identified in support of service
continuity planning, but external dependencies are not.• While service continuity plans are being executed appropriately in
the organization, no evidence was provided to show that plans are being evaluated for their effectiveness in operation.
• …
Identify Gaps
Findings statements are generated for class A, B, and C appraisals
Findings statements are agreed by consensus of the full appraisal team
70© 2011 Carnegie Mellon University
Appraisal processPreparation Onsite Reporting
Lead Appraiser
• Develops appraisal plan
• Trains appraisal team• Coaches and
monitors evidence preparation*
• Plans and schedules interviews
Appraisal team:• Reviews evidence
(may collect additional evidence)
• Performs interviews• Characterizes
practices by consensus
Appraisal team:• Presents final findings
to sponsor – typically in MS Powerpoint
• Optionally produces a written report which may include detailed recommendations
Customer • Collects and prepares evidence*
• Supports interviews and additional evidence collection
Identify Gaps
* Evidence collection in advance of the onsite is the most efficient appraisal process, but may require substantial effort by the customer – this mode is called “verification.” Alternatively, the evidence can be collected during the onsite period in a mode called “discovery.”
71© 2011 Carnegie Mellon University
Analyzing gaps
To make sure that closing gaps makes sense,gaps should be analyzed:
• Is the cost for closing a gap worth the investment?
• Are there any efficiencies that can be realized by making the changes to close one or more gaps (efficiencies may include streamlining controls or compliance activities)?
• Which gaps are most important in the context of the objective?
• Are the organizational changes necessary to close the gaps within the bounds of sponsorship?
Output is a set of prioritized gaps to be closed
Recognize Objective
Determine Scope
Identify Gaps
Analyze Gaps
Implement Changes
Evaluate Results
72© 2011 Carnegie Mellon University
Implementing changes
Use model guidance• Subpractices and other informative material provide implementation
guidance
• Code of Practice Crosswalk highlights connections between CERT-RMM and relevant standards and codes of practice, which can serve as additional implementation guidance
• Generic practices in the model provide guidance for having the changes persist in the organization
Consider measurements that could be implemented with the changes to help monitor results and inform management
Recognize Objective
Determine Scope
Identify Gaps
Analyze Gaps
Implement Changes
Evaluate Results
73© 2011 Carnegie Mellon University
Evaluating results
Did we achieve the objective?
Did the changes stick? Can we be sure the new state will persist?
Are additional needs or objectives now apparent?
When should we make another improvement cycle?
If measurements were implemented, are they revealing positive trends?
Recognize Objective
Determine Scope
Identify Gaps
Analyze Gaps
Implement Changes
Evaluate Results
74© 2011 Carnegie Mellon University
Outline
Operational resilience and operational risk
CERT Resilience Management Model Introduction
CERT-RMM Architecture
Measuring maturity with CERT-RMM – the capability dimension
Service Continuity process area
Using CERT-RMM
Compliance process area
Summary and resources
75© 2011 Carnegie Mellon University
Summary and resources
76© 2011 Carnegie Mellon University
Key benefits of using CERT-RMM
Improve efficiency and effectiveness of operational risk management
Institutionalize resilience management processes using proven techniques
Establish a common language for resilience in your organization (or community)
Access an extensive body of knowledge for managing operational risk and resilience
Lower risk, lower cost
Confidence that processes will be sustained in times of stress
Effectively communicate and collaborate to achieve resilience
Confidence in completeness, flexibility, and scalability of approach
77© 2011 Carnegie Mellon University
But I’m already using ________
Most organizations already use one or more standards or practice bodies to support security and continuity activities.
CERT-RMM can complement your current efforts• Completeness: CERT-RMM may provide coverage or guidance not
included in your current practice bodies
• Scalability & flexibility: use only the parts that you need to support your improvement objective
• Stickiness: institutionalization guidance can be deployed to help you make current and improved practices persist and collaborate
78© 2011 Carnegie Mellon University
Potential next steps
Get the book
Take the course
Select a subset of the model that matches your current improvement objectives
Convene a small team to review the model content and identify gaps in your current activities
79© 2011 Carnegie Mellon University
ResourcesBook
Includes full model (v1.1) plus adoption guidance and perspectives from real-world use of the model.
Available at Amazon.com
www.cert.org/resilienceemail: [email protected]
Training
Introduction to the CERT Resilience Management Model (3-day course)
• Public courses
- Feb 14-16, 2012 (DC)
- July 16-18, 2012 (Pittsburgh)
- Oct 2-4, 2012 (DC)
• Private onsite courses are also available
www.sei.cmu.edu/training/P66.cfm
Lead appraiser apprenticeship program is also available to certify people in leading CERT-RMM-based appraisals
80© 2011 Carnegie Mellon University
Contact information
David White
CERT Resilient Enterprise Management [email protected]
SEI Customer Relations
For general [email protected]
David Ulicne
For information about [email protected]
Joe McLeod
For information about working with [email protected]
www.cert.org/resilience
81© 2011 Carnegie Mellon University
Backup materials
82© 2011 Carnegie Mellon University
CERT-RMM Use ScenarioUsing selected process areas to improve incident management
83© 2011 Carnegie Mellon University
Scenario: improve incident management
Objective: improve incident management capability
A quick scan through CERT-RMM reveals several process areas that would assist with this objective
• Incident Management and Control• Risk Management• Monitoring• Service Continuity
84© 2011 Carnegie Mellon University
CrisisCriteria
IncidentCriteria
Incident Management and Control defines
Event Incident
Closure
Crisis
Event – one or more occurrences, possibly minor, that affect assets and have the potential to disrupt operations
Incident – an event (or series of events) of higher magnitude that significantly affects assets and requires action to limit impact
Crisis – an incident where the impact is rapidly escalating or immediate
Closure – should actively occur for all events, incidents, and crises when no further actions are needed.
85© 2011 Carnegie Mellon University
Incident Management and Control
In most organizations, many event streams need to be watched to effectively provide early warning and to detect incidents and crises.
How do we build an effective approach?
Event stream
Event stream
Event stream
Event stream
Event stream
86© 2011 Carnegie Mellon University
Risk Management -1
Risk Management guides the identification of sources and categories of risk that matter to the organization, for example:
Network intrusions
Malware
Extreme weatherMass illness
Supply disruption
87© 2011 Carnegie Mellon University
Event stream
Event stream
Event stream
Event stream
Event stream
Risk Management -2
These sources of risk should inform the event streams if they are likely to lead to incidents or crises
Network intrusions
Malware
Extreme weatherMass illness
Supply disruption
88© 2011 Carnegie Mellon University
Monitoring
Monitoring guides the implementation of data collection and sharing activities. In this example, it will provide guidance on implementing the infrastructure to monitor these event streams.
Network intrusions
Malware
Mass illness
Supply disruption
Extreme weather
89© 2011 Carnegie Mellon University
CrisisCriteria
IncidentCriteria
Risk Management -3
Risk Management practices produce criteria for measuring the potential impact of risks.
Network intrusions
Malware
Mass illness
Supply disruption
Extreme weather
Risk measurement criteriainform
90© 2011 Carnegie Mellon University
CrisisCriteria
IncidentCriteria
Incident Management and Control process
Incident
Incident
Closure
Crisis
Practices from Incident management and Control produce a consistent process for managing incidents and crises
Consistent incident management process, including closure
Network intrusions
Malware
Mass illness
Supply disruption
Extreme weather
91© 2011 Carnegie Mellon University
CrisisCriteria
IncidentCriteria
Service Continuity
Incident
Incident
Closure
Crisis
Service Continuity practices produce plans to ensure the continuity of operations in the event of disruptions. Continuity plans will be triggered during incidents or crises. Collaboration is needed to ensure that plans are effectively triggered.
Service continuity plansTriggers
Triggers
Network intrusions
Malware
Mass illness
Supply disruption
Extreme weather
92© 2011 Carnegie Mellon University
CrisisCriteria
IncidentCriteria
Incident Management system• Incident Management and Control• Risk Management• Monitoring• Service Continuity
Incident
Incident
Closure
Crisis
Network intrusions
Malware
Mass illness
Supply disruption
Extreme weather
Four process areas that can help us develop an effective incident management system in our organization
93© 2011 Carnegie Mellon University
CERT-RMM for AssuranceFocusing CERT-RMM on early life-cycle activities for building resilience in
94© 2011 Carnegie Mellon University
RTSE – Resilient Technical Solution Engineering
Ensure that software and systems are developed to satisfy their resilience requirements
95© 2011 Carnegie Mellon University
RTSE specific goals
Goal Goal Title
RTSE:SG1 Establish guidelines for resilient technical solution development
RTSE:SG2 Develop resilient technical solution development plans
RTSE:SG3 Execute the plan
96© 2011 Carnegie Mellon University
RTSE: Building in versus bolting on
Requires organizational intervention
Extends resilience requirements to assets that are to be developed
Creates requirements for quality attributes
Attempts to reduce the level of operational risk
Extends across the life cycle
97© 2011 Carnegie Mellon University
RTSE: Designing and testing for resilience• Performing resilience controls planning and design
• Incorporating resilience controls into architecture design
• Designing resilience-specific architecture
• Adopting secure coding practices
• Processes for detecting and removing defects
• Designing testing criteria to attest to asset resilience
• Testing resilience controls
• Designing service continuity plans during the development process
98© 2011 Carnegie Mellon University
RTSE influences
BSIMM2 bsimm.com
Open Web Applications Security Project (OWASP) Software Assurance Maturity Model www.owasp.org
Microsoft Security Development Life Cycle www.microsoft.com/security/sdl/
DHS Process Reference Model for Assurance Mapping to CMMI-DEV V1.2 https://buildsecurityin.us-cert.gov/swa/procresrc.html
99© 2011 Carnegie Mellon University
CERT-RMM for software assurance