Summary of Changes - FEMA.gov · DRAFT FOR REVIEW AND COMMENT ONLY. DRAFT FOR REVIEW AND COMMENT...

DRAFT FOR REVIEW AND COMMENT ONLY


Summary of Changes 1 2 The 2020 revision of the Homeland Security Exercise and Evaluation Program includes input from the whole 3 community, a revised chapter on Program Management replacing the Training and Exercise Planning Workshop 4 with an Integrated Preparedness Planning Workshop, more information on evaluation, and the addition of several 5 tables and figures. The intent of the revision is to provide a more comprehensive and inclusive document for the use 6 of the whole community to use in exercise: program management, design and development, conduct, evaluation, and 7 improvement planning. 8 9 Comments from the Community 10 We received the first set of comments for the HSEEP revision, to include: 11

• 18 agencies and organizations including the public and private sector, provided comments; 12 • 3 Critical, 218 Substantive, 45 Administrative, and 16 Unmarked; 13 • An overwhelming majority of the comments regarded grammatical errors, unclear and inconsistent 14

definitions 15 • Standardization of terms; and 16 • Lastly, several comments requested additional context to various sections to clarify and be more 17

specific. 18 Integrated Preparedness Planning Workshops 19

• FEMA proposes changing the Training and Exercise Planning Workshop (TEPW) within the Program 20 Management Section of HSEEP to an Integrated Preparedness Planning Workshop (IPPW) 21

• The current TEPW model does not support integration of risk assessments, planning efforts, grant 22 priorities, and other preparedness activities to drive exercise activities 23

• The proposed IPPW model aligns the concept of the National Preparedness System with exercise 24 program management by ensuring that preparedness activities are fully integrated into exercise 25 planning and design 26

• Exercises are a driver of other preparedness activities. It has been shown that many exercise sponsors 27 do not have a plan to deliver their capabilities for the scenario and training is a frequently-cited area for 28 improvement 29

• The IPPW would ensure risk assessments, plans, and training are scheduled and conducted prior to 30 conducting an exercise in a more integrated, concerted effort allowing exercises to serve their 31 validation function 32

Evaluation/Improvement Planning 33 • FEMA has enhanced the Evaluation and Improvement Planning section to fill gaps in guidance and 34

support continuous improvement efforts 35 • An analysis of AARs received into the HSEEP inbox and common feedback received during technical 36

assistance engagements reveal that jurisdictions are not receiving enough guidance on how to collect, 37 analyze, and document meaningful observations that drive corrective actions and continuous 38 improvement 39

• SMART Corrective Actions were created to help drive actionable corrective actions 40 Other Changes 41

• In place of text, several tables and figures were added to provide a more utility to the audience. 42 • The glossary is updated for standardization of terms and acronyms 43

44 The goal is to provide comprehensive and consistent guidance and tools that preparedness partners can use for 45 exercises 46 47 Thank you in advance for your review and commentary. We look forward to hearing your input. 48

49



50

HOMELAND SECURITY EXERCISE AND EVALUATION PROGRAM October 16 version

Draft for Review and Comment Only

Federal Emergency Management Agency


DRAFT FOR REVIEW AND COMMENT ONLY i

Table of Contents 51 Summary of Changes ............................................................................................................................................ i 52 Introduction and Overview .................................................................................................................................. 5 53

Purpose of Document ................................................................................................................................................ 5 54 Role of Exercises ....................................................................................................................................................... 5 55 Applicability and Scope of Document ....................................................................................................................... 6 56 Supersession .............................................................................................................................................................. 6 57 Doctrine Organization .............................................................................................................................................. 6 58 Revision Process ....................................................................................................................................................... 7 59

1. HSEEP Fundamentals ........................................................................................................................................ 8 60 Overview ................................................................................................................................................................... 8 61 Fundamental Principles ............................................................................................................................................ 8 62 The HSEEP Exercise Cycle ....................................................................................................................................... 8 63

Program Management ........................................................................................................................................ 9 64 Exercise Design and Development ...................................................................................................................... 9 65 Exercise Conduct ................................................................................................................................................. 9 66 Exercise Evaluation ............................................................................................................................................. 9 67 Improvement Planning ...................................................................................................................................... 10 68

2. Program Management .................................................................................................................................. 11 69 Overview.................................................................................................................................................................. 11 70 Engage Senior Leaders ............................................................................................................................................ 11 71 Multi-year Preparedness Priorities .......................................................................................................................... 12 72

Integrated Preparedness Planning Workshop (IPPW) ...................................................................................... 12 73 Multi-year Integrated Preparedness Plan (IPP) ....................................................................................................... 14 74

Progressive Approach ....................................................................................................................................... 15 75 Discussion-Based Exercises ............................................................................................................................... 15 76 Operations-Based Exercises .............................................................................................................................. 17 77

Rolling Summary of Outcomes ................................................................................................................................ 19 78 Manage Exercise Program Resources ..................................................................................................................... 19 79

Exercise Budget Management .......................................................................................................................... 19 80 Program Staffing ............................................................................................................................................... 19 81 Other Resources ................................................................................................................................................ 20 82

3. Exercise Design and Development .................................................................................................................. 21 83 Overview.................................................................................................................................................................. 21 84 Exercise Foundation ................................................................................................................................................ 21 85 Exercise Planning Activities ..................................................................................................................................... 22 86

Exercise Planning Team Considerations ............................................................................................................ 25 87


DRAFT FOR REVIEW AND COMMENT ONLY ii

Exercise Planning Team Positions ..................................................................................................................... 26 88 Exercise Design ....................................................................................................................................................... 29 89

Exercise Purpose ............................................................................................................................................... 29 90 Exercise Scope ................................................................................................................................................... 29 91 Evaluation Parameters ...................................................................................................................................... 31 92 Scenario ............................................................................................................................................................. 31 93 Exercise Documentation ............................................................................................................................ 32 94 Planning for Media and Public Affairs ............................................................................................................... 34 95

Exercise Development ............................................................................................................................................. 35 96 Planning for Discussion-Based Exercise Logistics .............................................................................................. 35 97 Planning for Operations-Based Exercise Logistics ............................................................................................. 36 98 Planning for Discussion-Based Exercise Facilitation .......................................................................................... 38 99 Planning for Operations-Based Exercise Control .............................................................................................. 39 100 Planning for Exercise Evaluation ....................................................................................................................... 42 101 Preparing for Exercise Conduct ......................................................................................................................... 43 102

4. Exercise Conduct ............................................................................................................................................ 44 103 Overview.................................................................................................................................................................. 44 104 Exercise Play Preparation ........................................................................................................................................ 44 105

Setup for Discussion-Based Exercises ............................................................................................................... 44 106 Setup for Operations-Based Exercises .............................................................................................................. 44 107

Exercise Play ............................................................................................................................................................ 46 108 Conduct for Discussion-Based Exercises ........................................................................................................... 46 109

Figure 4.1: Sample Discussion-Based Exercise Organization................................................................................. 46 110 Conduct for Operations-Based Exercises .......................................................................................................... 47 111

Figure 4.2: Sample Operations-Based Exercise Organization ............................................................................... 48 112 Wrap-Up Activities .................................................................................................................................................. 49 113

Player Hotwash ................................................................................................................................................. 49 114 Debrief .............................................................................................................................................................. 50 115

5. Evaluation ...................................................................................................................................................... 51 116 Overview.................................................................................................................................................................. 51 117 Evaluation Planning................................................................................................................................................. 51 118

Evaluation Team ................................................................................................................................................ 52 119 Evaluation Documentation ............................................................................................................................... 53 120 Exercise Evaluation Guide Development .......................................................................................................... 54 121

Observation and Data Collection............................................................................................................................. 55 122 Observation ....................................................................................................................................................... 55 123 Data Collection .................................................................................................................................................. 55 124

Data Analysis .......................................................................................................................................................... 55 125 After-Action Report (AAR) ....................................................................................................................................... 56 126

6. Improvement Planning ................................................................................................................................... 58 127


DRAFT FOR REVIEW AND COMMENT ONLY iii

Overview.................................................................................................................................................................. 58 128 SMART Corrective Actions ....................................................................................................................................... 58 129 After-Action Meeting ............................................................................................................................................... 58 130 Improvement Plan ................................................................................................................................................... 59 131 Corrective Action Tracking and Implementation ..................................................................................................... 59 132 Using Improvement Planning to Support Continuous Improvement ....................................................................... 60 133

Acronyms ........................................................................................................................................................... 61 134 Glossary of Terms .............................................................................................................................................. 63 135

136


DRAFT FOR REVIEW AND COMMENT ONLY iv

Figures and Tables 137 Figure 1.1: The HSEEP Cycle 1.2 138 Figure 2.1: Integrating Preparedness Cycle and the Exercise Cycle 2.1 139 Figure 2.2: Factors for Consideration in Developing Exercise Program Priorities 2.3 140 Figure 3.1: Entering the Exercise Cycle 3.1 141 Figure 3.2: Sample of an Exercise Planning Team 3.6 142 Figure 3.3: Priorities, Objectives, and Capabilities 3.10 143 Figure 3.4: Sample of a Facilitation Structure 3.16 144 Figure 3.5: Sample of a Control Structure 3.17 145 Figure 3.6: MSEL and Event Types 3.18 146 Figure 4.1: Sample of Discussion-Based Exercise Organization 4.3 147 Figure 4.2: Sample of Operations-Based Exercise Organization 4.4 148 Figure 5.1: Sample of an Evaluation Team Organization 5.2 149 Figure 5.2: Development of an Exercise Evaluation Guide 5.4 150 Figure 6.1: Exiting the Exercise Cycle and Entering the Preparedness Cycle 6.1 151 Table 2.1: Discussion-Based Exercise Types 2.4 152 Table 2.2: Operations-Based Exercise Types 2.6 153 Table 3.1: Planning Meetings 3.2 154 Table 3.2: SMART Objectives 3.10 155 Table 3.3: Exercise Documentation 3.11 156 Table 3.4: Other Exercise Documentation 3.12 157 Table 4.1: Brief Types and Descriptions 4.2 158 Table 5.1: Evaluation Documentation 5.3 159 Table 5.2: After-Action Report Documentation 5.6 160 Table 6.1: After-Action Meeting 6.2 161 Table 6.2: Improvement Plan Documentation 6.2 162 163


DRAFT FOR REVIEW AND COMMENT ONLY 5

Introduction and Overview 164

Purpose of Document 165

The Homeland Security Exercise and Evaluation Program (HSEEP) provides a set of 166 guiding principles for exercise programs as well as a common approach to program 167 management, design and development, conduct, evaluation, and improvement 168 planning. An exercise is an event or activity delivered through discussion or action to 169 develop, assess, or validate capabilities to achieve planned objectives. Exercises are a 170 fundamental component of preparedness by providing the whole community1 with the 171 opportunity to shape planning, assess and validate capabilities, and address areas for 172 improvement. 173 Through HSEEP, exercise program managers can develop, execute, and evaluate 174 exercises that address the priorities established by a jurisdiction’s/organization’s 175 leaders. The National Preparedness Goal2, strategy documents, threat and hazard 176 identification/risk assessment processes, capability assessments, and the results from 177 previous exercises and real-world events impact the priorities. These priorities guide 178 the overall direction of a progressive exercise program, where individual exercises 179 anchor to a common set of priorities or objectives, which increase in complexity over 180 time. These priorities guide the design and development of individual exercises and 181 allow planners to identify and align objectives to capabilities

3 evaluated during the 182 exercise. Exercise evaluation assesses the ability to meet the objectives and 183 capabilities by documenting strengths, areas for improvement, capability performance, 184 and corrective actions in an After-Action Report/Improvement Plan (AAR/IP). 185 Through improvement planning, jurisdictions/organizations take the corrective actions 186 identified through exercise and real-world events to improve plans, build and sustain 187 capabilities, and maintain readiness. Stakeholders are reminded of the importance of 188 implementing corrective actions iteratively throughout the progressive exercise cycle. 189 In this way, the use of HSEEP—in line with the National Preparedness Goal, the 190 National Preparedness System, and stakeholder preparedness priorities—supports 191 efforts across the whole community that improve the nation’s capacity to build, 192 sustain, and deliver capabilities. 193

Role of Exercises 194

Exercises play a vital role in preparedness by enabling the whole community to; test 195 and validate plans and capabilities; and identify both capability gaps and areas for 196 1 The whole community means involving people (individuals and families including those with access and functional needs, businesses, faith-based and community organizations, nonprofit groups, schools and academia, media outlets, and all levels of government including state, local, tribal, territorial, and federal partners) in the development of national preparedness documents and ensuring their roles and responsibilities are reflected in the content of the materials. 2 Refer to Federal Emergency Management Agency Preparedness https://www.dhs.gov/presidential-policy-directive-8-national-preparedness 3 Capabilities are distinct critical elements necessary to achieve the specific mission areas of prevention, protection, mitigation, response, and recovery. Capabilities provide a common vocabulary describing the significant functions required to deal with threats and hazards that must be developed and executed across the whole community to ensure national preparedness. To review the core capabilities from the Federal Emergency Management Agency, reference: https://www.fema.gov/core-capabilities

https://urldefense.com/v3/__https:/www.dhs.gov/presidential-policy-directive-8-national-preparedness__;!cYEOeHUTKcc!bRSkk_FvE115OD8YI5gG4y13PzuSf3JRtFUf9vwpBqov8NgDDF96tnXPHi4RdPu6MA$






improvement. A well-designed exercise provides a low-risk environment to test 197 capabilities, familiarize personnel with roles and responsibilities, and foster 198 meaningful interaction and communication across jurisdictions/organizations. 199 Exercises bring together and strengthen the whole community to prevent, protect, 200 mitigate, respond, and recover4, also known as the five mission areas, from all-201 hazards. Overall, exercises help the whole community address the priorities 202 established by a jurisdiction’s/organization’s leaders; validate capabilities; and 203 evaluate progress towards meeting preparedness efforts. 204

Applicability and Scope of Document 205

The HSEEP doctrine is flexible, scalable, adaptable, and is for use by any stakeholders 206 across the whole community. The HSEEP doctrine is applicable for exercises across 207 all mission areas. Using HSEEP supports the National Preparedness System5 by 208 providing a consistent approach to exercises and measuring progress toward building, 209 sustaining, and delivering capabilities. 210 The foundation of the HSEEP doctrine incorporates lessons learned and best practices 211 from the exercise community and current policies and plans that support training, 212 technology systems, tools, and technical assistance. 213 HSEEP doctrine is based on best practices and is supported by training, technology systems, 214 tools, and technical assistance. Exercise practitioners are encouraged to apply and adapt the 215 HSEEP doctrine to meet their specific needs. 216

Supersession 217

The 2020 iteration of the HSEEP doctrine supersedes the 2013 version. 218

Doctrine Organization 219

The organization of the HSEEP doctrine follows: 220

• Chapter 1: The HSEEP Fundamentals describes the basic principles and 221 methodology of HSEEP. 222

• Chapter 2: Program Management provides guidance for conducting a 223 Training and Exercise Planning Workshop and developing a Multi-year 224 Training and Exercise Plan. 225

• Chapter 3: Exercise Design and Development describes the methodology 226 for developing exercise objectives, conducting planning meetings, 227 developing exercise documentation, and planning for exercise logistics, 228 control, and evaluation. 229

• Chapter 4: Exercise Conduct provides guidance on setup, exercise play, 230 and wrap-up activities. 231

4 The National Preparedness Goal identified five mission areas prevention, protection, mitigation, response, and recovery, in which the thirty-two core capabilities exist. Reference: https://www.fema.gov/mission-areas 5 The National Preparedness System includes identifying and assessing risks; estimating the level of capabilities needed to address those risks; building or sustaining the required levels of capability; developing and implementing plans to deliver those capabilities; validating and monitoring progress; and reviewing and updating efforts to promote continuous improvement. Reference: https://www.fema.gov/national-preparedness-system

https://www.fema.gov/national-preparedness-goal

https://www.fema.gov/core-capabilities



• Chapter 5: Evaluation provides the approach to exercise evaluation 232 planning and conduct through data collection, analysis, and development of 233 an After-Action Report. 234

• Chapter 6: Improvement Planning addresses corrective actions identified 235 in the exercise Improvement Plan and the process of tracking corrective 236 actions to resolution. 237

Revision Process 238

The Department of Homeland Security, Federal Emergency Management Agency 239 reviews the HSEEP doctrine and methodology every six years or as needed, to make 240 necessary modifications and incorporate lessons learned. 241



1. HSEEP Fundamentals 242

Overview 243

The HSEEP doctrine consists of fundamental principles that frame a common 244 approach to exercises. The intent is to enhance consistency in exercise conduct and 245 evaluation while ensuring exercises remain a flexible, accessible way to improve our 246 preparedness across the nation. 247

Fundamental Principles 248

Applying the following principles to the management of an exercise program and 249 executing individual exercises is important to an effective evaluation of capabilities: 250

• Senior Leader Guidance. The early and frequent engagement of senior 251 leaders is the key to the success of any exercise program. Senior leaders 252 provide the overarching guidance and direction for the exercise and 253 evaluation program as well as specific intent for individual exercises. 254

• Capability-based, Objective Driven. Jurisdictions/Organizations can use 255 exercises to evaluate current capability levels/targets and identify gaps. 256 Exercises focus on assessing performance against capability-based 257 objectives. 258

• Progressive Exercise Planning Approach. A progressive approach 259 includes the use of various exercises aligned to a common set of program 260 priorities and objectives with an increasing level of complexity over time. 261 Progressive exercise planning does not always imply a linear progression of 262 exercise types. 263

• Whole Community Integration. The use of HSEEP encourages exercise 264 planners, where appropriate, to engage the whole community throughout 265 program management, design and development, conduct, evaluation, and 266 improvement planning. 267

• Informed by Risk. Identifying and assessing risks and associated impacts 268 helps jurisdictions/organizations identify and evaluate priorities, objectives, 269 and capabilities through exercises. 270

• Common Methodology. HSEEP includes a common methodology for 271 exercises that applies to all mission areas. The methodology enables 272 jurisdictions/organizations a shared understanding of program management, 273 design and development, conduct, evaluation, and improvement planning; 274 and fosters exercise-related interoperability and collaboration. 275

The HSEEP Exercise Cycle 276

HSEEP uses a common approach for planning and conducting individual exercises. 277 The following chapters contain more detailed descriptions of each phase as depicted in 278 Figure 1.1. 279



280 Figure 1.1: The HSEEP Exercise Cycle 281

Program Management 282 Program management involves a collaborative approach that integrates resources, 283 jurisdictions/organizations, and individuals to identify and achieve program priorities. 284 An effective exercise program maximizes efficiency, resources, time, and funding by 285 ensuring a coordinated and integrated approach to building, sustaining, and delivering 286 capabilities. Through the management of an exercise program, stakeholders provide 287 oversight for specific preparedness activities sustained over time. 288

Exercise Design and Development 289 During exercise design and development, planning team members use the intent and 290 guidance of senior leaders and the priorities to shape the individual or series of 291 exercises. Including risk and hazard assessments, plans and procedures, grant or 292 cooperative agreements, and relevant After-Action Reports and Improvement Plans 293 (AAR/IP), planners ensure that the exercise both test and validate a 294 jurisdiction’s/organization’s capabilities. 295

Exercise Conduct 296 After design and development activities are complete, the exercise is ready to occur. 297 Activities essential to conducting individual exercises include preparing for exercise 298 play, managing exercise play, and conducting immediate exercise wrap-up activities. 299

Exercise Evaluation 300 An effective evaluation assesses performance against exercise objectives and identifies 301 and documents strengths and areas for improvement relative to capabilities. Evaluation 302 is important and considered in all phases of the exercise planning cycle, beginning 303 when the exercise planning team meets to establish objectives and initiate exercise 304 design. 305



Improvement Planning 306 Improvement planning highlights identified strengths, areas for improvement and gaps 307 in planning, organizing, equipping, training, and exercising to create reasonable and 308 manageable Specific, Measurable, Achievable, Relevant, and Time-Bound (SMART) 309 corrective actions, items outlined in an Improvement Plan that is intended to resolve 310 areas for improvement. 311



2. Program Management 312

Overview 313

Program management is the process of overseeing and 314 integrating a variety of exercises over time. An effective 315 exercise program helps jurisdictions/organizations 316 maximize efficiency, resources, time, and funding by 317 ensuring that exercises are part of a coordinated, integrated 318 approach to building, sustaining, and delivering 319 capabilities. The integrated approach is called multi-year 320 planning and begins when senior leaders, working with 321 whole community stakeholders, identify and develop a set 322 of multi-year preparedness priorities informed by existing 323 assessments, strategies, and plans. These long-term 324 priorities help exercise planners design and develop a 325 progressive program of individual exercises to build, 326 sustain, and deliver capabilities. 327 Effective program management promotes a multi-year 328 approach to: 329

• Engage elected and appointed officials; 330 • Establish multi-year preparedness priorities; 331 • Use a progressive approach; 332 • Develop a multi-year Integrated Preparedness 333

Plan (IPP); 334 • Maintain a rolling summary of exercise 335

outcomes; and 336 • Manage exercise program resources. 337

Through effective program management, each exercise 338 becomes a supporting component of a larger exercise program with overarching 339 priorities. Figure 2.1 shows the relationship and linkage of the preparedness cycle and 340 exercise cycle. Exercise practitioners are encouraged to apply and adapt HSEEP 341 doctrine on program management to meet their specific needs. 342

Engage Senior Leaders 343

Engaging senior leaders in the exercise process is critical in providing both the 344 strategic direction for the exercise program and specific guidance for individual 345 exercises. Senior leaders ensure the highest level of support and align to the whole 346 community needs and priorities. Engage senior leaders early and often in an exercise 347 program, starting with the development of exercise program priorities at the Integrated 348 Preparedness Planning Workshop (IPPW). In developing individual exercises, the 349 exercise planning team should continue to engage the appropriate senior leaders 350 throughout the exercise planning cycle to achieve the leaders’ vision for the exercise. 351

Figure 2.1: Integrated Preparedness and Exercise Cycle



Multi-year Preparedness Priorities 352

Senior leaders develop an exercise program from a set of strategic, high-level 353 priorities. The risk and capability assessments, findings and corrective actions from 354 previous events, and external requirements like regulations and grant guidance 355 develop these priorities. Once determined, priorities guide the development of exercise 356 objectives, related training, staffing, resourcing, and other planning activities ensure 357 individual exercises test and validate preparedness in a progressive and coordinated 358 way. 359

Integrated Preparedness Planning Workshop (IPPW) 360 Purpose 361 The purpose of the Integrated Preparedness Planning Workshop (IPPW) is to use the 362 guidance provided by senior leaders to identify and set preparedness priorities and 363 develop a multi-year schedule of preparedness activities that align with other planning 364 efforts. 365 The process confirms the: 366

• Coordination of whole community initiatives; 367 • Prevent duplication of effort; 368 • Promotion the efficient use of resources; 369 • Avoidance of overextending key agencies and personnel; and 370 • Maximization the efficacy of preparedness appropriations. 371

The Integrated Preparedness Planning Workshop (IPPW) occurs on a periodic basis 372 (for example, annual or biennial) depending on the needs of the program and any grant 373 or cooperative agreement requirements. The Integrated Preparedness Planning 374 Workshop (IPPW) establishes the strategy and structure for an exercise program and 375 preparedness efforts. In addition, the Integrated Preparedness Planning Workshop 376 (IPPW) sets the foundation for the planning, conduct, and evaluation of individual 377 exercises. 378 Participation 379 When identifying stakeholders, exercise program managers should consider 380 individuals from jurisdiction/organizations throughout the whole community, 381 including but not limited to: 382

• Senior leaders responsible for providing direction and guidance for exercise 383 program priorities, planning activities, applicable training, and those 384 responsible for providing resources to support preparedness efforts; 385

• Representatives from relevant disciplines that would be part of the exercises 386 or any real-world events, including appropriate local, regional, or federal 387 department and agency representatives; 388

• Individuals with administrative responsibility relevant to exercise conduct; 389 and 390



• Representatives from volunteer, nongovernmental, nonprofit, or social 391 support organizations, including advocates for children, seniors, individuals 392 with disabilities, those with access and functional needs, racially and 393 ethnically diverse communities, people with limited English proficiency, and 394 animals. 395

Once the comprehensive set of stakeholders is identified, exercise program managers 396 can include them in the exercise program by regularly participating in the Integrated 397 Preparedness Planning Workshop (IPPW). 398 Conduct of the Integrated Preparedness Planning Workshop (IPPW) 399 When developing preparedness priorities and the multi-year schedule at the Integrated 400 Preparedness Planning Workshop (IPPW), stakeholders should engage 401 jurisdiction/organizational senior leaders early in the process to obtain their intent and 402 guidance. 403 The Integrated Preparedness Planning Workshop (IPPW) participants also review and 404 consider various factors such as: 405

• Jurisdiction-specific threats and hazards (for example, Threat and Hazard 406 Identification and Risk Assessment, and local risk assessments); 407

• Areas for improvement identified from real-world events and exercises; 408 • External requirements such as state or national preparedness reports, 409

homeland security policy, and industry reports; and 410 • Accreditation standards (for example, hospital accreditation requirements), 411

regulations, or legislative requirements. 412 Figure 2.2 illustrates some of the specific factors for consideration in developing 413 exercise program priorities. 414



415 Figure 2.2: Factors for Consideration in Developing Exercise Program Priorities 416

Drawing on the above factors and capabilities, the workshop facilitator leads a group 417 stakeholder discussion to review exercise program priorities and outline training and 418 exercise priorities shared across multiple jurisdictions/organizations. The group should 419 also develop a multi-year schedule of training and exercise activities designed to meet 420 those priorities. 421 At the end of the Integrated Preparedness Planning Workshop (IPPW), exercise 422 program managers will have a clear understanding of specific multi-year training and 423 exercise program priorities, and any available information on previously planned 424 training and exercises that align to those priorities. This combined set of information 425 develops a multi-year Integrated Preparedness Plan (IPP). 426

Multi-year Integrated Preparedness Plan (IPP) 427

Once the training and exercise program priorities are outlined, stakeholders develop 428 the multi-year Integrated Preparedness Plan (IPP). The Integrated Preparedness Plan 429 (IPP) is the structure for combining efforts across components of the preparedness 430 cycle to make sure that a jurisdiction/organization has the capabilities to handle threats 431 and hazards. Through effective program management, each planning6, training, and 432 exercise activity becomes a supporting component of the larger preparedness 433 priorities. The long-term priorities help exercise planners design and develop a 434

6 Guidance on conducting operational planning is available within the Federal Emergency Management Agency’s Comprehensive Preparedness Guide 101: Developing and Maintaining Emergency Operations Plan



progressive program of individual exercise that target resources to where they are 435 most useful. 436

Progressive Approach 437 A progressive, multi-year exercise program enables jurisdictions/organizations to 438 participate in a series of increasingly complex exercises with each exercise building 439 upon the previous one. Regardless of exercise type, each exercise within the 440 progressive series is linked to a set of common program priorities and designed to test 441 associated capabilities. Identifying training requirements as part of the integrated 442 preparedness planning process, jurisdictions/organizations address known shortfalls 443 prior to exercising capabilities. The progressive approach (with exercises that build 444 upon one another and supported at each step with training resources) ensures 445 jurisdictions/organizations select the appropriate exercise type. Effective planning of 446 exercises and integration of the necessary training reduce the waste of limited exercise 447 resources and serve to address known shortfalls prior to the conduct of the exercise. 448 The following sections describe the different types of exercises. 449

Discussion-Based Exercises 450 Discussion-based exercises include seminars, workshops, tabletop exercises, and 451 games. These types of exercises familiarize players with, or develop new, plans, 452 policies, agreements, and procedures. Discussion-based exercises focus on strategic, 453 policy-oriented issues. Facilitators or presenters lead the discussion, keeping 454 participants moving towards meeting the exercise objectives. 455 Table 2.1 provides the important information needed for each type of discussion-456 based exercise: 457

Discussion-Based Exercises

Seminar – A discussion-based exercise that orients participants to or provides an overview of authorities, strategies, plans, policies, procedures, protocols, resources, concepts, and ideas.

Purpose • Orient participants to, or provide an overview of, authorities, strategies, plans, policies, procedures, protocols, resources, concepts, and ideas.

Structure

• Usually in the form of multiple presentations, subject matter expert panel or case study discussions

• Lecture-based • Led by a seminar leader/presenter • Limited feedback or interaction from participants

Participant Goals

• Orient participants to new or existing plans, policies, or procedures • Gain awareness or assess interagency capabilities or inter-jurisdictional operations • Construct a common framework of understanding

Conduct Characteristics

• Minimal time constraints • Effective for small or large groups

Outcomes • Valuable for developing or making major changes to existing plans or procedures. • Awareness of the capabilities of interagency or inter-jurisdictional operations. • An After-Action Report (AAR)/Improvement Plan (IP)



Workshop – A discussion-based exercise often employed to develop policy, plans, or procedures.

Purpose • Increased participant interaction with focus on achieving or building a product. • Should have clearly defined objectives, products, or goals and focus on a specific

issue.

Structure

• A group of individuals in a venue conducive to discussion • Lectures, presentations, panel or case study discussions, or decision support tools • Facilitated, working breakout sessions • Led by a workshop facilitator/presenter

Participant Goals • Develop a product as a group • Obtain consensus • Collect or share information


• effective for small and large groups • Broad attendance by relevant stakeholders • Conducted based on clear objectives/goals • More participant discussion than lecture-based seminar • Frequently uses break-out sessions to explore parts of an issue with similar groups

Outcomes

• Emergency Operations Plans (EOPs) • Mutual Aid Agreements • Standard Operations Procedures (SOPs) • Continuity of operations plans • Workshop Summary Report • An After-Action Report (AAR)/Improvement Plan (IP)

Tabletop Exercise – A discussion-based exercise in response to a scenario intended to generate a dialogue of various issues to facilitate a conceptual understanding, identify strengths and areas for improvement, and/or achieving changes in perceptions about plans, policies, or procedures.

Purpose • Generate discussion of various issues regarding a exercise scenario. • Facilitate conceptual understanding, identify strengths and areas for improvement

and/or achieving changes in perceptions.

Structure

• Scenario is presented and remains constant—it describes an event and brings discussion participants up to the simulated present time. Players apply their knowledge and skills to a list of problems presented by the facilitator; problems are discussed as a group; and resolution may be reached and documented for later analysis

• Plenary or breakout session(s) • Discussion led by a facilitator(s) • Presentation

Participant Goals

• Enhance general awareness • Enhance roles and responsibility understanding • Validate plans and procedures • Discuss concepts and/or assess types of systems in a defined incident


• Requires an experienced facilitator • In-depth discussion • Problem-solving environment

Outcomes • Recommended revisions to current policies, procedures, and plans. • All participants should be encouraged to contribute to the discussion and be

reminded that they are making decisions in a no-fault environment.



• An After-Action Report (AAR)/Improvement Plan (IP) Game – A discussion-based exercise that is a simulation of operations, that involve two or more teams, usually in a competitive environment, using rules, data, and procedures designed to depict an actual or hypothetical situation.

Purpose

• Simulation of operations that explore the consequences of player decisions and actions

• Identifying critical decision-making points is a major factor in the success of evaluating a game

Structure

• Usually in an environment that involves two or more teams, using rules, data, and procedures designed to depict an actual or hypothetical situation

• Decision-making may be either slow and deliberate or rapid and more stressful, depending on the exercise design and objectives

• The open, decision-based format of a game can incorporate “what if” questions that expand exercise benefits

• Depending on the game’s design, the consequences of player actions can be either pre-scripted or decided dynamically

Participant Goals • Explore decision-making processes and consequences • Conduct "what-if" analyses of existing plans • Evaluate existing and potential strategies


• No actual resources used • Often involves two or more teams • May include models and simulations on increasing complexity as the game

progresses • May or may not include pre-scripted activities

Outcomes • Useful for validating plans and procedures or evaluating resource requirements • An After-Action Report (AAR)/Improvement Plan (IP)

Table 2.1 Discussion-Based Exercise Types 458 Operations-Based Exercises 459 Operations-based exercises include drills, functional exercises, and full-scale 460 exercises. These exercises validate plans, policies, agreements, and procedures; clarify 461 roles and responsibilities; and identify resource gaps. Operations-based exercises are 462 actual reactions to an exercise scenario, such as initiating communications or 463 mobilizing personnel and resources. 464 Table 2.2 provides the important information needed for each type of operations-465 based exercise: 466

Operations-Based Exercises

Drill – An operations-based exercise often employed to validate a single operation or function.

Purpose • Coordinated, supervised activity to validate a specific function or capability in a single agency/organization often employed to validate a single operation or function.

Structure

• Provide training on new equipment, validate procedures, or practice and maintain current skills (for example, establish a community-designated disaster receiving center or shelter).

• Can be stand-alone or used as a series of drills to prepare organizations for collaboration in a full-scale exercise.

• Clearly defined plans, procedures, and protocols need to be in place.



Participant Goals

• Evaluate new procedures, policies, and/or equipment • Practice and maintain skills • Prepare for future exercises


• Immediate feedback • Realistic but isolated environment • Narrow focus • Results are measured against established standards

Outcomes • Determine if plans can be executed as designed, to assess whether more training is

required, or to reinforce best practices • An After-Action Report (AAR)/Improvement Plan (IP)

Functional Exercise – An operations-based exercise that is designed to validate and evaluate capabilities and functions while in a realistic, real-time simulated environment.

Purpose • Validate and evaluate capabilities, multiple functions and/or sub-functions, or interdependent groups of activities.

Structure

• Events are projected through a realistic exercise scenario with event updates that drive activity typically at the management level. Movement of personnel and equipment is usually simulated.

• Controllers typically use a Master Scenario Events List (MSEL) to ensure participant activity remains within predefined boundaries and ensure exercise objectives are accomplished. Simulators can inject scenario elements.

Participant Goals

• Validate and evaluate capabilities • Focused on plans, policies, and procedures


• Conducted in a realistic environment • Simulated deployment of resources and personnel • Use of SimCell and Master Scenario Events List (MSEL) • Include controller and evaluators

Outcomes

• Typically focused on exercising plans, policies, procedures, staffs, agreements, networks, and coordination or intelligence centers or agencies.

• Performance analysis • Cooperative relationships are examined • An After-Action Report (AAR)/Improvement Plan (IP)

Full-Scale Exercise – An operations-based exercise that is typically the most complex and resource-intensive of the exercise types and often involves multiple agencies, jurisdictions/organizations, and real-time movement of resources.

Purpose

• Often include many players operating under cooperative systems such as the Incident Command System (ICS) Unified Command.

• Focus on implementing and analyzing the plans, policies, and procedures that may have been developed in discussion-based exercises and honed during previous, smaller, exercises

Structure

• Events are projected through an exercise scenario with event updates that drive activity at the operational level.

• Personnel and resources may be mobilized and deployed to the scene • Involves multiple agencies, organizations, and jurisdictions. • Use of a MSEL drives player actions • The level of support needed may be greater than that needed for other types of

exercises.



Participant Goals

• Demonstrate roles and responsibilities as addressed in plans and procedures • Coordinate between multiple agencies, organizations and jurisdictions


• Rapid problem solving; critical thinking • Conducted in a realistic environment to mirror a real incident by presenting complex

problems • Mobilization of personnel and resources • Exercise site is usually large with many activities occurring simultaneously • Site logistics require close monitoring • Safety issues, particularly regarding the use of props and special effects, must be

monitored.

Outcomes • Useful for validating plans and procedures or evaluating resource requirements. • An After-Action Report (AAR)/Improvement Plan (IP)

Table 2.2: Operations-Based Exercise Types 467

Rolling Summary of Outcomes 468

A rolling summary report provides stakeholders with an analysis of issues, trends, and 469 key outcomes from all exercises conducted as part of the exercise program. In 470 addition, the rolling summary report is an analysis of trends across preparedness 471 activities as part of the Integrated Preparedness Plan (IPP). This report intends to serve 472 as a program management and communications tool informing stakeholders and 473 guiding the development of future preparedness activities. 474 To adequately address preparedness priorities exercise program managers should 475 periodically develop and distribute a rolling summary report. 476 A rolling summary report: 477

• Informs senior leaders on the progress of the preparedness priorities; 478 • Provides data to support preparedness assessments and reporting requirements; and 479 • Enables exercise planners to modify objectives and the exercise 480

schedule to reflect knowledge gathered from the exercises. 481

Manage Exercise Program Resources 482

An effective exercise program utilizes the full range of available resources. Exercise 483 program managers should ensure a planned exercise budget, appropriate program 484 staffing, and other needed resources. 485

Exercise Budget Management 486 Effective budget management is essential to the success of an exercise program and 487 for exercise managers to maintain awareness of available resources and expected 488 expenditures. In developing and maintaining an exercise program budget, exercise 489 program managers work with stakeholders to identify financial resources and define 490 monitoring and reporting requirements for individual exercises. 491

Program Staffing 492 Exercise program managers should identify the administrative and operational staff 493 needed to oversee the exercise program. The Integrated Preparedness Plan (IPP) is one 494 tool for determining exercise program staffing needs in addition to grant funds or other 495



programmatic considerations. Exercise program managers should also identify gaps 496 between staffing availability and staffing needs. Exercise program managers may 497 consider alternative means of procuring staff members, such as adding volunteers, 498 students from universities (for example, student nurses or emergency management 499 students), or interns. 500

Other Resources 501 Exercise program managers may consider other resources that can support exercises 502 such as: 503

• Information technology (for example, modeling and simulation capabilities); 504 • Exercise tools and resources (for example, document templates); 505 • Materials from previous exercises; 506 • Training courses; 507 • Mutual aid agreements, memoranda of understanding, and memoranda of 508

agreement; 509 • Technical assistance; and 510 • Equipment and props (for example, smoke machines, mannequins, moulage 511

kids, etc.). 512



3. Exercise Design and Development 513

Overview 514 In the design and development phase, exercise planners use the intent and guidance of 515 senior leaders and the exercise priorities to plan individual exercises. Exercise 516 planning teams apply the guidance to shape the major concepts and planning 517 considerations for a single exercise or series of exercises. 518 The steps of exercise design and development include: 519

• Setting the exercise foundation by reviewing senior leadership guidance, the 520 Integrated Preparedness Plan (IPP), and other factors to develop exercise-521 specific objectives and align capabilities within a scenario; 522

• Selecting participants for an exercise planning team and developing an 523 exercise planning timeline with milestones; 524

• Planning for exercise control and identifying evaluation requirements; 525 • Creating documentation; and 526 • Coordinating logistics. 527

Exercise planners apply and adapt the HSEEP doctrine to exercise design and 528 development to meet specific needs. 529

Exercise Foundation 530 The exercise foundation is a set of priorities that drive the exercise design and 531 development process. The priorities developed during the Integrated Preparedness 532 Planning Workshop (IPPW) enable exercise program 533 managers to adhere to a progressive exercise approach. These 534 priorities enable the exercise planning team to transition from 535 the Preparedness Cycle into the HSEEP Cycle as shown in 536 Figure 3.1. 537 The exercise planning team reviews and considers the 538 following items before the design phase: 539

• The senior leadership intent and guidance; 540 • The Integrated Preparedness Plan (IPP); 541 • Any relevant After-Action Reports and 542

Improvement Plans (AAR/IP) from real-world 543 events and exercises; 544

• The Threat and Hazard Identification and Risk Assessment or other risk, 545 threat, and hazard assessments; 546

• The Stakeholder Preparedness Review, or other capability assessments; 547 • The organization/jurisdiction plans and procedures; and 548 • Any grant or cooperative agreement requirements. 549

The exercise foundation builds and sustains a jurisdiction’s/organization’s capabilities 550 while including lessons learned during the exercise design process 551

Figure 3.1: Entering the Exercise Cycle



552

Exercise Planning Activities 553

This section describes the types of planning activities most useful in exercise design and 554 development. The exercise planning team members decide the type and number of planning 555 activities needed to successfully plan a given exercise based on scope and complexity. When 556 arranging meeting and exercise site locations, the planning team should consider individuals who 557 require assistance or accommodations during attendance. 558 Table 3.1 provides the important information needed for each type of discussion-based exercise: 559

Planning Meetings

Concept and Objectives Meeting – A meeting that is the formal beginning of the exercise planning process. It is held to identify the scope and exercise objectives. For less complex exercises and for organizations with limited resources, the C&O Meeting can be conducted in conjunction with the Initial Planning Meeting (IPM).

Meeting Focus

The formal beginning of the planning process. Program priorities and objectives are determined during the Concept and Objectives Meeting.

Discussion Points

• Exercise scope • Proposed objectives and aligned capabilities linked to threats and risks • Location, date, and duration • Participants and anticipated extent of play • Exercise Planning Team makeup • Assumptions and artificialities • Control and evaluation concepts • Security organizations and structure • Available resources • Logistics • Planning timeline and milestones • Local issues, concerns, and sensitivities

Exercise Tools

Meeting agenda and briefing which includes the background and rational for conducting the exercise.

Exercise Outcomes

• Exercise concept • Exercise timeline (group consensus) • Extent of participant play • Identification of planning team members • Planning timeline, milestones, and meeting dates

Follow-up Compile and distribute the C&O Meeting minutes and include the next meeting time, date, and location.

Initial Planning Meeting – A meeting that is held to refine the scope and exercise objectives. For less complex exercises and for organizations with limited resources, the C&O Meeting can be conducted in conjunction with the IPM.

Meeting Focus

Formal beginning of the development phase. The IPM also focuses on refining the scope and the objectives for the exercise.



Discussion Points

• Clearly defined objectives and aligned capabilities linked to threats and risks • Evaluation requirements, including EEG capability targets and critical tasks • Relevant plans, policies, and procedures to be evaluated • Exercise scenario • Modeling and simulation • Participants extent of play • Optimum duration of the exercise • Exercise planners’ roles and responsibilities • Decision to record exercise proceedings (audio or video) • Local issues, concerns, or sensitivities • Any discussion points typically covered during a C&OM if a one is not conducted • Consensus regarding the date, time, and location for next meeting

Exercise Tools

• Read-ahead • Agenda • Briefing for presenting an overview of the exercise and meeting discussion points • Threat and hazard identification • Proposed room layout (discussion-based) • Map of proposed venue sites (operations-based) with description of local environment • Copy of proposed timeline and milestones for design and development • Copies of presentation briefing for meeting

Exercise Outcomes

• Any outcomes from C&O Meeting; if a C&O Meeting was not conducted; • Clearly defined objectives and aligned capabilities • Initial capability targets and critical tasks (reviewed and confirmed prior to next meeting) • Scenario variables (threat, scope, venue, conditions) • List of participating organizations and extent of play • Draft SitMan/ExPlan • Identification and availability of al source documents (plans, policies, procedures) needed

for exercise documentation • Refined exercise planning timeline • Identification of available SMEs (scenario vetting) • Preferred communication methods among planning team • Clearly identified and assigned responsibility for logistical issues • List of tasks to be accomplished by next planning meeting to include date and responsible

planning team member • Agreed upon date, time, and location of next planning meeting

Follow-up

• Compile and distribute the IPM meeting minutes, including the next meeting time, date, and location

• Between meetings: planning team collaborates on assignments and prepares draft of exercise documentation

• Distribute draft documentation prior to the next scheduled meeting

Midterm Planning Meeting – A meeting that serves as a forum to develop exercise scenario details and timeline.



Meeting Focus

The Midterm Planning Meeting allows for the re-engagement of senior leaders to keep leaders up to date with progress, answer any questions they may have, and ensure alignment with guidance and intent. The MPM also ensures exercise organization, scenario and timeline development, scheduling, logistics, and administrative requirements.

Discussion Points

• Comments on draft exercise documentation • Construction of the scenario timeline – usually the MSEL – if an additional MSEL

Planning Meeting will not be held • Identification of exercise venue artificialities and/or limitations • Agreement on final logistical items • Assignment of additional responsibilities

Exercise Tools

• Agenda • Briefing for presenting an overview of the exercise and meeting discussion points • IPM meeting minutes • Draft scenario timeline • Draft documentation (ExPlan, C/E Handbook) • Other selected documentation needed to illustrate exercise concepts and provide planning

guidance

Exercise Outcomes

• Fully reviewed exercise documentation (SitMan or ExPlan) • Draft Facilitator Guide or C/E Handbook, including EEGs • Well-developed scenario to include injects (if no MSEL Planning Meeting is scheduled) • Agreement on the exercise site • Identified logistics planning requirements • Finalization of date, time, and location on the MSEL Planning Meeting and Final Planning

Meeting

Follow-up

• Compile and distribute the MPM meeting minutes including next meeting date, time, and location

• Between meetings the planning team collaborates on assignments and prepares draft of exercise documentation

• Distribute draft documentation prior to the next scheduled meeting

Master Scenario Events List Meeting– A meeting for operations-based exercises that serves as a forum to build the Master Scenario Events List in detail.

Meeting Focus

A chronological listing that supplements the exercise scenario with: • Event synopses • Expected participant responses • Objectives and capabilities to be addressed • Responsible personnel • Specific injects and methods used to provide them

Discussion Points

• Tasks, conditions, and standards required to meet objectives • Key events and critical tasks • Event originator, target player, expected player actions, and timeframe • Contingency injects to prompt player action (if needed)

Exercise Tools

• Agenda • Previous meeting minutes • Draft exercise documentation • Applicable plans, policies, procedures • Agreed upon template used to create the MSEL



Exercise Outcomes

• Key event injects and delivery timeline identified • Responsibility for conducting remaining events is assigned • Draft documentation revision • Venue selection agreement • Identified logistics planning requirements • Timeline for completion

Follow-up

• Compile and distribute the MSEL meeting minutes, including the next meeting date, time, and location

• Between meetings: planning team collaborates on assignments and prepares draft exercise documentation

• Distribute draft documentation prior to next scheduled meeting Final Planning Meeting – A meeting that serves as the formal end of the exercise planning process. It is held to finalize exercise documentation and logistics.

Meeting Focus

Should be conducted for all exercises to ensure that all elements of the exercise are ready for conduct. Prior to FPM, the planning team receives final drafts of all exercise materials. No major changes to the exercise’s design, scope, or supporting documentation should take place at or following the FPM.

Discussion Points

• Conduct a comprehensive final review • Approve all remaining draft documents (e.g., SitMan, MSEL, C/E Handbook) and

presentation materials • Resolve any open planning issues and identify last-minute concerns • Review all exercise logistical activities (e.g., schedule, registration, attire, special needs)

Exercise Tools

• Agenda • Briefing for presenting an overview of the exercise and meeting discussion points • Previous meeting minutes from IPM, MPM, and MSEL if needed • All draft exercise documents and documentation • Previously finalized documents

Exercise Outcomes

• Final approval of exercise documents and material for production • Identified issues resolved • Attendees understand and approve exercise processes and procedures • Task assignments and logistical elements, including facilities, equipment, and schedules

are confirmed

Follow-up

The planning team: • Finalized all publications • Prepares all supporting materials • Rehearses presentations and briefings • Prepares to conduct the exercise • Prior to the exercise, documentation and any additional instructions should be

disseminated to all appropriate personnel

Table 3.1: Planning Meeting Types 560

Exercise Planning Team Considerations 561 The exercise planning team manages and is responsible for exercise design, 562 development, conduct, and evaluation. The membership of an exercise planning team 563 fits the type and scope of an exercise. 564



The exercise planning team is a manageable size yet represents the full range of 565 participating jurisdictions/organizations and stakeholders. For multi-566 jurisdictional/organizational exercises, planning team members include representatives 567 from each functional area or relevant discipline. 568

Typically, planning team members are not exercise players. With limited resources, 569 exercise planning team members may act as both planners and players but careful not 570 to divulge sensitive exercise information. Planning team members also help with 571 producing and distributing pre-exercise materials and conducting exercise planning 572 meetings, briefing, and training sessions. 573

To design and develop exercises most effectively, exercise planning teams: 574 • Identify and understand the desired objectives and associated capabilities for 575

the exercise, and design and develop the exercise; 576 • Adhere to a clear organizational structure, roles and responsibilities, and 577

accountability to the exercise planning team leader; 578 • Use proven management practices, processes, and tools, such as project 579

plans and timelines, status reports, and other communications; 580 • Incorporate evaluation planning and subject matter experts from the start of 581

exercise design and development; 582 • Use subject-matter experts to develop a realistic and challenging scenario. 583

Support agencies and organizations, including advocates for children, seniors, 584 individuals with disabilities, those with access and functional needs, diverse 585 communities, and people with limited English proficiency, should participate 586 throughout the planning process. In doing so, exercise planners incorporate a broad set 587 of perspectives and promote early understanding of roles, responsibilities, and 588 planning assumptions. 589

Exercise Planning Team Positions 590 Regardless of the scale and complexity of an exercise, the exercise planning team 591 should adhere to a coherent organizational structure that delineates roles and 592 responsibilities. An Exercise Director provides strategic oversight and direction to the 593 exercise planning team leader who manages the exercise planning team. In developing 594 a structure, the Exercise Director considers adequate span of control. The same 595 personnel may execute multiple functions depending on available resources. 596 The structure shown in Figure 3.2 can vary to reflect the scope of the exercise and the 597 available resources and personnel of participating jurisdictions/organizations: 598



599 Figure 3.2: Sample Exercise Planning Team 600

• Senior Leader: An individual(s) that provide the overarching guidance and 601 direction for the exercise program and intent for individual exercises. 602

• Exercise Director: An individual that provides the strategic oversight and 603 direction on the HSEEP exercise cycle phases for an individual exercise. 604

• Exercise Planning Team Lead: An individual that manages the exercise 605 planning team and coordinates with the Exercise Director and senior leaders. 606

• Safety Controller: An individual, during operations-based exercises, 607 responsible to ensure that the exercise is conducted in a safe and secure 608 environment 609

• Design and Development: Responsible for compiling and developing all 610 exercise background and facilitation or control, provides technical or 611 functional expertise for scenario development, developing the simulation 612 construct as applicable, and working with the evaluation and 613 resource/support functions to ensure all exercise design and development 614 needs are met. 615 o Scenario: Responsible to develop a realistic scenario and/or ground 616

truth/scenario background information based on identified threats or 617 hazards using subject matter expertise, plans, policies, and procedures 618 to be exercised, and overall exercise objectives. 619

o Control (operations-based exercise): Responsible that plans and 620 accounts for the control structure needed for an operations-based 621 exercise. This includes master control cell, individual venue control, 622 and simulation and modeling requirements. Will work with the 623



evaluation component in completing the controller/evaluator 624 handbook. 625

o Facilitation (discussion-based exercise): Responsible for the 626 facilitator guide for discussion-based exercises once the conduct and 627 evaluation materials have been completed; managing the recruitment 628 and training of facilitators as necessary; and work with evaluation 629 planners to confirm the facilitator is properly directed regarding 630 objectives and areas of focus. 631

o Modeling and Simulation/GIS: Responsible for coordinating the use 632 of modeling, simulation, and GIS for both discussion and operations-633 based exercises. Ensures modeling is coordinated with the scenario 634 and ground truth elements, and that simulation is coordinated with the 635 control element during exercise design and development. 636

o Master Scenario Events List (operations-based exercise): 637 Responsible for working with all other elements of exercise design and 638 development, as well as non-participating jurisdictions and subject 639 matter experts, to design and develop the MSEL timeline and events to 640 properly drive exercise play. 641

• Evaluation: Responsible for the development of the overall exercise 642 evaluation construct and all evaluation documentation; assigns evaluation 643 staffing in conjunction with the exercise planning team. Collects and reviews 644 the policies, plans, and procedures to be examined during exercise conduct 645 to finalize exercise objectives, match capability targets to exercise 646 objectives, and determine critical tasks to be evaluated in order to complete 647 exercise evaluation guides 648 o Evaluation Structure: Responsible for the development of the 649

evaluation methods, structure, and plan used during the exercise to 650 effectively examine and validate exercise objectives. Works with the 651 team developing exercise evaluation guides to ensure structure will 652 meet exercise needs. 653

o Exercise Evaluation Guide Structure: Responsible for the 654 development of the EEGs used for the exercise. 655

• Resource/Support: Responsible for logistics, administration, facility, and 656 finance support for planning meetings, exercise documentation, and other 657 planning considerations. 658 o Facilities: Responsible for coordinating the needed venue locations for 659

exercises including meeting rooms, conference centers, training 660 locations, and locations for operations-based exercises. 661

o Administration: Responsible for meeting scheduling and 662 administrative support such as: distributing meeting invitations, read 663 ahead materials, and meeting minutes; providing and collecting sign in 664 sheets; and note taking during planning meetings. 665

o Logistics: Responsible to ensure venues are adequately equipped with 666 A/V equipment, tables and chairs, and coordinates refreshments if 667 required. 668



Exercise Design 669

The core components of design include clarifying the purpose, establishing the 670 exercise scope, setting objectives, developing evaluation requirements, creating a 671 scenario, developing documentation, and determining media and public affairs 672 guidance. The exercise planning meetings serve as the principal mechanism for 673 executing the major steps of exercise design. 674

Exercise Purpose 675 The purpose of the exercise derives from a set of key factors determined from: 676

• A multi-year Integrated Preparedness Plan (IPP); 677 • Any existing plans and procedures; 678 • The Threat Hazard Identification Risk Assessment or other risk, threat, or 679

hazard assessments; 680 • Any past exercise or real-world After-Action Reports and Improvement 681

Plans (AAR/IP); and 682 • The grant or cooperative agreement requirements. 683

A review of the factors ensures the exercise builds and sustains a 684 jurisdiction’s/organization’s capabilities while taking prior lessons learned into 685 consideration in the exercise design process. Once the review is complete, each 686 exercise adheres to the progressive approach and is designed with the appropriate level 687 of scope and complexity within the range of exercises described in the Integrated 688 Preparedness Plan (IPP). 689

Exercise Scope 690 Determining exercise scope enables planners to “right-size” an exercise to meet the 691 objectives while staying within the resource and personnel constraints of the 692 exercising jurisdictions/organizations. 693 The elements defining exercise scope include: 694

• The exercise type; 695 • The participation level; 696 • The exercise location; 697 • The exercise duration; and 698 • Other considerations (for example, plans, policies, procedures tested, etc.). 699

Exercise Type 700 The purpose of the exercises determines the exercise type. For instance, if the intent is 701 to review and discuss a new policy, plan, or set of procedures, a discussion-based 702 exercise may be appropriate. If the intent is to assess the responders’ ability to 703 implement a plan, policy, or set of procedures, an operations-based exercise may be 704 appropriate. 705



Participation Level 706 Participating jurisdictions/organizations provide information such as: dates and times 707 of participation, number of personnel involved, positions of personnel (operational, 708 agency directors), and the intended outcomes. Participation level limitations may 709 include scheduling conflicts, real-world events, or other competing requirements. 710 Consider simulation to alleviate the participation level limitations. To ensure a 711 successful exercise, an Extent of Play Agreement outlines the agreement to participate. 712 Exercise Location 713 Discuss and decide suitable locations for the exercise to avoid limiting the scope or 714 defining artificialities required to simulate real-world events. Type of exercise will 715 determine the type of locations needed. Discussion-based exercise require facilities 716 with tables and chairs and operations-based require various venues. 717 Exercise Duration 718 When selecting the exercise duration, the planning team should determine how long 719 the exercise will take to address the exercise objectives effectively. In general, 720 discussion-based exercises are shorter than functional and full-scale exercises. 721 Resources also play a role in determining exercise duration. 722 Other Considerations 723 Additional exercise activities that may be useful to a jurisdiction/organization. Clearly 724 defining the exercise scope early in the design process helps exercise planners keep 725 the exercise to a manageable and realistic level. 726

Exercise Objectives 727 Based on direction from senior leadership, the exercise planning team selects one or 728 more exercise program priorities on which to focus an individual exercise. These 729 priorities drive the development of exercise objectives, which are distinct outcomes 730 that a jurisdiction/organization wishes to achieve during an exercise. The objectives 731 are aligned to a common set of capabilities which enables: 732

• A systematic tracking of progress throughout exercise programs and cycles; 733 • A standardized exercise data collection informing preparedness assessments; 734

and 735 • The fulfillment of grant or funding-specific reporting requirements. 736

Figure 3.3 shows the relationship between exercise program priorities, exercise 737 objectives, and capabilities. 738

Figure 3.3: Priorities, Objectives, and Capabilities 739



Exercise objectives should incorporate senior leadership intent, and exercise 740 participant’s plans, policies, and procedures, operating environment, and desired 741 outcomes. Planner should select a reasonable number of objectives to facilitate 742 effective scenario design, exercise conduct, and evaluation. An objective should be; 743 specific, measurable, achievable, relevant, and time-bound (SMART). Table 3.2 744 depicts guidelines for developing SMART objectives. 745

746 Table 3.2: SMART Guidelines for Exercise Objectives 747

Evaluation Parameters 748 The development of exercise evaluation parameters begins early in the exercise 749 process and helps to guide the development of the exercise scenario, discussion 750 questions or event timeline, and evaluation documentation. Once the exercise 751 objective are aligned to capabilities, the exercise planning team identifies the linked 752 capability target and critical tasks. Chapter 5: Evaluation provides additional details 753 regarding capability targets and critical tasks. 754

Scenario 755 A model or outline of the simulated sequence of events that drives participant 756 discussions or actions. The scenario can be a written narrative or depicted by an event 757 timeline. The scenario should be plausible, realistic, and challenging so participants 758 can meet the objectives. After developing the scope and objectives, the exercise 759 planning team should select and develop scenarios to enable an exercise to assess 760 objectives and capabilities. 761 The elements to a scenario are: 762

• The conditions allowing players to demonstrate their ability to meet 763 exercise objectives; 764



• The technical details necessary to accurately depict scenario conditions and 765 events (for example, the date and time of event and damage resulting from 766 the event); and 767

• A general context or comprehensive narrative. 768 Threat or Hazard 769 The identification of a threat or hazard originates from the organization’s/jurisdiction’s Threat 770 and Hazard Identification and Risk Assessment.7 Risk analyses enable 771 jurisdictions/organizations to identify potential events that stress preparedness capabilities. The 772 type of threat or hazard defines a scenario allowing for assessment of the objectives and 773 capabilities of the exercise. 774 Modeling and Simulation 775 A model is a representation of a system at a point in time or space intended to expand 776 an understanding of the real system. Simulation is a method of implementing the 777 performance of a model, or combination of models, over time. Modeling and 778 simulation support decision-making processes by providing human and computer 779 feedback to players during exercise play, thus dynamically representing the impact of 780 their decisions. Modeling and simulation may apply to situations where achieving 781 reality is difficult. The use of modeling and simulation realistically replicates variables 782 such as disease propagation, radiation, and chemical attacks. When incorporated into 783 the development of the scenario and overall exercise design, modeling, and simulation 784 brings versatility, cost savings, and fidelity to exercises.8 785

Exercise Documentation 786 Exercise documentation provides the framework for exercise conduct and evaluation. 787 The documentation helps the planning team apply the HSEEP Exercise Cycle. 788 Table 3.3 lists the most used exercise design and development documents (optional to 789 use) identified by the exercise type and relevant audience. 790

Discussion-Based Exercise Documentation

Document Description Distribution

Situation Manual A document that provides background information and serves as the primary reference material for exercise participants (core document for discussion-based exercises)

All Participants

Player Handout A document that supplements the Situation Manual and presentation providing players a quick-reference guide sometimes referred to as a placemat”

Players

Facilitator Guide A document that outlines instructions and key issues discussed and utilized by the exercise Facilitator to move participants through exercise play

Exercise Facilitator(s)

Presentation A document that provides background information for participants during exercise conduct All Participants

7 For further guidance on identifying and assessing risks and associated impacts, please refer to the U.S. Department of Homeland Security Comprehensive Preparedness Guide 201: Threat and Hazard Identification and Risk Assessment Guide, Third Edition, May 2018. 8 For more information please visit https://gis.fema.gov/Model-and-Data-Inventory/index.html



Exercise Evaluation Guide(s)

A document that provides a tool to organize exercise observations pertaining to the evaluation criteria Evaluators

Participant Feedback Form

A document that provides a mechanism to collects input from all participants following an exercise and supports the development of an After-Action Report and Improvement Plan (AAR/IP)

All Participants

791 Operations-Based Exercise Documentation


Exercise Plan A document that provides general information to exercise participants including exercise objectives and scope, not including the scenario information

Players and Observers

Player Handout A document that supplements the Exercise Plan and provides players with a quick-reference guide sometimes referred to as a pocketsize trifold or bifold

Players

Ground Truth Document

A document comprised of the detailed elements of a scenario that must remain consistent during exercise development and conduct to ensure that realism is maintained and objectives can be achieved.

All Participants

Controller & Evaluator Handbook

A document that provides specific exercise information and guidance for controllers and evaluators, may be a stand-alone document or supplement to the Exercise Plan, and adjusted accordingly

Controllers and Evaluators

Master Scenario Events List (MSEL)

A list of chronological events provided to players to drive exercise play, a single document containing events (injects and expected actions)

Controllers, Evaluators, and Simulator

Extent of Play Agreement

An agreement outlining the participating jurisdiction’s/organization’s level of play during an exercise

Exercise Planning Team

Exercise Evaluation Guides

A tool to organize exercise observations pertaining to the evaluation criteria Evaluators

Control Staff Instructions (COSIN)

A document from the controller portion of the C/E Handbook for large exercises, it provides detailed guidelines, procedures, responsibilities, and a management structure for exercise control, simulation, and support.

Controllers, Simulators, and Exercise Support

Evaluation Plan (EvalPlan)

A document for evaluators that provides guidance, instructions, methodology, and structure on evaluating and observing during an exercise.

Evaluators


A mechanism to collects input from all participants following an exercise and supports the development of an After-Action Report and Improvement Plan (AAR/IP)

All Participants

Table 3.3: Discussion-Based and Operation-Based Exercise Documents 792 Areas to consider when developing the exercise documentation are: 793

• Classification. Understand the importance of specific requirements for 794 security marking rules and requirements. 795

• Accessibility. Verify consideration and accessibility of presentations and 796 documents (for example, large print, compact disc, Braille), closed 797 captioning or another form of text display, or the provision of sign language 798 interpreters. 799



• Distribution. Understand the types of information and the dissemination of 800 information needs consideration (for example, the printing and circulating 801 classified scenario details). 802

Table 3.4 describes other execration documentation used during an exercise where 803 applicable: 804

Other Exercise Documentation

Document Description Distribution Table Tents An item on each table identifying the functional area seated at that table All Tables

Name Badge An item that each participant wears to clearly identify their name and agency/organization All Participants

Liability Waiver

A document for each actor, an individual who simulates a specific role, such as a disaster casualty victim, to add realism to an exercise, should receive a waiver form prior to the exercise. Signing a waiver form releases liability for all exercise planners and participants. Exercising entities should use discretion when recruiting Actors under the age of 18-years-old because of additional challenges and concerns related to liability. If the exercise requires volunteers younger than 18-years-old, parents or legal guardians must sign their waiver forms.

Actors

Photo Consent Form

A document for photographs prior to the exercise. Signing the form releases permission for all exercise participants photographed and their photo used for informational and educational purposes.

All Participants

Weapons and Safety Policy

A document for all exercises, where applicable, should employ a written weapon and safety policy in accordance with applicable state or local laws and regulations. Exercise sponsors should coordinate the application of a weapons and safety policy with the appropriate safety and legal departments as necessary.

All Participants

Symptomology Card

An item containing the signs and symptoms that an exercise actor will portray, as well as information helpful for players.

Actors and Players

Press Release A document that the exercise planning team develops for the media and the public to inform about the exercise.

Media and Public

Table 3.4: Other Exercise Documentation 805 Planning for Media and Public Affairs 806 Media and public affairs can be important participants for an exercise. Media and 807 public affairs may be players or observers. Exercise sponsors should work to 808 incorporate media related issues into exercise planning. 809

Press Release 810 Before an exercise, the exercise planning team develops a written press release to 811 disseminate to media outlets, including web-based and social media outlets, 812 Observers, senior leadership, and other very important persons, as appropriate. The 813 press release informs the media and the public about general exercise information. The 814 press release does not contain detailed scenario information, such as the type of threat 815 or hazard, nor contain information that might hinder meeting exercise objectives if a 816 participant read the release. 817

The contents of a media or public information release include the following: 818



• An introduction, including sponsor and exercise program information; 819 • The exercise scope and objectives; 820 • Any general scenario information; and 821 • The participating agencies or disciplines. 822

Public Announcement 823 Public announcements should occur before any exercise involving public space or 824 space viewable to the public and helps avoid public confusion. The announcement will 825 also help the public avoid congestion near the exercise site by providing suggestions 826 for alternate routes. Examples of announcements are local media, community alert and 827 notification systems, and directly on signs near the exercise site. 828 Media Policy 829 The agency or jurisdiction/organization sponsoring the exercise should decide whether 830 to invite media representatives to the exercise. If invited, media representatives 831 conduct interviews with exercise planners and participants prior to the exercise. 832 At discussion-based exercises, media representatives should not be present during the 833 discussion of any potentially sensitive information. Also, avoid the filming of exercise 834 conduct, which may hinder discussion or the flow of play. 835

Exercise Development 836

Exercise development involves planning for the critical elements of exercise conduct: 837 logistics, 838 control, and evaluation. Exercise logistics are important to create a smooth and 839 seamless exercise. Exercise logistics include venue, badging and identification, 840 parking, transportation, and designated areas, and media. 841

Planning for Discussion-Based Exercise Logistics 842 Logistics for discussions-based exercises include venue, registration and badging, 843 table and breakout identification, parking, and media, public affairs, and VIPs. 844 Venue 845

Facility and Room 846 Conduct meetings, briefings, and exercises in facilities appropriate for the exercise 847 scope and attendance. Reserve facilities solely for exercise purposes, free from 848 distractions, and accessible to all participants. 849 Exercise planners should account for the following considerations: 850

• Ensure enough tables and chairs for every participant; 851 • Arrange tables to best suit the meeting or exercise (for example, U-shaped 852

layout for exercises requiring facilitation and participant interaction); 853 • Select a facility with room acoustics encourages discussion; and 854 • Select a facility with the accessibility of parking and restrooms for all participants. 855



Audio/Visual Requirements 856 Identify audio/visual requirements during the development phase, including 857 individuals assigned to ensure equipment is properly functioning. Audio/visual 858 equipment may include a projector, screens, microphones, and speakers. 859

Supplies, Food, and Refreshments 860 Procure supplies such as writing utensils, notepads, easels, copies of plans and 861 procedures, name badges, and any other necessary equipment before the exercise and 862 provided to participants. The exercise planning team should consider, in accordance 863 with applicable funding guidance or venue policies, whether to provide food and 864 refreshments for participants. 865 Registration and Badging 866 Participants register upon arrival and for security purposes should wear a form of 867 identification such as uniforms or badges with name and jurisdiction/organization. 868 Participants should, at a minimum, provide their name, jurisdiction/organization, 869 telephone number, and e-mail address. The exercise planning team retains copies of 870 the sign-in sheets for follow-up correspondence. 871 Table and Breakout Identification 872 The exercise planning team or exercise sponsor places name tents on tables prior to 873 Start of Exercise to ensure proper seating arrangements. Use table tents at each table to 874 identify the jurisdiction/organization or functional area. 875 Parking 876 Clearly label established parking areas for participants arriving in personally owned 877 vehicles. If needed, law enforcement personnel may direct vehicles to proper parking 878 areas. 879 Media, Public Affairs, and VIPs 880 Media, public affairs, and VIPs personnel should always have an escort. Identify a 881 designated location for the media prior to the start of the exercise. 882

Planning for Operations-Based Exercise Logistics 883 Logistics for operations-based exercises include venues; registration and badging; 884 parking and transportation; actors; and media. 885 Venues 886

Facilities and Exercise Areas 887 Operations-based exercises often have several areas for exercise conduct. 888 These areas may include: 889

• Player Venues 890 o Exercise Play Area. A location where player activities and tasks are 891

demonstrated during an exercise. 892 o Response Route. A location for responding emergency units from the 893

assembly area to the staging area or other exercise sites during a 894 response-focused exercise. 895



• Controller Venues 896 o Exercise Assembly Area. A location for participants and resources to 897

gather and dispatch, managed by controller(s). 898 o Master Control Cell. A location where overall coordination is 899

managed between venue control cells, simulation cell and other control 900 areas. 901

o Venue Control Cell. A location where controllers manage individual 902 injects designed for their relevant players. 903

o Simulation Cell. A location from which controllers deliver messages 904 representing actions, activities, and conversations of an individual, 905 agency, or organization that is not participating in the exercise. 906

o Emergency Medical Services Area. A location is for real-world 907 response support, such as treatment sector areas, ambulance staging, 908 and transportation coordination points. 909

o Observer and Media Area. A location for observers and real-world 910 media representatives with a view of the exercise but prevents them 911 from interfering with exercise play. 912

Audio/Visual Requirements 913 Identify audio/visual requirements during the design phase, including individuals 914 assigned to ensure equipment is properly functioning. 915

Supplies, Food, and Refreshments 916 Exercise planners should not assume participants will bring necessary supplies with 917 them. Procure any supplies (for example, vests, clipboards, signage, writing utensils, 918 notepads, easels, copies of plans and procedures, name badges, and any other 919 necessary equipment) before the exercise conduct and provided to participants. 920 The exercise planning team should consider food and refreshments for participants at 921 all venues. Hydration is an important aspect of an operations-based exercise. 922 Registration and Badging 923 Participants register upon arrival and for security purposes should wear a form of 924 identification such as uniforms or badges with name and jurisdiction/organization. 925 Participants should, at a minimum, provide their name, jurisdiction/organization, 926 telephone number, and e-mail address. The exercise planning team retains copies of 927 the sign-in sheets for follow-up correspondence. 928 Parking and Transportation 929 Clearly label established parking areas for participants arriving in personal owned 930 vehicles at all venues. If needed, law enforcement personnel may direct vehicles to 931 proper parking areas and transportation to preposition controllers, players, and actors 932 in advance of the exercise. 933



Actors 934 Actors add realism and prompt players to react to a scenario event. Exercise planning 935 team members can recruit from various organizations9. 936 Prior to the exercise, actors should receive the following: 937

• The liability waiver for signature; 938 • The actor instructions; including information on when to arrive, where to 939

report, and other logistical details; and 940 • The symptomatology cards, when applicable. 941

Media, Public Affairs, and VIPs 942 Media, public affairs, and VIPs personnel should always have an escort. Identify a 943 designated location for the media prior to the start of the exercise. 944

Planning for Discussion-Based Exercise Facilitation 945 Exercise facilitation is a method that uses a facilitator/presenter with functional or 946 subject-matter expertise and awareness of appropriate plans and procedures for 947 discussion-based exercises. Typically occur in a plenary session or in breakout 948 groups. 949 Staffing 950 The planning team identifies the number of facilitators needed for exercise conduct. 951 Facilitators deliver scenario information and maintain a focused discussion. If 952 participants divide into groups for part of conduct, assign a table Facilitator to each 953 group. 954 Facilitation Structure 955 The facilitation structure is the framework that allow facilitators to coordinate amongst one 956 another. Figure 3.4 shows a sample facilitation structure. 957

Figure 3.4: Sample Exercise Facilitation Organization 958 • Facilitator/Presenter: An individual responsible for keeping a discussion focused on 959

exercise objectives and exploring all issues within the time allotted during discussion-960 based exercises or an individual designated to deliver information in a structured 961 setting. 962

9 Examples of organizations are local colleges and universities, medical and nursing schools, drama clubs, theaters, civic groups, emergency response academies, Community Emergency Response Teams, and federal and state military units. Including volunteer actors from within the access and functional needs population provides an opportunity to exercise in a variety of operational environments.



• Breakout (as needed): A location where the exercise conduct is taking 963 place. 964 o Facilitator/Presenter: Additional individual(s) responsible for keeping a 965

discussion focused on objectives for each of the breakout groups as needed. 966 Prior to the exercise, the facilitator conducts a walk-through of the exercise design, 967 event timeline, delivery methods, and facilitation techniques. 968

Planning for Operations-Based Exercise Control 969 Exercise control maintains exercise scope, pace, and integrity during conduct under 970 safe and secure conditions. Elements of exercise control include controller staffing, 971 structure, training, communications, safety, and security. 972 Staffing 973 The planning team identifies the number of controllers needed during the exercise to 974 deliver and track information. As a guiding principle, place at least one controller at 975 every venue whenever possible to help ensure a safe exercise with proper security 976 controls. Resource constraints may make placing a controller at every site challenging, 977 therefore multitasking personnel as both a controller and an evaluator is an option. 978 Control Structure 979 The control structure is the framework that allows controllers and simulators to 980 communicate and coordinate at multiple locations. The control structure ensures 981 exercise play progresses to meet exercise objectives, deliver, and track exercise 982 information. Figure 3.5 shows a sample control structure with multiple control cells. 983

Figure 3.5: Sample Control Organization 984

• Master Control Cell: A location where overall coordination is managed 985 between venue control cells, simulation cell, and other control areas. 986



o Lead Controller: An individual that monitors exercise progression, 987 communicates exercise activities throughout all venues, and 988 responsible for managing the control staff. 989

• MSEL Manager: An individual that manages the timely and accurate 990 delivery of injects, player expected actions, and decides on the need to 991 release contingency injects. 992

• Simulation Cell (SimCell): A location from which controllers 993 deliver scenario messages representing actions, activities, and 994 conversations of an individual, agency, or organization that is 995 not participating in the exercise 996

• SimCell Controller: An individual that coordinates with the 997 MSEL manager, or may act as the MSEL manager, and directs 998 simulators and actors. 999

• Ground Truth Advisor: An individual responsible for ensuring 1000 that the scenario details remain consistent during exercise 1001 conduct. 1002

• Venue Control Cell: A location where controllers manage 1003 individual injects designed for their relevant players. 1004

• Venue Controller: An individual that monitors exercise 1005 progression, communicates exercise activities throughout all 1006 venues, and responsible for managing the control staff. 1007

Control Cell 1008 In an exercise involving multiple jurisdictions/organizations in one location, a control 1009 cell serves as the central node for sharing information among controllers at the various 1010 sites. A control cell compiles all the information into a common exercise picture. If an 1011 exercise contains multiple geographic locations, consider establishing several venue 1012 control cells to communicate and coordinate through a master control cell. When an 1013 exercise requires the establishment of multiple venue control cells, defining the roles 1014 and relationships and the decision-making hierarchy, is important. Ideally, a control 1015 cell will contain a point of contact or a liaison representing each participating 1016 jurisdiction/organization. In exercises involving both classified and unclassified 1017 information, separate control cells with appropriate security firewalls set up to 1018 appropriately for classified and unclassified information is required 1019

Simulation Cell 1020 The Simulation Cell is a working location for several qualified professionals who 1021 represent the non-participating jurisdictions/organizations. A Simulation Cell delivers 1022 injects, receives player responses, and provides information in place of 1023 nonparticipating jurisdictions/organizations that would likely participate actively if 1024 exercise events were real. The professionals are knowledgeable of the 1025 jurisdictions/organizations portrayed and deliver injects in a realistic manner. 1026 Depending on the type of exercise, the Simulation Cell may require a telephone, 1027 computer, e-mail account, radio, or other means of communication. Moreover, if an 1028 exercise uses a Simulation Cell to drive exercise play, a determination is made on how 1029 to staff and integrate the SimCell into the broader control structure. 1030 Master Scenario Events List 1031



A MSEL is a chronological list of the events that drives exercise play in an operations-1032 based exercise. 1033 Each MSEL entry includes the following at a minimum: 1034

• A designated scenario time; 1035 • An event synopsis; 1036 • A controller responsible for delivering the inject, with controller or evaluator 1037

special instructions (if applicable); 1038 • An intended player; 1039 • An expected participant response; 1040 • The exercise objective; and 1041 • Notes section (for controllers and evaluators to track actual events against 1042

those listed in the MSEL, with special instructions for individual controllers 1043 and evaluators). 1044

Scenario timelines listed in the MSEL should be as realistic as possible and based on 1045 input from subject matter experts. If the activity occurs sooner than the MSEL writers 1046 anticipated, then controllers and evaluators should note the time the activity occurred, 1047 but play should not be interrupted. 1048 Figure 3.6 shows the MSEL and event types. 1049

Figure 3.6: MSEL and Event Types 1050 The types of descriptive MSEL events that support exercise play include: 1051

• Inject. Master Scenario Events List event introduced to a player by the 1052 control staff, representing non-playing entities, to build the exercise 1053 environment based on the exercise scenario and to drive operations-based 1054 exercise play. 1055

• Expected Action. A MSEL event represents an anticipated action to be 1056 taken by a player during the exercise. 1057

• Contingency Inject. A MSEL event introduced to a player by a control staff 1058 when a key player expected action did not occur as planned to provide an 1059 additional opportunity to meet exercise objectives. 1060

The MSELs have long formats, short formats, or include both. A short-format MSEL 1061 lists the injects in a single row in a spreadsheet format. The single rows are a quick-1062 reference guide during exercise play or projected onto a large screen in a control cell 1063



or Simulation Cell. A long-format MSEL includes a more detailed description of 1064 expected actions and the exact scripting language for actors and simulators. 1065

Controller Training 1066 Controller training includes a summary of the exercise design including the exercise 1067 objectives, a MSEL walk-thru, delivery methods, venues, timeline, communications 1068 plan, and evaluation criteria. 1069

Communications 1070 The C/E Handbook or COSIN provides a communication plan for controllers and 1071 instructions on; who they will communicate with, what they need to communicate, and 1072 how they will communicate. Communication can occur though many forms during 1073 conduct of an exercise such as phone, radio, e-mail, face-to-face, network system, or a 1074 combination. Depending on the method used, the essential equipment should be 1075 provided to the controllers and the control cells. 1076 Safety and Security 1077 Controllers also play an important role in ensuring that the exercise is a safe and 1078 secure environment. In exercises involving potentially dangerous field play or the use 1079 of classified materials, the control team designates a Safety Controller(s) to focus on 1080 those areas of control. 1081

Safety 1082 The most important consideration in planning any exercise is safety. Dependent on the 1083 type of exercise will depend on the level of safety required. 1084 The planning team should consider the following to ensure a safe environment: 1085

• Appointing a Safety Controller(s); 1086 • Dedicating real-world medical (not involved in the exercise) and emergency 1087

procedures to contact and deploy ambulance support; 1088 • Outlining safety requirements and policies to include other safety issues 1089

outside the scope of the exercise (weather, heat stress, hypothermia, etc.) 1090

Security 1091

Since most exercises are sensitive in nature, securing the site is an important item for 1092 the planning team to consider. Local law enforcement can provide site security where 1093 appropriate and should be included in the planning process. For exercises that include 1094 classified information, planners should identify and adhere to appropriate security 1095 standards to ensure that this information is not compromised. 1096

Planning for Exercise Evaluation 1097 Thorough planning and organization prior to an exercise are imperative to effective 1098 and successful exercise evaluation. The exercise planning team identifies evaluation 1099 elements early in the exercise design process. Also, during exercise development, an 1100 evaluation team determines the appropriate structure, organizes the team 1101 appropriately, and develops a comprehensive plan to address the evaluation of the 1102 exercise. 1103



Although evaluators receive pre-exercise training during the controller and evaluator 1104 briefing, evaluators may need additional sessions to walk-through the following areas: 1105

• Objectives and Exercise Evaluation Guide assignment; 1106 • Additional scenario details to support evaluator’s objectives; and 1107 • Data collection techniques. 1108

Chapter 5: Evaluation provides additional details regarding evaluation planning and 1109 training. 1110

Preparing for Exercise Conduct 1111 The exercise planning team finalizes all aspects of the exercise to prepare for conduct. 1112 Preparations include finalizing materials, rehearsing presentations and briefings, and 1113 training participants for the exercise. Prior to the exercise, the appropriate personnel 1114 (for example, presenters, facilitators, controllers, evaluators, simulator) receive 1115 documentation and additional instructions needed for conduct. 1116 To prevent jeopardizing mission performance in response to real-world events, the 1117 exercise planning team should maintain a contingency process to halt, postpone, or 1118 cancel an exercise as necessary. If the conduct of the exercise is put at risk, any efforts 1119 to respond to real-world events, or real-world events hinder the conduct of the 1120 exercise, the Exercise Director and exercise planning team should convene and 1121 determine the appropriate course of action. Following decision on a final course of 1122 action, the Exercise Director should communicate that course of action to all exercise 1123 planners, participants, and other key stakeholders through all formal communications 1124 mechanisms. 1125



4. Exercise Conduct 1126

Overview 1127 Exercise conduct involves activities such as preparing for exercise play, managing 1128 exercise play, and conducting immediate exercise wrap-up activities. Exercise 1129 practitioners apply and adapt the HSEEP doctrine on exercise conduct to meet specific 1130 needs. Throughout the exercise conduct process, the engagement of senior leadership 1131 by practitioners confirms the exercise follows in line with the original guidance and 1132 intent. 1133

Exercise Play Preparation 1134

Setup for Discussion-Based Exercises 1135 Members of the exercise planning team assigned to exercise setup should visit the 1136 exercise site before the event to arrange the room, test the audio/visual equipment, and 1137 discuss administrative and logistical issues. Prior to exercise conduct, the exercise 1138 planning team delivers the necessary exercise materials and equipment, which may 1139 include the following: 1140

• Any Situational Manuals or other written materials for exercise participants; 1141 • The presentations; 1142 • The appropriate audio/visual equipment including televisions, projectors, 1143

projection screens, microphones, and speakers; 1144 • The table tents for each table; 1145 • The name tents for each participant; 1146 • The badges identifying the role of each exercise participant; 1147 • The sign-in sheets; and 1148

The Participant Feedback Forms 1149 A rehearsal of the exercise structure and presentations helps to ensure an 1150 understanding of facilitator and evaluator roles and responsibilities, event timing, 1151 audio/visual equipment, and location specific details including access and security. 1152 On the day of the exercise, planning team members should arrive several hours before 1153 Start of Exercise to setup activities and arrange for registration. 1154

Setup for Operations-Based Exercises 1155 Depending on the scope of the simulated environment, the appropriate exercise 1156 planning team members should begin event setup as many days before the exercise as 1157 necessary. A communication check is important prior to the start of the exercise to 1158 ensure all forms of communications are working correctly. Prior to exercise conduct, 1159 the exercise planning team delivers the necessary exercise materials and equipment, 1160 which may include: 1161

• Arranging the briefing rooms; 1162 • Testing the audio/visual equipment; 1163 • Placing props and effects; 1164



• Marking the appropriate exercise areas and the perimeters; and 1165 • Checking for potential safety issues. 1166

A rehearsal of the exercise structure is vital to ensure an understanding of controller 1167 and evaluator responsibilities, transportation, event timing, 1168 audio/visual/communications equipment, and location specific details including access 1169 and security. 1170 On the day of the exercise, all exercise planning team members should arrive several 1171 hours before Start of Exercise to complete any remaining logistical or administrative 1172 items and setup registration. 1173

Briefings 1174 Before an exercise, briefings educate participants about their roles and responsibilities. 1175 By scheduling separate briefings for senior leadership, controllers and evaluators, 1176 actors, players, and observers, exercise planning team members can avoid giving 1177 extraneous material to different groups and ensure the design, development, and 1178 conduct of an exercise aligns to the senior leadership guidance. 1179 Table 4.1 describes the exercise brief titles and descriptions: 1180

Brief Description

Senior Leadership Brief

Briefing occurs during the design and development and before the conduct of an exercise. The exercise planning team leader periodically consults with the senior leadership within the exercise planning team to ensure the exercise aligns with leadership intent.

Controller and Evaluator Brief

Briefing is generally conducted before operations-based exercises. It begins with an exercise overview and then reviews the exercise location and area, schedule of events, scenario, control concept, controller and evaluator responsibilities, instructions on completing EEGs, and any miscellaneous information. Additional training for evaluators may be conducted.

Actor Brief

Briefing occurs before the exercise, prior to the Actors taking their positions. The Actor Controller leads the Actor Briefing, a meeting generally conducted before the exercise, providing actors with; an exercise overview, safety, real emergency procedures, acting instructions, schedule, Identification badges, and symptomatology cards. distributes identification badges and symptomatology cards, and includes the following: an exercise overview, safety information, real emergency procedures, symptomatology, acting instructions, and the schedule.

Player Brief

Shortly before the Start of Exercise, a controller conducts a briefing for all players to address individual roles and responsibilities, exercise parameters, safety, security badges, and any remaining logistical exercise concerns or questions. Participant handouts and Exercise Plans or Situational Manuals, depending on the type of exercise being conducted, are often distributed during this briefing. Following the exercise, controllers ensure that appropriate players attend the post-exercise hotwash in their respective functional area.

Observer Brief

Briefing occurs the day of an exercise and informs Observers and very important persons about the exercise background, scenario, schedule of events, Observer limitations, and any other miscellaneous information. Often, Observers are unfamiliar with public safety procedures and have questions about the activities, therefore designating someone, such as a public information officer to answer questions prevents Observers from interrupting exercise participants.

Table 4.1: Types of Exercise Briefs and Descriptions 1181



Exercise Play 1182

Conduct for Discussion-Based Exercises 1183 For a discussion-based exercise, conduct entails presentation, facilitation, and discussion. 1184 Facilitators and evaluators report their activities to the Lead Facilitator or the Lead Evaluator, 1185 depending on the position being held. Figure 4.1 shows a sample of a discussion-based exercise 1186 organization. 1187

Figure 4.1: Sample Discussion-Based Exercise Organization 1188

• Exercise Venue: A location designated as the primary facility or 1189 geographic area where participants gather to conduct an exercise. 1190 o Senior Leader: Individual(s) that provide the overarching guidance 1191

and direction for the exercise program and intent for individual 1192 exercises 1193

o Exercise Director: An individual that provides the strategic oversight 1194 and direction on the HSEEP exercise cycle phases for an individual 1195 exercise. 1196

o Safety Controller: An individual, during operations-based exercises, 1197 responsible to ensure that the exercise is conducted in a safe and 1198 secure environment. 1199

• Lead Facilitator/Presenter: An individual that oversees all facets of the 1200 facilitation process or the presentation(s) to include recruiting and assigning 1201 additional facilitators or presenters. 1202 o Facilitator/Presenter: An individual responsible for keeping a 1203

discussion focused on exercise objectives and exploring all issues 1204 within the time allotted during discussion-based exercises or an 1205 individual designated to deliver information in a structured setting 1206

• Lead Evaluator: An individual that oversees all facets of the evaluation 1207 process to include recruiting, assigning, and training evaluators. 1208



o Evaluator/Note-taker: An individual chosen based on their expertise 1209 in the functional areas, to observe and collect exercise, and analyze 1210 results 1211

• Resource Lead: An individual responsible for obtaining proper venues, 1212 equipment, and supplies for exercise conduct as well as providing support 1213 for media and VIP observers. 1214 o Facilities Lead: Responsible for managing exercise venue 1215

considerations such as set up, tear down, table and breakout room 1216 assignments as required. 1217

o Administration Lead: An individual responsible for managing the 1218 registration process, printed documents, sign in sheets, and badges. 1219

o Logistics Lead: An individual responsible to ensure proper room 1220 function and set up for audio/visual requirements as well as obtaining 1221 necessary equipment, food, and drinks and works with the facilities 1222 lead. 1223

Conduct activities for a discussion-based exercise include: 1224 • Presentation is a crucial vehicle for conveying information to the players 1225 • Facilitators help evaluators collect useful data by keeping discussions focused on 1226

exercise objectives, capabilities, capability targets, and critical tasks 1227 • Facilitators make sure all issues are explored within the time allotted 1228 • Ensuring a safe and secure exercise environment 1229

All facilitators should take and compile notes relevant to the groups’ discussions. If multiple 1230 facilitators were used, the Lead Facilitator will collect all notes from each of the facilitators. 1231 This information will be used to generate the After-Action Report/Improvement Plan (AAR/IP) 1232 and exercise notes. 1233

Conduct for Operations-Based Exercises 1234 During conduct of operations-based exercises, the exercise planning team leader often serves the 1235 Lead Controller. Controllers and evaluators report activities to the Lead Controller who is 1236 responsible for monitoring exercise progression, communicates exercise activities throughout all 1237



venues, and manages the control staff. Figure 4.2 shows a sample of an operations-based 1238 exercise organization. 1239

Figure 4.2: Sample Operations-Based Exercise Organization 1240

• Main Exercise Venue: A location designated as the primary facility or 1241 geographic area where participants gather to conduct an exercise. 1242 o Senior Leader: Individual(s) that provide the overarching guidance 1243

and direction for the exercise program and intent for individual 1244 exercises 1245

o Exercise Director: An individual that provides the strategic oversight 1246 and direction on the HSEEP exercise cycle phases for an individual 1247 exercise. 1248

o Safety Controller: An individual, during operations-based exercises, 1249 responsible to ensure that the exercise is conducted in a safe and 1250 secure environment. 1251

• Master Control Cell: A location where overall coordination is managed 1252 between venue control cells, simulation cell, and other control areas. 1253 o Lead Controller: An individual that monitors exercise progression, 1254

communicates exercise activities throughout all venues, and 1255 responsible for managing the control staff. 1256

• Simulation Cell (SimCell): A location from which controllers 1257 deliver scenario messages representing actions, activities, and 1258 conversations of an individual, agency, or organization that is 1259 not participating in the exercise 1260

• SIMCELL Controller: An individual that coordinates with the 1261 MSEL manager, or may act as the MSEL manager, and directs 1262 simulators and actors. 1263

• Ground Truth Advisor: An individual responsible for ensuring 1264 that the scenario details remain consistent during exercise 1265 conduct. 1266

• Venue Control Cell: A location where controllers manage 1267 individual injects designed for their relevant players. 1268

• Venue Controller: An individual that monitors exercise 1269 progression, communicates exercise activities throughout all 1270 venues, and responsible for managing the control staff. 1271

• MSEL Manager: Directs the delivery of injects and decides on 1272 the need for creation of contingency injects. Responsible for 1273 timely and accurate delivery of all exercise injects. 1274

• Lead Evaluator: An individual that oversees all facets of the evaluation 1275 process to include recruiting, assigning, and training evaluator 1276 o Venue Evaluator: An individual responsible for observing exercise 1277

activity/play in their assigned exercise venue, area, or for a specific 1278 activity. 1279

• Resource Lead: An individual responsible for obtaining proper venues, 1280 equipment, and supplies for exercise conduct as well as providing support 1281 for media and VIP observers. 1282



o Facilities Lead: An Individual responsible to work closely with the 1283 logistics section to manage exercise venue considerations such as set 1284 up, tear down, scheduling, and traffic. 1285

o Administration Lead: An individual responsible for managing the 1286 registration process, printed documents, sign in sheets, and badges. 1287

o Logistics Lead: An individual responsible to ensure proper room 1288 function and set up for audio/visual requirements as well as obtaining 1289 necessary equipment, food, and drinks and works with the facilities 1290 lead. 1291

Conduct activities for an operations-based exercise include: 1292

• Ensuring a safe and secure exercise environment 1293 • Exercise documents that outline roles, responsibilities, and a communication plan for 1294

the control staff 1295 • Importance of communications between the Simulation Cell, venue controllers, and 1296

MSEL manager to ensure pace of the exercise play is appropriate 1297 • Importance of the Exercise Assembly Area Controller remains in close communication 1298

with other controllers throughout the exercise 1299 • The Exercise Assembly Area Controller ensures that units are deployed based on the 1300

timeline that is created by the planning team 1301 • Security for the exercise is maintained for all players 1302

The Exercise Director and the planning team should have a contingency plan to pause, postpone, 1303 or cancel in the event of a real-world emergency impeding the exercise. 1304

Wrap-Up Activities 1305 Performing thorough exercise wrap-up ensures the collection of all relevant data to 1306 support effective evaluation and improvement planning. Immediately after the 1307 exercise, the exercise planning team members should conduct a short debrief to 1308 discuss the level of satisfaction with the exercise, any issues, or concerns, and propose 1309 improvements. Planners should collect exercise attendance lists, provide copies to the 1310 exercise planning team leader, collect Participant Feedback Forms, and develop 1311 debrief notes. 1312

Player Hotwash 1313 A meeting that provides an opportunity to discuss exercise strengths and areas for 1314 improvement immediately following the conduct of an exercise. A hotwash provides 1315 an opportunity for exercise participants to discuss exercise strengths and areas for 1316 improvement immediately following the conduct of an exercise. An experienced 1317 Facilitator leads the hotwash and ensures the discussion remains brief and 1318 constructive. The information gathered during a hotwash contributes to the After-1319 Action Report/Improvement Plan (AAR/IP) process and any exercise suggestions can 1320 improve future exercises. For operations-based exercises, a hotwash occurs for each 1321 functional area by that functional area’s controller or evaluator immediately following 1322 an exercise. A hotwash also provides an opportunity for players to gain clarification 1323 on exercise play at other exercise sites or in different functional areas. 1324



A hotwash provides opportunities to distribute, fill-out, and collect Participant 1325 Feedback Forms, which helps to generate the After-Action Report/Improvement Plan 1326 (AAR/IP). 1327 Debrief 1328 Controller/Evaluator Debrief is a meeting for controllers and evaluators to assemble 1329 after the exercise to discuss observations and provide information to create a shared 1330 understanding of the exercise. The Controller/Evaluator Debrief provides a forum for 1331 facilitators (if exercise is discussion-based), functional area controllers and evaluators 1332 to review the exercise. 1333 The exercise planning team leader leads the Controller/Evaluator Debrief giving each 1334 controller and evaluator an opportunity to provide an overview of the functional area 1335 observed and discuss the strengths and areas for improvement. The After-Action 1336 Report/Improvement Plan (AAR/IP) include the results captured and included in the 1337 debrief. During the debrief, facilitators or controllers and evaluators complete and 1338 submit a Participant Feedback Form. 1339 1340



5. Evaluation 1341

Overview 1342 Through exercise evaluation, jurisdictions/organizations assess the capabilities needed 1343 to accomplish a mission, function, or objective. The exercise evaluation ties objectives 1344 to priorities and assesses the performance of critical tasks to capability targets. 1345 Evaluation produces strengths and areas for improvement from exercises translating 1346 into corrective actions in the improvement planning phase. Exercise evaluation 1347 connects the exercise to improvement planning and feeds into the overall preparedness 1348 cycle. 1349 Effective exercise evaluation should integrate through HSEEP exercise cycle and 1350 involves: 1351

• Planning for exercise evaluation; 1352 • Developing exercise evaluation guides that tie into exercise objectives; 1353 • Observing the exercise and collecting exercise data during exercise conduct; 1354 • Analyzing collected data to identify strengths and areas for improvement; and 1355 • Reporting exercise outcomes in an After-Action Report. 1356

Using a common approach to evaluation supports consistent and meaningful reporting 1357 of exercise results. 1358

Evaluation Planning 1359 Evaluation planning begins during the initial planning phases of the exercise, as 1360 described in Chapter 3: Exercise Design and Development. Identifying clear 1361 evaluation requirements early in the planning process will ensure that the design, 1362 development, and conduct of the exercise best support an effective evaluation. 1363 Exercise planners should collaborate to ensure a consistent approach for evaluating an 1364 exercise. Also, engage senior leadership early in the evaluation planning to identify 1365 any focus areas, concerns, and specific evaluation requirements. The Evaluation Plan 1366 is a document for evaluators that provides guidance, instructions, methodology, and 1367 structure on evaluating and observing during an exercise. 1368 Planning an exercise evaluation includes: 1369

• Selecting Lead Evaluator and define evaluation team requirements; 1370 • Developing Exercise Evaluation Guides, which include SMART exercise 1371

objectives, links to capabilities, capability targets, and critical tasks; 1372 • Identifying the evaluation team, assigning evaluators roles and 1373

responsibilities, and providing training; 1374 • Developing and finalizing evaluation documentation; and 1375 • Conducting a pre-exercise brief. 1376

Through the evaluation planning process, an evaluation team develops a thorough plan 1377 addressing exercise evaluation process. 1378



Evaluation Team 1379 Once the planning team defines evaluation requirements, the Lead Evaluator oversees 1380 the recruiting, assigning, and training of evaluators. 1381 The evaluation requirements should specify the following: 1382

• The numbers of evaluators needed; 1383 • The type of subject matter expertise the evaluators should possess; 1384 • The evaluator assignments; and 1385 • The types of training or instruction needed prior to the exercise. 1386

Team Composition 1387 Whenever possible, evaluators should have experience and subject matter expertise in 1388 their assigned functional area. The Exercise Planning Team Lead should appoint a 1389 Lead Evaluator to oversee all facets of the evaluation process. The Lead Evaluator 1390 participates fully as a member of the exercise planning team and is familiar with the 1391 exercise’s objectives, mission areas, and capabilities associated with the plans, 1392 policies, and procedures, incident command, and decision-making processes. 1393 The Lead Evaluator determines the structure of the exercise evaluation team based on 1394 the scope of the exercise, the exercise objectives, associated capabilities, capability 1395 targets, and critical tasks evaluated during the exercise. The number of evaluators is 1396 based on the number of Exercise Evaluation Guides and venues. Some exercise venues 1397 may need specific security clearance levels. Exercises that involve multiple 1398 jurisdictions/organizations and multiple venues should consider assigning venue leads, 1399 as illustrated by the example provided in Figure 5.1. A venue is either a jurisdiction, a 1400 specific emergency operations center, or another exercise location. The evaluators at 1401 each of the venues will support the Lead Evaluator and evaluate activities at their 1402 assigned sites 1403

Figure 5.1: Sample Exercise Evaluation Team Organization 1404

o Lead Evaluator: An individual that oversees all facets of the evaluation 1405 process to include recruiting, assigning, and training evaluator 1406

• Venue: A location where the exercise conduct is taking place. 1407 o Evaluator: An individual chosen based on their expertise in the 1408

functional areas, to observe and collect exercise, and analyze results 1409 Consider the exercise scope and objectives when selecting the number of individuals 1410 for evaluation support. For exercises of limited scope and fewer objectives and 1411 capabilities, the Lead Evaluator and one additional person may satisfy evaluation 1412 support. For more complex or larger exercises with a greater number of objectives and 1413



capabilities, more individuals may be necessary for evaluation and After-Action 1414 Report development. 1415 Training 1416 Evaluator training10 should include guidance on observing exercise discussion or 1417 operations and criteria for inclusion of data in the final exercise analysis. Effective 1418 evaluator training enables a shared understanding of how data will be collected, used, 1419 and contributed to the evaluation of the exercise. 1420 Evaluator training includes reviewing the following: 1421

• Any general information about the exercise, including scope, objectives and 1422 aligned capabilities, scenario, and schedule; 1423

• Any relevant evaluator documentation (for example, Situational Manual, 1424 Controller/Evaluator Handbook, evaluation tools); and 1425

• The appropriate plans, policies, procedures, agreements, or other information 1426 that are the focus of the exercise. 1427

Before exercise play begins, the Lead Evaluator should meet with all evaluators to 1428 verify roles, responsibilities, and assignments, and to provide any significant updates. 1429 The briefing provides an opportunity for evaluators to ask questions and to ensure 1430 complete understanding of their roles and responsibilities. The briefing may occur in 1431 conjunction with exercise controllers, as a Controller/Evaluator Brief, a meeting 1432 before an operations-based exercises that provides an exercise overview and 1433 responsibilities of controllers and evaluators, to include instructions on delivering the 1434 MSEL and completing Exercise Evaluation Guides (EEGs). Depending on the 1435 organization of the exercise, briefings may occur at more than one exercise venue. 1436

Evaluation Documentation 1437 The exercise planning team and Lead Evaluator should determine the tools, data 1438 collection methods, and documentation needed. Exercise documentation varies with 1439 the level of complexity and scope; exercise requirements establish the type of 1440 evaluation documentation needed. 1441 Table 5.1 provides the document, description, and distribution of evaluation 1442 documentation: 1443

Evaluation Documentation


Facilitator/Evaluator Handbook

For discussion-based exercises, provides specific exercise information and guidance for facilitators and evaluators.

Facilitators and Evaluators

Controller/Evaluator Handbook

For operations-based exercises, provides specific exercise information and guidance for controllers and evaluators, may be a stand-alone document or supplement to the Exercise Plan, and adjusted accordingly.

Controllers and Evaluators

10 For more detailed information about evaluator training visit the following link: https://training.fema.gov/is/courseoverview.aspx?code=IS-130.a

https://training.fema.gov/is/courseoverview.aspx?code=IS-130.a

https://training.fema.gov/is/courseoverview.aspx?code=IS-130.a



Exercise Evaluation Guide

Provides a tool to organize exercise observations pertaining to the evaluation criteria. Evaluators


Provides a mechanism to collects input from all participants following an exercise and supports the development of an After-Action Report/Improvement Plan (AAR/IP).

All Participants

Table 5.1: Evaluation Documentation 1444 Exercise Evaluation Guide Development 1445 Exercise Evaluation Guides provide a consistent tool to guide exercise observation and data 1446 collection. Exercise Evaluation Guides align to exercise objectives and capabilities and list the 1447 relevant capability targets and critical tasks. 1448 Figure 5.2 shows the relationship from program management to the development of Exercise 1449 Evaluation Guides (EEGs). 1450

Capability targets are the 1451 performance thresholds 1452 established by the 1453 jurisdiction/organization for 1454 each capability. Capability 1455 targets are composed of three 1456 parts: a critical task, an impact, 1457 and a timeframe. Identified 1458 targets are part of a 1459 jurisdiction/organization’s 1460 Threat and Hazard 1461 Identification and Risk 1462 Assessment or other threat and 1463 hazard identification or risk 1464 assessment processes. 1465 Critical tasks are 1466 distinct specific 1467

actions needed to achieve the capability target. Critical tasks may derive from the 1468 jurisdictional/organizational operations plans, standard operating procedures, or 1469 discipline-specific standards. These tasks reflect the types of activities emergency 1470 managers routinely plan for and are broadly applicable to a wide range of threats and 1471 hazards. 1472 The impact and timeframe metrics for a capability target are ongoing. 1473 Capability targets and critical tasks may draw from the National Preparedness Goal, or 1474 jurisdiction’s/organization’s capability estimate, risk assessment product, or plans. 1475 The purpose of an Exercise Evaluation Guide is to: 1476

• Streamline data collection and provide a consistent process for assessing preparedness; 1477 • Assist jurisdictions/organizations to map exercise results to exercise 1478

objectives, capabilities, capability targets, and critical tasks for additional 1479 analysis and assessment; and 1480

Figure 5.2: Development of an Exercise Evaluation Guide



• Support development of the After-Action Report. 1481

Observation and Data Collection 1482

Exercise observation and data collection can differ between discussion-based exercises 1483 and operations-based exercises. Discussion-based exercises often focus on issues 1484 involving plans, policies, and procedures. Evaluators (or in some cases, note-takers) 1485 collect observations from participant discussions. Operations-based exercises focus on 1486 issues affecting the operational execution of capabilities and critical tasks, and 1487 implementation of planned policies and procedures. Evaluators collect and record 1488 participant actions, which form the analytical basis for determining the successful 1489 demonstration of critical tasks and capability targets. 1490

Observation 1491 Exercise evaluators should observe exercise activity in a non-attribution environment, 1492 in accordance with the evaluation training and Exercise Evaluation Guides. 1493 Evaluators observe many of the following topics: 1494

• The use of plans, policies, and procedures; 1495 • The roles and responsibilities, and authorities of the government agencies, 1496

jurisdictions, and private organizations; 1497 • Any pertinent decisions made or decision-making process; 1498 • The activation or implementation of processes and procedures, requests for 1499

resources, use of mutual aid agreements, etc.; and 1500 • The information sharing with other agencies and the public. 1501

Data Collection 1502 During discussion-based exercises, facilitators help evaluators collect useful data by 1503 keeping discussions focused on exercise objectives and goals. 1504 Evaluators should retain their notes and records to support the development of the 1505 After-Action Report. As necessary, the Lead Evaluator may assign evaluators to 1506 collect additional data such as plans, policies, procedures, Incident Action Plans, and 1507 other documentation associated with the exercise. The data is critical to providing an 1508 overall understanding of the exercise. 1509

Data Analysis 1510

The goal of data analysis is to evaluate the ability of participants to perform 1511 capabilities and to determine the success of meeting the exercise objectives. During 1512 data analysis process, the evaluation team consolidates the data collected and 1513 determines whether participants met capability targets and associated critical tasks. 1514 Evaluators consider participant performance against all targets to determine the overall 1515 ability to perform capabilities. Also, the evaluation team takes notes on the course of 1516 exercise play, demonstrated strengths, and areas for improvement. The notes provide 1517 the evaluators with what and why events happened. 1518 After the initial data analysis, evaluators examine each capability target not met and 1519 associated critical task with the aim of identifying a root cause. A root cause is the 1520 source of or underlying reason behind an identified issue toward which the evaluator 1521



can direct an improvement. The evaluator should attempt to trace the origin of each 1522 event back to earlier events and their respective causes. 1523 Evaluators should consider the following questions: 1524

• Were the capability targets met? If not, what factors contributed to not 1525 meeting the capability targets? 1526

• Did discussion or activities suggest the critical tasks were met capability 1527 targets? If not, what was the impact or consequences? 1528

• Do current plans, policies, and procedures support critical tasks and 1529 capability targets? Were participants familiar with these documents? 1530

Analyzing events in the above sequence help evaluators determine the underlying 1531 cause of issues and inform an organization’s corrective actions to remedy the issue. 1532

After-Action Report (AAR) 1533 The AAR is a document summarizing key information related to evaluation. The 1534 length, format, and development timeframe of the AAR depend on the exercise type 1535 and scope. These parameters should be determined by the exercise planning team 1536 based on the expectations of senior leaders. The focus of the AAR is the analysis of 1537 capabilities and generally includes basic exercise information, such as the exercise 1538 name, type of exercise, dates, locations, participating organizations, mission area(s), 1539 specific threat or hazard, a brief scenario description, and the name of the exercise 1540 sponsor and Point of Contact. 1541 The AAR should include an overview of performance related to each exercise 1542 objective and associated capabilities, while highlighting strengths and areas for 1543 improvement. Therefore, evaluators should review their evaluation notes and 1544 documentation to identify the strengths and areas for improvement relevant to the 1545 participating organizations’ ability to meet exercise objectives and demonstrate 1546 capabilities. 1547 Table 5.2 provides the document, description, and distribution of After-Action Report: 1548

After-Action Report Documentation


After-Action Report

After-Action Reports include basic exercise information, such as the exercise name, type of exercise, dates, location, participating jurisdictions/organizations, mission area(s), specific threat or hazard, a brief scenario description, and the name of the exercise sponsor and point of contact.

Stakeholders

Table 5.2: After-Action Report Documentation 1549 Upon completion, the evaluation team provides the draft After-Action Report to the 1550 exercise sponsor, who distributes the draft to participating jurisdictions/organizations. 1551 Senior leadership or their designees, review and confirm observations identified in the 1552 formal After-Action Report and determine which areas for improvement require 1553 further action. The areas for improvement requiring action are those continuously 1554 impeding capability performances if left unresolved. As part of the improvement 1555 planning process, senior leadership identifies corrective actions to bring areas for 1556



improvement to resolution and determine the organization with responsibility for those 1557 actions. Chapter 6: Improvement Planning describes the process in further detail 1558 1559



6. Improvement Planning 1560

Overview 1561

After the evaluation phase concludes, jurisdictions/organizations reach a consensus on 1562 identified strengths and areas for improvement and develop a set of actions directly 1563 addressing capability gaps. Improvement planning enables 1564 the exercise planner to transition from the HSEEP Cycle into 1565 the Preparedness Cycle. Actions identified during 1566 improvement planning help to strengthen elements of a 1567 jurisdiction’s/organization’s capability to plan, organize, 1568 equip, train, and exercise. In this way, improvement planning 1569 activities can help shape a jurisdiction’s/organization’s 1570 preparedness priorities and support continuous improvement. 1571 The transition is shown in Figure 6.1. 1572

SMART Corrective Actions 1573 After analyzing exercise data, jurisdictions/organizations 1574 perform an additional qualitative assessment to identify 1575 potential corrective actions. Corrective actions are concrete, actionable steps intended 1576 to resolve capability gaps and shortcomings identified in exercises or real-world 1577 events. In developing corrective actions, senior leadership or their designees should 1578 review and revise the draft After-Action Report, as needed, prior to the After-Action 1579 Meeting to confirm that the issues identified by Evaluators are valid and require 1580 resolution. An After-Action Meeting that serves as a forum to review the revised AAR 1581 and the draft IP. Participants should seek to reach final consensus on: strengths, areas 1582 for improvement, draft corrective actions, concrete deadlines, and owners/assignees 1583 for implementation of corrective actions. The reviewer then identifies issues within 1584 their jurisdiction’s/organization’s authority and assumes the responsibility for acting 1585 on the identified issues. Lastly, the reviewer determines an initial list of appropriate 1586 corrective actions to resolve the identified issues. 1587 The jurisdiction/organization’s reviewer may use the following questions to guide the 1588 discussion when developing corrective actions: 1589

• What changes to plans and procedures will improve performance? 1590 • What changes to organizational structures will improve performance? 1591 • What changes to management processes to improve performance? 1592 • What changes to equipment or resources will improve performance? 1593 • Was training provided and did the training affect performance? 1594 • What training improves performance? 1595 • What are the lessons learned for approaching similar problems in the future? 1596

After-Action Meeting 1597 The purpose of the After-Action Meeting is to serve as a forum to review the revised 1598 After-Action Report and the draft Improvement Plan. Participants should seek to 1599

Figure 6.1: Exiting Exercise Cycle and Entering the Preparedness Cycle



reach final consensus on: strengths, areas for improvement, draft corrective actions, 1600 concrete deadlines, and owners/assignees for implementation of corrective actions. 1601 Table 6.1 provides the meeting focus, discussion points, tools, outcomes, and follow-1602 up actions for and After-Action meeting: 1603

After-Action Meeting Meeting Focus Discussion Points Tools Outcomes Follow-up

After-Action Meeting is an interactive session providing attendees the opportunity to discuss and validate the analytical findings and corrective actions in the draft After-Action Report/Improvement Plan (AAR/IP)

• Review and revise After-Action Report draft the Improvement Plan

• Discuss exercise results

• Identify areas for improvement

• Identify corrective actions

• Draft After-Action Report

• Draft Improvement Plan

• Final consensus on strengths and areas for improvement

• Consensus on draft corrective actions

• Concrete deadlines for the implementation of corrective actions

• Assign the corrective actions to preparedness stakeholders

Corrective actions are tracked and reported

Table 6.1: After-Action Meeting 1604 Participant jurisdictions/organizations are responsible for developing implementation 1605 processes and timelines, and keeping their senior leadership informed of the 1606 implementation status. 1607

Improvement Plan 1608

The final Improvement Plan includes all consolidated corrective actions. The 1609 Improvement Plan may be part of the appendix to the After-Action Report. The After-1610 Action Report/Improvement Plan (AAR/IP) is then final and distributed to exercise 1611 planners, participants, and other preparedness stakeholders, as appropriate. 1612 Table 6.2 provides the document, description, and distribution of improvement 1613 planning documentation: 1614

Improvement Planning Documentation Document Description Distribution

Improvement Plan Identify specific corrective actions, assigns them to responsible parties, and establishes target dates for their completion.

Exercise planners, participants, and preparedness stakeholders (as appropriate)

Table 6.2: Improvement Planning Documentation 1615

Corrective Action Tracking and Implementation 1616

Corrective actions captured in the After-Action Report/Improvement Plan (AAR/IP) 1617 should be tracked and continually reported on until completion. These efforts are part 1618 of a more comprehensive continuous improvement process that applies before, during, 1619 and after an exercise. Stakeholders should also ensure a system is in place to validate 1620 previous corrective actions that were successfully implemented. Over time, corrective 1621 actions effectively integrated back into the preparedness cycle will yield continuous 1622 improvements during future exercises and real-world events. 1623



Using Improvement Planning to Support Continuous Improvement 1624

Continuous improvement includes: 1625

• A Consistent Approach. Jurisdictions/Organizations should employ a 1626 consistent approach for continuous improvement-related activities across 1627 applicable mission areas—prevention, protection, mitigation, response, and 1628 recovery. A consistent approach enables a shared understanding of 1629 terminology, functions, processes, and tools. The approach also fosters 1630 continuous improvement-related interoperability and collaboration across a 1631 jurisdiction/organization’s components. 1632

• Supporting Preparedness. Jurisdictions/Organizations support the 1633 development and sustainment of capabilities across the whole community, 1634 which contribute to supporting preparedness. Continuous improvement 1635 activities also ensure that jurisdictions/organizations can support 1636 assessments of preparedness in a timely, actionable, and meaningful way. 1637

• An Effective Issue Resolution and Information Sharing. Through 1638 improvement planning, jurisdictions/organizations complete continuous 1639 improvement action items at the lowest level possible while facilitating the 1640 sharing of strengths and areas for improvement. 1641

• An Application across Operational Phases. The functions, processes, and 1642 tools apply to all operational phases, including: 1643

• A near-real time collection and analysis during real-world events or 1644 exercises; 1645

• A post-event/exercise analysis; and 1646 • A trend analysis across multiple events and exercises over time. 1647

Application of the above principles and the conduct of improvement planning 1648 ultimately support the preparedness cycle. Continually examining the implementation 1649 of corrective actions, jurisdictions/organizations can identify capability gaps and 1650 determine what corrective actions require validation through exercises. Improvement 1651 planning activities can help shape a jurisdiction/organization’s preparedness priorities 1652 and support continuous improvement in the building and sustaining of capabilities. 1653



Acronyms 1654 Abbreviation Description

A/V Audio/Visual AAM After-Action Meeting AAR After-Action Report C/E Controller/Evaluator C&O Concept and Objectives COSIN Control Staff Instructions DHS U.S. Department of Homeland Security EEG Exercise Evaluation Guide EndEx End of Exercise EOC Emergency Operations Center EvalPlan Evaluation Plan ExPlan Exercise Plan FE Functional Exercise FEMA Federal Emergency Management Agency FPM Final Planning Meeting FSE Full-Scale Exercise HSEEP Homeland Security Exercise and Evaluation Program ICS Incident Command System IP Improvement Plan IPM Initial Planning Meeting MPM Midterm Planning Meeting MSEL Master Scenario Events List NEP National Exercise Program NGO Nongovernmental Organization NIMS National Incident Management System POC Point of Contact SimCell Simulation Cell SitMan Situational Manual SMART Specific, Measurable, Achievable, Relevant, Time-bound SME Subject-Matter Expert SOP Standard Operating Procedure StartEx Start of Exercise THIRA Threat and Hazard Identification and Risk Assessment IPP Integrated Preparedness Planning IPPW Integrated Preparedness Planning Workshop TTX Tabletop Exercise VIP Very Important Person



XPA Extent of Play Agreement



Glossary of Terms 1655

The table below provides a description for commonly used terms in HSEEP. 1656 1657

A Term Description

Actor An individual who simulates a specific role, such as a disaster casualty victim, to add realism to an exercise.

Actor Briefing A meeting generally conducted before the exercise, providing actors with; an exercise overview, safety, real emergency procedures, acting instructions, schedule, Identification badges, and symptomatology cards.

After-Action Meeting (AAM)

A meeting that serves as a forum to review the revised AAR and the draft IP. Participants should seek to reach final consensus on: strengths, areas for improvement, draft corrective actions, concrete deadlines, and owners/assignees for implementation of corrective actions.

After-Action Report (AAR)

A document that summarizes key information related to evaluation should include an overview of performance related to each exercise objective and associated capabilities. The lead evaluator and exercise planning team draft the AAR and submit it to meeting participants before the AAM.

B

Best Practices An item that is peer-reviewed techniques, procedures, and solutions that prove successful and are solidly grounded in actual experience in plans, operations, equipment, training, and exercises.

C Capabilities-Based Planning

A process to build capabilities suitable for a wide range of threats and hazards while working within an economic framework that necessitates prioritization and choice. Capabilities-based planning is the basis for guidance such as the National Preparedness Goal.

Capability An ability to be planned, organized, equipped, trained, and exercised through personnel or resources to achieve an intended target.

Capability Target A performance threshold established by the jurisdiction/organization for each capability.

Concept and Objectives (C&O) Meeting

A meeting that is the formal beginning of the exercise planning process. It is held to identify the scope and exercise objectives. For less complex exercises and for organizations with limited resources, the C&O Meeting can be conducted in conjunction with the IPM.

Contingency Inject A MSEL event introduced to a player by a control staff when a key player expected action did not occur as planned to provide an additional opportunity to meet exercise objectives.

Control Staff Instructions (COSIN)

A document for the Controller portion of the C/E Handbook for large exercises, it provides detailed guidelines, procedures, responsibilities, and a management structure for exercise control, simulation, and support.

Controller/Evaluator (C/E) Brief

A meeting before an operations-based exercises that provides an exercise overview and responsibilities of controllers and evaluators, to include



instructions on delivering the MSEL and completing EEGs.

Controller/Evaluator (C/E) Debrief

A meeting for controllers and evaluators to assemble after the exercise to discuss observations and provide information to create a shared understanding of the exercise.

Controller/Evaluator (C/E) Handbook

A document that provides the roles and responsibility guidelines, procedures, instructions, and structure for exercise controllers and evaluators.

Controller An individual who manage exercise play and monitor the pace of the exercise. Controllers may prompt or initiate certain player actions to maintain exercise pace and flow.

Core Capabilities Distinct critical elements necessary to achieve the specific mission areas of prevention, protection, mitigation, response, and recovery as outlined in the National Preparedness Goal.

Corrective Action An item outlined in an IP that is intended to resolve areas for improvement.

Critical Tasks An item where distinct specific actions needed to achieve the capability target.

D Design and Development A phase responsible for compiling and developing all exercise background and

facilitation or control, provides technical or functional expertise for scenario development, developing the simulation construct as applicable, and working with the evaluation and resource/support functions to ensure all exercise design and development needs are met.

Drill An operations-based exercise often employed to validate a single operation or function.

E End of Exercise (EndEx) An event that concludes an exercise.

Evaluation Responsible for the development of the overall exercise evaluation construct and all evaluation documentation; assigns evaluation staffing in conjunction with the exercise planning team. Collects and reviews the policies, plans, and procedures to be examined during exercise conduct to finalize exercise objectives, match capability targets to exercise objectives, and determine critical tasks to be evaluated to complete exercise evaluation guides.

Evaluation Plan (EvalPlan)

A document for evaluators that provides guidance, instructions, methodology, and structure on evaluating and observing during an exercise

Evaluation Structure Responsible for the development of the evaluation methods, structure, and plan used during the exercise to effectively examine and validate exercise objectives. Works with the team developing exercise evaluation guides to ensure structure will meet exercise needs.

Evaluator/Note Taker An individual chosen based on their expertise in the functional areas, to observe and collect exercise, and analyze results.

Exercise An event or activity delivered through discussion or action to develop, assess, or validate capabilities to achieve planned objectives.

Exercise Assembly Area A location for participants and resources to gather and dispatch, managed by controller(s).



Exercise Assembly Area Controller

An individual responsible for the logistical organization of the exercise assembly area, including placement locations for units entering the exercise assembly area, release of dispatched units into the field, and coordination of routes and overall safety within the assembly area.

Exercise Director An individual that provides the strategic oversight and direction on the HSEEP exercise cycle phases for an individual exercise.

Exercise Evaluation Guide (EEG)

A document for evaluators that provides a structured format to collect relevant exercise observations and data.

Exercise Evaluation Guide (EEG) Structure

Responsible for the development of the EEGs used for the exercise.

Exercise Plan (ExPlan) An operations-based exercise document for players that includes and exercise overview, objectives and aligned capabilities, roles and responsibilities, logistics, schedule, and communications plan.

Exercise Planning Team Individuals that are responsible for the design, development, conduct, evaluation, and improvement planning for an individual exercise.

Exercise Planning Team Lead

Individual that manages the exercise planning team and coordinates with the Exercise Director and senior leaders.

Exercise Play Area A location where player activities and tasks are demonstrated during an exercise.

Exercise Play Rules Items that identify the parameters that exercise participants follow during an exercise.

Exercise Program Manager

An individual responsible for overseeing and integrating preparedness activities for a jurisdiction/organization over time

Expected Action A MSEL event represents an anticipated action to be taken by a player during the exercise.

F Facilitated Discussion A method that uses a facilitator with functional or subject-matter expertise and

awareness of appropriate plans and procedures for discussion-based exercises.

Facilities Responsible for coordinating the needed venue locations for exercises including meeting rooms, conference centers, training locations, and locations for operations-based exercises.

Facilities Lead An individual responsible to ensure proper room function and set up for audio/visual requirements as well as obtaining necessary equipment, food, and drinks and works with the facilities lead.

Facilitator/Presenter An individual responsible for keeping a discussion focused on exercise objectives and exploring all issues within the time allotted during discussion-based exercises or an individual designated to deliver information in a structured setting.

Final Planning Meeting (FPM)

A meeting that serves as the formal end of the exercise planning process. It is held to finalize exercise documentation and logistics.

Full-Scale Exercise (FSE) An operations-based exercise that is typically the most complex and resource-intensive of the exercise types and often involves multiple agencies, jurisdictions/organizations, and real-time movement of resources.



Functional Exercise (FE) An operations-based exercise that is designed to validate and evaluate capabilities and functions while in a realistic, real-time simulated environment.

G Game A discussion-based exercise that is a simulation of operations, that involve two or

more teams, usually in a competitive environment, using rules, data, and procedures designed to depict an actual or hypothetical situation.

Ground Truth A document comprised of the detailed elements of a scenario that must remain consistent during exercise development and conduct to ensure that realism is maintained and objectives can be achieved.

Ground Truth Advisor An individual responsible for ensuring that the scenario details remain consistent during exercise conduct.

H Homeland Security Exercise and Evaluation Program (HSEEP)

A document that provides a set of guiding principles for exercise programs, as well as a common approach to exercise program management, design and development, conduct, evaluation, and improvement planning.

I Improvement Plan (IP) A document that includes consolidated list of corrective actions, responsible

parties, and timeline for completion.

Initial Planning Meeting (IPM)

A meeting that is held to refine the scope and exercise objectives. For less complex exercises and for organizations with limited resources, the C&O Meeting can be conducted in conjunction with the IPM.

Inject A Master Scenario Events List event introduced to a player by the control staff, representing non-playing entities, to build the exercise environment based on the exercise scenario and to drive operations-based exercise play.

Integrated Preparedness Plan (IPP)

A document for combing efforts across components of the preparedness cycle to make sure that a jurisdiction/organization has the capabilities to handle threats and hazards.

Integrated Preparedness Planning Workshop (IPPW)

A meeting that establishes the strategy and structure for an exercise program and sets the foundation for the planning, conduct, and evaluation of individual exercises.

L Lead Controller An individual that monitors exercise progression, communicates exercise

activities throughout all venues, and responsible for managing the control staff.

Lead Evaluator An individual that oversees all facets of the evaluation process to include recruiting, assigning, and training evaluators.

Logistics Responsible to ensure venues are adequately equipped with A/V equipment, tables and chairs, and coordinates refreshments if required.

Logistics Lead An individual responsible to ensure proper room function and set up for audio/visual requirements as well as obtaining necessary equipment, food, and drinks and works with the facilities lead.

M Master Control Cell A location where overall coordination is managed between venue control cells,

simulation cell, and other control areas.



Main Exercise Venue A location designated as the primary facility or geographic area where participants gather to conduct an exercise.

MSEL Manager An individual that manages the timely and accurate delivery of injects, player expected actions, and decides on the need to release contingency injects.

Master Scenario Events List (MSEL)

A document or system, for an operations-based exercise, that contains a chronological list of events that drives exercise play.

MSEL Meeting A meeting for operations-based exercises that serves as a forum to build the Master Scenario Events List in detail.

Midterm Planning Meeting (MPM)

A meeting that serves as a forum to develop exercise scenario details and timeline.

Mitigation The capabilities necessary to reduce loss of life and property by lessening the impact of disasters.

N National Preparedness Goal (Goal)

A document that defines what it means for the whole community to be prepared for all types of disasters and emergencies. The NPG describes 32 core capabilities that address the greatest risks to the nation.

National Preparedness System

A method that outlines an organized process for everyone in the whole community to move forward with preparedness activities and achieve the National Preparedness Goal.

O Objectives An item with a distinct outcome that a jurisdiction/organization wishes to achieve

during an exercise. An objective should be: specific, measurable, achievable, relevant, and time-bound (SMART).

Observer An individual that does not directly participate in but may watch selected segments of the exercise as it unfolds.

Observer Briefing A meeting that informs observers and Very Important Persons (VIPs) about the exercise background, scenario, schedule of events, limitations, and any miscellaneous information.

Observer/Media Area A location where observers and real-world media representatives can view the exercise but not interfere with exercise play.

P Participants Individual(s) that includes all players, facilitators/controllers, evaluators,

simulators, actors, observers, VIPs, media, and other staff members involved in conducting an exercise.


A document that the exercise team uses to collect observed strengths, areas for improvement, and input about exercise conduct and logistics from participants.

Pause Exercise (PauseEx) An event that temporarily suspends exercise play.

Planning Meetings A meeting(s) that serve as a structured event or forum for completing the major milestones of exercise design and development.

Player An individual that has an active role in the exercise by either discussing or performing their regular roles and responsibilities in response to a scenario.

Player Briefing A meeting before an exercise that provides players an overview of individual roles and responsibilities, exercise parameters, safety, security, and any remaining logistical exercise concern or questions.



Player Hotwash A meeting that provides an opportunity to discuss exercise strengths and areas for improvement immediately following the conduct of an exercise.

Preparedness The continuous cycle of planning, organizing, training, equipping, exercising, evaluating, and taking corrective action to ensure effective coordination.

Presenter An individual designated to deliver information in a structured setting.

Prevention The capabilities necessary to avoid, prevent, or stop a threatened or actual act of terrorism.

Program Management A process of overseeing and integrating preparedness activities for a jurisdiction/organization over time.

Props Items that are nonfunctional replications of objects used during the exercise that require certain actions from players.

Protection The capabilities necessary to secure the homeland against acts of terrorism and manmade or natural disasters.

R Recovery The capabilities necessary to assist communities affected by an incident to

recover effectively.

Resource Lead An individual responsible for obtaining proper venues, equipment, and supplies for exercise conduct as well as providing support for media and VIP observers.

Resource/Support Responsible for logistics, administration, facility, and finance support for planning meetings, exercise documentation, and other planning considerations

Response The capabilities necessary to save lives, protect property and the environment, and meet basic human needs after an incident has occurred.

Rolling Summary Report A document that provides stakeholders with an analysis of issues, trends, and key outcomes from all exercises conducted as part of the exercise program.

Root-Cause Analysis A method used to trace the origin of an event back to earlier events and their respective causes.

S Safety Controller An individual, during operations-based exercises, responsible to ensure that the

exercise is conducted in a safe and secure environment.

Scenario A model or outline, either written or depicted by an event timeline, of the simulated sequence of events that drive the player’s discussion or actions.

Scope An item that is defined by exercise type, participation level, exercise duration, and exercise location, that enables planners to meet exercise objectives while staying within the resource and personnel constraints.

Seminar A discussion-based exercise that orients participants to or provides an overview of authorities, strategies, plans, policies, procedures, protocols, resources, concepts, and ideas.

Senior Leader Individual(s) that provide the overarching guidance and direction for the exercise program and intent for individual exercises.

Simulation A method of implementing the performance of a model, or combination of models, over time.



Simulation Cell (SimCell) A location from which controllers deliver scenario messages representing actions, activities, and conversations of an individual, agency, or organization that is not participating in the exercise.

Simulators An individual that coordinates with the MSEL manager, or may act as the MSEL manager, and directs simulators and actors.

Situation Manual (SitMan)

A document for discussion-based exercises that provides contextual background information and supports the scenario narrative for participants.

Span of Control A method that defines the ideal number of functions, people, or things, for which an individual or organization is responsible.

Special Effects Items used for operations-based exercises that require trained and licensed personnel and special permission for use.

Sponsor Individual(s) who funds and hosts the exercise

Start of Exercise (StartEx)

An event that marks the official beginning of an exercise.

Subject-Matter Expert (SME)

An individual who has special skills and knowledge about a function or topic.

Symptomatology Card An item containing the signs and symptoms that an exercise actor will portray, as well as information helpful for players.

T Tabletop Exercise (TTX) A discussion-based exercise in response to a scenario intended to generate a

dialogue of various issues to facilitate a conceptual understanding, identify strengths and areas for improvement, and/or achieving changes in perceptions about plans, policies, or procedures.

V Venue A location where the exercise conduct is taking place.

Venue Control Cell A location where controllers manage individual injects designed for their relevant players.

Venue Evaluator An individual responsible for observing exercise activity/play in their assigned exercise venue, area, or for a specific activity.

W Whole Community Individuals to include families, businesses, faith-based and community

organizations, non-profit groups, schools and academia, media outlets, and all levels of government who are involved in preparedness efforts.

Workshop A discussion-based exercise often employed to develop policy, plans, or procedures.

1658

Summary of Changes - FEMA.gov · DRAFT FOR REVIEW AND COMMENT ONLY. DRAFT FOR REVIEW AND COMMENT...

Documents

Transcript of Summary of Changes - FEMA.gov · DRAFT FOR REVIEW AND COMMENT ONLY. DRAFT FOR REVIEW AND COMMENT...