Experience.

Lawrence Suda

Using Simulations to Train Future Project Leaders at NASA

Performance Simulations

• People learn by doing in a risk-free environment

• Learners:– are encouraged to experiment– learners drive the experience– get realistic and individualized

feedback throughout the experience– must master certain behaviors to

achieve higher overall performance metrics

Business Simulations

Project Leadership ExperienceAdaptive Project LeadershipApplied Agile Project ManagementGlobal Project LeadershipPortfolio & Program LeadershipBusiness AcumenVirtual Project LeadershipChange LeadershipOperations ManagementStrategic Business ManagementSupply Chain Management

Simulations Mapped to Levels

4

Business Simulations Scope

Large-Scale Team Exercise – 24 to 35 + hours to completeSimulation Integrated into Learning Design – Mixed MethodologiesOpen-ended, Multi-layered, Multi-dimensionalSystems and Detailed Levels of ComplexityInstructor Facilitated but Learner DrivenClassroom or Virtual or BothTeams Are Not Competing

Validation

Vetted over 30 years with world class organizations and literature review

– NASA’s PM Model– NASA Project Mirror: Assessment & Development Center– GE Project Leader Research & Model– Nortel Networks PM Model– MITRE– Royal Bank of Canada– Department of Energy– Merck Leadership Model– Management Research Group Leadership Model– NASA Team Study & Assessment– NASA Leadership Model– US Navy Project Execution Study– PMI Career Development Framework– Alcatel-Lucent– Hewlett-Packard

Transitioning Into Project Leadership

TechnicalDiscipline Actions

Level 1 Level 2 Level 3 Level 4

Program/Project Manager Levels

Program/ProjectManagement Actions

Low

High

Dimensions

• Technical --- Leadership• Technical --- Abstract• Productive --- Practical• Narrow --- Broad• Task Orientation --- World View• Defined Goals --- Moving Goals• Discipline Specific --- Multidisciplinary

TechnicalDiscipline Actions

Level 1 Level 2 Level 3 Level 4

Program/Project Manager Levels

Program/ProjectManagement Actions

Low

High

Major Clusters Competency ResearchFocus Groups & Individual Interviews

Personal Leadership Team Leadership NASA Agency Acumen Communication Effectiveness Strategic Perspective and Thinking Project Management Skills and Knowledge

Team Learning

Team Learning

Shared Meaning

JointPlanning

PublicReflection

Coordinated Action

Workshop Themes and Modules

Learning Objectives

The overall purpose of the workshop is to improve our understanding of human and project complexities and to develop effective approaches to leading complex projects. More specifically we want to develop skills in:

• Improving project team performance• Leading in complex environments• Developing adaptive leadership skills• Developing defensible, flexible, adaptive plans• Using information to make better decisions• Identifying and managing project tradeoffs• Understanding unintended consequences of decisions• Recognizing when to focus on technical versus adaptive problems•Learning from successes and failures

Typical Agenda

Introduction Plenary

Case Diagnosis Simulation (Teams)

Case Discussion Plenary

Theory Plenary

Planning Simulation Teams

Execution Simulation Teams

Debrief Plenary

Application Plenary

Day 1 Day 2 Day 3 Day 4 Day 5

Introduction

Leadership Context Setting

360 Feedback

Team leadership

Mind Mapping

Film

Scheduling

Resource Planning

Film

Personal Styles

Team Styles

Presentation

Project Review

Team Work & Knowledge Transfer

Film

After Action Reviews

Feedback To Teams

Reflections

Action Planning

Lunch Lunch Lunch Lunch Lunch

Introduction to project simulation

DefinitionScheduling

Film

Risk Management

Quality & Budget

Management

Project Execution

Simulation

Simulation

Simulation

Simulation

Simulation

Complete Simulation

NASA Project Leadership Lab

Simulation

SimulationSimulation

Simulation

4-Day Learning Design

Short Lectures

• Defensible Planning• Project Execution

– Problem Solving, Decision Making

• Team Leadership– Performance, Motivation, Start-

up & Maintenance

• Adaptive Leadership

Planning Phase

Hear Lectures & Simulation Planning

Scheduling Resource Risks Quality Budgeting

Create & Defend Plan

                    Time (week number)  CR       CR       CR       CR       CR       CR              TR                                                    

    1-Jan 8-Jan15-Jan 22-Jan 1-Feb 8-Feb 15-Feb 22-Feb

1-Mar

8-Mar

15-Mar

22-Mar 1-Apr 8-Apr

15-Apr 22-Apr

1-May

8-May

15-May 22-May 1-Jun 8-Jun 15-Jun 22-Jun 1-Jul 8-Jul

  Staff Quality   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Task ID# ppl SD

HW PR CC

TW IM

TEC                                                      

1 2 X   X     X     2 2 2                                              3 2     X     X           2 2 2                                        6 2         X X                 2 2 2 2 2                              7 1 X   X     X                 1 1 1 X X X X X X X                    8 2           X                       1 1 1 X X X X X X X              9 3 X   X     X                           3 3 3                        16 2   X X                                       2 2 2 2 2              19 2         X X X                                     2 2 X X X        20 3   X X X   X X                                         3 3 3        21 2         X X X                                               2 2                                                                           2 1   X   X               1 1 1 X X X                                  4 1       X                     1 1 X X X                              5 2   X                         2 2 X X X                              10 2/V   X   X X                       2 2 2 2 2 X X X                    11 2/V   X   X X                       2 2 2 2 2 X X X                    12 2/V   X     X                   2 2 X X X X X X X X                    13 2       X     X                   2 2 2 X X X X X X X                14 2   X                                       2 2 X X X                15 2   X   X     X                                 2 X X X                                                                                     17 4     X       X                               4 4 4 X X              18 2     X       X                               2 2 2 X X              

                                                                       Total staff                   2 2 2 3 3 3 8 8 9 9 9 8 7 5 10 10 10 4 4 3 3 3 2 2    

    SDHW PR CC

TW IM

TEC Note                                                    

Senna   7 7 9 6 5 6 9   1 1 1 3 3 3                                        Mayes   4 4 8 0 8 6 6   1 1 1 3 3 3                                        Franz   2 3 7 4 8 0 4   CC CC HW 2 2 2                                        Banks   3 7 4 3 4 1 4                                                      Dance   7 4 8 3 7 7 8                                                      Bilko   6 5 7 3 6 0 6                                                      Inc                                                                      Bovis                                                                      Brown                                                                      Samin                                                                      Unter                                                                      Ltd                                                                      Burns   6 5 6 0 7 0 6                                                      Candy   9 6 9 5 10 8 9                                                      Mindy   10 9 3 9 3 0 7                                                      

Example Tools – Scheduling & Resourcing

In the workshopparticipants develop plans using both computer andmanual tools

18

Planning

4-person teams (instructor shown) develop an overall strategy and approach. They develop a detailed plan that includes: project priorities, critical path schedule, workload leveling, human resources plan, training, risk management, quality, budgeting. They must defend their strategy and plans.

19

Project Execution Phase

Hear Lectures & Simulation Control

Problem Solving, Decision Making Decision Trade-offs Re-planning Control Cycle Control Tools

Implementation

Teams make tactical decisions on three levels as they implement their plan: Project, Task and Staff. During the exercise, they must actively manage the project as conditions change as a result of their decisions and unknown events that create more issues and problems. Note: Instructor can introduce more issues and problems. Participants report status on a monthly basis. They have control tools to assist them as they proceed through the simulation exercise.

21

Some Typical Disruptions

– Resignations & Absenteeism– Resources pull for other projects– Large Scope Change– Scope Creep– Customer wants more meetings– Sponsor wants project sooner– New technology doesn’t work– Team performance is low– Key staff member resigns– Budget cuts– Estimates are wrong– Poor Morale

Project Competition

Performance Summary

KPI PLANNED ACTUAL

Final Cost 693,000 875,000

Completion Week / Date June 17 June 20

Open Quality Issues 0 – 2 0

Quality Issues Reported 30 – 40 47

Quality Issues Fixed 30 – 40 47

Labor Cost / Hour 64 74

Technology Cost / Hour 60 75

Overtime Costs 6,315 20,486

Training Costs 10,000 4,000

Utilization % 83 87

Performance % 100 83

Absence Rate % 2 1

Communication % 6 4

Number of Resignations 1 1

Staff Pulled Away Weeks 6-8 5

25

Simulation Debrief

• What were your results? (brief overall description)• Did you keep your risk mitigation plan updated?• What contributed to your successes?• What were your major simulation / project challenges?• How did you overcome any challenges?• Describe your team dynamics.• Did you capitalize on team diversity?• Did you notice movement through the team development process?

• Lead in ambiguous, complex environments

• Improve individual performance

• Develop team effectiveness and improve team performance

• Develop defensible, flexible plans

• Align project work with organization’s strategy

• Think strategically and conceptually

• Translate strategies into concrete actions

• Plan systematically and establishes objectives and priorities

• Manage risk -- knows when too much risk is being taken, when the right amount is being taken, know the consequence and communicate to the team and to leadership

• Learn, develop and model new team behaviors

• Develop personal leadership skills

• Demonstrate sensitivity and encourage innovation and creativity

Reported Learning

NASA Comparison of PM62, PM61, PM60, PM59 Participants’ Mean Ratings

3

3.2

3.4

3.6

3.8

4

4.2

4.4

4.6

4.8

5

1 2 3 4 5

PM62 PM61 PM60 PM59

2

2.2

2.4

2.6

2.8

3

3.2

3.4

3.6

3.8

4

6 7 8 9 10

PM62 PM61 PM60 PM59

MeanRating

MeanRating

Questions about Instructor and Session

1. Instructor Effectiveness2. Discussion were effective in reinforcing

learning (not asked in PM59)3. Activities were effective in reinforcing

learning (not asked in PM59)4. Instructor stimulated thinking5. Overall rating of learning experience

Questions about Information Presented in Session

6. Relevant to job7. Likeliness to apply back on the job8. Usefulness in increasing knowledge/skill to be

an effective Project Manager9. Value to assist in the achievement of goals as

a Project Manager10. Usefulness back on the job

• PMI Global Registered ProviderAdvanced PM = 21 to 35 PDU’s

• American Council on Education (ACE) Accredited for Graduate Credits– Stevens Institute of Technology– Drexel University– Duke CE– e-Cornell– EADA (Barcelona Spain)

Accreditations

Application

• High Potential Leadership Training• New Employee Orientation• Introduction of New PM Initiatives• Kick-off or Wrap-up of Conferences• Create Project-based Organization• Culture Change• PM Certification Capstone

Manager Involvement - Getting Participants’ Managers to be part of the learning process.

Training Time Compression - Pressures to reduce training time and time away from work.

Work Pressures on Participants – Work while at training

Travel Funds – Budgets are reduced.

Budget Pressures – Develop funds for new work reduced.

Training Outsourced to One or Few Vendors.

Measuring Effectiveness – ROI.

Technology Challenges & Learning Design Challenges – Particularly with Large Scale Sims in Virtual Environments

Changing Organization, Program, Project Dynamics – e.g. Agile PM

Current & Future Challenges

Thank You!

Lawrence SudaPalatine Group, Inc.New York City, NY

Lsuda@thepalatinegroup.com

Useful Links:

www.seriousplayconference.comwww.seriousgamesdirectory.comwww.seriousgamesassociation.com

Contact:

sbohle@seriousgamesassociation.com