Patterson.dan
-
Upload
nasapmc -
Category
Economy & Finance
-
view
13.499 -
download
1
Transcript of Patterson.dan
Using the Risk Register in Integrated Cost/Schedule Risk Analysis with Monte Carlo Simulation
David T. Hulett, Ph.D.Hulett & Associates, LLC
Dr. Dan Patterson, PMPAcumen
NASA PM ChallengeDaytona Beach, FL
February 24-25, 2009
© 2009 Hulett & Associates, AcumenPM
© 2009 Hulett & Associates, AcumenPM
Agenda
• Explain “Risk Factors” approach• Apply Risk Factors to schedule and cost risk• Apply Risk Factors to simple space vehicle
development schedule as an example
© 2009 Hulett & Associates, AcumenPM
Limitations with the Traditional 3-point Estimate of Activity Duration
• Typical schedule risk analysis starts with the activity that is impacted by risks– Estimates the 3-point estimate for optimistic, most
likely and pessimistic duration– Implies the risk is 100% likely with uncertain impact
• Which risks cause the most overall schedule risk? Cannot say directly, but indirectly:– Sensitivity to activity durations– Criticality of activity durations
© 2009 Hulett & Associates, AcumenPM
Some Problems with Traditional Approach
• Can tell which activities are crucial, but not directly which risks are driving
• Makes poor use of the Risk Register that is usually available
• Cannot decompose the overall schedule risk into its components BY RISK– Ability to assign the risk to its specific risk
drivers helps with communication of risk causes and risk mitigation
© 2009 Hulett & Associates, AcumenPM
We Propose a Different Approach: Start with the Risks Themselves
• Drive the schedule risk by the risks already prioritized in the Risk Register
• For each risk, specify:– Probability it will occur (NEW)– Impact on time if it does (multiplicative factors, allows it to
affect different duration activities)– Assign to the activities it will affect
• Starting with the risks themselves gives us benefits– Links qualitative analysis to the quantitative analysis– Estimates the impact of specific risks for prioritized
mitigation purposes– Correlations between activities modeled automatically
© 2009 Hulett & Associates, AcumenPM
Simple Example of Risk Register Risks
• Use the Risk Factors module in Pertmaster 8• Collect probability and impact data on risks• Map the risks to activities
© 2009 Hulett & Associates, AcumenPM
Risk Factors Mechanics (1)
• The risk factor is assigned to one or several activities, affecting their durations by a multiplicative factor– E.g., the factor may be .90 for optimistic, 1.0 for most
likely and 1.25 for pessimistic– These factors multiply the schedule durations of the
activities to which they are assigned • Risks can be assigned to one or more activities• Activity durations can be influenced by one or
more risks
© 2009 Hulett & Associates, AcumenPM
Risk Factors Mechanics (2)
• Risk Factors are assigned a probability of occurring on any iteration – When the risk occurs, the factor used is chosen at
random from the 3-point estimate and operates on all activities to which it is assigned
– When not occurring on an iteration the risk factor takes the value 1.0, a neutral value
• When an activity is influenced by more than one risk, their factors are multiplied together, if they happen, on any iteration
© 2009 Hulett & Associates, AcumenPM
Risk Factor Probability is 100%, Factor can be + or -
90 95 100 105 110 115
Distribution (sta rt o f in te rva l)
0
50
100
150
200
250
Hits
0% 90
5% 94
10% 95
15% 96
20% 97
25% 98
30% 99
35% 99
40% 100
45% 101
50% 101
55% 102
60% 103
65% 104
70% 104
75% 105
80% 106
85% 108
90% 109
95% 111
100% 115
Cum
ulat
ive
Freq
uenc
y
0 01 0 - C o n s tru c tio n : D u ra tio n
Here the Ranges are based on deviations + and – from the Plan. Probability is 100%
For the examples we use an activity with 100 days in the schedule
© 2009 Hulett & Associates, AcumenPM
Assigning a Probability Less than 100%
100 110 120 130
Distribution (start of interval)
0
200
400
600
800
1000
1200
Hits
0% 100
5% 100
10% 100
15% 100
20% 100
25% 100
30% 100
35% 100
40% 100
45% 105
50% 107
55% 109
60% 110
65% 111
70% 113
75% 114
80% 116
85% 118
90% 120
95% 123
100% 130
Cum
ulat
ive
Freq
uenc
y
0040 - Technology Design : Duration
95 100 105 110 115
Distribution (start of interval)
0
500
1000
1500
2000
Hits
0% 91
5% 97
10% 99
15% 100
20% 100
25% 100
30% 100
35% 100
40% 100
45% 100
50% 100
55% 100
60% 100
65% 100
70% 100
75% 100
80% 100
85% 101
90% 103
95% 107
100% 115
Cum
ulat
ive
Freq
uenc
y
0010 - Construction : Duration
Spike contains 70% of the probability
Spike contains 40% of the probability
© 2009 Hulett & Associates, AcumenPM
Assigning More than One Risk to an Activity
• If more than one risk is acting on an activity, the resulting ranges are the multiplication of the factors
• The activity duration range is derived from the risk factors that affect the risk– Model how the activity duration range is
generated– Focus on the causes of activity duration
ranges
© 2009 Hulett & Associates, AcumenPM
Two Risks affect One Activity using Factors that Occur 100%
100 110 120 130 140
Distribution (start of interval)
0
20
40
60
80
100
120
140
Hits
0% 93
5% 102
10% 104
15% 106
20% 108
25% 109
30% 110
35% 111
40% 113
45% 114
50% 115
55% 116
60% 117
65% 118
70% 119
75% 121
80% 123
85% 124
90% 127
95% 130
100% 144
Cum
ulat
ive
Freq
uenc
y
0040 - Technology Design : Duration
Range from 90 to 150 days, Peak about 113 days
© 2009 Hulett & Associates, AcumenPM
Two Risks with Less than 100% Probability Affecting one Activity
100 110 120 130 140
Distribution (start of interval)
0
100
200
300
400
500
600
700
800
900
1000
1100
Hits
0% 91
5% 97
10% 99
15% 100
20% 100
25% 100
30% 100
35% 100
40% 100
45% 101
50% 102
55% 104
60% 106
65% 108
70% 110
75% 111
80% 113
85% 116
90% 119
95% 123
100% 144
Cum
ulat
ive
Freq
uenc
y
0040 - Technology Design : Duration
The spike at 100 days represents (1) the likelihood that neither risk occurs and (2) the chance that 100 days is picked when one or both occur
Risk Factors Model How Correlation OccursCoefficients are Calculated (1)
© 2009 Hulett & Associates, AcumenPM
Risk #1P = 50%, Factors
.95, 1.05, 1.15
Activity A Activity B
Activities A and B Correlation is Calculated to be 100%
Risk Factors Model How Correlation OccursCoefficients are Calculated (2)
© 2009 Hulett & Associates, AcumenPM
Risk #1P = 50%, Factors
.95, 1.05, 1.15
Activity A Activity B
Activities A and B Correlation is Calculated to be 48%
Risk #2P = 25%, Factors
.8, .95, 1.05
Risk #3P = 45%, Factors
1.0, 1.1, 1.2
Correlation is modeled as it is caused in the projectCorrelation coefficients are generated, not guessed
© 2009 Hulett & Associates, AcumenPM
Sensitivity to the Risk Factors
The tornado diagram focuses on risks, not activities
© 2009 Hulett & Associates, AcumenPM
Simple 2-Stage Space Vehicle Schedule
Software used: Pertmaster v. 8
© 2009 Hulett & Associates, AcumenPM
Case Study: Simple Space Vehicle Development Schedule
• 87 month schedule• 11 work activities linked, 3 major milestones• Beginning 3 March 2008• PDR on 11 September 2009• CDR on 3 June 2011• Delivery to launch site 12 June 2015
© 2009 Hulett & Associates, AcumenPM
Two Types of Risk
• Background risk based on typical general risk, estimating error– Used Quick Risk of -5% and +10%
•• Discrete risksDiscrete risks derived from Risk Register– Summarized from detailed Risk Register– These have a probability of occurring and an
impact on specific activities if they do– Parallel to their Risk Register information– Risk Register is used in data collection
© 2009 Hulett & Associates, AcumenPM
Standard 3-point Range Representing Schedule Estimating Error
Background risk:
Optimistic -5% Pessimistic +10%
© 2009 Hulett & Associates, AcumenPM
Results with Schedule Estimating Error Only
Deterministic: 12JUN15 is <1%
P-80 is 30SEP15, about 3.5 monthslater than planned
Spread from P-5 to P-95 is 5JUL15 to 27OCT15 for 3.7 months
31/May/15 08/Sep/15 17/Dec/15
Distribution (start of interval)
0
100
200
300
400
Hits
0% 14/May/15
5% 06/Jul/15 10% 17/Jul/15
15% 24/Jul/15
20% 31/Jul/15 25% 07/Aug/15
30% 12/Aug/15
35% 18/Aug/15 40% 21/Aug/15
45% 26/Aug/15 50% 01/Sep/15
55% 04/Sep/15
60% 09/Sep/15 65% 14/Sep/15
70% 18/Sep/15
75% 24/Sep/15 80% 30/Sep/15
85% 07/Oct/15
90% 15/Oct/15 95% 27/Oct/15
100% 30/Dec/15
Cum
ulat
ive
Freq
uenc
y
Spacecraft ProgramEntire Plan : Finish Date
Risk Analysis on Space Vehicle ProjectRisk Factors are from Risk Register
© 2009 Hulett & Associates, AcumenPM
Time Impact Range Cost Impact Range
Probability Minimum Most Likely Maximum Minimum Most Likely Maximum
Requirements have not been decided
30% 95% 105% 120%
Several alternative designs considered
60% 95% 100% 115%
New designs not yet proven 40% 96% 103% 112%
Fabricaton requires new materials
50% 96% 105% 115%
Lost know‐how since last full spacecraft
30% 95% 100% 105%
Funding from Congress is problematic
70% 90% 105% 115%
Schedule for testing is aggressive
100% 100% 120% 130%
Cost Risk is based on immature data
100% 95% 105% 110%
Mapping Risks to Activities (1)
© 2009 Hulett & Associates, AcumenPM
Risk Requirements Definition
FS Preliminary Design
FS Final Design FS Fabrication
Test FS Engine
Requirements Not Complete X
Alternative Designs Possible X
Designs Not Proven XNew Materials in Fabrication X
Lost Know-How XFunding Problematic X X X XTesting Schedule Aggressive X
Cost Estimate is based on Immature Data
X X X X X
Mapping Risks to Activities (2)
© 2009 Hulett & Associates, AcumenPM
RiskUS Preliminary Design
US Final Design US Fabrication US Test Integration Integratio
n Testing
Requirements Not CompleteAlternative Designs Possible X
Designs Not Proven X
New Materials in Fabrication X
Lost Know-How X XFunding Problematic X X X X X X
Testing Schedule Aggressive X X
Cost Estimate is based on Immature Data
X X X X X X
© 2009 Hulett & Associates, AcumenPM
Results Adding Risk Factors to the Background Risk
Baseline 12JUN 15 is only 3% likely
The 80th percentile (P-80) is 26MAY16, 11.5 months later
Spread P-5 to P-95 is 13JUL15 to 5OCT16, for 15.5 months
20/Feb/15 08/Sep/15 26/Mar/16 12/Oct/16 30/Apr/17
Distribution (start of interval)
0
50
100
150
200
250
300
350
Hits
0% 21/Jan/15
5% 13/Jul/15
10% 27/Aug/15
15% 25/Sep/15
20% 15/Oct/15
25% 02/Nov/15
30% 16/Nov/15
35% 01/Dec/15
40% 16/Dec/15
45% 30/Dec/15
50% 15/Jan/16
55% 04/Feb/16
60% 24/Feb/16
65% 15/Mar/16
70% 06/Apr/16
75% 02/May/16
80% 26/May/16
85% 01/Jul/16
90% 11/Aug/16
95% 05/Oct/16
100% 03/May/17
Cum
ulat
ive
Freq
uenc
y
Spacecraft for PMChallenge 2009Entire Plan : Finish Date
© 2009 Hulett & Associates, AcumenPM
Activity Tornado Chart from All-In Simulation
Risky Activities:Fabrication, Integration, Final Design, Preliminary Design, TestingThis is the typical sensitivity analysis of 3-point estimating
61%
62%
63%
69%
69%
76%
76%
78%
80%
81%00025 - US Fabrication
00011 - FS Fabrication
00028 - Integration
00009 - FS Final Design
00023 - US Final Design
00021 - US Preliminary Design
00007 - FS Preliminary Design
00012 - Test FS Engine
00029 - Integration Testing
00026 - US Test
Spacecraft for PMChallenge 2009Duration Sensitivity
© 2009 Hulett & Associates, AcumenPM
Risk Factor Tornado from All-In Simulation
The main RISK, however, is funding from Congress, which affects all activities. This is the main risk to mitigate, if possible
6 - Funding from Congress is problematic
4 - Fabricaton requires new materials
3 - New designs not yet proven
7 - Schedule for testing is aggressive
5 - Lost know-how since last full spacecraft
2 - Several alternative designs considered
1 - Requirements have not been decided
0.00 0.20 0.40 0.60 0.80
Correlation
Driving Schedule Risk Factors
Contribution of Each Risk to the Time Contingency (1)
© 2009 Hulett & Associates, AcumenPM
Explain the Contingency to the P-80
P-80 Date Contribution of Risk
All Risks In 26-May-16 Days Saved % of Contingency
Specific Risks Taken Out in Order
Funding Risk 22-Jan-16 125 36%
Testing Schedule is Aggressive 1-Dec-15 52 15%
Design Requires New Materials 28-Oct-15 34 10%
New Design Risk 15-Oct-15 13 4%
Alternative Design Risk 6-Oct-15 9 3%
Requirements Risk 1-Oct-15 5 1%Lost Know How Risk 30-Sep-15 1 0%
Background Schedule Estimating Risks
Background Risk 12-Jun-15 110 32%
Total Contingency 349 100%
© 2009 Hulett & Associates, AcumenPM
Contribution of Each Risk to the Time Contingency (2)
26/
May
/16
20/
Jan/
16
01/
Dec
/15
28/
Oct
/15
1
6/O
ct/1
5
06/
Oct
/15
0
1/O
ct/1
5
01/
Oct
/15
3
0/Se
p/15
12/
Jun/
15
06/ Apr/ 15 15/ Jul/ 15 23/ Oct/ 15 31/ Jan/ 16 10/ May/ 16 18/ Aug/ 16 26/ Nov/ 16 06/ Mar/ 17
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Cum
ulat
ive
Pro
babi
lity
g g
Resources and Cost
© 2009 Hulett & Associates, AcumenPM
Estimate is $6.86 billion
Cost Risk Results
© 2009 Hulett & Associates, AcumenPM
$7,000,000,000 $8,000,000,000 $9,000,000,000
Distribution (start of interval)
0
50
100
150
200
250
300
350
400
450
500
550
600
650
Hits
0% $6,189,493,454
5% $6,861,086,752
10% $7,007,130,725
15% $7,101,752,271
20% $7,188,049,098
25% $7,258,107,546
30% $7,331,963,666
35% $7,391,997,593
40% $7,452,396,460
45% $7,514,273,968
50% $7,568,332,545
55% $7,630,117,222
60% $7,687,432,085
65% $7,749,200,097
70% $7,813,239,105
75% $7,890,603,866
80% $7,976,966,619
85% $8,088,750,149
90% $8,234,937,806
95% $8,429,356,166
100% $9,283,008,684
Cum
ulat
ive
Freq
uenc
y
Spacecraft for PMChallenge 2009Entire Plan : Cost
Baseline $6.86 billion is only 5% likely
The 80th percentile (P-80) is $7.98 B for a $1.1 B contingency
Spread P-5 to P-95 is $6.86 to $8.43, for $1.57 B
Contribution of Each Risk to the Cost Contingency
© 2009 Hulett & Associates, AcumenPM
Contributions of Individual Risks to Cost RiskAll Risks $ millionsCost Risk is based on immature data 287Funding from Congress is problematic 277Fabricaton requires new materials 168Schedule Estimate inaccurate 110Schedule for testing is aggressive 63New designs not yet proven 31Lost know‐how since last full spacecraft 2Several alternative designs considered 1Requirements have not been decided 0
Scatter Diagram of Time and Cost
© 2009 Hulett & Associates, AcumenPM
3%
1% 94%
2%
20/Feb/15 31/May/15 08/Sep/15 17/Dec/15 26/Mar/16 04/Jul/16 12/Oct/16 20/Jan/17 30/Apr/17
Entire Plan: Finish
3%
$6,200,000,000
$6,400,000,000
$6,600,000,000
$6,800,000,000
$7,000,000,000
$7,200,000,000
$7,400,000,000
$7,600,000,000
$7,800,000,000
$8,000,000,000
$8,200,000,000
$8,400,000,000
$8,600,000,000
$8,800,000,000
$9,000,000,000
$9,200,000,000E
ntir
e Pl
an: C
ost
5%
Spacecraft for PMChallenge 2009
12/ Jun/ 15
$6,861,500,000
Deterministic Point Inside both limits Outside both limits
There is a 94% chance of overrunning both cost and scheduleNotice the slope of the scatter –time drives cost in this model
Probabilistic Cash Flow
© 2009 Hulett & Associates, AcumenPM
Deterministic Cost: $6,861,500,000
Det
erm
inis
tic F
inis
h: 1
2/Ju
n/15
26/ Jul/ 08 11/ Feb/ 09 30/ Aug/ 09 18/ Mar/ 10 04/ Oct/ 10 22/ Apr/ 11 08/ Nov/ 11 26/ May/ 12 12/ Dec/ 12 30/ Jun/ 13 16/ Jan/ 14 04/ Aug/ 14 20/ Feb/ 15 08/ Sep/ 15 26/ Mar/ 16 12/ Oct/ 16 30/ Apr/ 17
Time
0
1,000,000,000
2,000,000,000
3,000,000,000
4,000,000,000
5,000,000,000
6,000,000,000
7,000,000,000
8,000,000,000
9,000,000,000
Cum
ulat
ive
Resource Flow for CostFilter: Entire Plan
Mean P20 P80
This monthly probabilistic cash flow can be compared to the budget to adjust the spending patterns when considering risk
Probabilistic Cash Flow Compared to Planned
© 2009 Hulett & Associates, AcumenPM
© 2009 Hulett & Associates, AcumenPM
Summary (1)
• The focus is on the risks, not their impact• Risks “explain” the need for a contingency• Management appreciates this focus on
risks• Risk interviews are conducted at 10,000
foot level, where people typically think of risk
• Interviews go faster, stick to the substance
© 2009 Hulett & Associates, AcumenPM
Summary (2)
• Use Risk Register for quantitative analysis• Specific risks can be quantified and assigned to
schedule activities– Quantification is probability and impact– A risk can affect several activities– An activity can be affected by several risks
• Risk Factors can be combined with other more traditional approaches such as 3-point estimates for background risk or probabilistic branching
Summary (3)
• Schedule uncertainty creates cost uncertainty
• Analysis of cost simultaneously with time requires inserting the budget/resources into the schedule and simulating both together
• More accurate cost risk analysis and full appreciation of the role of schedule risk in creating cost risk
© 2009 Hulett & Associates, AcumenPM
Using the Risk Register in Integrated Cost/Schedule Risk Analysis with Monte Carlo Simulation
David T. Hulett, Ph.D.Hulett & Associates, LLC
Dr. Dan Patterson, PMPAcumenpm
NASA PM ChallengeDaytona Beach, FL
February 24-25, 2009
© 2009 Hulett & Associates, AcumenPM