Using Simulation to Manage Software Delivery Risk
-
Upload
focusedobjective -
Category
Business
-
view
685 -
download
1
description
Transcript of Using Simulation to Manage Software Delivery Risk
![Page 1: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/1.jpg)
Using Simulation to Manage Software Delivery Risk
Effective Modeling and Simulating Kanban and Scrum Projects using
Monte Carlo Techniques
Troy Magennis [email protected]
@AgileSimulation
![Page 2: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/2.jpg)
Are we meeting our commitments?
Schedule Risk
![Page 3: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/3.jpg)
What team skillset additions (or losses) have the biggest
impact?
Staff Risk
![Page 4: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/4.jpg)
What are the top three risks jeopardizing delivery?
Risks
![Page 5: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/5.jpg)
![Page 6: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/6.jpg)
Arm my teams (and yours) with the tools and techniques
to answer these questions
My Mission
And manage risk more effectively
![Page 7: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/7.jpg)
• Currently: Founder and CTO Focused Objective • Previously: Vice President of Technology (Arch)
• Travelocity and Lastminute.com • Director Architecture, Corbis • Various: Automotive, Banking
Contact: @AgileSimulation and @t_magennis FocusedObjective.com [email protected]
![Page 8: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/8.jpg)
![Page 9: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/9.jpg)
DEFINITIONS, HISTORY AND USE What, when, who, why
![Page 10: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/10.jpg)
A model is a tool used to mimic a real world process
A tool for low-cost experimentation
Definition: Model
![Page 11: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/11.jpg)
A technique of using a model to determine a result given a
set of input conditions
Definition: Simulation
![Page 12: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/12.jpg)
Performing a simulation of a model multiple times using
random input conditions and recording the frequency of
each result occurrence
Monte Carlo Simulation
![Page 13: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/13.jpg)
DEMO: VISUAL MODEL SIMULATION DEMO: MONTE-CARLO SIMULATION
Simple to more complex model and simulation of a software project
In case of demo disaster, press here…
![Page 15: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/15.jpg)
When there is no correct single answer (knowable in
advance) or when the time/effort taken to compute an answer is beyond realistic
When to use Monte Carlo Simulation
![Page 16: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/16.jpg)
When a range of input conditions can MASSIVELY
alter the final outcome
When to use Monte Carlo Simulation
![Page 17: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/17.jpg)
![Page 18: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/18.jpg)
High risk industries Natural resource exploration, insurance,
finance, banking, pharmaceutical…
Who Uses Monte Carlo Simulation
Software Development == High Risk! Just look at our reputation, and on-time, on-budget success rate…
![Page 19: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/19.jpg)
APPLYING MONTE CARLO METHODS TO SOFTWARE DEV
![Page 20: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/20.jpg)
Why? To Answer Tough Questions…
Date and cost forecasts Impact of staff hire/loss
Cost of defects Cost of blocking events
… And my three 1:1 questions each week!
![Page 21: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/21.jpg)
But doesn’t it require estimates?
Yes, but very few…
MUST: Estimate major risks
SHOULD: Column cycle-times and story counts
![Page 22: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/22.jpg)
We need to estimate risk events
We spend all our time estimating here
1 2 3
**Major risk events have the predominate role in deciding where deliver actually occurs **
![Page 23: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/23.jpg)
Is it Accurate?
1. Gin still equals Gout
2. Doesn’t suffer from the “Flaw of Averages”
![Page 24: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/24.jpg)
Flaw of Averages
50% Possible Outcomes
50% Possible Outcomes
![Page 25: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/25.jpg)
Freq
ue
ncy
of
Res
ult
1
5
10
15
20
25
Major issue: Race condition, third party component failure…
The average release date!!!
Major Risk Event Shifts Delivery Shape Right
Developer Estimates
![Page 26: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/26.jpg)
We need to estimate risk events
See model example…
We spend all our time estimating here
1 2 3
**Major risk events have the predominate role in deciding where deliver actually occurs **
![Page 27: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/27.jpg)
Risk likelihood changes constantly
1 2 3
95th Confidence
Interval
![Page 28: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/28.jpg)
Risk likelihood changes constantly
1 2 3
95th Confidence
Interval
![Page 29: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/29.jpg)
Risk likelihood changes constantly
1 2 3
95th Confidence
Interval
![Page 30: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/30.jpg)
Risk likelihood changes constantly
1 2 3
95th Confidence
Interval
![Page 31: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/31.jpg)
DEMO: FORECASTING (DATES & COST) DEMO: SENSITIVITY (COST OF DEFECTS) DEMO: STAFF IMPACT (STAFF RISK) In case of demo disaster or no internet, press here…
![Page 32: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/32.jpg)
BEST PRACTICES AND TIPS
![Page 33: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/33.jpg)
Model (a little)
Visually Test
Monte-Carlo Test
Sensitivity Test
The Model Creation
Cycle
![Page 34: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/34.jpg)
Baseline
Make Single
Change
Compare Results
Make Informed
Decision(s)
The Experiment
Cycle
![Page 35: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/35.jpg)
Best Practice 1
Start simple and add ONE input condition at a time.
Visually / Monte-carlo test
each input to verify it works
![Page 36: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/36.jpg)
Best Practice 2
Find the likelihood of major events and estimate delay E.g. vendor dependencies,
performance/memory issues, third party component
failures.
![Page 37: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/37.jpg)
Best Practice 3
Only obtain and add detailed estimates and opinion to a
model if Sensitivity Analysis says that input is material
![Page 38: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/38.jpg)
Best Practice 4
Use a uniform random input distribution UNTIL sensitivity
analysis says that input is influencing the output
![Page 39: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/39.jpg)
Best Practice 5
Educate your managers’ about risk. They will still want a “single” date for planning, but let them decide 75th or
95th confidence level (average is NEVER an option)
![Page 40: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/40.jpg)
Q1. Are we meeting our commitments?
Q2. What are the top three risks jeopardizing on-time delivery?
Q3. What skillsets do your next three hires need to have?
Is the likelihood of the models forecast date increasing or decreasing?
Top three items in the Sensitivity report
Skills applicable to the top three WIP limit increases that cause the biggest reduction in forecast
![Page 41: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/41.jpg)
Call to action
• Read these books
• Download the software FocusedObjective.com
• Follow @AgileSimulation
• Learn: http://strategicdecisions.stanford.edu/
![Page 42: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/42.jpg)
Questions?
My Contact Details and to get these slides, the software or the book used in this session -
FocusedObjective.com
Follow: @AgileSimulation and @t_magennis
![Page 43: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/43.jpg)
![Page 44: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/44.jpg)
BASICS OF MODELING AND SIMULATION
Return to main presentation…
![Page 45: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/45.jpg)
Manual Kanban Model & Simulation
Backlog Design Develop Test
Deployed
2
1 – 2 days 1 – 2 days 1 – 2 days
1
2 3 4
PLUS: For this manual example, at least 1 defect, blocking event and scope-creep item.
5
![Page 46: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/46.jpg)
Day 1
Backlog Design Develop Test
Deployed
2
1 – 2 days 1 – 5 days 1 – 2 days
1 Day picked at random for this columns cycle-
time range
![Page 47: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/47.jpg)
Day 2
Backlog Design Develop Test
Deployed
2 1 day
1 – 2 days 1 – 5 days 1 – 2 days
![Page 48: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/48.jpg)
Day 3
Backlog Design Develop Test
Deployed
2 1 day
1 – 2 days 1 – 5 days 1 – 2 days
![Page 49: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/49.jpg)
Day 4
Backlog Design Develop Test
Deployed
2 1 day
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 50: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/50.jpg)
Day 5
Backlog Design Develop Test
Deployed
2 1 day
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 51: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/51.jpg)
Day 6
Backlog Design Develop Test
Deployed
2 1 day
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 52: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/52.jpg)
Day 7
Backlog Design Develop Test
Deployed
2 1 day
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 53: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/53.jpg)
Day 8
Backlog Design Develop Test
Deployed
2 1 day
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 54: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/54.jpg)
Day 9
Backlog Design Develop Test
Deployed
2
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 55: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/55.jpg)
Day 10
Backlog Design Develop Test
Deployed
2
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 56: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/56.jpg)
Day 11
Backlog Design Develop Test
Deployed
2
1 – 2 days 1 – 5 days 1 – 2 days
Added Scope
![Page 57: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/57.jpg)
Result versus Frequency (50 runs)
More Often
Less Often Result Values – For example, Days
15 10 20
Fre
qu
en
cy o
f R
esu
lt
1
5
10
15
20
25
![Page 58: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/58.jpg)
Result versus Frequency (250 runs)
More Often
Less Often Result Values – For example, Days
15 10 20
Fre
qu
en
cy o
f R
esu
lt
1
5
10
15
20
25
![Page 59: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/59.jpg)
Result versus Frequency (1000+ runs)
More Often
Less Often Result Values – For example, Days
15 10 60
Fre
qu
en
cy o
f R
esu
lt
1
5
10
15
20
25
![Page 60: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/60.jpg)
Central Limit Theorum
Return to main presentation…
![Page 61: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/61.jpg)
Flaw of Averages
50% Possible Outcomes
50% Possible Outcomes
Return to main presentation…
![Page 62: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/62.jpg)
Software Development Model
Model Columns & WIP
Work
Defects
Blocking Events Added
Work
Staff Vacations
…
1
2
3 4
5
6
7
Return to main presentation…
![Page 63: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/63.jpg)
Return to main presentation…
![Page 64: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/64.jpg)
SIMULATION EXAMPLES
Return to main presentation…
![Page 65: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/65.jpg)
unlikely
certain
Forecasts Return to main presentation…
![Page 66: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/66.jpg)
unlikely
certain
Forecasts
50% Possible
Outcomes
50% Possible Outcomes
Return to main presentation…
![Page 67: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/67.jpg)
Actively Manage
Ignore for the moment
Sensitivity Report Return to main presentation…
![Page 68: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/68.jpg)
Staff Skill Impact Report
Explore what staff changes have the greatest impact
Return to main presentation…
![Page 69: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/69.jpg)
Return to main presentation…
![Page 70: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/70.jpg)
MULTI-MODAL RESULT MODEL
![Page 71: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/71.jpg)
Return to main presentation…
![Page 72: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/72.jpg)
<setup> <backlog type="custom" > <deliverable name=“work"> <custom count="10">Build website</custom> </deliverable> <deliverable name="performance issues, add caching" skipPercentage="50"> <custom count="10" >Rework: Performance Issues</custom> </deliverable> </backlog> <columns> <column id="1" estimateLowBound="1" estimateHighBound="5" wipLimit="1">Develop</column> <column id="2" estimateLowBound="1" estimateHighBound="5" wipLimit="1">Test</column> </columns> </setup>
Return to main presentation…
![Page 73: Using Simulation to Manage Software Delivery Risk](https://reader033.fdocuments.in/reader033/viewer/2022060108/554f8e16b4c9052a518b5250/html5/thumbnails/73.jpg)
Return to main presentation…