Barely Sufficient Portfolio Management Todd Little So many decisions, more time than we thought.
Todd Little
description
Transcript of Todd Little
Todd Little
Uncertainty Surrounding the
Cone of Uncertainty
IEEE Software, May/June 2006
Managing the Coming Storm Inside the Tornado
When will we get the requirements?All in good time, my little pretty, all in good timeBut I guess it doesn't matter anyway
Doesn't anybody believe me?
You're a very bad man!
Just give me your estimates by this afternoon
No, we need something today!
I already promised the customer it will be out in 6 months
No, we need it sooner.
Not so fast! Not so fast! ... I'll have to give the matter a little thought. Go away and come back tomorrow
Ok then, it will take 2 years.
Team Unity
Project Kickoff
We’re not in Kansas Anymore
My! People come and go so quickly here!
I may not come out alive, but I'm goin' in there!
The Great and Powerful Oz has got matters well in hand.
"Hee hee hee ha ha! Going so soon? I wouldn't hear of it! Why, my little party's just beginning!
Developer HeroReorg
Testing
The Cone of Uncertainty (Boehm)
0.1
1
10
Feasibility Concept ofOperation
RequirementsSpec
ProductDesign Spec
Detail DesignSpec
AcceptedSoftware
Minimum
Maximum
0.5
2
Cone of Uncertainty: The amount of possible error in a software project estimate, which is very large in the early stages of a project and shrinks dramatically as the project nears completion – Steve McConnell
Hurricane Rita
Landmark Product Suite
Common Model Representation
Well data
Production data
Seismic data
Velocity data
Reservoir /Fluid data
Structural /Stratigraphic data
Common Model Representation
Data from LGC
Initial Estimate vs. Actual Project Duration (from LGC Portfolio Database)
y = 1.6886x
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700 800 900 1000
Initial Estimate
Ac
tua
l
LGC Data
Ideal
Linear (LGC Data)
Data from Tom DeMarco
It’s déjà vu all over againIndustry data from Tom DeMarco
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600 700 800 900 1000
Estimated Effort
Ac
tua
l E
ffo
rt
Actual
2X
Ideal
Cumulative Distribution Curve for Actual/Estimate (DeMarco)
DeMarco Cumulative Distribution Function
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1 1 10
Ratio
CD
F P
rob
ab
ilit
y
Data
Log Normal Curve
p(10) 0.79p(50) 1.74p(90) 3.81
CDF Distribution Curve (LGC)
Landmark Graphics Cumulative Distribution of Portfolio Projects
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1 1 10
Ratio of Actual to Estimate
Cu
mm
ula
tive
Pro
bab
ility
p(10) 0.96p(50) 1.76p(90) 3.23
Probability Distribution Curve
Distribution Curve of Actual/Estimated (DeMarco data vs. LGC)(Demarco data is Effort/Effort; LGC data is Duration/Duration)
0
0.5
1
1.5
2
2.5
0 1 2 3 4 5 6 7 8
(Actual/Estimated)
Fre
qu
en
cy
DeMarco
LGC
Log Normal Distribution
Estimation Accuracy follows a Log Normal distribution
0.1 1 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Median
Mean
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50.1 1 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50.1 1 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
“It’s tough to make predictions, especially about the future.”Yogi Berra, Niels Bohr
That idea is so damned nonsensical and impossible that I'm willing to stand on the bridge of a battleship while that nitwit tries to hit it from the air.
Newton Baker, U.S. secretary of war in 1921, reacting to the claim of Billy Mitchell (later Brigadier General Mitchell) that airplanes could sink battleships by dropping bombs on them.
“It’s tough to make predictions, especially about the future.”Yogi Berra, Niels Bohr
Heavier-than-air flying machines are impossible.Lord Kelvin, British mathematician, physicist, and president of the British Royal Society, spoken in 1895.
How does Estimation Accuracy Improve Over Time? (Boehm)
0.1
1
10
Feasibility Concept ofOperation
RequirementsSpec
ProductDesign Spec
Detail DesignSpec
AcceptedSoftware
Minimum
Maximum
0.5
2
0
200
400
600
800
1000
1200
0 100 200 300 400 500 600
Ideal
From Start
Post Env
Post Plan
Post Dev
So what does LGC data look like?
Landmark Cone of Uncertainty
But is Uncertainty Really Reduced?
“Take away an ordinary person’s illusions and you take away happiness at the same time.”
Henrik Ibsen--Villanden
Cumulative Distribution (CDF) Curve
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1 1 10 100
Initial
Post Env
Post Plan
Post Dev
Log Normal
Remaining Uncertainty
The Pipe of Uncertainty
0.1
1
10
Envisioning Planning Developing Stabilizing
Minimum
Maximum
0.5
2
Does Landmark Suck at Estimation?
A severe depression like that of 1920-21 is outside the range of probability.Harvard Economic Society, Weekly Letter, November 16, 1929.
Does Landmark Suck at Estimation?
I think there is a world market for about five computers.Thomas J. Watson, chairman of IBM, 1943.
Does Landmark Suck at Estimation?
They couldn't hit an elephant at this dist…General John B. Sedgwick, Union Army Civil War officer's last words, uttered during the Battle of Spotsylvania, 1864
Estimation Quality Factor (EQF)
Elapsed Time
Val
ue to
be
Est
imat
ed
Actual Value
Initial Estimate
Actual End Date
Link to article by Tim Lister
Blue Area
Red AreaEQF =
EQF from Lister/DeMarco
EQF Comment
1.8 Typical disaster project
3.8 Median of Medians
5.0 Pretty Good
8.0Excellent sustained Median EQF
10.0 Not Sustained
EQF Distribution Curve (LGC)
Landmark Graphics Cumulative Distribution of EQF
-0.2
0
0.2
0.4
0.6
0.8
1
0.1 1 10 100
EQF
CD
F/P
rob
Raw EQF
LogNormal
p(10) 2.80p(50) 4.78p(90) 11.68
EQF for duration has a theoretical minimum of 2.0
We slip one day at a time, EQF=2
Elapsed Time
Val
ue to
be
Est
imat
ed
Actual Value
Initial Estimate Actual End Date
Blue Area
Red AreaEQF =
(EQF-2) Distribution Curve (LGC data)
Landmark Graphics Cumulative Distribution of EQF
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1 1 10 100
EQF
CD
F/P
rob
EQF Data - 2
Log Normal Curve
p(10) 2.80p(50) 4.78p(90) 11.68
From the home office in Duncan, Oklahoma
Top Ten reasons why we are late in 2008
Dubai, UAEDubai, UAE
10: Requirements, what Requirements?
What you want, baby I got it
R-E-Q-U-I-R-E Find out what it means to me
Top Ten reasons why we are late in 2008
9: Dependencies on other groups that were late
Top Ten reasons why we are late in 2008
8: Over-optimistic Schedule Estimation
Always look on the bright side of code. . . . . . .Always look on the bright side of code. . . . . . .The code’s a piece of $#!^, when we look at itWe can always overlook a minor kink . . . .It probably compiles, it might even link . . .Surely that must mean it doesn’t stink
Top Ten reasons why we are late in 2008
7: Those weren’t MY estimates
How low can you go!
Scheduling Ritual
Top Ten reasons why we are late in 2008
6: Not enough testers or documentation resources.
Who needs them anyway? We put those bugs--I mean features--in there on purpose. Besides, it was difficult to program, it should be difficult to use.
Top Ten reasons why we are late in 2008
5: Offshore and Outsourcing issues
My source code lies over the ocean, My source code lies over the ocean, My source code lies over the sea .My source code lies over the sea .
My source code lies over the ocean, My source code lies over the ocean, Oh bring back my source code to meOh bring back my source code to me
. . . . .. . . . .Bring Back, Bring Back, Bring Back, Bring Back,
oh bring back my source code to me, to meoh bring back my source code to me, to meBring Back, Bring Back, Bring Back, Bring Back,
oh bring back my source code to meoh bring back my source code to me
Top Ten reasons why we are late in 2008
4: One word, Ch-ch-ch-changes
Top Ten reasons why we are late in 2008
3: I can’t get no, System Admin I can’t get no, CM action ‘cause I try, ..and I try, ….and I try, ……and I try….
Top Ten reasons why we are late in 2008
2: You didn’t give me the headcount that you promised
Top Ten reasons why we are late in 2008
1: Weren’t you doing the backups!?1: Weren’t you doing the backups!?
Top Ten reasons why we are late in 2008
You can get better
How to get better
Start Measuring Understand bias Collaborate Manage Uncertainty
Start Measuring
Understand Bias
"What gets us into trouble is not what we don't know. It's what we know for sure that just ain't so.“
» Mark Twain
Don’t know thatwe don’t know
Knowable
Unknowable
Uncertainty
Know that we know
Know that we don’t know
Don’t know that we know
Don’t know thatwe don’t know
Knowable
Unknowable
Uncertainty
Know that we know
Know that we don’t know
Don’t know that we know
UncertaintyManagement
Planning Risk Management
p10 p50
p90
Just Double the Estimate
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.1 1 10
Actual/Estimate Ratio
CD
F
Defending an Unpopular Schedule
http://www.stevemcconnell.com/ieeesoftware/bp03.htm
Developers tend to be temperamentally opposed to the use of negotiating tricks. Such tricks offend their sense of technical accuracy and fair play. Developers don't want to offer lopsidedly high initial estimates even when they know that customers, marketers, or bosses will start with lopsidedly low bargaining positions.
Steve McConnell
We want this
Negotiation Bias
"It is difficult to get a man to understand something when his salary depends upon his not understanding it.“
» Upton Sinclair:
Trust: The Cooperative or Non-Cooperative Game?
Customer perception of Team's ability to Deliver
Desire to set aggressive targets
Team’s ability to Deliver against targets
o
o
s
R
Customer perception of Team's ability to Deliver
Team’s ability to Deliver against targets
Test 1 (Jørgensen)
Group Guidance Result
A 800
B 40
C 4
D None 160
Test 1
Group Guidance Result
A 800 300
B 40 100
C 4 60
D None 160
Test 2
Group Guidance Result
A Minor Extension
B New Functionality
C Extension 50
Test 2
Group Guidance Result
A Minor Extension
40
B New Functionality
80
C Extension 50
Test 3
Group Guidance Result
A Future work at stake, efficiency will be measured
B Control 100
Test 3
Group Guidance Result
A Future work at stake, efficiency will be measured
40
B Control 100
Collaborate: Planning Poker
Collaborative Estimation Activity
Modified Fibonacci Sequence 0,½,1,2,3,5,8,13,20,40,100
,∞,?
Interactive version at www.planningpoker.com
Exercise
Low Med High
Distance from Las Vegas to Houston, Texas
Height of the Empire State Building
Population of Sweden
U.S. Oil Consumption/day
Water in a 100 gallon vat filled with sand
Exercise
Low Med High
Distance from Las Vegas to Houston, Texas
1222
Height of the Empire State Building
1453
Population of Sweden 9MM
U.S. Oil Consumption/day 20MM
Water in a 100 gallon vat filled with sand
35
Managing for Uncertainty
We expect uncertainty and manage for it through iterations, anticipation, and adaptation.
The Cone of Uncertainty
Uncertainty
Time
Val
ue
ContractMinimum scope for the release at the latest date that it can be released.
PlanPlanned scope for the
release at the optimal time that it can be released.
TargetBest possible scenario if
everything went perfectly.
The A/B/C List sets proper expectations
A MUST be completed in order to ship the product.
B SHOULD be completed in order to ship the product.
C MAY be completed prior to shipping the product if time allows.
Only “A” features may be committed to customers.
“A” features must fit in a p90 confidence schedule. No more than 50% of the planned effort can be allocated to “A” items
A
A/B/C List
50% 100%
Backlog Plan
Typical Delivery
25%
A B C
B C D
50% 25%
Target Delivery Date
A/B/C List
50% 100%
Backlog Plan
Uncertainty Risk
25%
A B C
B C D
50% 25%
Target Delivery Date
A
Uncertainty
Time
Val
ue
C
B
A
Uncertainty
Time
Val
ue
Failure
MaximalSuccess
ScheduleFlexible
ScopeFlexible
MinimalSuccess
Estimation accuracy improves(Eveleens and Verhoef)
Estimation accuracy constant(Eveleens and Verhoef)
Estimation accuracy decreases(Eveleens and Verhoef)
Take Aways
Estimation follows log-normal patterns There is a cone, sorta Relative uncertainty can reduce, but
often may stay relatively constant Improvement is possible Watch out though
Successful Projects?
Customer Satisfaction
Contact
Todd Little Senior Development Manager, Landmark
Graphics Corporation, www.lgc.com [email protected] www.toddlittleweb.com
Coming Soon from Addison-Wesley!
Stand Back and Deliver: A Leaders Guide to the Agile Enterprise
Your Questions?
The Cone of Uncertainty