Risk And Relevance 20080414ppt
description
Transcript of Risk And Relevance 20080414ppt
®
IBM Software Group
© 2008 IBM Corporation
Risk and Relevance
Dr. Murray CantorDistinguished Engineer, IBM Rational [email protected]
IBM Software Group | Rational software
22
Value is being created by software/IT
Transportation Equipment
Hospitals
Chemical Manufacturing
Software/Information
Food Manufacturing
Telecommunications
Computers/Electronics
$0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00
US Industry Revenue in $B
US Census Bureau
IBM Software Group | Rational software
33
Software provides huge value for all the other industries
US Industry Revenue in $B
US Census Bureau
Transportation Equipment
Hospitals
Chemical Manufacturing
Software/Information
Food Manufacturing
Telecommunications
Computers/Electronics
$0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00
IBM Software Group | Rational software
44
US Industry Revenue in $B
US Census Bureau
Transportation Equipment
Hospitals
Chemical Manufacturing
Software/Information
Food Manufacturing
Telecommunications
Computers/Electronics
$0.00 $100.00 $200.00 $300.00 $400.00 $500.00 $600.00 $700.00 $800.00
Software provides huge value for all the other industries
This does not include value created by
back office/operations support
IBM Software Group | Rational software
55
The concern is we are not creating value efficiently
For example, in a European Services Strategy Unit 2007 report studied105 outsourced public sector ICT projects with significant cost overruns, delays and terminations, total value of contracts is £29.5 billion
Results Cost overruns totaled £9.0 billion
57% of contracts experienced cost overruns
The average percentage cost overrun is 30.5%
33% of contracts suffered major delays
30% of contracts were terminated
12.5% of Strategic Service Delivery Partnerships have failed
How much value was deliveredby all the successful efforts?
IBM Software Group | Rational software
66
How should we meet the challenge of efficiently delivering value?
Focusing on avoiding overruns? Leads to risk avoidance
limits opportunity for creating value, to be relevant
Rather, lets discuss how to apply engineering discipline to creating value
IBM Software Group | Rational software
77
History has shown we do not get the estimates ‘right’Boehm COCOMO Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
100 1000 10000 100000
Size (SLOC)
Eff
ort
(S
taff
-mo
nth
s)
Jones Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
1.00 10.00 100.00 1000.00 10000.00
Size (FP)
Eff
ort (
LM)
Basili & Bailey Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
100 1000 10000 100000 1000000
Size (SLOC)
Eff
ort
(L
M)
ISBSG Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
1.00 10.00 100.00 1000.00 10000.00
Size (FP)
Effo
rt (L
M)
Rone Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
1.00 10.00 100.00 1000.00 10000.00
Size (SLOC)
Eff
ort
(S
M)
From George Stark, Paul Oman, “A comparison of parametric Software Estimation Models using real project data”, in press
IBM Software Group | Rational software
88
History has shown we do not get the estimates ‘right’Boehm COCOMO Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
100 1000 10000 100000
Size (SLOC)
Eff
ort
(S
taff
-mo
nth
s)
Jones Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
1.00 10.00 100.00 1000.00 10000.00
Size (FP)
Eff
ort (
LM)
Basili & Bailey Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
100 1000 10000 100000 1000000
Size (SLOC)
Eff
ort
(L
M)
ISBSG Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
1.00 10.00 100.00 1000.00 10000.00
Size (FP)
Effo
rt (L
M)
Rone Effort Estimation Accuracy
1.00
10.00
100.00
1000.00
1.00 10.00 100.00 1000.00 10000.00
Size (SLOC)
Eff
ort
(S
M)
From George Stark, Paul Oman, “A comparison of parametric Software Estimation Models using real project data”, in press, 2007
Variance dominates
Maybe there is a reason
IBM Software Group | Rational software
99
“Make everything as simple as possible, but not simpler.”
- Albert Einstein
“We should be guided by theory, not by numbers.”
- W. Edward Deming
Eternal wisdom
“To measure is to know. If you can not measure it, you can not improve it.”
- Lord Kelvin
“There are risks and costs to a program of action. But they are far less than the long-range risks and costs of comfortable inaction.”
- John F. Kennedy
IBM Software Group | Rational software
1010
The value conversation can vary
Future business efficiency Investing now to receive future savings, capacity, responsiveness
Operational risk avoidance Investing now to avoid future business/IT risks, e.g., security, privacy, continuity …
Business impact Investing now to affect future top line
Treacy Framework
Mission capabilities
ProductInnovation
OperationalEfficiency
ClientIntimacy
Your Business?
IBM Software Group | Rational software
1111
The value conversation can vary
ProductInnovation
OperationalEfficiency
ClientIntimacy
Your Business?
Future business efficiency Investing now to receive future savings, capacity, responsiveness
Operational risk avoidance Investing now to avoid future business/IT risks, e.g., security, privacy, continuity …
Business impact Investing now to affect future top line
Treacy Framework
Mission capabilities
All can be monetized
IBM Software Group | Rational software
1212
Just enough theory
Money is how one measures value – need to monetize
The future cost, value are uncertain
Cost and value are random variables
Need to consider current Net Present Value of program
Results need only to be accurate enough to drive desired behavior: Assuming managed risk to deliver value
IBM Software Group | Rational software
1313
How much is an incomplete development program worth?
Conventional wisdom is the accountants answer. provides no opportunity for ongoing value management
Can only discuss cost, not value
0 6 Ship date
Conventional WisdomV
alu
e
Little opportunity to show relevance
IBM Software Group | Rational software
1414
Imagine (if you will) you could sell your program
The buyer would spend money now to get the right to invest in completing the program – like a call option
How would one reason about the fair price? The buyer, reasoning like an investor, would like to know
Expected cost to complete
Expected value received
The riskThis is what the economists call“incomplete market reasoning”
IBM Software Group | Rational software
1515
The Net Present Value of a development program
RI = revenue, benefits stream
Ej = development expense stream
Mk = maintenance expenses stream
The rR, rM, rE are discount rates accounting for the time value of money
NPV, Ri, Ej, Mk are all random variables
0111 )1()1()1(
Er
E
r
M
r
RNPV
m
jj
E
jp
kk
M
kn
ii
R
i
IBM Software Group | Rational software
1616
The Net Present Value of a development program
RI = revenue, benefits stream
Ej = development expense stream
Mk = maintenance expenses stream
0111 )1()1()1(
Er
E
r
M
r
RNPV
m
jj
E
jp
kk
m
kn
ii
R
i
NPV, Ri, Ej, Mk are all random variables
So, how can value be created?
IBM Software Group | Rational software
1717
Expense risk: Imagine you have 12 months to deliver a business critical system
Your estimators tell you it will be done in 11 months
What do you do with the information? Rest easy, believing there is no risk?
0 6 12
IBM Software Group | Rational software
1818
Maybe you realize that program parameters (cost, schedule, effort, quality, …) are random variables Area under curve describes probability of measurement
falling in range
0.80
1.20
1.60
2.00
2.40
2.80
3.20
3.60
4.00
4.40
4.80
5.20
5.60
6.00
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0.00
0.40
Likelihood of actual value falling in range
is area under curve
IBM Software Group | Rational software
1919
Imagine you have 12 months to deliver a business critical systems
So you ask for the distribution and discover there is some uncertainty
0 6 12
IBM Software Group | Rational software
2020
Imagine you have 12 months to deliver a business critical systems
In fact there is less than 50% chance of making the date
48%
0 6 12
IBM Software Group | Rational software
2121
0 6 12
Then what?
Move out the date to improve likelihood of shipping?
15
95%
IBM Software Group | Rational software
2222
0 6 12
Then what?
Or move in the estimate by sacrificing quality or content?
95%
IBM Software Group | Rational software
2323
The estimate variance reflects current state of understanding Source
Lack of knowledge
Lack of confidence
Reduction of variance reflects Increased knowledge about
Client needs
Technology
Team capability
Good decisions 0.80
1.20
1.60
2.00
2.40
2.80
3.20
3.60
4.00
4.40
4.80
5.20
5.60
6.00
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0
0.00
0.40
IBM Software Group | Rational software
2424
Then what?
Determine the source of the variance
Over the project lifecycle, reduce the variance to improve likelihood of shipping
90%
0 6 12
IBM Software Group | Rational software
2525
0 6 12
Then what?
Over the lifecycle, reduce the variance further to improve likelihood of shipping
95%
IBM Software Group | Rational software
2626
To summarize so far
Cost, schedule are random variables
Variance is the measure of risk
Progress measured by reduction of risk
0111 )1()1()1(
Er
E
r
M
r
RNPV
m
jj
E
jp
kk
m
kn
ii
R
i
Medium Variance
Time
Var
ian
ce i
n C
ost
, S
ched
ule
…
Project Lifecycle Phases
Inception Elaboration Construction Transition/Maintenance
High Variance Low Variance
80% reduction96% reduction
IBM Software Group | Rational software
2727
Benefits: Delivered value is also a distribution
0111 )1()1()1(
Er
E
r
M
r
RNPV
m
jj
E
jp
kk
m
kn
ii
R
i
Shortfall in stakeholder needs
Value is improved by increasing upside variance0 Planned $Value
Reuse opportunities
IBM Software Group | Rational software
2828
Simple inputs
Expert opinion Just guess using expert opinion
Use optimistic, pessimistic, nominal assumptions in other models
Function point, Use case point, Cocomo
Enter into triangular distributions
Historical data Use other distributions if you have them
IBM Software Group | Rational software
2929
Model output
In this case Expected NPV = $4,324K
No chance > $14,924K, < ($4,837k)
Mean 4,324.45
Standard Deviation 2,738.59
Variance 7,499,853.36
0% -4,836.65
10% 818.52
20% 1,959.10
30% 2,804.66
40% 3,541.42
50% 4,249.81
60% 4,972.52
70% 5,739.09
80% 6,661.06
90% 7,928.97
100% 14,923.77
IBM Software Group | Rational software
3030
No initial risk – little opportunity to add value, stay relevant
NPV increases when you invest in Improving likelihood of delivery (reduce variance of costs)
Improving range of value at delivery by reuse of architecture, services … (increase upside variance of benefits)
0 6 Ship date
Conventional WisdomV
alu
e
NPV/Options Theory
0 Planned $Value0 Planned $Value
Medium Variance
Time
Varia
nce
in C
ost,
Sche
dule
…
Project Lifecycle Phases
Inception Elaboration Construction Transition/Maintenance
High Variance Low Variance
80% reduction96% reduction
IBM Software Group | Rational software
3131
So, what can you do?
Steer your project to success Measure, analyze and embrace estimation variance
Tailor governance to your risk/value context Determine if governance approach is paying off
Getting more or less value from agile, non-agile projects?
Invest with discipline Detect early when risks are not paying off – abandon projects that are not delivering
Have improved stakeholder conversations Risk and value with funders
Conveys more complete information for better collaboration Inputs/assumptions with program staff
Asking experts for likely and ranges elicits more complete information
– Forces needed discipline
– Wide ranges engenders discussion on where to focus efforts
IBM Software Group | Rational software
3232
Actual Path
Steer your project to success
Uncertainty in Stakeholder
Satisfaction Space
Initial Planned Content
Initial Planned Path
Uncertainty in delivery
Each iteration should reduce variance and increase NPV
IBM Software Group | Rational software
3333
Tailor the governance solution
Kind of value for organization processes
Kind of projects Full variance – foster discovery and learning
Medium Variance – architecture alignment, lean methods
Low variance – focus on automation, cost
Medium Variance
Little discovery
Var
ianc
e in
Cos
t, S
ched
ule… Inception Elaboration Construction Transition/Maintenance
HighVariance
LowVariance
New platform architecture
Service Delivery
Code Maintenance
New capability
with existing architecture
Some discovery
Much discovery
IBM Software Group | Rational software
3434
Tailoring governance is a process
Ass
ess
Imp
lemen
t
Manage
Plan
Governance Lifecycle Dep
loy
ControlMonitor
An
alyz
eE
valu
ate
Des
ign
Set Objective
Agree with stakeholders on value metrics
Choose internal measures (klocs, churn …) that contribute to business value and manage risk
Collect measures Compute
- Value- Productivity: Value per
programmer months
Reassign responsibilities, authorities to improve productivity
Tailor organization decision rights, responsibilities to risk areas
IBM Software Group | Rational software
3535
Adjust Governance Solution for each risk area
Medium Variance
Time
Var
ian
ce i
n C
ost
, S
ched
ule
…
Adjust Governance Solution for each risk area
Inception Elaboration Construction Transition/Maintenance
High Variance Low Variance
New Platform
New Capability on existing
platform
Maintenance and small change
requests
Focusing on artifacts, not process.
Constrain behavior creating uncertainty
Architectural alignment reduces
uncertainty due to diseconomy
of scaleAutomate, enforce
processes to reduce
uncertainty
IBM Software Group | Rational software
3636
Disciplined Investment: Apply quantitative portfolio analysis, not only scorecards
-200
0
200
400
600
800
1000
1200
-2 0 2 4 6 8 10 12
Liklihood of Success
Val
ue
Project 1
Project 2
Project 3
Project 4
Project 5
Project 7
Project 8
Project 9
Project 6
Project 10
Cost
IBM Software Group | Rational software
3838
Some final thoughts
Staying relevant requires smart risk taking
Value can only be reliably delivered if it measured and managed
Value delayed is value denied
In the end, these techniques do allow us to “Take risks and add value”
IBM Software Group | Rational software
39
For more information
Contact Clay Williams, manager of the Governance Science Research Group at IBM Watson Research
[email protected], 914-784-7457
IBM Software Group | Rational software
4040
© Copyright IBM Corporation 2008. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
Learn more at:
IBM Rational software
IBM Rational Software Delivery Platform
Process and portfolio management
Change and release management
Quality management
Architecture management
Rational trial downloads
Leading Innovation Website
IBM Rational TV
IBM Rational Business Partners