Business Value of
Agile Methods
Using Return on InvestmentDr. David F. Rico, PMP, CSM
2
Agenda
Introduction Sources of Business Value
Planning for Business Value Surveys of Business Value
Measures of Business ValueModels of Business ValueEstimation of Business Value
Comparison of Business Value Summary of Business Value
3
Author DoD contractor with 25+ years of IT experience B.S. Comp. Sci., M.S. Soft. Eng., D.M. Info. Tech. Large NASA & DoD programs (U.S., Japan, Europe)
* Published five textbooks and over 15 articles on various topics in return on investment, information technology, agile methods, etc.
4
Purpose Provide an overview of the business value of
Agile Methods using return on investment: Business value is an approach for estimating the
tangible and intangible worth of organizational assets Business value is an appraisal of intellectual assets
such as knowledge, experience, and skills Business value is a technique for determining the
complete worth of an investment to an enterprise Business value is a method of determining the health
and well-being of a firm in the long-run Business value includes employee, customer,
supplier, alliance, management, and societal value
5
What is Agility? A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness,
and ease of movement; nimbleness Agility is the ability to create and respond to change
in order to profit in a turbulent business environment Agility is reprioritizing for maneuverability because of
shifting requirements, technology, and knowledge Agility is a very fast response to changes in customer
requirements through intensive customer interaction Agility is the use of adaptability and evolutionary
delivery to promote rapid customer responsiveness Agility is a better way of developing products using
collaboration, teamwork, iterations, and flexibility
6
What are Agile Methods? ‘Adaptable’ software development methodologies ‘Human-centric’ method for creating business value ‘Alternative’ to large document-based methodologies
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
alsoknown as
CustomerCollaboration
Individuals &Interactions
WorkingSoftware
Respondingto Change
CustomerInteraction
High-Performance Teams
IterativeDevelopment
Adaptabilityor Flexibility
ContractNegotiation
Processes& Tools
ComprehensiveDocumentation
Followinga Plan
Agile Methods‘Values’
alsoknown as
alsoknown as
alsoknown as
valuedmore than
valuedmore than
valuedmore than
valuedmore than
Agile Methods‘Principles’
Traditional Methods‘Values’
7
Essence of Agile Methods High degree of customer & developer interaction Highly-skilled teams producing frequent iterations Right-sized, just-enough, and just-in-time process
Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.
Adaptability or Flexibility
High-Performance Teams
Iterative Development
Customer Interaction
8
Why use Agile Methods? Adaptability to changing market/customer needs Better cost efficiencies and fastest time-to-market Improved quality, satisfaction, and project success
Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org
9
Antecedents of Agile Methods Rooted in management evolution from early 1900s Evolved from software methods from 1950s/1960s Spinoffs of NPD/RAD approaches from the 1980s
Rico, D. F., Sayani, H. H., & Field, R. F. (2008). History of computers, electronic commerce, and agile methods. In M. V. Zelkowitz (Ed.), Advances in computers: Emerging technologies, Vol. 73. San Diego, CA: Elsevier.
1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004
J oint Application Development
Rapid Systems Development
New DevelopmentRhythm
Rapid Application Development
Partic ipatoryDesign
Synch-n-Stabilize
Internet Time
J udo Strategy
Rational Unified Process
10
Agenda
Introduction
Sources of Business ValuePlanning for Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business Value
Comparison of Business Value Summary of Business Value
11
Types of Agile Methods Crystal Methods and Scrum 1st Agile Methods Extreme Programming swept the globe by 2002 Scrum/Extreme Programming hybrids are popular
Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.
Feature Lists (Customer Needs), Domain Model (Object Orientation), Inspection (Peer Review)
Use Cases , Domain Models, Frequent Delivery, Reflection Workshops, Risk Management
DSDM
Easel
IBM
Beck
De Luca
Millington
Sutherland
Cockburn
Author
Release Planning, Onsite Customers, Iterations,Pair Programming, Test-Driven Development
User Involvement, Time Boxes and Prototypes (Iterations), Testing and Quality Assurance
Backlogs (Feature Lists), Daily Scrums, Sprints (Iterations), Retrospectives (Post Mortems)
Major Features
Chrysler
Nebulon
Firm
1998
1997
1993
1993
1991
Year
Feature-Driven Development
CrystalMethods
Extreme Programming
Dynamic Systems
Development
Scrum
Method
12
Crystal Methods Created by Alistair Cockburn in 1991 Has 14 practices, 10 roles, and 25 products Scalable family of techniques for critical systems
Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley.
13
Scrum Created by Jeff Sutherland at Easel in 1993 Has 5 practices, 3 roles, 5 products, rules, etc. Uses EVM to burn down backlog in 30-day iterations
Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.
14
Dynamic Systems Develop. Created by group of British firms in 1993 15 practices, 12 roles, and 23 work products Non-proprietary RAD approach from early 1990s
Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.
15
Feature Driven Development Created by Jeff De Luca at Nebulon in 1997 Has 8 practices, 14 roles, and 16 work products Uses object-oriented design and code inspections
Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.
Develop anOverall Model
Build aFeatures List
Plan byFeature
Design byFeature
Build byFeature
Iteration
16
Extreme Programming Created by Kent Beck at Chrysler in 1998 Has 28 practices, 7 roles, and 7 work products Popularized pair programming and test-driven dev.
Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.
UserStories
ArchitecturalSpike
ReleasePlanning
IterationAcceptance
TestsSmall
Releases
Spike
TestScenarios
SystemMetaphor
CustomerApproval
LatestVersion
ReleasePlan
NextIteration
BugsNew
StoriesRequirements
UncertainEstimates
ConfidentEstimates
XPExtreme Programming
17
Extreme Programming (cont’d) RELEASE PLANNING — Best Practice
Created by Kent Beck at Chrysler in 1998 Lightweight project management framework Used for managing both XP and Scrum projects
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
Release Plan Iteration Plan
UserWrite
StoriesUser
Estimate
StoriesUser
Split
StoriesUser
Order
StoriesUser
Analyze
StoriesDev.
Create
TasksDev.
Accept
TasksDev.
Estimate
Tasks
18
PAIR PROGRAMMING — Best Practice Term coined by Jim Coplien in 1995 Consists of two side-by-side programmers Highly-effective group problem-solving technique
Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.
MOVE PEOPLE AROUND
REFACTOR MERCILESSLY
CONTINUOUS INTEGRATION
CREATE AUNIT TEST
We NeedHelp
ChangePair
SimpleCode
ComplexCode
New UnitTests
NewFunction-
ality
FailedUnitTest
PassedUnitTest
Extreme Programming (cont’d)
19
REFACTORING — Best Practice Term coined by William Opdyke in 1990 Process of frequently rewriting source code Improves readability, maintainability, and quality
Extreme Programming (cont’d)
Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.
20
TEST-DRIVEN DEV. — Best Practice Term coined by Kent Beck in 2003 Consists of writing all tests before coding Ensures all source code is verified and validated
Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.
Extreme Programming (cont’d)
21
Extreme Programming (cont’d) CONT. INTEGRATION — Best Practice
Term coined by Martin Fowler in 1998 Process of automated build/regression testing Evaluates impact of all changes against entire system
Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley
BuildIntegration
Server
VersionControlServer
BuildScripts
UsesWatches
BuildStatus
ProvidesDeveloper A
Developer B
Developer C
CommitsChanges
CommitsChanges
CommitsChanges
Compile Source Code
Run Unit Tests
Run Coverage Tests
Static Code Analysis
Build Database
Generate Help Files
Archive and Deploy
22
Agenda
IntroductionSources of Business Value
Planning for Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business Value
Comparison of Business Value Summary of Business Value
23
Release Planning Deliverables Used in both Extreme Programming and Scrum Lightweight framework of Agile planning products Ranges from release plans down through unit tests
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
No. Deliverable Description
1. Release Plan Fluid informal roadmap (or program plan) for planning software releases (usually containing one or more iterations).
2. Iteration Plan Informal project plan that divides an iteration into user stories and development tasks (usually spanning two to three weeks)
3. User Story A software requirement that has value to end-users or customers (usually a simple sentence written on an index card)
4. Metaphor A simple narrative about how the whole system works (usually written as a sentence or paragraph with major objects)
5. Development Task A development activity necessary to satisfy a user story (usually a numbered list of software development activities)
6. Acceptance Test An end-user test to determine if an iteration satisfies its acceptance criteria (usually written and executed by customers)
7. Unit Test A development test to determine if software components are working properly (usually written and executed by developers)
24
Release Plan Fluid, informal roadmap for planning releases Includes dates for releases, iterations, and stories Must prioritize, split, estimate, and order user stories
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
Release Plan
<Release Plan>: <Release 1, 2, n>
<Release>: <Iteration 1, 2, n>
<Iteration>: <Story 1, 2, n>
Release PlanRelease Iteration
1
2
3
4
n
1
1
2
2
n
Story
01 thru 06
07 thru 12
13 thru 18
19 thru 24
25 thru nn
25
Iteration Plan Plan that divides iterations into development tasks Each iteration is one to three weeks in duration Iteration plans updated using daily standups
Iteration Plan
<Iteration Plan>: <Story 1, 2, n>
<Story>: <Task 1, 2, n>
<Task>: <Developer 1, 2, n>
<Status>: <Days Complete>
Iteration PlanStory
1
1
2
2
n
Task
1
2
3
4
n
Developer
Bob
Sue
Mary
John
n
Status
1/3
2/3
3/3
3/3
n/n
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
26
User Story A function or feature of value to a customer An estimable and testable software requirement Six user stories should be implemented per iteration
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
<Title of User Story>
As a <Type of User> I can <Goal of User> so that <Objective of User>
Make a Reservation
As a customer, I can make a reservation so that I can
perform personal travel
27
System Metaphor Simple story about how the whole system works Overarching 10,000 foot view of system architecture Pushes the system into a sense of coherent cohesion
System Metaphor
<Metaphor>: <Object 1>, <Object 2>, <Object n>
System Metaphor
Shopping Cart: Item, Description, Barcode, Price, Quantity, Subtotal, Tax, Discount, Total, etc.
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
28
Development Tasks Detailed steps for implementing user stories User stories are decomposed into technical tasks Brainstormed by developers to last two to three days
<Title of Development Task>
<Action of Developer> a <Software Unit> using
<Technology>
Splash Screen
Design a splash screen using PhotoShop
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
29
Acceptance Tests Black-box, functional tests to be performed Specified by customers during iteration planning Run when user stories and unit tests are completed
<Title of User Story>
Verify <Type of User> can <Satisfy their Goals and
Objectives>
Make a Reservation· Verify customers can establish a
reservation
· Verify customers can change a reservation
· Verify customers can cancel a reservation
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
30
Unit Tests A test written from the developer’s perspective Each task is implemented by two programmers Unit tests are developed prior to implementation
<Title of Development Task>
Verify <Type of User> can <Satisfy Task> when <Condition Occurs>
Make a Splash Screen· Verify customers can see splash
screen when they visit website
· Verify customers can see company logo when splash screen executes
· Verify customers can skip splash screen when they want to enter site
Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.
31
Agenda
IntroductionSources of Business ValuePlanning for Business Value
Surveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business Value
Comparison of Business Value Summary of Business Value
32
Surveys of Agile Methods Numerous surveys of Agile Methods since 2003 AmbySoft and Version One collect annual data Generally include both hard and soft benefits
Rico, D. F. (2008). What is the return-on-investment of agile methods? Retrieved February 3, 2009, from http://davidfrico.com/rico08a.pdf
Johnson 131
Barnett 400
Begel, et al. 492
Rico, et al. 250
Ambler 642
Wolf, et al. 207
Hanscom 3,061
Author Size
93%
45%
14%
81%
82%
78%
74%
Productivity
88%
43%
32%
80%
72%
74%
68%
Quality
49%
23%
16%
75%
72%
72%
38%
Cost
Shine
Agile Journal
Microsoft
UMUC
AmbySoft
IT Agile
Version One
Organization
2003
2006
2007
2007
2008
2008
2008
Year
67% 65% 49%Average
33
Shine Technologies Survey of 131 international respondents Extreme Programming (58%) and Scrum (8%) 85% of respondents were experts in Agile Methods
Johnson, M. (2003). Agile methodologies: Survey results. Victoria, Australia: Shine Technologies.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Produc tivity
Quality
Satisfac tion
Cost
Improvement
34
Agile Journal Survey of 400 international respondents Extreme programming (28%) and Scrum (20%) 80% using Agile Methods to deliver maximum value
Barnett, L. (2006). And the agile survey says. Agile Journal, 1(1).
0% 10% 20% 30% 40% 50%
Time to Market
Quality
Alignment
Cost
Improvement
35
Microsoft Survey of 492 Microsoft respondents Scrum (65%) and Extreme Programming (5%) 65% using Agile Methods in virtual distributed teams
Begel, A., & Nagappan, N. (2007). Usage and perceptions of agile software development in an industrial context: An exploratory study. Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, Madrid, Spain, 255-264.
0% 10% 20% 30% 40% 50% 60% 70%
Communication
Time to Market
F lexibility
Quality
Satisfac tion
Produc tivity
Improvement
36
UMUC Survey of 250 international respondents 70% of respondents using Agile Methods 83% of were from small-to-medium sized firms
Rico, D. F., Sayani, H. H., Stewart, J. J., & Field, R. F. (2007). A model for measuring agile methods and website quality. TickIT International, 9(3), 3-15.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Satisfac tion
Produc tivity
Cyc le Time
Quality
Cost
Improvement
37
AmbySoft Survey of 642 international respondents 69% of firms had adopted an Agile Method 62% were from firms with less than 1,000 people
Ambler, S. W. (2008). Agile adoption survey. Retrieved October 17, 2008, from http://www.ambysoft.com/downloads/surveys/AgileAdoption2008.ppt
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Produc tivity
Satisfac tion
Projec t Success
Quality
Cost
Virtual Success
Improvement
38
IT Agile Survey of 207 respondents in Germany Scrum (21%) and Extreme Programming (14%) 97% of respondents are satisfied with Agile Methods
Wolf, H., & Roock, A. (2008). Agile becomes mainstream: Results of an Online Survey. Object Spektrum, 15(3), 10-13.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Flexibility
J ob Satisfac tion
Learning on the J ob
Produc tivity
Projec t Status
Quality
Improvement
39
Version One Survey of 3,061 respondents from 80 countries Scrum (49%), Scrum/XP (22%), and XP (8%) 68% from small firms and 57% distributed
Version One. (2008). The state of agile development: Third Annual Survey. Alpharetta, GA: Author.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Cyc le Time
Projec t Visibility
Produc tivity
Quality
Maintainability
Cost
Improvement
40
Agenda
IntroductionSources of Business ValuePlanning for Business ValueSurveys of Business Value
Measures of Business ValueModels of Business ValueEstimation of Business Value
Comparison of Business Value Summary of Business Value
41
Measures of Business Value A major principle of Agile Methods is creating value ROI is the measure of value within Agile Methods There are seven closely related ROI measures
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Metric Definition Formula Costs
Sum of Costs Total amount of money spent
n
iiCost
1
Benefits Sum of Benefits
Total amount of money gained
n
iiBenefit
1
B/CR Benefit to Cost Ratio
Ratio of benefits to costs
Costs
Benefits
ROI Return on Investment
Ratio of adjusted benefits to costs
%100
Costs
CostsBenefits
NPV Net Present Value
Discounted cash flows
Years
iYears
i CostsRateDiscount
Benefits
10)1(
BEP Breakeven Point
Point when benefits exceed costs
1CostsNewCostsOld
CostsNew
ROA Real Options Analysis
Value gained from strategic delay
YearsRateeCostsdNBenefitsdN 21
d1 = [ln(Benefits Costs) + (Rate + 0.5 Risk2) Years] Risk Years, d2 = d1 Risk Years
42
Costs Total amount of money spent on Agile Methods Includes training, coaching, automated tools, etc. Minimally, includes the dev. effort of Agile Methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
n
iiCost
1
43
Benefits Total amount of money gained from Agile Methods Includes economic benefit from using new system Minimally, includes maintenance rework savings
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
n
iiBenefit
1
44
Benefit to Cost Ratio Ratio of total benefits to total costs of Agile Methods Includes development, maintenance, and business Minimally, benefits should be larger than all costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Costs
Benefits
45
Return on Investment Ratio of adjusted benefits to costs of Agile Methods Benefits are adjusted downward using total costs Minimally, benefits should be larger than costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
%100
Costs
CostsBenefits
46
Net Present Value Discounted benefits of using Agile Methods Future benefits are discounted due to inflation Minimally, future benefits should exceed all costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Years
iYears
i CostsRateDiscount
Benefits
10)1(
47
Breakeven Point Point when benefits exceed costs of Agile Methods Point where slope of benefits and costs intersect Minimally, old costs should exceed new costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1CostsNewCostsOld
CostsNew
48
Real Options Analysis Iterative benefits gained from using Agile Methods Future benefits are increased because of risks Minimally, benefits should exceed costs
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
YearsRateeCostsdNBenefitsdN 21
d1 = [ln(Benefits Costs) + (Rate + 0.5 Risk2) Years] Risk Years d2 = d1 Risk Years
49
Agenda
IntroductionSources of Business ValuePlanning for Business ValueSurveys of Business Value
Models of Business ValueMeasures of Business ValueEstimation of Business Value
Comparison of Business Value Summary of Business Value
50
Software Lifecycle Costs 1:10:100 is a classical ratio of dev. to maint. hours Defects have negative multiplicative effect on cost A conservative and contemporary ratio is 1:6:30
Boehm, B. W. (1981). Software engineering economics. Englewood Cliffs, NJ: Prentice-Hall.Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.
200
100
50
20
10
5
2
1
Phase in Which Error was Detected and Corrected
Re
lati
ve
Co
st
t o F
ix E
r ro
r
Requirements Design CodeDevelopment
TestAcceptance
TestOperation
51
Software Cost Models Cost estimation models still in use today Used to estimate effort of Traditional Methods Adjusted average of 5,088 used for ROI estimation
Benediktsson, O., & Dalcher, D. (2005). Estimating size in incremental software development projects. Journal of Engineering Manufacture, 152(6), 253-259.
COCOMO-O
COCOMO-S
COCOMO-E
COCOMO-II
Walston-Felix
Bailey-Basili
Doty
Source
Average
Months 2.4 KLOC 1.05
Months 3.0 KLOC 1.12
Months 3.6 KLOC 1.20
Months 2.9 KLOC 1.10
Months 5.2 KLOC 0.91
Months 5.5 + 0.73 KLOC 1.15
Months 5.288 KLOC 1.047
Model10,000
10,000
10,000
10,000
10,000
10,000
10,000
LOC26.93
39.55
57.06
36.51
42.27
15.81
58.92
39.58
Months4,667.60
6,854.94
9,889.73
6,328.20
7,326.31
2,740.66
10,213.48
6,860.13
Hours2.24
3.30
4.75
3.04
3.52
1.32
4.91
3.30
Years
* (6,854.94 + 7,326.31 + 2,740.66 + 10,213.48) 4 0.75 5,087.89
52
Total Lifecycle Costs 0.51 hours/line of code for Traditional Methods 10% defect inject rate (1,000 defects/10 KLOC) 67% of defects in test (33% in maintenance)
Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.In, H. P., et al. (2006). A quality-based cost estimation model for the product line life cycle. Communications of the ACM, 49(12), 85-88.McCann, B. (2007). The relative cost of interchanging, adding, or dropping quality practices. Crosstalk, 20(6), 25-28.
Step1.
2.
3.
4.
5.
6.
7.
Total Lifecycle Cost Model0.51 Size + 30 IR Size – 30 IH – 5.47 TH + IH + TH
0.51 Size + 30 10% Size – 30 IH – 5.47 TH + IH + TH
0.51 Size + 3 Size – 30 IH – 5.47 TH + IH + TH
0.51 Size + 3 Size – 29 IH – 4.47 TH
3.51 Size – 29 IH – 4.47 TH
3.51 10,000 – 0 – 4.47 3,651.48
18,751.48* hours or $1,875,148* 5,087.89 Development Hours + 3,651.48 Test Hours + 10,012.11 Maintenance Hours
53
Agile Productivity Studies Productivity data found in 26 Agile Methods studies Studies conducted from 2002 to the present time Average productivity 21.2374 LOC per hour
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
No. Author(s)1. Abrahamsson 2003 XP Case Study 19.2550
2. Abrahamsson & Koskela 2004 XP Case Study 16.9000
3. Back, Hirkman, & Milovanov 2004 XP Experiment 8.0000
24. Cohn 2008 Scrum Case Study 5.9050
25. Jones 2008 Scrum Case Study 5.7400
26. Sutherland 2006 Scrum Case Study 4.6858
Average 21.2374
Year Method Type LOC/Hour
54
Agile Productivity Models Based on 13 studies of Extreme Programming (XP) Also based on 7 studies of pair programming (PP) “Pair programming” had highest productivity
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1. XP 13
2. TDD 2
3. PP 8
4. Scrum 3
5. Agile 26
No. Method Pts.
3.5000
12.3800
15.4667
4.6858
3.5000
Low
16.1575
29.2800
33.4044
5.4436
21.2374
Median
43.0000
46.1800
86.4502
5.9050
86.4502
High
LOC 16.1575
LOC 29.2800
LOC 33.4044
LOC 05.4436
LOC 21.2374
Productivity
55
Agile Quality Studies Defect density found in 21 studies of Agile Methods Studies conducted from 2002 to the present time Average quality 1.7972 defects per KLOC
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
No. Author(s)1. Abrahamsson 2003 XP Case Study 2.1450
2. Abrahamsson & Koskela 2004 XP Case Study 1.4300
3. Back, Hirkman, & Milovanov 2004 XP Experiment 0.7000
19. Cohn 2008 Scrum Case Study 2.9000
20. Jones 2008 Scrum Case Study 8.5000
21. Schatz & Abdelshafi 2005 Scrum Case Study 0.4350
Average 1.7972
Year Method Type Def/KLOC
56
Quality Models Based on 10 studies of Extreme Programming (XP) Also based on 6 studies of pair programming (PP) “Extreme Programming” had the highest quality
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1. XP 10
2. TDD 2
3. PP 6
4. Scrum 3
5. Agile 21
No. Method Pts.
0.0032
0.6100
0.3250
0.4350
0.0032
Low
0.7466
2.1550
2.3550
3.9450
1.7972
Median
2.1450
3.7000
5.8500
8.5000
8.5000
High
0.7466 KLOC 30
2.1550 KLOC 30
2.3550 KLOC 30
3.9450 KLOC 30
1.7972 K LOC 30
Quality
57
Agenda
IntroductionSources of Business ValuePlanning for Business ValueSurveys of Business Value
Measures of Business ValueModels of Business Value
Estimation of Business ValueComparison of Business Value
Summary of Business Value
58
Agile Lifecycle Costs Costs based on productivity and quality models Development costs based on LOC productivity rate Maintenance costs based on defects KLOC MH
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1. XP
2. TDD
3. PP
4. Scrum
5. Agile
No. Method Agile Lifecycle Cost Models
(10,000 05.3858 + 0.7466 10 30) 100
(10,000 09.7600 + 2.1550 10 30) 100
(10,000 11.1350 + 2.3550 10 30) 100
(10,000 05.4436 + 3.9450 10 30) 100
(10,000 07.9311 + 1.7972 10 30) 100
$167,109
$160,459
$302,052
$180,002
Costs
$208,069
* XP, TDD, and PP reduced by two-thirds to moderate effects of laboratory conditions
59
Agile Lifecycle Benefits Benefits based on total traditional less agile costs Traditional costs based LOC dev. maint. effort Traditional costs credited testing effort applied
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
1. XP
2. TDD
3. PP
4. Scrum
5. Agile
No. Method
$1,708,039
$1,714,690
$1,573,096
$1,695,146
Benefits
$1,667,079
Agile Lifecycle Benefit Models
(10,000 3.51 – 3,651.48 4.47) 100 – $208,069
(10,000 3.51 – 3,651.48 4.47) 100 – $167,109
(10,000 3.51 – 3,651.48 4.47) 100 – $160,459
(10,000 3.51 – 3,651.48 4.47) 100 – $302,052
(10,000 3.51 – 3,651.48 4.47) 100 – $180,002
60
Extreme Programming Costs based on avg. productivity and quality Productivity moderated from 16.1575 to 5.3858 Costs were $208,069, benefits were $1,667,079
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Formula
(10,000 5.3858 0.7466 10 30) 100
$1,667,079 $208,069
($1,667,079 – $208,069) $208,069 100%
($1,667,079 5) 1.055) – $208,069
$208,069 ($1,875,148 $208,069 – 1)
NORMSDIST (2.24) $1,667,079 –NORMSDIST (0.59) $208,069 EXP (–5% 5)
(10,000 3.51 – 3,651.48 4.47) 100 – $208,069
Value
$208,069
8:1
701%
$1,235,446
$10,064
$1,529,066
$1,667,079
Metric
Costs
B/CR
ROI
NPV
BEP
ROA
Benefits
5
1i(
61
Test Driven Development Costs based on avg. productivity and quality Productivity moderated from 29.2800 to 9.7600 Costs were $167,109, benefits were $1,708,039
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Formula
(10,000 9.7600 + 2.1550 10 30) 100
$1,708,039 $167,109
($1,708,039 – $167,109) $167,109 100%
($1,708,039 5) 1.055) – $167,109
$167,109 ($1,875,148 $167,109 – 1)
NORMSDIST (2.76) $1,708,039 –NORMSDIST (1.58) $167,109 EXP (–5% 5)
(10,000 3.51 – 3,651.48 4.47) 100 – $167,109
Value
$167,109
10:1
922%
$1,311,874
$6,430
$1,580,490
$1,708,039
Metric
Costs
B/CR
ROI
NPV
BEP
ROA
Benefits
5
1i(
62
Pair Programming Costs based on avg. productivity and quality Productivity moderated from 33.4044 to 11.135 Costs were $160,459, benefits were $1,714,690
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Formula
(10,000 11.1350 + 2.3550 10 30) 100
$1,714,690 $160,459
($1,714,690 – $160,459) $160,459 100%
($1,714,690 5) 1.055) – $160,459
$160,459 ($1,875,148 $160,459 – 1)
NORMSDIST (2.95) $1,714,690 –NORMSDIST (1.86) $160,459 EXP (–5% 5)
(10,000 3.51 – 3,651.48 4.47) 100 – $160,459
Value
$160,459
11:1
969%
$1,324,283
$5,919
$1,590,927
$1,714,690
Metric
Costs
B/CR
ROI
NPV
BEP
ROA
Benefits
5
1i(
63
Scrum Costs based on avg. productivity and quality Productivity data remained the same at 5.4436 Costs were $302,052, benefits were $1,573,096
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Formula
(10,000 5.4436 + 3.9450 10 30) 100
$1,573,096 $302,052
($1,573,096 – $302,052) $302,052 100%
($1,573,096 5) 1.055) – $302,052
$302,052 ($1,875,148 $302,052 – 1)
NORMSDIST (1.97) $1,573,096 –NORMSDIST (-0.27) $302,052 EXP (–5% 5)
(10,000 3.51 – 3,651.48 4.47) 100 – $302,052
Value
$302,052
5:1
421%
$1,060,084
$21,682
$1,441,741
$1,573,096
Metric
Costs
B/CR
ROI
NPV
BEP
ROA
Benefits
5
1i(
64
Agile Methods Costs based on avg. productivity and quality Productivity data resulted in average of 7.9311 Costs were $180,002, benefits were $1,695,146
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
Formula
(10,000 7.9311 + 1.7972 10 30) 100
$1,695,146 $180,002
($1,695,146 – $180,002) $180,002 100%
($1,695,146 5) 1.055) – $180,002
$180,002 ($1,875,148 $180,002 – 1)
NORMSDIST (2.52) $1,695,146 –NORMSDIST (1.16) $180,002 EXP (–5% 5)
(10,000 3.51 – 3,651.48 4.47) 100 – $180,002
Value
$180,002
9:1
842%
$1,287,817
$7,483
$1,562,126
$1,695,146
Metric
Costs
B/CR
ROI
NPV
BEP
ROA
Benefits
5
1i(
65
Agenda
IntroductionSources of Business ValuePlanning for Business ValueSurveys of Business Value
Measures of Business ValueModels of Business ValueEstimation of Business Value
Comparison of Business ValueSummary of Business Value
66
Data for Agile Methods Agile Methods were ranked based on ROI Agile Methods with high quality had lower ROI Agile Methods with high productivity had high ROI
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
Method Costs Benefits B/CR ROI NPV BEP ROA
PP $160,459 $1,714,690 11:1 969% $1,324,283 $5,919 $1,590,927
TDD $167,109 $1,708,039 10:1 922% $1,311,874 $6,430 $1,580,490
Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,562,126
XP $208,069 $1,667,079 8:1 701% $1,235,446 $10,064 $1,529,066
Scrum $302,052 $1,573,096 5:1 421% $1,060,084 $21,682 $1,441,741
67
ROI of Agile Methods Agile Methods were ordered based on ROI Agile Methods had a high ROI value of 969% Agile Methods yielded an average ROI of 842%
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
842%
701%
421%
0%
200%
400%
600%
800%
1,000%
PP TDD Agile XP Scrum
922%969%
68
Data for Traditional Methods Traditional Methods were ranked based on ROI Methods with good cost and quality had higher ROI Agile Methods had better ROI than heaviest methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
Method Costs Benefits B/CR ROI NPV BEP ROA
PSP sm $105,600 $1,755,148 17:1 1,562% $1,414,174 $945 $1,672,907
TSP sm $148,400 $1,706,648 12:1 1,050% $1,329,379 $5,760 $1,591,127
Inspections $82,073 $897,499 11:1 994% $695,067 $51,677 $833,681
Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,556,997
SW-CMM® $311,433 $1,153,099 4:1 270% $687,030 $153,182 $998,013
ISO 9001 $173,000 $566,844 3:1 228% $317,828 $1,196,206 $486,750
CMMI® $1,108,233 $1,153,099 1:1 4% -$109,770 $545,099 $891,412
69
ROI of Traditional Methods Traditional Methods were ordered using ROI Traditional Methods had high ROI value of 1,562% Agile Methods had better ROI than heaviest methods
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
1,562%
1,050%994%
842%
270% 228%
4%0%
300%
600%
900%
1,200%
1,500%
1,800%
PSPsm TSPsm Inspections Agile SW-CMM® ISO 9001 CMMI®
70
Data for All Methods Software methods were ranked based on ROI Methods with good cost and quality had best ROI Best Agile and Traditional Methods had similar ROI
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
Traditional
Traditional
Agile
Traditional
Agile
Agile
Traditional
Agile
Traditional
Traditional
Type Costs Benefits B/CR ROI NPV BEP ROAMethod
Agile
PSP sm $105,600 $1,755,148 17:1 1,562% $1,414,174 $945 $1,672,907
TSP sm $148,400 $1,706,648 12:1 1,050% $1,329,379 $5,760 $1,591,127
Inspections $82,073 $897,499 11:1 994% $695,067 $51,677 $833,681
PP $160,459 $1,714,690 11:1 969% $1,324,283 $5,919 $1,590,034
TDD $167,109 $1,708,039 10:1 922% $1,311,874 $6,430 $1,578,575
Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,556,997
XP $208,069 $1,667,079 8:1 701% $1,235,446 $10,064 $1,513,332
Scrum $302,052 $1,573,096 5:1 421% $1,060,084 $21,682 $1,389,810
SW-CMM® $311,433 $1,153,099 4:1 270% $687,030 $153,182 $998,013
ISO 9001 $173,000 $566,844 3:1 228% $317,828 $1,196,206 $486,750
CMMI® $1,108,233 $1,153,099 1:1 4% -$109,770 $545,099 $891,412
71
ROI of All Methods Software methods were ordered by ROI Agile Methods had a high ROI value of 969% Traditional Methods had high ROI value of 1,562%
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
1,562%
1,050%994% 969%
922%842%
701%
421%
270% 228%
4%0%
300%
600%
900%
1,200%
1,500%
1,800%
PSPsm TSPsm Inspections PP TDD Agile XP Scrum CMM® ISO 9001 CMMI®
72
Unadjusted ROI of All Methods Are data based on unrealistic laboratory conditions? Are productivity data from lab studies optimistic? Are total lifecycle costs closer to 1:10:100?
4,133%
3,272%3,103%
2,826%
1,788%1,607% 1,499%
871%
580%
229% 173%
0%
500%
1,000%
1,500%
2,000%
2,500%
3,000%
3,500%
4,000%
4,500%
PSPsm Inspect XP TSPsm Agile TDD PP SW-CMM® Scrum ISO 9001 CMMI®
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls
73
Agenda
IntroductionSources of Business ValuePlanning for Business ValueSurveys of Business Value
Measures of Business ValueModels of Business ValueEstimation of Business ValueComparison of Business Value
Summary of Business Value
74
Benefit Summary Agile (138 pt.) and Traditional Methods (99 pt.) Agile Methods fare better in all benefits categories Agile Methods 459% better than Traditional Methods
Agile Methods Traditional MethodsLow Median HighCategory
ROI 240% 2,633% 8,852%
Satisfaction 70% 70% 70%
Quality 10% 70% 1,000%
Productivity 14% 122% 712%
Schedule 11% 71% 700%
Cost 10% 26% 70%
Low Median HighCategory
ROI 200% 470% 2,770%
Satisfaction -4% 14% 55%
Quality 7% 50% 132%
Productivity 9% 62% 255%
Schedule 2% 37% 90%
Cost 3% 20% 87%
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.
75
Cost of Quality Apply traditional reliability and quality theory Defects are inexpensive to remove early in cycle Late bug removal has negative, multiplicative effect
Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI) strategies. Rome, NY: DACS.
Inspection Cost (57X PSP)
Softw
are
Defe
cts
PSP
Test Cost (138X PSP)
Ad Hoc (326X)
Analysis Design CodeUnitTest
System Test
ComponentTest
CustomerUse
PSP Cost (326X lower than Ad Hoc)
76
Real Options NPV models losses of Traditional Methods Real options model profits from Agile Methods Agile Methods incur less initial risk and higher ROI
Fichman, R. G., Keil, M., & Tiwana, A. (2005). Beyond valuation: Options thinking in IT project management. California Management Review, 47(2), 74-96.
0.00
0.05
0.10
0.15
0.20
0.25
- $9M - $7M - $5M - $3M - $1M $1M $3M $5M $7M $9M
Prob
abili
ty
NPV of Traditional Methods = $1.1 Million
0.00
0.05
0.10
0.15
0.20
0.25
- $9M - $7M - $5M - $3M - $1M $1M $3M $5M $7M $9M
Prob
abili
ty
Real Options of Agile Methods = $1.4 Million
77
Agile vs. Traditional Metrics Agile Methods are a fundamentally new paradigm Agile Methods are “not” lighter Traditional Methods They should not be viewed through a Traditional lens
Rico, D. F. (2009). Metrics for agile methods. Retrieved February 7, 2009, from http://davidfrico.com/agile-metrics.pdf
Customer Interaction
Iterative Development
High-Performance Teams
Adaptability or Flexibility
valuedmore than
valuedmore than
valuedmore than
valuedmore than
Agile
Met
rics
Traditional Metrics
Contracts
Documentation
Processes
Project P lans
· Interaction frequency· Communication quality· Strength of Relationship
· Customer trust· Customer loyalty· Customer satisfaction
· Team skills· Team motivation· Team cooperation
· Team trust· Team cohesion· Team communications
· Iteration size· Iteration number· Iteration frequency
· Productivity· Defect density· Software reliability
· Organizational flexibility· Management flexibility· Developer flexibility
· Process flexibility· Design flexibility· Technology flexibility
· Fixed-fee contracts· Engineering changes· Contract change orders
· Standards compliance· Process Maturity Level· Manufacturing practices
· Document standard· Lifecycle/phase reviews· Quality assurance audits
· Cost Compliance· Scope Compliance· Schedule Compliance
78
New Book Guide to Agile Methods for business leaders Communicates business value of Agile Methods Rosetta stone to Agile Methods for Traditional folks
* Rosetta stone to the business value and culture of Agile Methods for executives, managers, and thought leaders in the field of software methods.
THE BUSINESS VALUE OF AGILE METHODS
Maximizing ROI with Right-Sized, Just-Enough, and Just-in-Time Processes and Documentation
DR. DAVID F. RICO, DR. HASAN H. SAYANI
AND DR. SAYA SONE
Forward by Dr. Jeffrey V. Sutherland
Table of Contents
1. Introduction2. Values of Agile Methods3. History of Agile Methods4. Antecedents of Agile Methods5. Types of Agile Methods6. Practices of Agile Methods7. Agile Project Management8. Agile Software Engineering9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Costs of Agile Methods14. Benefits of Agile Methods15. ROI of Agile Methods16. NPV of Agile Methods17. Real Options of Agile Methods18. Conclusion
79
References Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI)
strategies. Rome, NY: DACS.
Rico, D. F. (2002). How to Estimate ROI for Inspections, PSP, TSP, SW-CMM, ISO 9001, and CMMI. Software Tech News, 5(4), 23-31.
Rico, D. F. (2002). The Return on investment in quality. TickIT International, 4(4), 13-18.
Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.
Rico, D. F. (2005). Practical metrics and models for return on investment. TickIT International, 7(2), 10-16.
Rico, D. F. (2006). A framework for measuring the ROI of enterprise architecture. International Journal of End User Computing, 18(2), 1-12.
Rico, D. F. (2007). Optimizing the ROI of enterprise architecture using real options. In S. Clarke (Ed.), End user computing challenges and technologies: Emerging tools and applications. Hershey, PA: Information Science Reference.
Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? An analysis of extreme programming, test-driven development, pair programming, and scrum (using real options). TickIT International, 10(4), 9-18.
Solingen, R. A., & Rico, D. F. (2006). Calculating software process improvement’s return on investment. In M. V. Zelkowitz (Ed.), Advances in computers: Quality software development, Vol. 66 (pp. 1-41). San Diego, CA: Elsevier.
80
Contact Information Website: http://davidfrico.com Biography: http://www.linkedin.com/in/davidfrico Capabilities: http://davidfrico.com/rico-capability.pdf
Top Related