Gauging the ROI of Your Agile Organization Lee Cunningham Director, Enterprise Agile Enablement VersionOne
June 2014
Understanding Improving
© 2014 VersionOne 2
About me…
• Lee Cunningham • Lean-agile practitioner, coach,
advisor • Some traditional hats along the way:
Software Engineer, Development Manager, QA Manager, Release Manager, Project/Program Manager
• USAF • SPC, PMP, PMI-ACP, CSM • [email protected]
© 2014 VersionOne 3
14% ROI standards “well defined”
ROI? Compared to what?
33% ROI standards “not defined” or “poorly-defined”
Standish Group, DARTS 2007
© 2014 VersionOne 4
Is ROI even being measured?
64% “rarely” or “never” perform post-project value evaluation
Standish Group, DARTS 01-2013
© 2014 VersionOne 5
…yet
72% report that their project evaluations are “somewhat accurate”
Standish Group, DARTS 01-2013
© 2014 VersionOne 6
So, what they have told us is…
I HARDLY EVER EVALUATE MY PROJECTS --
BUT WHEN I DO, MY EVALUATIONS ARE SOMEWHAT
ACCURATE.
STAY CLUELESS, MY FRIENDS…
© 2014 VersionOne 7
A word about statistics
http://tylervigen.com/
© 2014 VersionOne 8
The Product Owner?
© 2014 VersionOne 9
Takeaways
• Agile is about business -- not software.
• We need to be able to talk about agile practices in terms that senior organizational leadership would appreciate.
© 2014 VersionOne 10
“Agile Organization”?
For our discussion, we’re defining “Agile Organization” as a system formed around an agile value stream, whether a portfolio, program, or a team.
© 2014 VersionOne 11
Cycle Time & Lead Time
© 2014 VersionOne 12
ROI =
Revenue – [Operating Expense + Total Variable Expenses] Investment
How much we are paid for it How much we are paid for it – How much it cost to make it How much we are paid for it – How much it cost to make it What it cost for the raw materials used to make it
© 2014 VersionOne 13
Why projecting ROI is hard to do
Revenue – [Operating Expense + Total Variable Expenses] Investment
© 2014 VersionOne 14
Challenges to calculating Revenue
• New Revenue: new revenue from new customers
• Incremental Revenue: new revenue from existing customers
• Retained Revenue: revenue from existing customers we would have lost had we not undertaken this
• Operational Efficiencies: reduced costs resulting from having undertaken this
© 2014 VersionOne 15
Basic “Concept to Cash” Value Stream
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
© 2014 VersionOne 16
The essence of throughput economics
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
Until we’ve gotten paid, all we’ve done is spend money.
Frequency and Value
© 2014 VersionOne 17
And we can’t forget….
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
rework -$
Defects -$
© 2014 VersionOne 18
Mapping to ROI components
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
rework -$
Defects -$
Investment Revenue Operating and Variable Expenses
© 2014 VersionOne 19
If we want to increase ROI…
Revenue – [Operating Expense + Total Variable Expenses] Investment
Increase the Numerator
…and/or Decrease the Denominator
© 2014 VersionOne 20
If we want to increase ROI…
Revenue – [Operating Expense + Total Variable Expenses] Investment
Focus here
…and/or Decrease the Denominator
Expenses and Investment can only be reduced so far until Revenue is no longer possible.
Increase the Numerator
Revenue, at least hypothetically, has no ceiling.
© 2014 VersionOne 21
“Revenue? But we don’t sell our software – it’s for
an internal customer…”
© 2014 VersionOne 22
Factors affecting ROI components
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
-$ -$ -$ -$ +$
rework -$
Defects -$
Investment Revenue Operating and Variable Expenses
• Effort expended in ideation • Effort expended in developing Epics, Stories • Quality of Epics, Stories
• Effort expended in analysis & design • Quality of Epics, Stories • Degree of team empowerment • Maturity of engineering practices & discipline • Delivery/deployment capability • Volume of escaped defects • Quality of support process*
• Release frequency • Customer-perceived value • Quality of Epics, Stories • Volume of escaped defects
© 2014 VersionOne 23
What to measure and
where
© 2014 VersionOne 24
What to Measure What that indicates
“Concept to team-ready” cycle time • How much the development, grooming, and maintenance of features and stories is costing us
• How long we’re allowing the value of the features and stories to decay
Story/Feature acceptance rate The effectiveness of the backlog generation and grooming process
What to measure: Investment
© 2014 VersionOne 25
What to Measure What that indicates
“Team-ready-to-Deployed” cycle time • How much the process of taking a team-ready story or feature to the customer is costing us
• How long we’re allowing the value of the features and stories to decay
Escaped Defect rate • How much it is directly costing us to fix the defects
• The opportunity cost of fixing those defects
What to measure: Expenses
© 2014 VersionOne 26
What to Measure What that indicates
“Concept to cash” cycle time (total lead time)
How much the complete ideation-to-payment value stream is costing us
Cycle-time by story/feature value How long the more valuable stories & features are being allowed to decay in value
Release frequency • How long revenue-generating opportunities are being deferred
• How long direct customer feedback is being deferred
Escaped defect rate Risk exposure to retained, incremental, and new revenue
What to measure: Revenue
© 2014 VersionOne 27
Cycle time shows up a lot as something to measure. Why?
Key: Cycle Time
• Revenue
• Inventory (epics, stories in the backlog)
• Feedback
The value of all of these decays with time:
© 2014 VersionOne 28
The best single indicator of system ROI is the trend in lead time
associated with that value stream…
…because all of the lower-level things we measure (e.g., defect
rate, internal cycle times) directly affect lead time.
Key: Cycle Time
© 2014 VersionOne 29
No matter what we might wish, the capability of the system today is what it is.
Key: Measure trends, not snapshots in time
Instead of just asking “Why is it taking us x days to deliver a feature?”
ask “Are we delivering things that are truly valuable to our customers faster now than we were 3 months ago?”
© 2014 VersionOne 30
So how can we improve ROI?
Idea Design & Code Test
Deliver/ Deploy
Valuable, Working, Delivered Features
Decrease Investment
Increase Revenue
Decrease Operating and Variable Expenses
• Competent and committed Product Ownership • Effective and efficient backlog development and grooming skills
• Trained and Empowered teams • Mature and disciplined agile engineering practices • Robust delivery/deployment capability • Solid definition of “Done” • Robust support process*
• Value focus • Frequent Releases to customer
Decrease Lead Time!
© 2014 VersionOne 31
Suggested reading for thought leadership aggregation
© 2014 VersionOne 32
Q&A
© 2014 VersionOne 33
On Behalf of Everyone at VersionOne…
Thanks!
Top Related