Metrics for the lean agile pm
-
Upload
yuval-yeret -
Category
Documents
-
view
1.871 -
download
0
description
Transcript of Metrics for the lean agile pm
Metrics for the LEAN/Agile Product Manager
Yuval Yeret
Main things we want to pay attention to
• Performance of the Production floor – covered elsewhere (Simple KPIs Slides)
• Performance of the Product Management group: – Business Value–Wastes related to PM– Technical Debt
Business value
• We care about outcome – features delivered, adopted, used, paid for
• How can we measure this? • Manage a kanban at the high level features level,
that tracks when features are adopted, and upon first paying customer.
• Then see how much WIP of features not yet adopted we have, LEAd/Cycle time to adoption, features that we dropped on the way.
Debt
• A lot of time debt is taken due to PM decision
• We want to track how much debt we have, and take action to minimize it.
• E.g. we need to release CustomerFeatureX now, so we don’t “automate tests”/”code it correctly”, so every work on ModuleY which is used in FeatureX is slower, until this is fixed
Tracking debt in kanban
• Have debt card type that is created when debt is taken on
• Track amount of debt versus overall WIP/Backlog
• See whether stable, improving, worsening trend
• Decide on policy for dealing with debt – WIP Limit, etc.
• Track the cycle time and WIP for debt cards to see whether they get the SLA they deserve
Wastes related to PM
• Waiting for PM• PM related Churn / Context switching /
Expediting• Sunk Costs• Rework due to late feedback by PM
Waiting for PM
• Look at the CFD, observe the size of the PM-related queues over time. – Especially Pending PM Review which is in the
middle of DEV/Test– And Ready-MMFs as well as DEV Ready in some
cases which depend on PM approval– Advanced – in the cycle time performance
report, focus on PM areas
• When looking at exceptions to Cycle time, participate in the root cause analysis, and see if interaction with PM was part of the long cycle time.
PM related Churn / Context switching / Expediting
• Add Expedited class of service– Can be used by PM to
override priorities in DEV WIP – just to top of queue, don’t override current WIP
• Add emergency class of service– Can also override current
WIP
• Assumption – – This is value trumps flow.
We give up efficiency when we use these COSs
Measuring the effect of value over flow COS
• Look at cycle times for different kinds of classes of service
• Look at distribution of different COS in the WIP
Look at amount of changes in scope
• Replace – need visualization that shows scope changes in content
• Add – can simply look at total scope for a “Release” and observe whether its growing
Case Study – Typical release behavior
1 2 3 4 5 60
20
40
60
80
100
120
140
160
180
200
Total Required WorkLinear (Total Required Work)Actual DonePlanned ScopePlanned DoneOriginal + Dark MatterLinear (Original + Dark Matter)
Added Scope
Growth in Feature
Cost / Dark Matter
• Dark Matter – Is where we thought a feature costs X
• But then, during breakdown, analysis, creating iteration stories, we understand it actually costs X+D
• Then, PM decides whether to scope to fit down to X again, or D is worth it.
• Worthwhile tracking our behaviour on this, and learning from it.
• What is the right D number/percent? Good question!
• Can be observed in the CFD for a release.
Sunk Costs
• Add a LANE that collects features/stories that are “ON HOLD” – the Recycle Bin in the archive area
• The amount of work done on them is the sunk cost
• Amount of work hard to measure, so use alternative:– CYCLE TIME – look at cycle time for end
lane being the recycle bin
Rework due to late feedback by PM
• Will appear as high cycle times• Will appear as moving back cards on
the board (need to find way to measure)
• Can use special Class of service / card type to identify these kinds of stories better for measurement/tracking purposes
Workload compared to DEV
• See how much workload is in PM compared to DEV
• Look for trends and major changes that can indicate:– Bottleneck in PM– Idle and slack capacity – expect to see PM seeing
clients/customers at those times
Release OVERHEAD
• “How often do we release? What does it cost us?”
• The Usual Suspects– PM wants to release more often. He wants
the ready feature to be out there as soon as possible.
– R&D usually wants to limit the amount of releases, as they cost a lot, and R&D doesn’t like to do hardening
How often do we release
• On kanban, simply add a card type of “Release” and flow it thru the board to signify releases.
• The size should be the hardening cost planned for this release.
• based on SPC and other charts, you can understand:– Plan versus actual on hardening costs/times/dates– Frequency of actual releases– Ratio of hardening work compared to feature work
(see next slide for a view on this)
V3.0 V3.5 V4.0 V4.50
20
40
60
80
100
120
Indirect/Other Ef-fort
Effort on Stabi-lization
Effort on New Features
Release
% o
f R
ele
ase B
ud
get/
Eff
ort
Release Overhead
• This metric shows how much effort is spent on releasing versus developing.
• The aim is to reduce the overhead of each release, such that the organization can increase the frequency of releases to meet business expectations.
Reducing the release overhead
• Two things we need to do:– Reduce the overhead of each release–Make sure our release frequency makes
sense
Reducing the release overhead of each release
• Invest in reducing legacy hardening debt– As the PM you’ll be asked to invest. – Ask for a plan that associates investment of X days with Y days
of reduction in hardening cost. – Decide what is your investment horizon– Based on the horizon, the X/Y ratio, and the current frequency of
releases, make your decision– Typical areas of investment - Continuous Integration, Automation
of EVERYTHING (Including platform matrix, performance, any thing that is currently in the hardening plan)
• Avoid hardening debt while developing new features• Build quality in – don’t let defects wait for the end• Consider different types of releases – e.g. Majors,
Feature/Service Packs, Patch Bundles– And associate the relevant risk-driven hardening work for each
Does our release frequency make sense?
• First step – have the visibility– How many releases– What kinds of releases – scheduled major
“Trains”, emergency fixes “Ambulance?”, “special delivery/Taxis”
• Second step – lay out what the business actually needs and is willing to pay for
• Are those aligned?• A lot of the times, you will see “Trains” and
“Taxis”. Think of investing in a “Subway” – a predictable frequent release mechanism