Post on 15-Jan-2015
description
metrics as a
learn and change agent
Gaetano Mazzanti @mgaewsj
Gama-Tech
as an agile coach
I am often asked to define/introduce KPIs
“why?”
no process chaos
code & fix
Case 1 “uhm, ehm, we
need to measure if and how much we are improving”
traditional
rigid process top-down
Case 2 “we
need to measure if and how much
we are improving; company policies enforce using
KPIs*”
*typically for performance appraisals
unfortunately things are not so simple and linear
ordered chaotic complex
system
any network that has coherence,
it may be fuzzy, it may have purpose
agent
anything which acts within the system:
individual, group, idea, etc.
systems: some definitions
ordered chaotic complex
ordered
system constrains agents
complex
system lightly constrains agents
agents modify system by their interaction
with it and each other, they co-evolve
chaotic
agents underconstrained &
independent of each other
no process chaos
code & fix
ordered chaotic complex
no metrics
no process chaos
code & fix
no metrics traditional
rigid process top-down
ordered chaotic complex
metrics for a linear, deterministic world
no process chaos
agile
traditional
code & fix rigid process
top-down
ordered chaotic complex
no metrics
metrics for a linear, deterministic world
???
complex systems
exhibit one or more properties (behavior) not obvious from the
properties of the individual
interconnected parts they are composed of
product development is complex
“self-organizing, non-linear, feedback systems are
inherently unpredictable, they are not controllable“
D.Meadows
so what are we supposed to do?
we can watch, learn and work
with the system
learn & change
metrics
how?
loop!
PDCA Loop
plan
do
check
act
Shewhart & Deming
OODA Loop
orient
decide act
observe
J.Boyd
LeanStartup Loop
Ideas
Build
Product
Measure
Data
Learn
are we building the right product?
E.Ries
when looping
minimize total time through the loop
learn fast fail fast! “biggest failure is failure to learn” (M.Poppendieck)
single loop learning
results actions how
efficiency doing things right
incremental change
lead to
which shape future
single loop drawbacks
delays defects
plan
uncertainty emerging information
unexpected impediments
more detailed planning rigid change management
overburden (waste)
double loop learning
results actions values, assumptions
why
effectiveness doing the right things question assumptions radical change
guide
lead to new/improved
efficiency doing things right incremental change
how
Chris Argyris
step outside the system!
timebound metrics learn, change, move on
results actions values, assumptions
define metric* set expiration date
goal ok or expiration
date passed?
*actionable, accessible, auditable, time-bound
do not focus on the metric itself, it is just a mean to understand/learn and change
measures have value only if they inform decisions or motivate action
be careful
how do you know that the results you are seeing are related to the changes you have made?
how do you know you are learning correctly from those changes?
what’s going on?
open bugs
time
“bugs show lack of quality not its presence” Gojko Adzic removing what you don’t want (i.e. bugs) does not imply getting what you do want
some hints
measure what scares you measure outcome not output i.e. how much value created vs how many stories done
defects trending down vs how many unit tests created
but still
organizations want to measure individuals => people game metrics ⇒ “if you do X I will give you Y” reduces motivation 1) no autonomy: feeling controlled by who
gives rewards 2) being payed for doing something: may
imply it’s not worth doing for its own sake
Hawthorne Effect
how to avoid this?
no bonus, appraisals, etc. shared goals (let the team find how to reach them)
visualize feedback as informational and not controlling
give unexpected rewards if you really have, let the team do individual appraisals
metrics
quadrants
inward & outward looking metrics
R&D Business &
Other Stakeholders
inward looking
outward loooking
boundary objects
feedback
metric
boundary objects
boundary object [sociology]: something that helps different communities exchange ideas and information. could mean different things to different people but allows coordination and alignment
business R&D
metrics quadrants
Team Maturity
Business
Product P
rocess
inward looking
outward looking
& feedback
metrics quadrants
Team Maturity
Business
Product P
rocess
boundary objects
metrics quadrants
Team Maturity
Business
Product P
rocess
fragile
boundary objects
agile
metrics quadrants
Team Maturity
Business
Product P
rocess
metrics quadrants
Lead Time Cycle Time
Quality of Service (SLA) Throughput
Business Value
WIP Cadence
CI Failures Rework
Impediments RetrospecHves
Morale
Revenues ROI
Customer SaHsfacHon
Code Quality Technical Debt Test Coverage
Team Maturity
Business
Product P
rocess
Bugs?
metrics quadrants
Lead Time Cycle Time
Quality of Service (SLA) Throughput
Business Value
WIP Cadence
CI Failures Rework
Impediments RetrospecHves
Morale
Revenues ROI
Customer SaHsfacHon
Code Quality Technical Debt Test Coverage
Team Maturity
Business
Product P
rocess
what!? no
velocity?
Bugs?
fragility
code quality
technical debt
lack of advanced engineering practices (i.e. TDD, CI) => rework
reduce
code quality evolution
code quality evolution
agility being agile is not the goal, it’s a mean if you are really interested there are plenty of agility tests on the Internet: Nokia Test Scrum Open Assessment - ScrumAlliance Agile Maturity Model Agile Evaluation Framework Comparative Agility Assessment etc.
impediments, retrospectives, reviews
# of questions answered
# of questions asked
# action items addressed
# action items assigned (at previous meetings)
# of WTFs
WTF!?
WTF!?
?
metrics
queues
queues are bad
increase cycle time
risk variability overhead
reduce quality
motivation
stop starting start finishing
cumulative flow diagram
time in queue (cycle time)
queue size (WIP)
time
cumulative quantity
arrivals
departures (throughput)
source: Donald Reinertsen
cumulative flow diagram WIP is a leading indicator
time
cumulative quantity
WIP
cycle time
cumulative flow diagram large batches large queues
time
cumulative quantity
cumulative flow diagram small batches small queues
time
cumulative quantity
Kanban board
backlog to do in progress done 2 3
cycle time
cycle time = WIP throughput
if you can’t see it you can’t manage it
no WIP limit -> queue!
2 3 ready backlog to do in progress done
slack (%)
optimize flow absorb variation
flow related metrics active WIP - buffered WIP tasks that are really in progress – task waiting to be handed-off (#,%,% of time spent) process efficiency active time / cycle time technical debt WIP / standard WIP # of projects a person works in parallel (should be 1!!!)
visualizing tasks dynamics
backlog to do in progress done 2 4
1 2 3 4 days inactive task
0
5
10
15
20
25
30
35
Backlog
To Do
In Progress
Done
cumulative flow diagram
WIP
cycle time
done
in progress
to do
backlog
time
# u
ser
stor
ies
throughput
not so helpful?
-‐4
-‐3
-‐2
-‐1
0
1
2
3
4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
in
out
single column dynamics
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
In Progress
WIP
Kanban board dynamics
control charts
source: Samuli Heljo
metrics
easy but powerful
42
Happiness Index
Mon Tue Wed Thu Fri
Tom J K J L J Anne J J J J J Paul L J K J J Joe J J J J K Eva J J J L J
niko-niko calendar
feedback board
leading or lagging?
Pizza Index
Pizza = Overtime => not good Steve Denning
how long since?
you talked to a customer last useful retrospective you learned something at work your boss last freaked out last critical bug
52 days
6 weeks
2 days
3 days
1 week
and don’t forget
bus factor # of key developers that need to be hit by a bus to kill a project
“per una vera mille sono finte” F. De André
“for every true one thousands are fake”
Gaetano Mazzanti
Gama-Tech @mgaewsj
info@gama-tech.net