The Broken State of Process Improvement in Software Development

35
Agile India 2014 BROKEN STATE OF PROCESS Why software process improvement is broken… and what to do about it…

description

Experimentation and the scientific method are very valuable for validating business opportunities. The Lean Startup and Lean UX are driving new thought patterns in the software development world. However, despite all the new thinking on product, rarely do organizations apply these techniques to their processes, opting for inefficiently adding more steps ad nauseum until productivity has ground to a halt. We’ll cover the following topics: What are processes, and why do we need them How processes affect delivery and productivity Scale and process inefficiency Process experimentation techniques Designing productive processes

Transcript of The Broken State of Process Improvement in Software Development

Page 1: The Broken State of Process Improvement in Software Development

A g i l e I n d i a 2 0 1 4

BROKEN STATE OF PROCESSWhy software process improvement is broken… and what

to do about it…

Page 2: The Broken State of Process Improvement in Software Development

A BIT ABOUT ME…

ThoughtWorker Product Manager by trade See a lot of processes Unfortunate Six Sigma Black Belt Fascinated by process wonkery

!2

Page 3: The Broken State of Process Improvement in Software Development

GOAL: !

TRY, MEASURE, LEARNSoftware development processes… Be incremental about your process, just like your delivery

!3

Page 4: The Broken State of Process Improvement in Software Development

HOW WE THINK ABOUT PROCESS IMPROVEMENT IS BROKENToo many people like me are telling you what you *should* do…

!4

Page 5: The Broken State of Process Improvement in Software Development

WE ARE OBSESSED WITH…

▫︎ Small batches, watch your queues ▫︎Measure the cost of delay ▫︎ Pair ▫︎ Scrum/Kanban/SAFe ▫︎Do TDD ▫︎Do CI ▫︎Do CD ▫︎ LeanUX ▫︎ Burn up ▫︎ Run experiments ▫︎ Build, Measure, Learn ▫︎ Control chart !!!

!5

Frameworks Methodologies Practices

All great guide posts but…

Page 6: The Broken State of Process Improvement in Software Development

WHAT’S THE POINT?

Why do we have processes in the first place?

!6

Hans Splinter - http://www.flickr.com/photos/archeon/4885276155/ via cc

Page 7: The Broken State of Process Improvement in Software Development

REDUCE VARIATION IN A BUSINESS OUTCOMEYour process shouldn’t get in the way…

!7

Page 8: The Broken State of Process Improvement in Software Development

A BAD PROCESS

Doesn’t achieve the outcome !

OR !

Doesn’t reduce variation in that outcome !

!8

Page 9: The Broken State of Process Improvement in Software Development

WHAT ARE YOUR OUTCOMES THEN?

!9

Facilitate a decentralized, empowered, collaborative workplace

the most critical aspects of being adaptive leaders. These are

Value(Releasable Product)

Figure 4: The Agile Triangle

Jim Highsmith’s Agile Triangle Speed

Quality

Performance

Compliance

User Experience

Responsiveness

Learning

Customer Satisfaction

Page 10: The Broken State of Process Improvement in Software Development

PRINCIPLES OF PRODUCT DEV FLOW

Read it Particularly the chapter on variation ■ Pay-off functions

■ Pay-off asymmetries

■ Some of it is wrong :-)

!10

Reducing variation isn’t bad, reducing it irrespective of the outcome is bad

Page 11: The Broken State of Process Improvement in Software Development

HOW DO PROCESSES AFFECT YOUR OUTCOMES?Let’s talk about throughput to make things easy…

!11

Page 12: The Broken State of Process Improvement in Software Development

MARGINAL BENEFIT OF ADDITIONAL PROCESS

!12

It turns negative! Assuming small discrete steps as a simplification…

Page 13: The Broken State of Process Improvement in Software Development

ALL VARIATION IS REDUCED

!13

Page 14: The Broken State of Process Improvement in Software Development

INSTITUTIONAL MEDIOCRITY

!14

Page 15: The Broken State of Process Improvement in Software Development

REAL EXAMPLE - CUSTOMER CONVERSATION

Problem: Poor Unit Testing Solution: Attach tests to stories, fill out form !

Cost: 300 Devs X 10mins per check-in X 200 Days X 3 checkins per day = !

30,000 Hrs or 15 Person Years!15

Page 16: The Broken State of Process Improvement in Software Development

STANDARDIZATIONUt oh… Let’s challenge a scaling core belief

!16

Page 17: The Broken State of Process Improvement in Software Development

SCALED INSTITUTIONALIZED MEDIOCRITY

!17

Page 18: The Broken State of Process Improvement in Software Development

DIFFERENT TEAM OUTCOMES?

How can you go “faster” if need to follow “slow” processes? !

Do you have “special”teams?

!18

http://en.wikipedia.org/wiki/File:Neus1.jpg via cc

Page 19: The Broken State of Process Improvement in Software Development

WHAT HAPPENS WHEN OUTCOMES CHANGE?

!19

Felix Burton - http://flickr.com/photos/65328860@N00/14320717 via cc

Speed

Quality

Performance

Compliance

User Experience

Responsiveness

Learning

Customer Satisfaction

Org process rollouts tend to be big bang.

Page 20: The Broken State of Process Improvement in Software Development

STANDARDIZATION CREATES RESPONSIBILITY DISCONNECT

Rarely do leaders blame the process for poor outcomes

!20A2gemma - http://www.flickr.com/photos/a2gemma/1448178195/ via cc

Page 21: The Broken State of Process Improvement in Software Development

REPORTING IS NOT AN OUTCOMEIf you have a bunch of process or unnecessary standardization to facilitate the “right” report for managers or the PMO, you’re doing it wrong… Good reporting = side effect of good process

!21

Page 22: The Broken State of Process Improvement in Software Development

SCALE ISN’T AN EXCUSE FOR PROCESS COMPLEXITYScale brings complexity. Just like in a good product, your job is to strive for simplicity despite the complexity. Hide complexity from your teams.

!22

Page 23: The Broken State of Process Improvement in Software Development

WHY DO WE STANDARDIZE?

▫︎Common language across teams

▫︎ People fungibility

▫︎ Reporting

▫︎Management convenience

▫︎ Investment management

!23

Page 24: The Broken State of Process Improvement in Software Development

SOLUTIONS?Questions and practices as guide posts!

!24

Page 25: The Broken State of Process Improvement in Software Development

How complex is your process?

!

Can a random team member accurately describe your process?

!25

Page 26: The Broken State of Process Improvement in Software Development

Are your processes any good?

!

Are you hitting your outcomes?

Are you hitting your outcomes consistently?

!26

Page 27: The Broken State of Process Improvement in Software Development

Do you have feedback loops across organizational hierarchy?

!

Do your standards change?

Is there a visible feedback mechanism?

!27

Page 28: The Broken State of Process Improvement in Software Development

How much time do you spend administering your processes a

day?

!

Do you “work the spreadsheet”?

Does reporting take days?

!28

Page 29: The Broken State of Process Improvement in Software Development

TRY, MEASURE, LEARN

!29

Try

Measure

Learn

Ideas

StepsData

http://theleanstartup.com/principles

Page 30: The Broken State of Process Improvement in Software Development

EXPERIMENTS

▫︎ Clear hypothesis/problem statement

▫︎ Ability to measure the result

▫︎ Constant monitoring of biz outcomes

▫︎ Stop things that hurt outcomes or affect variation (in an unwanted direction)

▫︎ Learn a bit about stats

▫︎ Rinse, repeat

!30

Page 31: The Broken State of Process Improvement in Software Development

MINIMUM VIABLE PROCESS

▫︎ Start from the minimum amount of process required to get the outcome you want.

!

▫︎ Tweak until you get the right consistency (process costs)

!

▫︎ Carefully add to support for other goals, as marginal additions eventually turn negative (try to subtract things, and keep measuring outcomes!)

▫︎ The Improvement Kata looks interesting (I haven’t used it

!31

Page 32: The Broken State of Process Improvement in Software Development

MINIMUM VIABLE STANDARDIZATION

▫︎ Start from the minimum amount of standardization for visibility and communication. (e.g. 3 key metrics, or common names for work items) !

▫︎Make sure your base standardization gives teams plenty of wiggle room. !

▫︎Carefully add, measuring the outcomes!32

Page 33: The Broken State of Process Improvement in Software Development

SHAMELESS PLUG

!33

Now O

SS!

Page 34: The Broken State of Process Improvement in Software Development

TRY, MEASURE, LEARN !

PROCESSES HAVE COST !

KNOW YOUR OUTCOMESThen have fun with all those frameworks, methodologies, and practices

!34

Page 35: The Broken State of Process Improvement in Software Development

Twitter: @twchad

THANK YOU!