Managing Software Debt

Continued Delivery of High Value as Systems Age

Chris SterlingTechnology Consultant / Agile Coach /

Certified Scrum TrainerSterling Barton, LLC

Web: www.SterlingBarton.comEmail: chris@sterlingbarton.com

Blog: www.GettingAgile.com Follow Me on Twitter: @csterwa

Hash Tag for Presentation: #swdebt

1Friday, September 24, 2010

© 2009-2010,

Topics Being Covered

Problems Found with Aging Software

Software Debt Explained

Technical Debt

Quality Debt

Configuration Management Debt

Design Debt

Platform Experience Debt

The Wrap Up

A Story of What is Possible

2

2Friday, September 24, 2010

© 2009-2010,

Problems Found with Aging Software

Software gets difficult to add features to as it ages

Business expectations do not lessen as software ages

Software must remain maintainable and changeable to meet needs of business over time

3

3Friday, September 24, 2010

© 2009-2010,

Lack of emphasis on software quality attributes contributes to decay

4

4Friday, September 24, 2010

© 2009-2010,

The “Rewrite”, “NextGen” or “Like-to-like Migration”

“It will be easy since we worked on the original version” - although we understand the domain we will be fighting with new features, technology, tools, and processes

“We don’t have any other options” - Refactoring and test automation are potential alternatives to like-to-like migrations.

5

5Friday, September 24, 2010

© 2009-2010,

Limited Platform Expertise

Risk and costs increase as expertise becomes more limited for aging software platforms.

6

6Friday, September 24, 2010

© 2009-2010,

Costs for Release Stabilization Increase Over Time

0

125000

250000

375000

500000

Release 1

Release 2Release 3

Release 4Release 5

Release 6

Cost of Fixing Defects Cost for Feature Dev

7Friday, September 24, 2010

© 2009-2010,

Extreme Specialization

Knowledge and capability to maintain legacy software decays with time

Costs to maintain rarely used software platforms are higher

Leads to waiting for people in specialized roles to finish their tasks in support of development effort

8

8Friday, September 24, 2010

Software Debt

9

Creeps into software slowly and leaves organizations with liabilities

9Friday, September 24, 2010

© 2009-2010,

Software Debt Creeps In

10

10Friday, September 24, 2010

© 2009-2010,

Software Debt Creeps In

11

11Friday, September 24, 2010

© 2009-2010,

Software Debt Creeps In

12

12Friday, September 24, 2010

© 2009-2010,

Managing Software Debt – an Overview

13

13Friday, September 24, 2010

© 2009-2010,

Managing Software Debt

14

It is impossible to stop software debt from creeping into our software

Managing debt in software is based on putting frequent feedback mechanisms in place for code, quality assurance, configuration management, design, and organization of people on project team

Feedback mechanisms should be frequent, automated, and easy to use to support their continued use or modified to new needs

14Friday, September 24, 2010

© 2009-2010,

Types of Software Debt

Technical Debt

Quality Debt

Configuration Management Debt

Design Debt

Platform Experience Debt

15

15Friday, September 24, 2010

Technical Debt

16

Issues in software that will impede future development if left unresolved

16Friday, September 24, 2010

© 2009-2010,

* Ward Cunningham on “Technical Debt”

Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring.

Technical Debt doesn’t include deferred functionality, except possibly in edge cases where delivered functionality is “good enough” for the customer, but doesn’t satisfy some standard (e.g., a UI element that isn’t fully compliant with some UI standard).

* Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt

17

17Friday, September 24, 2010

© 2009-2010,

My Definition of “Technical Debt”

“Technical debt is the decay of component and inter-component

behavior when the application functionality meets a minimum standard

of satisfaction for the customer.”

18

18Friday, September 24, 2010

© 2009-2010,

Regression Costs - Manual vs. Automated

19

19Friday, September 24, 2010

© 2009-2010,

Principles of Executable Design

20

The way we design can always be improved.

We’ll get it “right” the third time.

We will not get it “right” the first time.

Design and construct for change rather than longevity.

Lower the threshold of pain.

If we are not enhancing the design then we are just writing a bunch of tests.

20Friday, September 24, 2010

Quality Debt

A lack of quality will lessen the value per feature added over time

21

21Friday, September 24, 2010

© 2009-2010,

Accrual of Quality Debt with Releases

22

22Friday, September 24, 2010

© 2009-2010,

Break/Fix Only Prolongs the Agony

23

23Friday, September 24, 2010

© 2009-2010,

Effect of Project Constraints on Quality

24

24Friday, September 24, 2010

© 2009-2010,

Effect of Project Constraints on Quality

24

24Friday, September 24, 2010

© 2009-2010,

Acceptance Test-Driven Development

25

25Friday, September 24, 2010

A Fit Case Study

Cost reduction using Fit for test automation and data conversion

26

26Friday, September 24, 2010

© 2009-2010,

Manual Regression Testing

Testing was taking 75 person hours during 2 full test runs consisting of:

Comprehensive manual regression testing

Data conversion and validation

Cost for testing was $17,000 each iteration

27

27Friday, September 24, 2010

© 2009-2010,

Introducing Fit into Testing Process

After 8 iterations team had introduced healthy amount of Fit fixtures and automated tests

Reduced 70+ hour test runtime down to 6 hours which now included:

Fit automated regression testing

Data conversion and validation automated with Fit fixtures

Reduced cost of testing each iteration from $17,000 to $7,000

28

28Friday, September 24, 2010

Configuration Management Debt

Unpredictable and error-prone release management

29

29Friday, September 24, 2010

© 2009-2010,

Traditional Source Control Management

30

30Friday, September 24, 2010

© 2009-2010,

Traditional Source Control Management

30

Main Branch

30Friday, September 24, 2010

© 2009-2010,

Traditional Source Control Management

30

Main Branch

Version 1Branch

Integrate forVersion 2

CodeComplete

30Friday, September 24, 2010

© 2009-2010,

Traditional Source Control Management

30

Main BranchDebt

Death March

Version 1Branch

Integrate forVersion 2

CodeComplete

30Friday, September 24, 2010

© 2009-2010,

Traditional Source Control Management

30

Main BranchDebt

Death March {Debt accrues quickly within stabilization periods

Version 1Branch

Integrate forVersion 2

CodeComplete

30Friday, September 24, 2010

© 2009-2010,

Flexible Source Control Management

31

31Friday, September 24, 2010

© 2009-2010,

Flexible Source Control Management

31

Main Branch

31Friday, September 24, 2010

© 2009-2010,

Flexible Source Control Management

31

Main Branch

Version 1

31Friday, September 24, 2010

© 2009-2010,

Flexible Source Control Management

31

Main Branch

Version 1 Version 2

31Friday, September 24, 2010

© 2009-2010,

Flexible Source Control Management

31

Main Branch

Version 1 Version 2{Not Easy! Must have proper infrastructure to do this.

31Friday, September 24, 2010

© 2009-2010,

Continuous Integration

32

32Friday, September 24, 2010

© 2009-2010,

Scaling to Multiple Integrations

33

33Friday, September 24, 2010

© 2009-2010,

The Power of 2 Scripts: Deploy and Rollback

34

Deploy Rollback

34Friday, September 24, 2010

Design Debt

Design decays when not attended to so design software continually

35

35Friday, September 24, 2010

© 2009-2010,

* Abuse User Stories

36

Implement Securityfor User Information

* From “User Stories Applied” presented by Mike Cohn Agile 2006

36Friday, September 24, 2010

© 2009-2010,

* Abuse User Stories

36

Implement Securityfor User Information

As a Malicious Hacker I want to steal credit card information so that I can make fraudulent charges

* From “User Stories Applied” presented by Mike Cohn Agile 2006

36Friday, September 24, 2010

Platform Experience Debt

Silos of knowledge and increased specialization will increase cost of maintenance over time

37

37Friday, September 24, 2010

© 2009-2010,

How to Combat Platform Experience Debt

Ignore it (I do not suggest this!) Surround existing functionality with automated functional tests

Wrap platform interfaces with adapters

Transfer knowledge of platform to more people

Rewrite on more current platform

Move thin slices of functionality to newer platform

Start platform upgrade discussions and rearrange teams into known effective team

38

~OR~

38Friday, September 24, 2010

© 2009-2010,

Team Configuration Patterns

Virtual Architect Pattern

Integration Team Pattern

Component Shepherd Pattern

Team Architect Pattern

39

39Friday, September 24, 2010

© 2009-2010,

Virtual Team Pattern

EnterprisePlanning

40

40Friday, September 24, 2010

© 2009-2010,

Virtual Team Pattern

Pros

Share architecture ideas and needs across teams

Based on verbal communication

Cons

Usually singles out special Team Member role

Could lead to top-down architecture decisions

IT may gain extensive influence and begin to run Product Backlog prioritization for architecture needs

41

41Friday, September 24, 2010

© 2009-2010,

Integration Team Pattern

42

Feature Development

Integrate Features

All features are implemented

and integrated every iteration

42Friday, September 24, 2010

© 2009-2010,

Integration Team Pattern

Pros

Reduces complexity on Feature Teams

Forces delivery from Integration Team instead of interface and deployment designs

Cons

Perpetuates specialized roles

Don’t always work on highest value Product Backlog items

43

43Friday, September 24, 2010

© 2009-2010,

Component Shepherd Pattern

44

44Friday, September 24, 2010

© 2009-2010,

Component Shepherd Pattern

Pros

Share more knowledge within organization to minimize platform experience debt

Work on highest value Product Backlog items

Cons

Not always optimal as using individual knowledge

Difficult to learn multiple systems across Teams

45

45Friday, September 24, 2010

© 2009-2010,

Feature Team Pattern

46

46Friday, September 24, 2010

© 2009-2010,

Feature Team Pattern

Pros

Team owns architecture decisions

Decisions are made close to implementation concerns

Cons

May not have appropriate experience on Team

Team could get “stuck” on architecture decisions

47

47Friday, September 24, 2010

What is possible?

High quality can be attained and enables accelerated feature delivery.

48

48Friday, September 24, 2010

© 2009-2010,

A Story: Field Support Application

2000+ users access application each day

Application supports multiple perspectives and workflows from Field Support Operations to Customer Service

Team of 5 people delivering features on existing Cold Fusion platform implementation

Migrating to Spring/Hibernate in slices while delivering valuable features

36 2-week Sprints, 33 production releases, and only 1 defect found in production

So, what was the defect you say? Let me tell you…

49

49Friday, September 24, 2010

Lets wrap this up...

What should I take away from this?

50

50Friday, September 24, 2010

© 2009-2010,

Principles for Managing Software Debt

Maintain one list of work

Emphasize quality

Evolve tools and infrastructure continually

Improve system design always

Share knowledge across the organization

And most importantly, get the right people to work on your software!

51

51Friday, September 24, 2010

Thank you

Questions and Answers

52

52Friday, September 24, 2010

© 2009-2010,

Chris Sterling – Sterling Barton, LLC

Technology Consultant, Agile Consultant and Certified Scrum Trainer

Developer of AgileEVM ( www.AgileEVM.com ), a project portfolio decision support tool

Consults on software technology, Agile technical practices, Scrum, and effective management techniques

Innovation Games® Trained Facilitator

Open Source Developer and Consultant

Software technology, architecture, release management, monitoring, and design consulting for Agile Teams

53

Email: chris@sterlingbarton.comwww.AgileEVM.com

Web: http://www.sterlingbarton.comBlog: http://www.gettingagile.com

Follow me on Twitter: @csterwa

53Friday, September 24, 2010