Agile by the numbers · Agile Practices Adoption Rates range of readers, not just AgilistsSuccess...

1

© 2010 IBM Corporation

Agile by the Numbers:What’s Really Going On Out There

Scott W. AmblerChief Methodologist for Agile and Lean, IBM Rationalwww.ibm.com/developerworks/blogs/page/amblertwitter.com/scottwambler

2© 2010 IBM Corporation

Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance

� Development and Quality

� Modeling and Documentation

� Communication

� Scaling agile

� Parting Thoughts


Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



The Surveys

� All survey data, original questions, and summary slide decks can be downloaded from www.ambysoft.com/surveys/� If you can’t look at the original questions and analyze the data yourself, how can you trust

the survey results?

� Some surveys were done via Dr. Dobb’s Journal (DDJ), a community with a wide range of readers, not just Agilists

� Some surveys, the Ambysoft ones, focused on just the agile community

� The source survey for each chart is indicated using graphics such as:

DDJ 2009 State of the IT Union

Ambysoft 2009 Agile Practices

Warning: You’ll be presented with a lot of information really

quickly!


Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



Most Effective Practices: Top 10 (out of 30)

26%

28%

35%

36%

39%

43%

44%

47%

47%

65%

Burndown Tracking

Potentially Shippable Software

Active Stakeholder Participation

Pair Programming

Retrospectives

Code Refactoring

Iteration Planning

Developer TDD

Daily Stand Up Meeting

Continuous Integration


2


Practices Easiest to Learn: Top 10 (out of 30)

21%

21%

21%

25%

27%

31%

32%

35%

38%

70%

Code Refactoring

Product Backlog

Release Planning

Pair Programming

Burndown Tracking

Iteration Planning

Iteration Demo

Retrospectives


Daily Stand Up Meeting

Ambysoft 2009 Agile Practices8© 2010 IBM Corporation

Practices Most Difficult to Learn : Top 10 (out of 30)

18%

18%

19%

20%

22%

24%

26%

28%

30%

37%


Release Planning

Executable Specifications


Database Refactoring


Initial Estimate and Schedule

Pair Programming

Aceptance TDD

Developer TDD



Practices Tried and Abandoned : Top 8 (out of 30)

10%

11%

14%

14%

14%

17%

21%

24%

Retrospectives


Initial Estimate and Schedule


Daily Stand Up Meetings


Burndown Tracking

Pair Programming

Ambysoft 2009 Agile Practices10© 2010 IBM Corporation

Practices People Want to Adopt: Top 10 (out of 30)

15%

16%

16%

17%

18%

19%

19%

21%

21%

27%

JIT Model Storming

Collective Code Ownership

Database Regression Testing



Pair Programming

Database Refactoring


Developer TDD

Acceptance TDD



Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



Number of Agile Projects Run (%)

18

45

19

8

5

6

Pilot

1 to 5

6 to 10

11 to 20

21 to 50

51+

DDJ 2008 Agile Adoption

�69% of organizations have adopted agile techniques

3


24%

14%

18%

9%

7%

8%

4%

1%

11%

2%

4%

None

1-10%

11-20%

21-30%

31-40%

41-50%

51-60%

61-70%

71-80%

81-90%

91-100%

What percentage of your development teams have adopted agile techniques?

�76% of organizations have adopted agile techniques

�On average, 44% of project teams in those are now doing agile

� In small orgs of 50 or less IT, it’s 53%

� In larger orgs, it’s 38%

DDJ State of the IT Union July 200914© 2010 IBM Corporation

4%

23%

17%

26%

11%

19%

Don't Know

No Agile Projects

Planning toDefine

No, Wish We HadIt

No, Don't SeeValue

Yes, commonlyapplied

Has your organization defined criteria to determine whether a team claiming to be "agile" really is?

Ambysoft 2009 Governance


What does it mean to be agile?

Disciplined agile teams:

1. Value: Produce a consumable solution on a regular basis which provides value to stakeholders.

2. Validation: Do continuous regression testing, and better yet take a Test-Driven Development (TDD) approach.

3. Stakeholders: Work closely with their stakeholders, or a stakeholder proxy, ideally on a daily basis.

4. Self organization: Are self organizing and work within an appropriate governance framework.

5. Improvement: Regularly reflect on, and measure, how they work together and then act to improve on their findings in a timely manner.

Survey says:

� Value: 94%

� Validation: 87%

� Stakeholders: 95%

� Self organization: 56%

� Improvement: 88%

� All 5: 53%

� Everything but self organization: 72%

Ambysoft 2010 How Agile Are You?


Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



Why Agile? Because it Works!

DDJ 2008 Agile Adoption 18© 2010 IBM Corporation

Project success rates

Bottom Line: Agile teams produce higher quality work, are quicker to deliver, are more likely to deliver the right functionality,

and more likely to provide greater ROI than traditional teams

AgileIterativeTraditionalAd-Hoc

0.8

0.8

2.7

0.4

0.8

0.2

1.8

2.3

4.0

3.0

5.6

5.0

4.4

3.9

6.0

4.9

Time

Money

Functionality

QualityIterative

Agile

Traditional

Ad-Hoc

DDJ 2008 Project Success

4


Why agile?

0% 20% 40% 60% 80% 100%

Traditional

Ad-Hoc

Agile

Iterative

Successful Challenged Failed



A Few More Thoughts About Success� 41% of people believe that canceling a troubled project is a success

� 69% of people had been on a project that they knew would fail right at the very beginning



Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



21%

16%

14%

16%

7%

13%

2%

12%

No initial estimates

Within 50%

Within 25%

Within 20%

Within 15%

Within 10%

Within 5% range

Must be "exact"

An organization’s typical approach to initial estimates on software development projects

�On average, estimates need to be within a +/-11% range

DDJ State of the IT Union July 2009


39%

4%

4%

14%

17%

10%

5%

7%

2%

No tracking against estimates

More than 75%

Within 75%

Within 50%

Within 25%

Within 20%

Within 15%

Within 10%

Within 5%

On average, the actual costs of software development projects compared to estimates

�On average, actuals came in within a +/- 19% range

DDJ State of the IT Union July 200924© 2010 IBM Corporation

Strategies which project teams use to stay out of trouble or when in trouble to help get them out of it

� “Questionable” strategies:

�18% pad the budget

�63% de-scope towards the end of the project to meet deadline

�34% ask for extra funds to complete the projects

�72% extend the schedule to deliver promised scope

�39% avoid scope creep wherever possible via a “change control/management”process

�10% change the original estimate to reflect the actuals

�18% change the original schedule to reflect the actuals

� Ethical strategies:

�12% take a “stage gate” approach to funding

�13% have a flexible budget from the beginning of the project

�26% have a flexible schedule from the beginning of the project

�32% have flexible scope from the beginning of the project


5


Approach to Initial Estimation

Don’t know3%

Detailed estimate based on reasonable guess6%

Detailed estimate based on agile estimation technique7%

Detailed estimate based on traditional estimation technique5%

High-level estimate based on reasonable guess of experienced person(s)

36%

High-level estimate based on agile estimation technique28%

High-level estimate based on traditional estimation technique9%

No initial estimate at all8%

Ambysoft 2009 Agile Project Initiation


How long did it take your project team to get started? (Average: 3.9 weeks)

10%

12%

3%

11%

17%

12%

19%

10%

8%

Don't Know

> 8 Weeks

7-8 Weeks

5-6 Weeks

4 Weeks

3 Weeks

2 Weeks

1 Week

<1 Week



Justifying Agile Projects81% had to justify their projects in some manner

5%

8%

14%

21%

29%

45%

48%

Estimate NPV

Considered Offshoring

Considered Commercial Packages

Show Operational Feasibility

Estimate ROI

Show Technical Feasibility

Show Stakeholder Concurrence



Ambysoft 2008 Practices and Principles0 50 100 150

Only Done

Done Defn

SustainablePace

WorkingSoftware

Strongly AgreeAgreeNeutralDisagreeStrong Disagree

1. Working software is the primary measure of progress for our agile teams2. Our agile teams are allowed to work at a sustainable pace3. Our agile teams identify what done means at the beginning of each iteration4. Our agile teams only take credit for work that is actually done at the end of each

iteration

Agile Project Management Principles


Project Management Practices

� Iteration planning (3.54)

� Daily Scrum Meeting (3.29)

� Prioritized worklist (3.08)

� High-level release planning (2.19)

� Retrospectives (1.84)

� One Product Owner (1.55)

� Burndown chart (1.51)

� Potentially Shippable Software (1.51)

� Status Reports (1.15)

� Story Board with Task Breakdowns (0.83)

Ambysoft 2008 Practices and Principles


0 50 100 150

Self-Org

Trusted

Support

Motivated


1. We build agile teams around motivated individuals2. Our agile teams are provided with the env. and support that they need to succeed3. Our agile teams are trusted to get the job done4. Our agile teams are self organizing

Cultural Environment


6


0 50 100 150

Adjustment

ReflectionStrongly AgreeAgreeNeutralDisagreeStrong Disagree

1. At regular intervals the team reflects on how to become more effective in future iterations2. The team actually adjusts its behavior in the next iteration by focusing on the highest priority

items

Process Improvement



3.1

9.2

32.8

16.7

22.8

7.2

1.7

1.4

5.6

< 1 Week

1 Week

2 Weeks

3 Weeks

4 Weeks

5-6 Weeks

7-8 Weeks

> 8 Weeks

No Iterations

Length of Iterations (% respondents)82% have iterations between 1 and 4 weeks in length



Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



How would you rate your IT governance program?

6%8%

19%

11%

20%

36%

Too early to tell

Generally helps

Neither helpful norharmfulGenerally harmful

Don't Know

No IT governanceProgram



2%

21%

15%

35%

37%

56%

Undefined, Don't Need Them

Undefined, and we Need Them

Undefined, but Part of Culture

Defined for Stakeholders

Defined for Operations andSupport

Defined for Development Teams

Are rights and responsibilities (R&R) defined for various groupswithin your organization?



4%

19%

51%

26%

Don't Know

Yes, majorityautomated

Yes, majoritymanual

No

Do your project teams collect metrics to enable project monitoring by senior management?


7


Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



Development Practices

� Coding Standards (2.30)

� Collective Code Ownership (1.97)

� Continuous integration (1.94)

� Database standards (1.86)

� UI standards (1.65)

� Pair programming (-1.34)



Quality Practices

� Code Refactoring (1.79)

� UI Testing (1.54)

� Automated Developer Testing (1.08)

� TDD (-0.08)

� UI Refactoring (-0.22)

� Database refactoring (-0.31)

� Automated Acceptance Testing (-0.87)

� Database regression testing (-1.03)

� Executable Specs (-1.43)



41%

48%

53%

71%

User Interface

Data

Security

Coding

Which organizational conventions/guidelines do development teams conform to?



Value of Various Work Products on Agile Teams (out of 5)

� Velocity (metric) (3.56)

� Defect Reports (3.61)

� Arch Spec - High Level (3.66)

� Customer Tests (3.87)

� Iteration Task List (3.87)

� Whiteboard Sketches (3.90)

� Developer Tests (3.96)

� Source Code (4.22)

� Working Software (4.22)

� Gantt Chart - Detailed (2.21)

� Gantt Chart - High Level (2.70)

� Arch Spec - Detailed (2.88)

� Requirements Spec - Detailed (2.90)

� Use Cases - Detailed (2.96)

� Paper models (3.22)

� Burn Down Chart (3.35)

� Test Plan (3.39)

� Use Cases - Light (3.52)

� Requirements Spec - High Level (3.52)



Ambysoft 2008 Test Driven Development

8


Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



Modeling Practices

� Active Stakeholder Participation (1.95)

� Requirements Envisioning (1.50)

� Architecture Envisioning (1.31)

� Documentation as Requirement (0.49)

� Model Storming (-0.84)



Agile Approach to Initial Requirements

Do some sort of initial modeling, or have initial models supplied, or leverage reference models

89%

Do some sort of initial modeling or have initial models supplied to them88%

Use industry models as a reference12%

Use enterprise models as a reference10%

Have initial requirements models supplied to them12%

Detailed initial requirements modeling30%

High-level initial requirements modeling76%



Agile Approach to Initial Architecture

Do some sort of initial modeling, or have initial models supplied, or leverage reference models

86%

Do some sort of initial modeling or have initial models supplied to them83%

Use industry models as a reference12%

Use enterprise models as a reference14%

Have initial architecture/design models supplied to them12%

Detailed initial architecture/design modeling25%

High-level initial architecture modeling70%



Primary Approach to Modeling

0% 20% 40% 60% 80% 100%

Agile

Iterative

Traditional

Ad-HocNo Modeling

Sketch to Think andCommunicateSketch and CaptureKey DiagramsSBMT for Docs

SBMT to GenerateCodeSBMT for Full TripEngineering


Did you need to produce a vision document (or similar) as part of project initiation?

Yes56%

No37%

Don't Know7%


9







0 50 100 150

Docum

enta

tion

Emer

gence

Envision

i ng Strongly Agree

AgreeNeutralDisagreeStrong Disagree

1. We do some initial requirements modeling at the beginning of agile projects for scoping and planning purposes

2. The requirements details emerge over time on our agile projects3. Our agile teams have an understanding of the correct balance of documentation or other artifacts for

delivery

Requirements and Documentation


Modeling vs TDD: Primary Strategy for Requirements Specification Is/Was (%)

0 10 20 30 40 50 60

AcceptanceTests

Detailed Docs

DetailedDiagrams

High-LevelDiagrams

Ad-HocTraditionalIterativeAgile

DDJ 2008 Modeling and Documentation


Ambysoft 2008 Practices and Principles0 50 100 150

Emergence

Architecture

Simplicity

Excellence


1. Our agile teams give continuous attention to technical excellence and good design2. Simplicity, the art of maximizing the amount of work not done, works well in practice for our

agile teams3. We do some initial architecture modeling at the beginning of agile projects to get going in the

right technical direction4. The architecture and design details emerges over time on our agile projects

Design and Architecture


Modeling vs TDD: Primary Strategy for Arch/Design Specification Is/Was (%)

0 20 40 60 80

AcceptanceTests

Detailed Docs

DetailedDiagrams

High-LevelDiagrams

Ad-HocTraditionalIterativeAgile


10


Percentage of Teams Creating Deliverable Documentation


0% 20% 40% 60% 80% 100%

Agile

Iterative

Traditional

Ad-Hoc

User manualTraining materialSystem Overview docOperations doc


What is the quality of the deliverable documentation produced by a development team?Rating: -10 (very low) to 10 (very high)

-3.9

-1.3

-1.3

-0.6

IterativeAgileTraditionalAd-Hoc

And the “quality” of the documentation is the same.

DDJ September 2009 State of the IT Union


Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



Effectiveness of Communication Strategies


Effectiveness of Communication Strategies(bigger the number the better)

1.321.08Email

1.621.34Videoconferencing

1.511.42Teleconference calls

1.861.84Overview documentation

0.152.10Online chat

1.892.54Overview diagrams

0.16-0.34Detailed Documentation

3.464.24F2F at Whiteboard

4.064.25Face to face (F2F)

With StakeholdersWithin Team




Feedback Cycle

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software

2. Our agile project teams welcome new or changing requirements, even just before delivery3. Project Stakeholders work closely with our agile teams and are readily available4. At regular intervals our agile teams demonstrate potentially shippable software to their

stakeholders

0 50 100 150Ship

pableStak

ehol

ders

Changin

g Req

Satisfy


11


Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile



0 20 40 60 80 100 120 140 160

1 to 5

6 to 10

11 to 20

21 to 50

51-100

101 to 200

200+

Attempt Success

Largest Team Size Attempted vs. Successful



Does your team have to comply to industry regulations?

Yes33%

No60%

Don't Know7%



Does your team follow a CMMI compliant agile process?

Yes9%

No78%

Don't Know13%



Agile and Legacy Systems

45%

51%

57%

78%

Working withLegacy Data

Evolving LegacySystems

Integrating WithLegacy Systems

Working WithLegacy in Some

Way



Domain Complexity

Straight-forward

Intricate,emerging

Compliance requirement

Low risk Critical,audited

Team size

Under 10developers

1000’s ofdevelopers

Co-located

Geographical distribution

Global

Enterprise discipline

Projectfocus

Enterprisefocus

Technical complexity

HomogenousHeterogeneous,

legacy

Organization distribution(outsourcing, partnerships)

Collaborative Contractual

Agile scaling factors (of the Agile Scaling Model)

Disciplined Agile

Delivery

Flexible Rigid

Organizational complexity

12


50%

32%

57%

93%

59%

19%

Maintenance

Ops/Support

Transition

Construction

ProjectInitiation

Projectselection

Full Lifecycle Agile

DDJ November 2009 State of the IT Union Survey


21%

30%

45%

47%

60%

Same city

Some at home

Co-located

Far located

Same building

Agile and geographical distribution



55%

73%

79%

70%

AverageCo-LocatedNear LocatedFar Located


� Agile approaches require high levels of trust and communication

� Distribution reduces trust and makes communication more difficult

� Distribution and large team size often go hand-in-hand

DDJ 2008 Project Success Survey



Co-Located, 42%

Same Building, 17%

Within Driving Distance, 13%

Some Very Distant, 29%

Don't Know, 1%



1%

1%

3%

3%

6%

12%

26%

77%

> 500

201 to 500

101 to 200

51 to 100

31 to 50

21 to 30

11 to 20

10 or less

Agile and team size

Team size:



1%

1%

1%

2%

9%

13%

IFRS

Basel II

SEC

FDA

HIPPA

Sarbanes-Oxley

Agile and regulatory


13


2%

2%

2%

7%

7%

9%

9%

D/MODAF

COBIT

TOGAF

ITIL

6 Sigma

CMMI

ISO 900x

Agile and governance frameworks



6%

18%

22%

35%

12%

8%

Don't know

Very complex

Complex

Mediumcomplexity

Straightforward

Pilot projectsonly

Agile and domain complexity



17%

25%

30%

37%

52%

67%

Outsourcing

Partner orgs

Other countries

Multiple divisions

Consultants

Same org

Agile and organizational complexity



15%

34%

34%

42%

49%

59%

62%

COTS

Stand alone

Multiple platform

Legacy data

Single platform

Greenfield

Integration

Agile and technical complexity



10%

11%

12%

17%

18%

32%

32%

32%

Portfolio Mgmt

Governance

Ent. Bus. Arch.

Ent. Data

SEPG

Support

Operations

Ent. Tech. Arch.

Agile and enterprise disciplines



Agenda

� The Surveys

� Agile Practices

� Adoption Rates

� Success Rates

� Management

� Governance



� Communication

� Scaling agile


14


14%

15%

29%

31%

32%

32%

33%

52%

54%

Mgmt Resistance

Stakeholder Resistance

Other visions

Specialization

C&C Culture

Lack of Trust

T&E

Stakeholder involvement

Waterfall culture

Organizational challenges faced when adopting agile

Source: Dr Dobb’s November 2009 State of the IT Union Survey80© 2010 IBM Corporation

What do you think about the concept of becoming a "certified master" after taking a two-day course?

78%

10%

12%

The Certification isObviouslyMeaninglessI Respect theCertification

I Don't Care EitherWay

Ambysoft 2009 Agile Certification


Question the Rhetoric

� There appears to be a difference between what people say they are doing and what they are doing

� Many of the concerns that the traditional community has regarding agile don’t appear to hold true

� There are many unfounded beliefs in both the traditional and the agile communities

� In the end, you need to identify what works well for you � Every organization is different


© Copyright IBM Corporation 2010. All rights reserv ed. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way.

IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

scott_ambler [at] ca.ibm.com

twitter.com/scottwambler

www.ibm.com/developerworks/mydeveloperworks/blogs/ambler/

www.ibm.com/rational/agile/

www.jazz.net

Agile by the numbers · Agile Practices Adoption Rates range of readers, not just AgilistsSuccess...

Documents

Transcript of Agile by the numbers · Agile Practices Adoption Rates range of readers, not just AgilistsSuccess...