________________________________________________________________________ Jonsson School of Engineering and Computer Science

Dr. Mark C. Paulk2015 ASEE Annual Conference, 21 February 2015

Adoption of Agile Methods by High Maturity

Organizations

2

Software CMM v1.1 (1987-2005)

Competent people (and heroics)

Defect PreventionTechnology Change ManagementProcess Change Management

Continuous process improvement

Product and process quality

Engineering processes and organizational support

Project management processes

Quantitative Process ManagementSoftware Quality Management

Organization Process FocusOrganization Process DefinitionTraining ProgramIntegrated Software ManagementSoftware Product EngineeringIntergroup CoordinationPeer Reviews

Requirements ManagementSoftware Project PlanningSoftware Project Tracking & OversightSoftware Subcontract ManagementSoftware Quality AssuranceSoftware Configuration Management

Level Focus Key Process Areas

Initial

Optimizing

1

Repeatable2

3

Managed4

5

Defined

3

Implications of Maturity

Initial

Repeatable

Defined

Process is informal and unpredictable

Project management system in place; performance is repeatable

Software engineering and management processes defined and integrated

Product and process are quantitatively controlled

Time/$/...

Time/$/...

Time/$/...

Optimizing Process improvement is institutionalized

Time/$/...

Time/$/...

Level Process Characteristics Predicted Performance

Managed

1

2

3

4

5

Better predictability… Less variability… Improved performance…

Empirical Data on Improvement By Maturity Level

Quality (reliability, defects) improves by roughly a factor of 2 (or more)

- C. Jones, “Software Benchmarking,” IEEE Computer, October 1995.

- K.D. Williams, "The Value of Software Improvement… Results! Results! Results!" SPIRE97, June 1997.

- R. Yacobellis, “Panel: Does SEI Level 5 Lead to High Quality Software?” COMPSAC 2001.

Effort for a given product decreases 15-21%, productivity increases, cycle time decreases

- B.K. Clark, “Quantifying the Effects on Effort of Software Process Maturity,” IEEE Software, November/December 2000.

- D.E. Harter, M.S. Krishnan, and S.A. Slaughter, “Effects of Process Maturity on Quality, Cycle Time, and Effort in Software Product Development,” Management Science, April 2000.

- L.H. Putnam, “Linking the QSM Productivity Index with the SEI Maturity Level,” QSM, 2000.

4

5

CMMI-DEV v1.3

Process is unpredictable,poorly controlled, and reactive

Process is characterized for projects and is oftenreactive

Process is characterizedfor the organization andis proactive

Process is measuredand controlled

Focus is on quantitativecontinuous processimprovement

Level Process Characteristics

Requirements ManagementProject Planning

Product & Process Quality Assurance

Configuration Management

Project Monitoring & ControlSupplier Agreement Management

Quantitative Project ManagementOrganizational Process Performance

Causal Analysis & Resolution

Process Areas

Requirements Development Technical Solution

Product Integration

ValidationVerification

Organizational Process Focus

Integrated Project Management

1Initial

2Managed

3Defined

4Quantitatively

Managed

5Optimizing

Measurement & Analysis

Organization Process DefinitionOrganizational Training

Risk ManagementDecision Analysis & Resolution

Organizational Performance Management

6

A Scrum Adoption Survey

184 distinct organizations responded to the 2011 Scrum adoption survey• 24 ML5• 2 ML4• 44 ML3• 61 ML2• 125 ML1 and unknown

128 projects adopting Scrum

M.C. Paulk, “A Scrum Adoption Survey,” ASQ Software Quality Professional, March 2013.

7

Organizational Size

≤25 26-50 51-75 76-100 101-200 201-300 301-500 501-1000

1001-2000

>20000

2

4

6

8

10

12

14

16

18

20

8

Organizational Size by Level

Org Size ML5 ML4 ML3 ML2 Unknown ≤25 1 0 0 2 15

26-50 0 0 0 1 1451-75 0 0 0 0 5

76-100 0 0 0 2 9101-200 2 0 2 3 10201-300 2 0 0 2 10301-500 4 1 0 1 9

501-1000 3 0 3 2 61001-2000 3 0 4 1 9

>2000 0 0 0 0 0

9

Software Engineering Methods

Scrum

Extreme Programming (XP)

Feature Driven Development (FDD)

Crystal methods (including Crystal Clear)

Team Software Process (TSP)

Unified Process (including RUP, AUP, OUP)

Other methods

0 40 80 120

10

Methods by Level A high percentage of high maturity organizations use Scrum.

Many high maturity organizations use XP, FDD, and the Unified Process.

Method ML5 ML4 ML3 ML2 Unknown

Scrum 17 0 12 16 86

Extreme Programming (XP) 5 0 6 6 16

Feature Driven Development (FDD) 5 0 2 4 11

Crystal methods (including Crystal Clear) 0 0 1 0 5

Team Software Process (TSP) 1 0 2 1 0

Unified Process (including RUP, AUP, OUP) 7 2 4 6 18

Other methods 8 1 10 8 87

11

Requirements Volatility Per Month

<1% per month

1-3% per month

3-5% per month

5-10% per month

10-20% per month

20-50% per month

>50% per month

0 5 10 15 20 25 30 35

12

Requirements Volatility by Level

Requirements Volatility ML5 ML4 ML3 ML2 Unknown<1% per month 4 0 6 4 131-3% per month 3 0 3 1 53-5% per month 3 0 0 1 55-10% per month 4 0 3 3 1710-20% per month 7 1 0 6 1720-50% per month 1 0 0 0 14>50% per month 5 0 1 1 7

Use of Scrum

13

We never heard of Scrum before

We are aware that Scrum exists

We are currently piloting Scrum

We have piloted Scrum but no decision about adoption has been made

We are currently deploying Scrum across the organization

Scrum is one of the standard methods we use

Scrum is the normal way we build software

0 10 20 30 40 50 60

14

Use of Scrum by Level

Few high maturity organizations are unaware of Scrum or uninterested in it.

Scrum Adoption ML5 ML4 ML3 ML2 UnknownWe never heard of Scrum before 1 0 1 0 4We are aware that Scrum exists 2 1 2 1 32We are currently piloting Scrum 5 0 0 1 9We have piloted Scrum but no decision about adoption has been made 1 1 4 1 9We are currently deploying Scrum across the organization 2 0 2 3 16Scrum is one of the standard methods we use 6 0 3 11 30Scrum is the normal way we build software 4 0 3 0 25

15

Scrum Team Size by Level

Team Size ML5 ML4 ML3 ML2 Unknown≤3 0 1 4 3 6

4 to 6 5 0 2 2 227 to 9 5 0 1 4 27

10 to 12 2 0 0 3 613 to 15 4 0 1 1 2

≥16 6 0 2 2 14

16

Quality of the Software

much lower

lower

about the same

higher

much higher

0 10 20 30 40 50 60 70

17

Quality by Level

Quality ML5 ML4 ML3 ML2 Unknownmuch lower 0 0 0 0 1

lower 1 0 1 2 3about the same 2 1 2 4 14

higher 12 0 5 5 37much higher 5 0 2 2 15

Quality tends to be higher for organizations adopting Scrum in general.

18

Cost

much lower

lower

about the same

higher

much higher

0 10 20 30 40 50 60

19

Cost by Level

Cost ML5 ML4 ML3 ML2 Unknownmuch lower 0 0 0 0 3

lower 7 0 3 5 22about the

same 13 1 5 5 29higher 0 0 1 1 9

much higher 0 0 0 2 7

Cost does not appear to be significantlyaffected for most organizations adopting Scrum.• lower for a noticeable percentage

20

Meeting Schedule Expectations

much lower

lower

about the same

higher

much higher

0 10 20 30 40 50 60

21

Schedule Expectations by Level

Schedule Expectations ML5 ML4 ML3 ML2 Unknownmuch lower 0 0 0 0 4

lower 2 0 0 1 3about the same 3 1 1 4 12

higher 12 0 6 6 33much higher 3 0 2 3 18

Meeting schedule expectations appears to be higher for organizations adopting Scrum in general.

22

Customer Satisfaction

much lower

lower

about the same

higher

much higher

0 10 20 30 40 50 60

23

Customer Satisfaction by Level

Customer Satisfaction ML5 ML4 ML3 ML2 Unknownmuch lower 0 0 0 0 1

lower 0 0 0 2 2about the same 1 0 1 4 9

higher 11 1 5 3 28much higher 8 0 3 5 26

Customer satisfaction tends to be higher for organizations adopting Scrum in general.

24

Concluding Thoughts

It appears that most high maturity organizations are aware of, have piloted, and have adopted agile methods.

Most particularly Scrum.

With good success in terms of customer satisfaction, quality, and meeting schedule expectations… and some impact on improving cost.

This survey is only a first step in exploring the adoption of agile methods by high maturity organizations…

25

Questions and Answers