Software Quality Processes – Part I

CSSE 376, Software Quality AssuranceRose-Hulman Institute of TechnologyMarch 16, 2007

2

Acknowledgments

Some material was taken from a tutorial by Mike Phillips of the Software Engineering Institutehttp://software.gsfc.nasa.gov/docs/CMMI%20Tutorial.pdf

Some material was also taken from the August 2004 SW-CMM Maturity Profilehttp://www.sei.cmu.edu/appraisal-program/profile/pdf/SW-CMM/2004augSwCMM.pdf

3

Outline

1. Why focus on Process?

2. CMMI - Introduction

Why Process?

5

Underlying Premise of Process Improvement

“The quality of a product is largely determined by the quality of the process that is used to develop and maintain it.”

Based on TQM principles as taught by Juran, Deming and Crosby.

6

Categories of Benefits

1. Improved schedule and budget predictability

2. Improved cycle time

3. Increased productivity

4. Improved quality (as measured by defects)

5. Increased customer satisfaction

6. Improved employee morale

7. Increased return on investment

8. Decreased cost of quality

7

Results: Boeing Effort Estimation

.

0 %

140%

-140%

....

.

..

. ... .

.

. .

. . . .

.. . .

. .

.

.. . . .. .. . . . . .... . . .. .

.. .

. ...

..

. .. .. ...... . .. . ... . .. . .. ..

Without Historical Data With Historical DataVariance between + 20% to - 145% Variance between - 20% to + 20%

(Mostly Level 1 & 2) (Level 3)

Ove

r/U

nd

er P

erce

nta

ge

.

(Based on 120 projects in Boeing Information Systems)

.. . .

.

.. .

...

. .

. ..

.. .

..

.. .. . .. . . . . .. . . . . .. .

... . .. . . . . . .. . . .. . . . .

. . . . .. . . . .. . . . . .. . . . .. . . . . .

. . . . . .. . . . .. . .

. . . . . . . .

. . . . . . . . .

. . . . . .. . . . . .

. . . . . .

Reference: John D. Vu. “Software Process Improvement Journey:From Level 1 to Level 5.” 7th SEPG Conference, San Jose, March 1997.

Improved Schedule and Budget Predictability

8

Project Cycle Times

0

250

500

750

1991

1992

1993

1994

1995

1996

Year

Avg

Wo

rkin

g

Day

s Req Def

Implement

Source: Software Engineering Div., Hill AFB, Published in Crosstalk May 1999

Improved Cycle Time

9

Source: Software Engineering Div., Hill AFB, Published in Crosstalk May 1999

Man-hours per LOC

0

0.2

0.4

0.6

0.8

1

1.2

A B C D E

Release

No

rmal

ized

Man

-ho

urs

Increased Productivity

10

Increased Productivity and Quality

11

Cartoon of the Day

Capability Maturity Model® Integration (CMMISM)

13

History (1/2)

In the beginning there was chaos...Department of Defense spent millions of

dollars on software that was never completed.Contractor selection was unscientific

Meanwhile, process gurus (Deming, Crosby, Juran) taught the Japanese how to improve manufacturing

14

History (2/2)

Software Engineering Institute (SEI) created Capability Maturity Model (CMM) for software, others were developed later: Systems engineering Software acquisition People

Increasing pressure to integrate all the models led to the Capability Maturity Model Integration (CMMI) However, CMMI can still be used only for software development

in organizations

15

The CMMI Project DoD sponsored collaboration

between industry, Government, SEI Over 100 people involved

U.S. Army, Navy, Air Force Federal Aviation Administration National Security Agency Software Engineering Institute ADP, Inc. AT&T Labs BAE Boeing Computer Sciences Corporation EER Systems Ericsson Canada Ernst and Young General Dynamics Harris Corporation Honeywell

KPMG Lockheed Martin Motorola Northrop Grumman Pacific Bell Q-Labs Raytheon Reuters Rockwell Collins SAIC Software Productivity Consortium Sverdrup Corporation TeraQuest Thomson CSF TRW

Staged Representation

17

The Maturity Levels

Process unpredictable, poorly controlled and reactive

Process characterized for projects and is often reactive

Process characterized for the organization and is proactive

Process measuredand controlled

Focus on processimprovement

Optimizing

QuantitativelyManaged

Defined

Performed

Managed

Optimizing

Defined

1

2

3

4

5

18

Maturity Levels Cannot Be Skipped

A level provides a necessary foundation for effective implementation of processes at the next level.

Higher level processes are easily sacrificed without the discipline provided by lower levels.

The effect of innovation is obscured in a noisy process.

19

How Long Does It Take?

For organizations that began theirCMM-based SPI effort in 1992 or later,the median time to move from:

maturity level 1 to 2 was 22 months maturity level 2 to 3 was 19 months maturity level 3 to 4 was 25 months maturity level 4 to 5 was 13 months

20

Why Does It Take So Long? (1/2) Training

Staff need to learn how to assess and change the process Management needs to learn how to support process

assessment and change Technical staff need to appreciate need for process

assessment and change

Assessment Process Collection of data Analysis of results

21

Why Does It Take So Long? (2/2) Changing the Process

Train staff Establish goals Measure Analyze Act on recommendations