Copyright QSM Associates, Inc. All Rights Reserved 1 Michael C. Mah Managing Partner QSM Associates,...

1Copyright QSM Associates, Inc. All Rights Reserved

Michael C. MahManaging Partner

QSM Associates, Inc.75 South Church Street

Pittsfield, MA 01201413-499-0988

Fax 413-447-7322e-mail: [email protected] Site: www.QSMA.com

Boston SPINOctober 19, 1999

Software Projects in CrisisDealing with Dynamics of Imposed Deadlines


Industry Overrun Statistics

$250 Billion + Spent on IT Application Development

31% of Projects Will Be Cancelled, Representing $81 Billion in Losses

52.7% of Projects Will Overrun by >189%

Only 16.2% On-time,Under Budget

But with only 42% Original Functionality!

* Source: Standish Group, Dennis MA, 1995


Other Noteworthy Trends

BILLION-DOLLAR OUTSOURCING DEALS DOUBLEIDC estimates that the number of large outsourcing contracts rose 100% between 1997 and 1998. (Source: *Computerworld*; www.idg.net/go.cgi?id=125306)

75% of organizations surveyed have backlogs running from seven months to over two years.

(Source: Chris Pickering's 1998 *Survey of Advanced Technology*;www.cutter.com/itgroup/reports/sat98.html)


Worldwide Software Productivity Trend

The Good News: Over the last decade, productivity has increased fairly consistently. 10% faster speed, 25% less cost, about every 2.5 years.

The Bad News: It’s Not Enough. Demand continues to outstrip capacity.

Companies are reporting growing backlogs ranging from 7 months to over 2 years.

The Badder News: Continued labor shortages. The More Badder News: Don’t expect any

schedule relief in an “Internet Speed” economy.


Reality Bites

“We don’t have the luxury of determining our schedules. They’re told to us. Then, given the time frame, we try to tell the client what we can build. In the end, we usually wind up working lots of overtime, because they want everything.”

- Manager of Development Wall St. Financial Firm


Anyone Know What This Is?


QSM Mixed Application Data Base

0.1

1

10

100

1000

100 1000 10000 100000 1000000 10000000

Effective Source Lines of Code

Calandar

Months

Real Time

Engineering

Info Systems

Impossible Zone

Are Deadlines/Plans in the “Impossible Zone”?


Good to Know...

"As impressive as growth of the software industry has been, it is outpaced by growth of software-related litigation. It is not unusual for a large software development organization today to have upwards of 50 active cases on its hands."

Tom DeMarco, Cutter Consortium Bulletin, July 16, 1998


Two “Old School” Waysof Estimation

Level of Effort, “Bottoms Up” Activity Based Method

Estimate and Add Up Person Months Force into Deadline

“Productivity Ratio” Based Method Divide Size into “Productivity” Calculate Effort Force Into Schedule


Example # 1Level of Effort

“Here’s how we get an estimate. First, we look at each requirement. We estimate the person hours for the work based on complexity. We have an Excel spreadsheet with a table for the person hours needed to make a simple, modest, or complex change, on a simple, modest, or complex program. So, we get something like 20 person hours for high level design, 32 person hours for detailed design, another 40 for coding and unit testing for a given requirement.”

“That comes out to 112 person hours. We multiply that by 30% to get the integration. That makes 145.”


Example # 1Level of Effort

“We add all the person hours up for the entire project. The grand total might add up to 34,000 person hours, or 200 person months. If we have a 10 month deadline, then it means we need to assign 20 people to the project.”

“If the schedule gets shortened on us to 6 months vs 10, then dividing 200 person months into a 6 month schedule means we need 33 people instead of 20.”


Example # 2“Productivity Ratio”

“We ascertained that this project will be 800 total new + modified Function Points. Our productivity is 10 FP/staff month. So this project will take 80 staff months. We have 8 people on the team. So it will take 10 months.”

“If the schedule gets shortened on us to 6 months vs 10, then dividing 80 staff months into a 6 month schedule means we need 13 people instead of 8.”


Violation of Brook’s Law

Manpower and Time are not interchangeable. That is, you can not cut schedule in half by doubling the number of staff.

- Fred Brooks, The Mythical Manmonth

(Corollary: You can not get a baby in 1 month with 9 women.)


Laws of Cause and Effect

The fundamental, or core metrics of time, cost, and quality, across small, medium, and large projects (size) are interdependent.

We, as managers and developers, have treated them as independent variables.

For example, we want to believe that we can keep a deadline fixed in spite of size growth. The think the effort is the effort, and reliability should not be impacted much (we hope).

Not only are they interdependent, but data has revealed to us that changes in one dimension can result in geometric impacts in another dimension


How DevelopmentLifecycles Behave

A Pop Quiz:

“If you tried to shorten the schedule on an application development project by adding staffto say, double (20 people versus 10), how muchwill you able to compress it? Will defects go upor down, and by how much?”


How SoftwareLifecycles Behave

Answer:

Schedules will only compress (nominally) byabout 20 percent.

Defects typically rise by about 6 fold.*

Rule of Thumb: “20/200/6x”

*Source: QSM Industry Database Statistics


When Time & Effort are “Fixed” Something’s Got to Give...

SizeTime

Effort

Defects


“Feature Creep” - Impact on Quality with Fixed Schedule

Time Profile

11

12

13

14

15

16

17

18

Mo

nth

s

1 2 3 4 5Solutions

FOC MTTD Profile

1.5

2.0

2.5

3.0

3.5

4.0

4.5

Day

s

1 2 3 4 5Solutions

Size Profile

60

70

80

90

100

110

ES

LOC

(thou

san

ds)

1 2 3 4 5Solutions

Solution 4

TimeEffortUinf CstPk StaffMTTDSize

14.06116.40

106712.922.15

87000

MonthsPM$ 1000PeopleDaysESLOC

100% Prob

38% Prob100% Prob 1% Prob

PI 17.0

Schedule Fixed at 14 Months

Size Increases in 5KSLOCIncrements from 72K to 92K

Quality Levels Cut in Half - MTTD 3.5 Days to 1.75 Days


Size

Probabilityon

Time

Scenario: Size Growth,or “Feature Creep”


Compress Schedule: Plan 1

uses 9 People

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7 8 9 10 11 12 13

Staffing 9 People 69 Staff Months MTTD = 3.2

Days

Elapsed Time - Months


Compress Schedule: Plan 2 uses 24 People

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7 8 9 10 11 12 13

Staffing

Elapsed Time - Months

24 People 144 Staff Months MTTD = 1.2 Days


Dangerous Productivity Metrics: Only 2 Dimensions

Lines of Code

Staff Month

Function Points

Staff Month


Summary: Both Ratios Are Equally Poor Measures

Size = 75,000 New and Modified SLOC 707 FP

PeakStaff

Sched(Mos) FP/PM

69

14243366

13.612.311.310.29.58.3

15.410.27.34.93.82.1

LOC/PM

16301086 773 520 398 225

LOC/DAY

254281306339364417

Effort(PM)

46 69 97144188333


SEI Special ReportCMU/SEI-95-SR-005

1) A Corporate Memory (Historical Database)

2) Structured Processes for Estimating Size and Reuse

3) Mechanisms for Extrapolating from Demonstrated Accomplishments on Past Projects


4) Audit Trails (Values for Cost Model Parameters Used to Produce Each Estimate Are Recorded and Explained)

5) Integrity in Dealing with Dictated Costs and Schedules

6) Data Collection and Feedback Processes That Foster Capturing and Correctly Interpreting Data from Work Performed

SEI Special ReportCMU/SEI-95-SR-005


The Role of Measurement, and Commitments

Estimation& Planning

Control&

Forecasting

Support FutureCommitments

ManageCommitmentCommitment

Analyze Performance on Commitment

HistoryRepository

Assess Viable Strategies

Monitor Status & ReplanPost Project Analysis

Make Commitment


Mapping to An “Efficiency”, or “Productivity Index”

Productivity Index

Functionality(Size)

Effort Time

~ Goodness(Quality/Defects)


Example PI CalculationSize = 375 Function

Points 35,000 LOC

Effort = 24 Person-Months

Time = 6 Months

*PI = SIZE

TIME EFFORT= 17

Project Staffing Profile

23

5 5 5

4

0

1

23

4

5

6

Jan-97 Feb-97 Mar-97 Apr-97 May-97 Jun-97


Productivity IndexIndustry Baselines - 1997

Category PI STD Dev

Business 17.3 +/- 4.1

System Software 13.7 +/- 4.9

Telecom 12.2 +/- 4.0

Scientific 12.1 +/- 3.5

Process Control 12.1 +/- 3.4

Command & Ctrl 11.3 +/- 4.3

Avionic 8.2 +/- 4.8 Real-time 7.8 +/-

3.8 Microcode 6.3 +/-

2.8

FinanceRetailInsuranceOthers


Given a Certain Development Efficiency (PI),

And Given the Deadline With a Team of “X” People ...

“Real World Estimation”

How Much Functionality Can We Build?

How Much Functionality Should We Promise?


Fitting Trendlines to DataQ SM M ixed Application Data Base


Calandar

Months

0.1

1

10

100

1000

100 1000 10000 100000 1000000 10000000

A v era g e

1 S ig m a L ess T im eT o p 1 6 % o f th e d a ta b a se

1 S ig m a M o r e T im eB o tto m 1 6 % o f th e d a ta b a se


Positioning Projects Vs.Industry Trends - Example

Q SM M ixed Application Data Base


Calandar

Months

0.1

1

10

100

1000

100 1000 10000 100000 1000000 10000000

C ompany XYZProject D ata

P ro je c ts P o s itio n R e la tiv e to In d u s try T re n d sU s in g th e A p p ro p ria te S iz e a n d A p p lic a tio n T y p e

M ain Build - ScheduleSystem Software D ata


In Summary...

Productivity is Multi-Dimensional. 2-Dimensional Measures Are Inadequate

Visual Explanations Are Vital

The Need to Quantify Size of New, Modified, and Total Delivered Functionality Will Arise

No One Size Measure is a “Silver Bullet”


In Summary...

Metrics of Schedule, Cost, and Quality are Interdependent.

Productivity Metrics in the Form of Ratios Are Dangerous. They Omit Time. Time = Deadline. Deadline is the Independent Variable, Upon Which the Dependent Variables Behave.


Establishing a Software Management Program:Lessons Learned

Management Awareness and Acuity are Essential

A Metrics “Champion” is Essential

Collect Core Metrics; “Know Your Capability”

Dull Metrics Means You’ve Got the Wrong Metrics

Turn Numbers Into Pictures, Show the Right

Dimensions, Reveal What is Hidden


RecommendedReading

Carleton, Anita, Park, Robert, and Goethert, Wolfhart , “The SEI Core Measures: Background Information and Recommendations for Use and Implementation” © 1994 The Journal of the Quality Assurance Institute.

Mah, Michael C., and Putnam, Lawrence H., “Softwareby the Numbers: An Aerial View of the SoftwareMetrics Landscape” © 1997 American Programmer.

Putnam, Lawrence H., and Myers, Ware, “Executive Briefing: Controlling Software Development” © 1996 IEEE Computer Society Press.

Tufte, Edward, “Visual Explanations, Images and Quantities,Evidence and Narrative” © 1997 Graphics Press.

Copyright QSM Associates, Inc. All Rights Reserved 1 Michael C. Mah Managing Partner QSM Associates,...

Documents

Transcript of Copyright QSM Associates, Inc. All Rights Reserved 1 Michael C. Mah Managing Partner QSM Associates,...