Software Acquisition Management SAM-450 EXEC SAM (1) George Prosnik DAU CDSC E&T Center Software...

Software Acquisition ManagementSAM-450 EXEC SAM (1)

George ProsnikDAU CDSC E&T Center

Software Management One Big Dilemma?

SAM Executive Seminar


SeminarOutline

Day 1 Day 2

8:30-9 a.m. Welcome & Overview

9-9:30 a.m.

9:30-10 a.mSoftware

"Maintenance"

10-10:30 a.m.Software Testing

Issues

10:30-11 a.mSoftware Quality

Factors

11-11:30 a.m.

11:30-12 noon

1-1:30 p.m.

1:30-2 p.m

2-2:30 p.m.

2:30-3 p.m

3-3:30 p.m.

3:30-4 p.m

4-4:30 p.m. Session Wrap-Up

12:00 to

1:00 p.m.

CMMI Guest Instructor

Defence Industry Guest Speaker

Spiral Models and Paradigms

LUNCH

Safety & Airworthiness Guest Speakers

Cost & Schedule Estimation Issues

Best Practices & "Lessons Learned"

Software Management: One Big Dilemma?

Acquisition Strategies

Software Measurement (Guest Instr)

LUNCH

COTS-based System Issues

Software Risk Management


EPMC“Slice”

EPMC

8:30-9 a.m. Welcome & Overview

9-9:30 a.m.

9:30-10 a.mSoftware

"Maintenance"

10-10:30 a.m.Software Testing

Issues

10:30-11 a.mSoftware Quality

Factors

11-11:30 a.m.

11:30-12 noon

1-1:30 p.m.

1:30-2 p.m

2-2:30 p.m.

2:30-3 p.m

3-3:30 p.m.

3:30-4 p.m

4-4:30 p.m. Session Wrap-Up

12:00 to

1:00 p.m.

CMM &

CMMI

Defence Industry Guest Speaker

Spiral Modelsand Paradigms

LUNCH

Safety & Airworthiness Guest Speakers

Cost & Schedule Estimation Issues

Best Practices &

Lessons Learned

Software Management:

One Big Dilemma?

Acquisition Strategies

Software Measurement

LUNCH

COTS-based System Issues

Software Risk Management


2.1.2 Software Disaster Defined*

In his book, Death March: The Complete Software Developer’s Guide to Surviving “Mission Impossible” Projects, software guru, Ed Yourdon, defines a software disaster, death march program as the follows.

“A death march project is one for which an unbiased, objective risk assessment (which includes an assessment of technical risks, legal risks, political risks, etc.) determines the likelihood of failure is >50%.”

Regrettably, Yourdon concludes that, “Death march projects are the norm, not the exception.” He further categorizes death march programs as “mission impossible,” “ugly,” “suicidal,” and “Kamikaze.”

* GSAM Version 3


2.1.2 Software Disaster Defined*

Stephen Flowers explains in his book, Software Failure: Management Failure, that there are varying degrees of software acquisition failure, including “flawed but usable,” “totally unusable,” “unused,” and “absolute disaster.” He says a program can be called a failure if it falls into the one (or more) of the following categories:

• Never used - On implementation, it does not perform as originally intended, or it is so user hostile it is rejected by users.• Cost exceeds benefits - The cost of development exceeds any benefits the system may bring during its useful life.• Not completed - Due to problems with system complexity, program management, or program longevity (where the development is no longer relevant due to advances in technology), the system is abandoned before it is completed.

* GSAM Version 3


Why Worry About Software?

Software is usually on the critical path Most of today’s systems key functionality has been allocated to the software Once software gets late, adding more resources typically makes it later (Brooks Law) It is a complex, conformable, changeable and invisible product with unique characteristics Cost and schedule overruns are very common for Software-Intensive Systems

Rev 3.1

Software Acquisition ManagementSAM-450 EXEC SAM (7)Rev 1.8

Software Impacts on Military Systems

0

20

40

60

80

100

1960 1964 1970 1975 1982 1990 2000

Percent Functions

Performed in Software

F-22

B-2

F-16

F-15

F-111A-7F-4


Software Productivity vs. “Size”*

0

2

4

6

8

10

12

14

16

10 40 160 640 2560 10240Software Size in Function Points

* Capers Jones, Becoming Best in Class, Software Productivity Research, 1995 briefing

FunctionPointsPerPersonMonth

Rev 1.7

Economies of Scale Don’t Seem to Apply!


Why Software is “hard” *

• Essential Differences

• RAM Distinctions

• Inherent Complexity

*Derived from Frederick Brooks, “No Silver Bullet: Essence and Accidents of Software Engineering”

• Complexity: no two parts alike• Conformity: conformance to external environment• Changeability: perception of infinite malleability• Invisibility: product is inherently unvisualizable

• Non-linear scaling rules• No uniform “size” metric • Impact of Brooks’ Law • No bounding natural laws


Software RAM Distinctions*

Reliability• Failures impact all systems• Failures are deterministic• Generally improves with age• Many competing reliability models with differing assumptions and low acceptance

Availability• Zero manufacturing costs• “Manufacturing” (copying) is defect-free• Many valid definitions of quality• Instantaneous failures• Redundancy and fault tolerance rare

Maintainability• Maintenance costs (less installation) are independent of number of systems fielded• Repair changes fielded product baseline• Standard software parts rare• Typically no salvage value• No Preventive Maintenance required

* Derived From: ANSI/AIAA Standard R-013-1992, Recommend Practice for Software Reliability


Reasons Why DoD Software Programs Get Into Trouble*

• Poor Requirements Definition– Lack of User Involvement in Development Process

– Inability of Users to Foresee Benefits Without Incremental Capability

• Inadequate Software Management & Control By Contractors– Failure to Establish “Team” with Vendors and Users

– Little Participation of Functional Area Experts

• Ineffective Subcontractor Management

• Lack of Consistent Attention to Software Process

• Software Architectures Ignored

• Poorly Defined and Controlled Interfaces (HW, Comm, Software)

• Assumption That Software Upgrades Can “Fix” Hardware Deficiencies (Without Assessment of Cost and Schedule Risks)

• Focus on Innovation Rather than Cost and Risk

• Limited or No Tailoring of Specifications Based on Continuing Cost-Benefit Evaluations

* Source: DSB Task Force on Acquiring Defense Software Commercially


Tri-service Assessment Initiative - Findings from Software Assessments

21 Programs 8 Army 9 Navy 2 Air Force 2 Other

~ 450 Findings

Management: Acq. Strategy Project Planning Program Management Contracting Communication

Tech Product: Product Line Product Requirements Quality Product Risk

0 50 100 150 200 250 300 350

1. Environment

2. Mission Requirements

3. Financial

4. Resources

5. Management

6. Technical Process

7. Technical Product

8. Schedule

9. User/Customer


Some Perspectives“Many previous studies have provided an abundance of valid conclusions and detailed recommendations. Most remain unimplemented. If the military software problem is real, it is not perceived as urgent.”

Report of the Defense Science BoardTask Force on Military Software, July 1987

• “Annual embedded software costs are $24-$32 billion and expected to be 20% of the DoD budget by 2008.” [GAO IMTEC-92-62BR]

• New major systems currently being developed depend on the successful development of millions of lines of software. However, to date, development of large amounts of software supporting complex systems has been riddled with significant problems [GAO IMTEC-93-13]

• A review of OT&E reports shows that…85% of software-intensive systems tested were immature, ineffective in a threat environment or difficult to maintain in field operations. [GAO NSAID-93-198]

• Do something already!! [Defense Science Board November 2000]


• “Software continues to grow in importance in our weapons systems; however, problems attributed to software remain a significant contributor to program cost, schedule and performance shortfalls.” E.C. Aldridge

• “If I were to select the most critical software need today, it is in the software tools and management techniques.“ J. Gansler

• “Software continues to grow in importance in our weapons systems; however, problems attributed to software remain a significant contributor to program cost, schedule and performance shortfalls.” E.C. Aldridge

• “If I were to select the most critical software need today, it is in the software tools and management techniques.“ J. Gansler

US DoD AT&L Perspectives


…..You name it and I can point to software problems. We are putting major, major efforts into developing our ability to manage software projects and we are now insisting that all of our managers have the same ability to as least assess software risk as they do engineering risk.

It was interesting that the year before I as at Edwards Air Force Base in the US looking at the joint fighter contenders and there was a group of about 20 people standing around these two aircraft, very state-of-the-art, fifth generation aircraft, and there was a discussion of the developmental problems that had gone on. Virtually everyone in that group, there were economists and there were fighter pilots and there were acquirers and logisticians and so on, everyone understood that debate that was going on about issues with hydraulic systems and so on and yet when it came to discussions about the software, and these aircraft are essentially software aircraft, the entire group switched off….

Speech by Mick Roche to Joint Logistics Organization


Typical Software Costs & Schedules*

COTSReleases

Small Commercial

COTS Products

ComplexCommercial

Products

Custom CommercialBusiness Systems

Custom DoDBusiness Systems

Commercial Engineering

Systems

DoDEngineering

Systems

Custom Commercial

Real-Time SystemsSmall DoD

Embedded Real-TimeSystems

Large DoD Embedded Real-Time Systems

TI

ME

TO

DEVELOP

C O S T TO D E V E L O P*Chart is based on the Report of the Defense Science Board Task Force on Acquiring Defense Software Commercially, July 1994

WHY?

Rev 1.1


Commercial Industry’s Software Acquisition Management “Track Record”*

*Source: STANDISH Group CHAOS Report (1994)

Over-budget and late (58%)

Completed OK (16%)

ProjectCancelled (31%)

Rev 3.0

“Over half of theseprojects have costoverruns of more than 189% and schedule overruns of 222%”


Resource Allocation Differences*

• Engineering 30% 50%

• Evaluation 20% 20%

• Management 15% 10%

• Meeting Support 15% 5%

• Documentation 15% 7%

• Customer/User Support 5% 8%

DoD Software

Commercial Software

*Sources: Magnavox, Capers Jones (Assessment and Control of Software Risks) and DSB Report: Acquiring Defense Software Commercially, USD (A&T)


The “Software Crises”: The Good News and The Bad News*• Since the 1960s there has been Continuous Wailing and Gnashing of Teeth about the “Software Crisis”• In the 1968 the concept of “Software Engineering” was introduced as the remedy to the “Software Crisis”.• But...articles in the literature are continuing to report that there are no “Silver Bullets” for the “Software Crisis”?????

The Good News: For most software efforts today there is no “Software Crisis”. These efforts aren’t pushing the technical envelope: they are covering conceptually simple ground in a technically straight-forward fashion.

The Bad News: For software developments today there is a “management of software development crisis”. Time after time after time conceptually simple, straight-forward efforts are late and over-run!

*From a General Dynamics VP’s briefing to an ARES class


Commercial Industry’s Software Acquisition Management “Track Record”: 6 Years Later*

*Source: CHAOS Report 2000 UpdateRev 2.0

Over-budget and late (49%)

Completed OK (28%)

ProjectCancelled (23%)

“Over half of theseprojects have costoverruns of more than 45% and schedule overruns of 63%”

WHY? “Microproject Management….Better User Participation…Executive Consensus-buildingWHY? “Microproject Management….Better User Participation…Executive Consensus-building

Software Acquisition Management SAM-450 EXEC SAM (1) George Prosnik DAU CDSC E&T Center Software...

Documents

Transcript of Software Acquisition Management SAM-450 EXEC SAM (1) George Prosnik DAU CDSC E&T Center Software...