Software Acquisition Management Guidelines

SOFTWARE ACQUISITION MANAGEMENT GUIDELINES

by

Daniel Svennberg

A Thesis submitted for the Degree of

Master of Science

Linköping, Sweden, 2001

The Department of Computer and Information Science

Linköping University

LiTH-IDA-Ex-01/32

Lars Ljungberg, Mats Ran

Advisors, Q-Labs AB Kristian Sandahl

Examiner, Linköping University

Copyright 2001 Daniel Svennberg

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means including, without limitation, photocopying, recording, or oth-erwise without the prior written permission of the author. Written per-mission is not needed if this publication is distributed for non-commercial purposes.

Linköpings Universitet, Institutionen för Datavetenskap

Q-Labs, Linköping and Maryland

Fraunhofer, Maryland, Center for Experimental Software Engineering

Software Acquisition Management Guidelines

Riktlinjer för ledning av programvaruupphandling

LiTH-IDA-Ex-01/32

Linköping University

Q-Labs AB

Fraunhofer, Inc.

i

Abstract

The thesis provides guidelines from the viewpoint of the cus-tomer for managing an acquisition project concerning an out-sourced development of a software product customized for the specific needs of the customer. Current frameworks for soft-ware acquisition management such as SA-CMM, IEEE 1062, ISO 12207, etc. as well as literature, articles, and interviews has been used as a basis for formulating the guidelines.

The resulting guidelines include acquisition team roles descrip-tions, checklists for artifacts, customizable processes, and cus-tomization factors. The processes contain guidelines for the fol-lowing issues: project steering, project management, planning and organizing, acquisition training, requirements manage-ment, risk management, contractor selection, contract manage-ment, product evaluation, transition to support, and post par-tum.

The guidelines can be used as a basis for planning an acquisi-tion project or be read as an overview over the many issues that concern software acquisitions.

ii

Acknowledgements

Many thanks to (in alphabetical order)…

Fraunhofer Institute, Jaak Urmi, John Marciniak, Kristian San-dahl, Lars Ljungberg, Linköping University, Mats Ran, Mikael Lindvall, Q-Labs, Victor R. Basili, Winifred Menezes, Östen Os-karsson, and everybody else

… for your help making this thesis possible.

D.S.

iii

Table of Contents

Abstract.................................................................................i

Acknowledgements ............................................................ii

1 Introduction ...............................................................1

1.1 Background.............................................................................1 1.2 Target audience......................................................................2 1.3 Objective .................................................................................2 1.4 Problems .................................................................................2 1.5 Scope........................................................................................3 1.6 Approach.................................................................................3 1.7 Outline.....................................................................................4

2 Software Acquisition Environment .........................6

2.1 The nature of software .........................................................6 2.2 Software acquisition.............................................................7 2.3 Software development .........................................................8 2.4 Outsourcing............................................................................9 2.5 Common problems .............................................................13 2.6 Frameworks ..........................................................................16

3 Customization .........................................................20

3.1 Customization levels ..........................................................20 3.2 Customization factors.........................................................21 3.2.1 Product characteristics............................................................22 3.2.2 Effort........................................................................................24 3.2.3 Contracting environment .......................................................26 3.2.4 Other factors ...........................................................................28

Table of Contents

iv Software Acquisition Management Guidelines

4 Roles ........................................................................29

4.1 Sponsor roles........................................................................29 4.1.1 Sponsor....................................................................................29 4.1.2 Collaborating customer...........................................................30 4.1.3 Product manager.....................................................................30 4.2 Managing roles ....................................................................30 4.2.1 Acquisition manager...............................................................30 4.2.2 Technical manager ..................................................................32 4.2.3 Contract manager ...................................................................32 4.2.4 Administrator .........................................................................32 4.3 Assisting roles......................................................................33 4.3.1 Acquisition expert...................................................................33 4.3.2 Technical expert ......................................................................33 4.3.3 Domain expert.........................................................................33 4.3.4 Legal expert .............................................................................34 4.3.5 Translator................................................................................34 4.3.6 End user ..................................................................................34 4.3.7 Maintenance and support staff ...............................................34 4.3.8 Independent verification and validation .................................35 4.3.9 System engineering and technical assistance .........................35 4.4 Executing roles.....................................................................36 4.4.1 Contractor ...............................................................................36 4.4.2 Collaborating contractor .........................................................37 4.4.3 Subcontractor..........................................................................37 4.4.4 Supplier ...................................................................................37

5 Artifacts ...................................................................38

5.1 Inception ...............................................................................39 5.1.1 Acquisition proposal ...............................................................39 5.1.2 Project specification ................................................................40 5.1.3 Acquisition plan......................................................................41 5.1.4 Risk list....................................................................................43 5.1.5 Requirements specification .....................................................44 5.1.6 Maintenance plan ...................................................................46 5.2 Solicitation............................................................................47 5.2.1 Request for proposal ................................................................47 5.2.2 Contractor evaluation criteria ................................................49 5.2.3 Proposal...................................................................................50 5.2.4 Contract...................................................................................51 5.3 Monitoring............................................................................53

Table of Contents

© Daniel Svennberg 2001 v

5.3.1 Artifacts from the contractor ..................................................53 5.3.2 Product evaluation plan..........................................................54 5.3.3 Product evaluation report .......................................................55 5.3.4 Change request........................................................................55 5.3.5 Project status report................................................................56 5.4 Finalization...........................................................................57 5.4.1 Deliverables .............................................................................57 5.4.2 Post partum plan.....................................................................57 5.4.3 Post partum.............................................................................58

6 Processes................................................................60

6.1 Overview...............................................................................60 6.2 Project steering ....................................................................66 6.3 Project management ...........................................................69 6.4 Planning and organizing ...................................................72 6.5 Acquisition training ...........................................................74 6.6 Requirements management ..............................................76 6.7 Risk management................................................................78 6.7.1 Risk identification ...................................................................80 6.8 Contractor selection ............................................................85 6.8.1 Contract types .........................................................................88 6.9 Contract management.........................................................96 6.9.1 Project status metrics............................................................100 6.9.2 Earned value..........................................................................102 6.10 Product evaluation ............................................................106 6.10.1 Product evaluation approaches .............................................109 6.11 Transition to support........................................................110 6.12 Post partum.........................................................................113

7 Results and summary ..........................................116

7.1 Guidelines summary ........................................................116 7.2 Outlook ...............................................................................122

8 Bibliography..........................................................124

A Frameworks coverage..........................................129

A.1 SA-CMM.............................................................................129 A.2 FAA-iCMM.........................................................................130 A.3 IEEE 1062.............................................................................131 A.4 ISO 9000-3 ...........................................................................132

Table of Contents

vi Software Acquisition Management Guidelines

A.5 ISO 12207 ............................................................................133 A.6 ISO 15504 ............................................................................134 A.7 Euromethod ........................................................................135

Index.......................................................................137

1

1 Introduction

“Much of present-day software acquisition procedures rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it

built, and install it. I think this assumption is fundamentally wrong, and that many software acquisition

problems spring from that fallacy.”

– Frederick P. Brooks, Jr.

This chapter states the background of the thesis, its objective, the approach, and outlines the contents of the report.

1.1 Background Many organizations have a need to acquire customized soft-ware products that meet special demands that no commercial off-the-shelf product can meet. Many times the most efficient and economically viable solution to obtain the software is to outsource the development to a contractor instead of develop-ing the software in-house.

In order to manage such a software acquisition project success-fully, many different non-trivial issues such as requirements elicitation and management, risk analysis and management, contractor selection, contract management, product evaluation, etc. need to be addressed. Several frameworks and standards have been developed for assisting and assessing such software acquisition efforts.

1 Introduction

2 Software Acquisition Management Guidelines

The purpose of this thesis is to gather practices from these frameworks and other sources and to present them as guide-lines for software acquisition project managers. The thesis along with the diploma thesis (Assmann, 2000) of a German student, Danilo Assmann, are parts of an international project on soft-ware acquisition between Linköping University in Sweden, Q-Labs in Sweden and USA, and the Fraunhofer Institute in Germany and USA. The thesis by Assmann deals mainly with the evaluation and selection of a contractor, while this thesis deals mainly with the management of an acquisition project.

It isn’t possible for a single master’s thesis to cover this subject in all its intricacies, therefore the guidelines should be consid-ered as an introduction to software acquisition management, and hopefully as an inspiration on further studies on how to manage software acquisition projects.

1.2 Target audience The primary audience group for this master’s thesis consists of those who are assigned the task of managing a software acqui-sition project. Naturally, all groups that are involved in soft-ware acquisition projects may find this thesis interesting.

1.3 Objective The objective of this master’s thesis is to provide guidelines from the viewpoint of the customer for managing an acquisition project concerning an outsourced development of a software product customized for the specific needs of the customer.

1.4 Problems The objective stated above can be broken down into seeking an-swers to the following problems:

• What problems are common?

1 Introduction

© Daniel Svennberg 2001 3

• What activities should be performed?

• What roles are conceivable?

• What artifacts are conceivable?

• How can the activities, roles, and artifacts be customized for different kinds of projects?

1.5 Scope The management of custom-made software product acquisi-tions will be examined – not services. Fully or partially devel-oped products are the main concern – not commercial off-the-shelf products. The acquirer’s point of view is mainly ad-dressed – not the contractor’s. Legal aspects will not be exam-ined further than contract contents. The issues will be examined from a business perspective instead of a governmental or mili-tary perspective.

1.6 Approach An introductory study of the subject was performed by reading the thesis of the German student (Assmann, 2000), the different frameworks listed in section 2.6, Frameworks, on page 16, web pages, and articles on software acquisition, and performing in-terviews with Jaak Urmi, Östen Oskarsson, and Mats Ran. This gave an understanding of the concepts of software acquisition management and a number of ideas on how to formulate the guidelines and provide answers to the problems stated above.

I found that the different sources gave quite a coherent picture of software acquisition management without many contradic-tions. The sources contained mainly different descriptions of the same issues. Like Assmann (2000) does in his thesis, I as-sume that the frameworks are useful in their own context and therefore are viable as input to the guidelines. Furthermore my goal was to extract and assemble the lion’s share of the advice

1 Introduction


given in the main frameworks SW-CMM, SA-CMM, FAA-iCMM, IEEE 1062, ISO 9000, ISO 12207, ISO 15504, and Eu-romethod, with as little filtering as possible, and then augment with advice from the other sources studied. Naturally, I had to integrate and arrange the advice into a suitable and coherent form for the purpose of the guidelines.

The ideas on how to describe the guidelines were presented and discussed with several people, including Kristian Sandahl, Lars Ljungberg, Danilo Assmann, John Marciniak, Jon Valett, Winifred Menezes, Mikael Lindvall, and others. These discus-sions motivated me to write the guidelines in their present form.

The guidelines described in this thesis are not a new model for software acquisition management, but an attempt to extract, as-semble, and integrate advice from the existing models and frameworks and present them as management processes, check-lists for artifacts, descriptions of roles, and customization fac-tors. The option would have been to describe the guidelines in a more prosaic fashion, but that would have made the sugges-tions for customization less logical and to the point.

1.7 Outline The following is a description of the contents of each chapter in the report:

Chapter 1, Introduction, on page 1 describes the background of the thesis, its objective, and approach.

Chapter 2, Software Acquisition Environment, on page 6 in-vestigates the nature of software, different types of software ac-quisitions, software development, outsourcing issues, problems with software projects, and software acquisition frameworks.

Chapter 3, Customization, on page 20 describes a number of factors that can be used as a guideline on what level of formal-ity is appropriate for the actual software acquisition project.

1 Introduction


Chapter 4, Roles, on page 29 describes the different roles that are conceivable for a software acquisition project.

Chapter 5, Artifacts, on page 38 contains checklists for the dif-ferent types of artifacts that are conceivable for a software ac-quisition project.

Chapter 6, Processes, on page 60 contains suggestions on cus-tomized software acquisition management processes.

Chapter 7, Results and summary, on page 116 summarizes the guidelines and contains an outlook on continued work.

6

2 Software Acquisition Environment

“A typical software project can present more opportunities to learn from mistakes than some people get in a lifetime.”

– Steve McConnell

This chapter introduces the concept of software acquisition, its nature, environment, and problems. It also lists some frame-works that deal with software acquisition management.

2.1 The nature of software There are a number of inherent characteristics of software that affect the acquisition and development of customized software in a profound way. The most salient of these characteristics, in my opinion, are the following:

Software is an almost infinitely malleable conceptual con-struct. As a consequence this makes it difficult to settle the requirements and makes the scope of the software prone to change.

Software has no physical features and is therefore difficult to visualize. Brooks (1995) tells us that this makes it hard to design the software and to communicate the design of the software, which in turn makes it difficult to estimate how long it will take to develop the software. It also



makes it difficult to see progress in the development of the software.

Software is extremely cheap to manufacture, as Reeves (1992) argues. All it takes is to transform the source code into executable code and clone as many copies as you wish. The actual costs for software lie in development and maintenance.

Software is extremely complex. This is, according to Brooks (1995), due to the fact that software has a very large number of possible states, which increases exponen-tially with size. This makes it very hard to get an over-view of the software, which in turn makes it hard to de-sign the software to fully meet all quality requirements and to have full control over the possible errors.

2.2 Software acquisition There are several different ways to acquire software. According to IEEE 1062 (1998), software products can be classified accord-ing to the degree to which the acquirer can specify the features of the software.

Commercial-off-the-shelf software, or COTS, is commercially available software. It is usually well defined and documented and a wide range of commercial users has demonstrated its fit-ness for use. The supplier isn’t normally willing to modify the software for the needs of a specific customer, and also controls the maintenance of the software. The cost to acquire the soft-ware is low to medium and the delivery time is immediate. The customer has no control of the quality characteristics of the software.

Modified-off-the-shelf software, or MOTS, is similar to COTS software with the difference that the supplier is willing to make modifications to the software product according to the cus-tomer’s requirements. The software product’s fitness for use has been demonstrated in similar applications. The customer



has some control of the maintenance of the product and its quality characteristics, i.e. the customized part. Delivery time is short to long and the cost to acquire the software is medium to high.

Fully developed software are unique, one-of-a-kind or low-volume applications fully customized for the specific needs of the customer. The software product’s fitness for use is unprece-dented, but the customer has full control over its maintenance and quality characteristics. The cost to acquire fully developed software is high and the delivery time is long.

The characteristics of these product classes are summarized in the table below:

Table 1. Software acquisition types. FD stands for fully developed software.

This thesis applies to MOTS and especially fully developed software products.

2.3 Software development To be brief, software development essentially consists of four activities: analysis, design, coding, and testing. There are a number of methods that describe how and when these activities should be performed, ranging from code and fix, which is the software development version of trial and error, to perhaps the

COTS MOTS FD*

Scope Fix Partially customizable

Fullycustomizable

Fitness-for-use Demonstrated Demonstrated in similar applications Unprecedented

Maintenance No control Some control Full control

Delivery time Immediate Short-Long Long

Acquisition cost (Not usage/maint. cost) Low-Medium Medium-High High

Quality (ISO 9126) Not controllable Partially controllable Mostly controllable

*) Partially to fully outsourced



most well known method, the waterfall method, which in its purest form is to perform the activities in the following sequen-tial manner: analyze, design, code, test, and never go back to a previous step.

More recently, iterative, evolutionary, or incremental develop-ment methods such as eXtreme Programming (Beck, 1999) have become popular. Following such a method means that the pro-gram is developed in a series of iterations or increments with a subset of the requirements implemented in each iteration; this has the benefit that you can more easily change your mind about the requirements as the program evolves. See the figure below for a comparison between the waterfall model, the spiral model and extreme programming.

Different software development methods suit different types of projects, but that is not for this report to examine. What is im-portant is that the contractor can produce good results with the method they use.

Figure 1. The development cycles of the waterfall model, the spiral model and extreme programming (Beck, 1999).

2.4 Outsourcing There are many different reasons for outsourcing a software development project, such as the following examples:

Analysis

Design

Coding

Testing

Waterfall Spiral XP

Scope



• It’s difficult to hire developers.

• It’s cheaper than hiring developers.

• Somebody else can develop the program faster, cheaper, and with higher quality than you can.

• You lack the necessary competence and resources.

• You want to use your staff for other tasks.

The following are examples of drawbacks with outsourcing:

• You lose some control over the software development process, which implies that the risk is higher; possibly you risk putting the fate of your competitiveness in the hands of the contractor.

• You don’t build up your own software development competence.

• You don’t get any indirect labor hours since you don’t have your own development staff.

Haimes, et al. (1997) states that there are three groups of partici-pants in a software acquisition project, each with separate goals:

• The end user wishes to have a product that function well in operation. Another possible goal for the end user is to get the product as quickly as possible.

• The customer, that is the participant that finances the pro-ject, can have several goals: Similarly to the user, the cus-tomer wishes to receive a software product that function well in operation. The customer also wants to minimize cost and deviation from time schedule. Furthermore, the customer wants to maximize return on investment and to have a well-functioning relationship with the contractor.



• The contractor on the other hand often wants to maximize profits and to maximize the potential for future earnings, such as additional contracts.

This gives us the optimization problem that a software acquisi-tion project consists of, with several of the goals overlapping and conflicting. For instance, the customer wants to maximize the technical performance of the product and on the same time minimize cost and deviation from time schedule. Furthermore, the cost minimization goal of the customer conflicts with the profits maximization goal of the contractor. Obviously trade-offs are necessary to have an operational contract.

Participant Objective

End user Maximize technical performance Minimize development time

Customer Maximize technical performance Maximize customer-contractor relation Maximize return on investment Minimize cost Minimize deviation from time schedule

Contractor Maximize profit Maximize future earnings

Table 2. Participants in a software acquisition project and their goals. Adapted from Haimes, et al. (1997).

The most common scenario when you think of an outsourcing project is that you have two parties involved – a single cus-tomer and a single contractor. Gallivan (1999) shows that this simple dyadic relationship is not always the case, as more com-plex outsourcing constellations are becoming more common-place. Gallivan (1999) gives a taxonomy of outsourcing constellations:

• The simple dyadic constellation consists of a single cus-tomer outsourcing to a single vendor.



• The multi-vendor constellation consists of a group of ven-dors collaborating in a project outsourced by a single cus-tomer.

• The co-sourcing constellation appears when a group of customers that have similar needs cooperate and jointly outsources a project to a single vendor.

• The complex constellation is a combination of the multi-vendor and co-sourcing constellations, where you have a group of customers jointly outsourcing to a group of col-laborating vendors.

Figure 2. A taxonomy of outsourcing relationships. (Gallivan, 1999)

To make the outsourcing scenario even more complex you can have one or more subcontractors involved and several support-ing contractors and suppliers.

Compared to the dyadic constellation, the multi-vendor rela-tionship allows vendor specialization and reduces the risk of a vendor behaving opportunistically since the other vendors are ready and willing to substitute a non-performing vendor.

C VC V C

V

V

V

C

V

V

V

V

V

V

C

C

C

V

C

C

C

C

C

C

V

V

V

V

C

C

C

V

V

V

V

V

V

C

C

C

C

C

C

One Vendor Many Vendors

OneClient

Simple Dyadic Multi-Vendors

Many Clients

Co-Sourcing Complex

One VendorOne Vendor Many VendorsMany Vendors

OneClientOneClient

Simple DyadicSimple Dyadic Multi-VendorsMulti-Vendors

Many ClientsMany Clients

Co-SourcingCo-Sourcing ComplexComplex



Drawbacks are increased coordination costs and more complex contracts, etc. (Gallivan, 1999)

Compared to the dyadic constellation, the co-sourcing relation-ship allows risk sharing and reduction, and cost and time sav-ings due to buyer economies of scale. Drawbacks are increased coordination costs and that it is more difficult to control information and keep secrets because of the interdependent nature of the relationship between the customers, etc. (Gallivan, 1999)

2.5 Common problems Marciniak and Reifer (1990) states that the main problems in software acquisitions are:

• Development costs exceeds budget. • Time schedule is overrun. • Outcome does not meet expectations.

The Standish Group (1995) “Chaos” report investigated how common these problems are. 8 380 software development pro-jects were surveyed with respondents representing small, me-dium, and large companies across major industry segments such as banking, health care, and federal organizations, etc. The report does not indicate the level of outsourcing in these soft-ware development projects, but the statistics will still give an idea of the extent to which software development projects suf-fer from these problems.

The findings were:

• 16% of the projects were completed on time and on budget, with all features and functions as initially speci-fied. These projects were labeled as successful.

• 53% of the projects were completed and operational but over budget, over the time estimate, and offered fewer



features and functions than originally specified. These projects were labeled as challenged.

• 31% of the projects were canceled at some point during the development cycle. These projects were labeled as canceled.

Figure 3. Findings from the Standish Group (1995).

In addition, the average project exceeded planned budget with 90% and schedule with 120%.

The root cause to these problems is primarily ineffective man-agement as stated in Brown, et al. (1998):

In 1987, the Defense Science Board concluded that these soft-ware acquisition problems came not from technical difficulties, but from poor management. Since 1987, the situation has gotten worse, not better. Software has gotten bigger, more complex, and more expensive, and ineffective management is still the root cause of much that afflicts software acquisition.

Below is a list of some common problems in software acquisi-tion projects. It is assembled and adapted from The Standish Group (1995), Ganssle (1998), Marciniak and Reifer (1990), Reifer (1997), and interviews with Urmi (2000), Oskarsson (2000), and Ran (2000):

• Laissez-faire – The customer does not take an active part in the project once the contract is signed.

Successful16%

Challenged53%

Canceled31%



• Administrative overload – Too much administratrivia to monitor contract compliance instead of focusing on tech-nical matters.

• Creeping scope – The customer keeps adding and chang-ing scope and functionality to a job in progress on a tight schedule with limited resources.

• Fragmentation – Members of the team, both customer and contractor, are pulled off to work on other projects at random times.

• Goldplating – Having overkill requirements or making sophisticated and new rather than simple and proven technical solutions.

• “I’m paid to engineer!” – The customer tells the contrac-tor how to do the job, not what the job is.

• Missing or bad indicators – Measurement of progress and overall performance is qualitative instead of quantita-tive. The performance levels of the indicators are poorly set.

• Who’s in charge? – Too many bosses, decisions aren’t made in a timely manner.

• No end user involvement – Not getting and incorporat-ing the end user’s point of view on the functionality and usability of the product.

• Poorly defined requirements – The customer gives an in-complete and unvalidated set of requirements to the con-tractor.

• Acquisition incompetence – Failure to understand the particular needs of software acquisitions.

• Overpromising – The marketing people of the contractor makes promises that the technical staff can’t keep.



• Lack of discipline – Reverting to an ad hoc development process when deadlines are getting close. “Gotta crank out that code!”

• Unrealistic expectations – Having an impossible sched-ule, and/or being unaware of the limitations of the tech-nology being used.

• Inadequate resources – Not having appropriate financial funding or lacking appropriate staff, tools, and equip-ment.

• No executive support – No support or interest in the pro-ject from top management.

• Lack of clear objectives – No clear statement of goal or vision causing project members to pull in different direc-tions.

• Ineffective communications – Lack of effective commu-nication channels, information not reaching the right peo-ple in a timely manner.

• Lack of competence – Not having the appropriate techni-cal and leadership skills.

• Friction – Cooperation is hindered by some of the in-volved parties not getting along for some reason.

All of these problems can probably be avoided by better soft-ware acquisition management practices. Also see the risk tax-onomy given on page 80.

2.6 Frameworks The following is a list of frameworks that in some way gives guidance on software acquisition management. Apart from standards, books and collections of guidelines are mentioned.



There are of course more frameworks that cover the subject than those listed1.

SW-CMM – The Capability Maturity Model for Software is one of the most well known frameworks for assessing the capabilities of software contractors. The purpose of the model is to help or-ganizations determine the maturity of their current software development capabilities. The model characterizes the level of an organization’s maturity based on the extent to which meas-urable and repeatable software engineering and management practices are institutionalized. The software acquisition activi-ties examined for this thesis are in the Software subcontract man-agement key process area of the model. See Paulk, et al. (1993).

SA-CMM – The Software Acquisition Capability Maturity Model is a framework developed to help organizations assess and im-prove their acquisition capabilities for software-intensive sys-tems. The structure of the model is consistent with SW-CMM in that it is a staged model with key process areas grouped in five maturity levels: initial, repeatable, defined, managed, and op-timizing. See Cooper, et al. (1999).

FAA-iCMM – The Federal Aviation Administration Integrated Capability Maturity Model is an effort to combine the features of SW-CMM, SA-CMM, and SE-CMM2 into one consistent model. It is intended to be used as a tool for organizations to evaluate their acquisition, management, and engineering practices and to devise improvements on them. See FAA-iCMM (1997).

CMMI – The Capability Maturity Model Integration framework is an effort to integrate existing maturity models. CMMI is under development and will eventually consist of a series of disciplines ranging from software engineering and systems engineering to

1 www.software.org/quagmire is a good starting point to find out more about available standards.

2 Systems Engineering Capability Maturity Model.



software acquisition, etc. A requirement for the CMMI frame-work is consistency with ISO 15504. This framework had not been extended to contain the software acquisition discipline when this thesis was written. See CMMI (2000).

IEEE 1062 – The IEEE Recommended Practice for Software Acquisi-tion is a standard that takes a comprehensive approach to giv-ing recommended practice for managing software acquisition projects regarding COTS, MOTS, and fully developed software. It covers the whole acquisition process from planning organiza-tional strategy to using the software and is compliant with ISO 12207. See IEEE 1062 (1998).

ISO 9000 is a series of standards for quality management and quality assurance. It is mainly intended to provide guidance in a situation where a contract between two parties require the demonstration of a contractor’s capability to develop, supply and maintain products. In this series, the standards that are im-portant for software development projects are ISO 9001 and ISO 9000-3. ISO 9001 – Quality systems – Model for quality assurance in design / development, production, installation and servicing deals with generic two-party contractual situations and ISO 9000-3 Guidelines for the application of ISO 9001 to the development, supply and maintenance of software gives guidelines for the application of ISO 9001 to software projects. ISO 9001 is roughly compara-ble to SW-CMM level 2. See SS-ISO 9000-3 (1992).

ISO 12207 – The Standard for information technology – Software life cycle processes takes a holistic and comprehensive approach to the whole life cycle of a software product. It includes a well-defined terminology for software life cycles that is intended to standardize the terminology used by the industry. The standard can be used to establish software acquisition, supply, develop-ment, operation, maintenance, and supporting processes. It is also intended to foster an improved understanding between customers and vendors and among the parties involved in the life cycle of a software product. Worth mentioning is the goal of facilitating world trade in software. It replaces older U.S. mili-tary standards such as DOD-STD-2167A and DOD-STD-2168. See IEEE/EIA 12207.0 (1996).



ISO 15504 – The Information Technology – Software Process As-sessment standard is a framework for assessing software proc-esses regarding acquisition, development, support, etc. It is in-fluenced by and similar to SW-CMM in such that it has a capability scale. It is different from the SW-CMM in that it has not a discrete pass/fail mechanism and that it is aimed at assessing individual processes and not organizations. It is strongly aligned with ISO 12207. See ISO/IEC 15504 (1998).

Euromethod is a framework with the intent to assist an acquisi-tion at the contractual level. It aims to help the customer and contractor understand each other, improve risk management, and harmonize the terminology used in an acquisition effort. According to Breu (1995), Euromethod is particularly aimed at guiding public procurement processes in the context of the open European market, but can be used for all types of software acquisition projects. See Euromethod (1996).

Other frameworks that provide important and useful advice for software acquisition management are the following: The Road to Successful ITS Software Acquisition (Salwin, 1998), Software Acqui-sition Management (Marciniak and Reifer, 1990), The Program Manager’s Guide to Software Acquisition Best Practices (Brown, et al., 1998), Guidelines for Successful Acquisition and Management of Software-Intensive Systems (Mosemann, 1996), A Guide to the Pro-ject Management Body of Knowledge (Duncan, 1996), and Defense Acquisition Deskbook (DAD, 2000).

20

3 Customization

“Depending on the circumstances, you should be as hard as diamond, flexible as willow, smooth-flowing like water or as

empty as space.”

– Morihei Ueshiba

This chapter investigates project categorization factors that in-dicate what level of formality is necessary for the actual soft-ware acquisition project.

3.1 Customization levels Software acquisition projects have different characteristics. Some projects are small, short-term, and easy to grasp, while others are the other way around. Cockburn (n.d.) states that

… each organization must develop its own way of working, and each project must get its own, tailored methodology, deal-ing with culture, location, criticality, and technology.

To have effective acquisition processes that give good results, it is beneficial to employ an adequate level of formality for those processes according to the characteristics of the actual project. For the purpose of this thesis I give suggestions on three levels of formality for the acquisition management processes. These levels are:

3 Customization


Minimal processes – The project is easily managed and the purpose of the acquisition processes is achieved with a low level of rigor. The processes contain just the activities without which the project would really be in trouble. This level is suit-able for low-risk projects.

Controlled processes – The project has a tangible complexity and a reasonable risk of going out of hands. To contain the complexity and risks, a higher level of formality is appropriate for the acquisition processes. The processes on this level contain the activities of the minimal processes and add some activities that are not absolutely necessary but make things more con-trolled and reliable. This level is suitable for medium-risk pro-jects.

Robust processes – The project is highly complex and has a strong tendency to get chaotic unless it is managed appropri-ately and performed with discipline. The processes contain ac-tivities that increase the formality and administrative effort but in return make things more robust and reliable. This level is suitable for high-risk projects.

An acquisition process level below minimal would imply ad hoc, laissez-faire processes, which is not recommended.

3.2 Customization factors To determine what level of formality is appropriate for your project you can look at the customization factors in the follow-ing sections. The factors have been inspired by D.I.R. (2000), ISO 12207 (1995), and Euromethod (1996) and should be viewed as guidelines and not exact values on what level to choose. It is important to look at several factors and determine which are most important for your project before you decide upon a for-mality level.

3 Customization


3.2.1 Product characteristics

The product characteristics factors obviously stem from the char-acteristics of the software product to be acquired.

Requirements volatility

Since software development is a creative process, and it is diffi-cult to anticipate all technical problems beforehand, change of requirements occur more or less in all software development projects. However, due to the nature of the problem the soft-ware is to solve, the total set of requirements can be more or less change-prone.

Minimal proc-esses suffi-cient

Stable requirements. Highly unlikely that the overwhelming majority of the requirements will change.

Controlled processes recommended

A significant subset of the requirements is vola-tile.

Robust proc-esses rec-ommended

Volatile requirements. High degree of uncer-tainty on a large or important subset of the requirements.

Program size

The estimated program size indicates the effort that is needed to build the software. The values given in the table below are to be considered as suggestions since program size depends on what language is used and other factors.


Small. Less than 32 KLOC1.

1 1 KLOC = 1 thousand lines of source code.

3 Customization


Controlled processes re-commended

Medium. 32 – 128 KLOC.


Large. More than 128 KLOC.

Criticality

Criticality is the impact malfunctioning software will have on people’s safety and the magnitude of financial loss that can re-sult, according to Oskarsson and Glass (1996).


People’s safety is not at risk, nor will any signifi-cant amount of money be lost if the software mal-functions.


People’s safety is not at risk, but there is a risk of moderate financial losses due to software mal-functioning.


The safety of people is at stake or huge amounts of money. A disaster occurs if the software doesn’t function properly.

Innovation

Oskarsson and Glass (1996) define innovative projects as those in which the problem is new and different and hasn’t been solved with computer programs before. Less innovative pro-jects use mature and proven technology.


Mature technology. Many similar applications exist.


The technology has been proven to work and is expanding but few similar applications exist.

3 Customization



Emerging unproven technology where the ques-tion is more on whether the problem can be solved. Unique application.

Transformational impact

Transformational impact is the depth of change that the deliv-ered software product will have on the user’s behavior and the user organization’s processes and products.


Minimal change for the user.


Moderate modifications or amendments to the user’s behavior or the internal work processes and products of the user organization.


Fundamentally changes the behavior of the user and the work processes and products of the user organization.

3.2.2 Effort

The effort customization factors delivery time, total cost, and de-velopment team size are implied by the estimated effort to build the software. The estimate is based on the product characteris-tics. The larger the effort is the higher formality is required to avoid unnecessary chaos.

Delivery time

Delivery time is the estimated calendar time for delivery of the finished software product. With a longer project there is a higher likelihood that new people will join the development team and others leave. Change in the requirements is also more likely. The values given in the table below are to be considered as suggestions.

3 Customization



18 months or less.


12 – 36 months.


24 months or more.

Total cost

Total cost is the estimated total cost to acquire the software in order of magnitude. The value ranges of low, medium, and high in the table below are difficult to specify since these values may change over time and are only given as suggestions.


Low. Less than $50,000.


Medium. $50,000 – $1,000,000.


High. More than $1,000,000.

Contractor development team size

The more people that are involved in the contractor’s develop-ment team the more communication is needed to coordinate the work, sort out misunderstandings, and resolve conflicts. The values in the table are not to be considered as absolute.


Less than 10 developers.

3 Customization



10 – 30 developers.


More than 30 developers.

3.2.3 Contracting environment

The contracting environment customization factors are related to the contractor and the organizational environment that the ac-quisition takes place in.

Organizations involved

The more organizations, such as stakeholders, supporting con-tractors, consultants, independent testing organizations, etc. are involved, the more communication is needed to coordinate the work.


Two parties involved in a simple dyadic relation-ship; customer and contractor.


More than two parties are involved; either sev-eral customers co-sourcing, or several contractors collaborating, or subcontractors are used, and possibly supporting contractors. The organiza-tional structure is not (too) complex.


Several organizations are involved in a complex organizational structure composed of some or all of the following: multiple customers, multiple contractors, multiple subcontractors, multiple supporting contractors, etc.

3 Customization


Contractor development team characteristics

The contractor’s development team characteristics are the ex-perience, constitution, and performance of the contractor’s de-velopment team.


Low staff turnover. Mostly experienced and well-trained personnel. Well functioning teamwork. Efficient and appropriate working methods. High performance.


Moderate staff turnover. Some inexperienced and untrained personnel. Teamwork could be func-tioning better. Working methods need some im-provement. Medium performance.


High staff turnover. Lots of inexperienced and untrained personnel. Teamwork not functioning optimally. Inefficient and inappropriate working methods. Low performance.

Experience with contractor

The type and level of experience with the contractor(s).


The customer and contractor have worked to-gether on several previous occasions and have a good relationship built on trust. The customer knows how the contractor works and vice versa.


First time working with a contractor with good results working for other customers, or mixed experiences with a previously used contractor.


First time working with a contractor that is not well known or has had bad results working for other customers, or bad experiences with a previ-ously used contractor.

3 Customization


Geographic dispersion

Management becomes more difficult with customer and con-tractor on different geographical sites.


The customer, contractor, and other involved parties work at the same site or can visit and work on each other’s site on a daily basis.


The customer, contractor, and other involved parties work at different sites, with their respec-tive teams located at the same site, and cannot meet on a daily basis.


The customer, contractor, and other involved parties have several teams involved in the project dispersed on several different sites and they can-not meet on a daily basis.

3.2.4 Other factors

The factors stated above are only suggestions and you will probably have to look at more factors to be able to customize the project appropriately. Other factors include: standards re-quired, laws and regulations, other projects, cultural differ-ences, application domain, system complexity, etc.

29

4 Roles

“No matter how much you want it to be a technical problem, it’s a people problem.”

– Unknown

Having good people and good teamwork is key to the success of the project. This chapter presents the different types of roles that may be necessary for a successful software acquisition pro-ject. One person can be assigned several roles. The roles de-scriptions are assembled from and based upon the different frameworks stated in appendix A, Marciniak and Reifer (1990), Salwin (1998), and the interviews with Urmi (2000), Oskarsson (2000), and Ran (2000).

4.1 Sponsor roles The sponsor roles are played by persons representing the or-ganizations that have the powers to initiate and cancel the ac-quisition project.

4.1.1 Sponsor

The sponsor initiates and supports the software acquisition pro-ject enthusiastically but also has the power to cancel it if it will cost more to finish than will be gained from finishing the pro-ject. The responsibilities of the project sponsor are:

4 Roles


• To define and communicate a goal and vision for the pro-ject.�

• To provide a stable and adequate funding and set the fi-nancial limits and other bounds for the project.

• To appoint an acquisition manager that has the ultimate responsibility for the success of the project. The sponsor should be careful not to interfere too much with the way the acquisition manager manages the project.�

4.1.2 Collaborating customer

Should the acquisition project be a co-sourcing project with several customer organizations involved, the collaborating cus-tomer is another project sponsor whose collaboration needs to be coordinated with the other sponsors.

4.1.3 Product manager

In some cases the software is part of a larger systems acquisi-tion with the product manager for that system playing the spon-sor role.

4.2 Managing roles The managing roles are played by the people on the customer side that are put in charge of managing, monitoring, and ad-ministering the proceedings of the acquisition project.

4.2.1 Acquisition manager

The acquisition manager is appointed by the sponsor and is ulti-mately responsible for the success of the project. Therefore, the acquisition manager has the final word on how the project should be performed. The tasks and responsibilities of the ac-quisition manager involves the following:

4 Roles


• To adapt and customize the acquisition strategy accord-ing to the project characteristics.

• To plan the project and execute it accordingly, re-plan as the project progresses, manage risks, and solve problems.

• To recruit and organize people for the acquisition team, perform teambuilding and training activities, praise and encourage people, smooth the path for everyone, and be supportive.

• To select contractor and supporting contractors.

• To negotiate and sign agreements with the contractor and supporting contractors.

• To manage the relationship with the contractor and other involved organizations built on moral, trust, communica-tion, and collaboration.

• To tell the contractor what the scope of the software is so that you and the contractor knows when you are done.

• To manage the budget for the project within the limits set by the sponsor.

• To coordinate the work for monitoring project progress and report to the project sponsor.

• To perform reality checks periodically. Take necessary ac-tions if the answers to the following questions aren’t satis-factory:

o Are we acquiring the right software? o Are we acquiring the software right? o Is the contractor building the right software? o Is the contractor building the software right?

4 Roles


• To make necessary trade-offs between delivery time, total cost, product scope, and product quality if deviation from the previous estimates and specifications of them should occur.

• To coordinate work for evaluating and accepting the product.

• To prepare for post-contract support and maintenance of the product.

4.2.2 Technical manager

For large projects that are technically complex or with very high quality demands, such as life-critical software, the acquisition manager can delegate responsibilities for requirements and quality issues to a technical manager.

4.2.3 Contract manager

For projects with many involved organizations or projects where the administrative burden of the contract is high, such as cost-reimbursement contracts, the acquisition manager can delegate responsibilities for contract management issues to a contract manager.

4.2.4 Administrator

The administrative burden depends on the size and type of pro-ject. One should be careful not to over-administrate. Other members of the acquisition team perform the tasks of the ad-ministrator role, but for some projects a specific person can be assigned this role. The responsibilities of the administrator role include:

• To maintain configuration and change management on project documents such as contract, requirements specifi-cation, project plan, risk list, etc.

4 Roles


• To document and file correspondence, meeting minutes, and decisions.

• To administer payments to the contractor and supporting contractors.

• To document and file measurements on progress, quality, requirements changes, etc.

4.3 Assisting roles The assisting roles are played by different experts and support contractors that can assist and give advise to the managing, steering, and executing roles.

4.3.1 Acquisition expert

An acquisition expert can be used for giving advice on how to plan and perform the acquisition project, what contracting ve-hicle to select, and to train the acquisition staff in software ac-quisition management issues.

4.3.2 Technical expert

A technical expert can be used to evaluate the requirements and to give independent cost and size estimates. Furthermore, the technical expert can be used to review technical documents and to audit the technical knowledge and capabilities of the contrac-tor. The contractor can use technical experts for areas that they lack competence in.

4.3.3 Domain expert

A domain expert is not necessarily a technical expert but knows the field in which the software is intended to be used very well. The domain expert can therefore be used to develop and vali-date the requirements for the software. Another task for the domain expert can be to develop user training for the software.

4 Roles


4.3.4 Legal expert

A legal expert is essential for giving advise on how to write the contract and to inform on the actual laws that regulates the con-tract and other matters concerning the project, such as intellec-tual property rights, patents, licensing, warranties, copyright, etc. The legal expert can also help during disputes and schisms with the contractor. The use of a legal expert with software-specific knowledge is money well spent, especially in interna-tional projects, according to Salwin (1998).

4.3.5 Translator

A translator is, according to Salwin (1998), a role that can be played by a person that is good at translating layman’s terms into technical terms and vice versa. This can be useful in speci-fying the requirements and transferring the requirements to the technical staff of the contractor. This person should be skilled at translating the needs of the customer into something that the developers can use to build the system.

4.3.6 End user

The end user is the raison d’être of the software, unless of course the software doesn’t interact directly with a human user. There-fore, it is imperative to involve the end user on the elicitation of the requirements for the software, on evaluation of the graphi-cal user interface, and on evaluation of the functionality of the software product.

4.3.7 Maintenance and support staff

The maintenance and support staff has the primary tasks of keep-ing the software running, making upgrades, fixing bugs, and adding new features as well as giving technical support to the end users. Therefore, it is important to get suggestions from the maintenance and support staff on what to require from the software to make it easier to maintain and troubleshoot. They should also be involved in evaluating the maintenance and troubleshooting capabilities of the software and to review the

4 Roles


documentation to see if it conveys the necessary information required to do their jobs.

The maintenance and support staff should be involved early in the project in order to be able to discuss how the maintenance and support organization should be organized and to facilitate a smooth transition of the software once it is delivered to the maintenance and support organization with maintained con-figuration management. This role can either belong to the cus-tomer organization or be outsourced.

4.3.8 Independent verification and validation

An independent verification and validation contractor, IV&V, can be hired by the customer to make an in-depth technical assess-ment of the delivered software. This is mainly useful when the software affects peoples’ health or lives or when large amounts of money are at stake should the software malfunction. IV&V naturally increases the costs for the project significantly.

4.3.9 System engineering and technical assistance

When the buyer or seller lacks adequate resources or special-ized expertise, the use of a supplemental support or services contractor such as a system engineering and technical assistance contractor, SETA, can be necessary. The services provided by a SETA contractor can range from planning the project to testing, measuring, quality assurance, etc.

The use of SETA contractors should be considered for the fol-lowing areas, according to Marciniak and Reifer (1990):

• Technical risk. • Independent testing. • Management support. • Integration.

4 Roles


4.4 Executing roles The executing roles are played by representatives from the con-tractors side, i.e. those developing the actual software product.

4.4.1 Contractor

The contractor can be a sole contractor or a prime contractor in case several collaborating contractors are used. Choose contrac-tor carefully – the contractor with the lowest bid or most opti-mistic schedule and budget estimates isn’t always the best choice. No management practices can make up for having a bad contractor. Once selected, the contractor should be seen as a team member and not an adversary. The responsibilities of the contractor are:

• To develop and deliver the requested software. Should the project be cancelled, what has been developed up to that point should be delivered.

• To demonstrate that the sufficient technical and manage-rial capabilities exists to be able to develop the software. Should subcontractors or supporting contractors be used, their competence and capabilities needs to be demon-strated as well.

• To provide a feasible plan and work breakdown struc-ture for the project as well as realistic estimates on total cost and delivery time.

• To show the customer that progress is being made by holding regular demonstrations of the so far developed software.

• To make sure that the requirements are understood cor-rectly.

• To foster an effective, functional, supporting, and col-laborative culture in the relationship with the customer built on moral and trust. According to Ran (2000), both

4 Roles


parties must realize that they have a mutual responsibil-ity for the success of the project and should refrain from taking advantage of each other. The involved parties should work as a team, jointly solve problems, and re-frain from blaming each other.

• To promptly and openly inform the customer of unan-ticipated problems and schedule changes.

• To openly and jointly discuss, review, and mitigate risks.

4.4.2 Collaborating contractor

Should more than one contractor be used, the collaboration be-tween the contractors needs to be coordinated. This could be accomplished by assigning one of the contractors as a prime contractor with the responsibility to coordinate the work of the collaborating contractors.

4.4.3 Subcontractor

The contractor can use subcontractors to deliver parts of the software. The competence and capabilities of the subcontractor needs to be evaluated, and you as a customer should have the right to decide which subcontractor should be used. If the con-tractor is using subcontractors, make sure that they are in-volved as soon as possible in the project.

4.4.4 Supplier

It might be necessary to deal with a supplier of COTS and/or hardware during the course of the project.

38

5 Artifacts

“Everything has been said before, but since nobody listens we have to keep going back and beginning all over again.”

– A. Gide

In the acquisition processes different documents, reports, deliv-erables, and other artifacts are used. This chapter contains sug-gestions for the contents of those artifacts. The artifacts have been organized into the phase that they are most likely to ap-pear for the first time of the following four generic software ac-quisition project phases: inception, solicitation, monitoring, and finalization. The contents of the artifacts have been assembled from the different frameworks stated in appendix A unless oth-erwise stated.

Many of the artifacts are documents, and there are many rea-sons to document something, such as the following:

• To communicate vital information or to explain how you think.

• To help remember something.

• To help you think – the process of writing something down in a lucid and understandable way makes you do a lot of thinking you wouldn’t have done otherwise, re-gardless if the document is needed later or not.

5 Artifacts


• To legally prove that a transaction has occurred or that a commitment exists.

There is always a risk of documenting too much, writing things you won’t need later, or having too much bureaucracy. There-fore it is necessary for you to evaluate the need of each artifact and the applicability of each artifact checklist item in order to customize the artifact for the actual project. Some of the artifacts can be used as checklists during meetings while others require some written documentation. It is also important to remember that some of the artifacts, such as the acquisition plan, the re-quirements specification, and the risk list are likely to change during the course of the project. Therefore it is necessary to have some form of configuration and change management of these artifacts.

5.1 Inception During the inception phase the needs of the software are identi-fied, the requirements are elicited, risks are analyzed, and the acquisition processes are planned.

5.1.1 Acquisition proposal

The purpose of the acquisition proposal is to present the needs for and the feasibility of the acquisition, and also what benefits and risks exists so that an informed decision can be made by the sponsor on whether to initiate the acquisition project or not. In addition to the frameworks stated in appendix A, this artifact is based upon Pressman (1997).

Contents checklist

Describe the organization’s needs for the software prod-uct and how they are related to the organization’s objec-tives. What is the background of the needs? What is the current situation? What justifies the needs for the soft-ware?

5 Artifacts


Conceptualize the software product and give operational scenarios. What will the software have done? What are the short-term and long-term consequences for acquiring the software? What are the advantages and drawbacks? Analyze the alternatives for acquiring the software: COTS, MOTS, internal development, outsourced devel-opment, joint venture development, enhancement of ex-isting software, or combinations of these alternatives. What are the risks, costs, and benefits for each alterna-tive? Also do a market study of the alternatives. Describe whether the benefits, short-term or long-term, outweigh the costs for acquiring the software? Analyze the technical feasibility of the software. Are there any developmental risks? Can the necessary technical re-sources such as skilled staff and equipment be made available? Does the technology exist to develop the soft-ware or is it better to wait? Determine any infringement, violation, or liability that could result from development of the system. Examine patents, copyrights, and other intellectual property rights. Describe any external constraints on the project, such as standards, regulations, other projects, interfacing systems, etc. Describe any public relations issues. Is there value in oth-ers knowing about this? How do we do that? Describe how the project will be funded and give a time frame for the project. State any priorities, assumptions, limitations, and trade-offs considered. List potential contractors. Give a motivated acquisition decision recommendation.

5.1.2 Project specification

The purpose of the project specification is to document the deci-sions made by the project sponsor regarding project goals,

5 Artifacts


bounds, expected outcome, etc. which serves as a decision basis for the acquisition manager.

Contents checklist

Describe the organization’s needs. Conceptualize the software product and give operational scenarios. What will the software have done? Define the purpose of the project. State the acquisition project’s goal and vision in specific, measurable, accepted, realistic, and time-based terms. Specify the expected out-come of the project in terms of total cost, delivery time, product scope, and product quality, etc. Specify the initially identified product requirements and their relative priorities. Describe any identified risks. Specify funding and time bounds. Identify and give instructions regarding interfaces and collaborations with other projects. Identify legal issues, such as intellectual property rights, etc. Specify any external constraints, such as standards, regu-lations, interfacing systems, etc. Give instructions on security, access to information, and confidentiality issues. List potential contractors and/or capabilities that can be used for identifying potential contractors, such as re-quired competence, etc. Describe routines for reporting project status. See the Project status report on page 56.

5.1.3 Acquisition plan

The purpose of the acquisition plan is to describe how the acqui-sition project will be managed.

5 Artifacts


Contents checklist

Describe the project’s background. Why is the project done? What is its purpose? State the project’s objectives. State the project funding and schedule bounds. Define the terminology that will be used in the acquisition project. Describe what tools, techniques, and methods will be used. Describe any standards, laws, practices, and conventions that are to be followed. Determine problem resolution procedures. Specify documentation requirements and procedures. Specify configuration and change management proce-dures. Describe collaborations and interfaces with external or-ganizations, support contractors, etc. and how the respon-sibilities are divided between the involved organizations. Specify communication and reporting channels. List im-portant addresses, etc. Determine acquisition project team inter-group coordina-tion and division of responsibilities. Outline a master budget for the different processes. Outline a master schedule describing the chosen acquisi-tion life cycle and appropriate milestones. Specify security, access to information and confidentiality procedures. Describe how public relations issues will be handled. Describe routines for reporting project status Describe how measurements will be collected, analyzed and used.

5 Artifacts


Plan each of the acquisition processes in chapter 6, Proc-esses, on page 60:

o Identify the process group members, their respec-tive responsibilities, and the organization of the group.

o Describe any collaborations and interfaces with other groups and organizations.

o Specify the scope and objectives of the process ac-tivities.

o Determine the working methods of the process group.

o Specify and schedule the process activities. Deter-mine who is responsible for each activity.

o Specify the budget for the process activities. o Describe other resources allocated for the process

activities in form of equipment, tools, materials, documents, facilities, etc.

o Determine how the process activities will be coordi-nated and how progress, problems, etc. will be re-ported and reviewed.

5.1.4 Risk list

The purpose of the risk list is to describe the identified risks, prioritize the risks and provide contingency and mitigation ap-proaches for each risk. The risk list should be maintained cur-rent and used as a tool for communicating risk issues between the involved parties. Each item on the risk list can contain sev-eral of the following:

Contents checklist

Risk identification. Risk classification. Date of last update. Risk description, i.e. condition and consequences. Probability of risk occurring.

5 Artifacts


Impact of risk occurrence. Risk exposure, i.e. probability × impact. Indicators of risk turning into a problem. Approaches to control, transfer, avoid, minimize, and mitigate risk. Name of person responsible for mitigation activities and date for last implementation of those activities. Status of indicators and mitigation activities. Contingency plan, what to do if the risk occurs. Contingency triggers, what conditions are necessary for the contingency plan to be implemented.

5.1.5 Requirements specification

The purpose of the requirements specification is to capture and describe the requirements of the software product and other de-liverables. The requirements can be described several different ways such as shall-statements, problem statements, use cases, or user stories1. The contents checklist is partially based upon Kar and Bailey (1996).

Contents checklist

State the project goals. Which goal is the most important one in case a trade-off decision needs to be made? Describe the software product in general terms; what is the background for acquiring the software, what is its us-age domain, what type of product is it, etc.

1 More information about requirements can be found in numerous web sites and books. See for instance “Managing Software Requirements: A Unified Approach” by Dean Leffingwell, Don Widrig, and Edward Your-don (ISBN 0201615932). Information on user stories can be found on: http://www.extremeprogramming.org/rules/userstories.html.

5 Artifacts


Describe the different kinds of users the software will have. How do the different types of users use the soft-ware? What will the software have done for the user? Are there any safety or security restrictions? Describe the target hardware for the software. Describe the external interfaces, to humans, hardware and other software. Define the terminology used in the specification. State any assumptions that have been made. State the functional requirements. How shall the software handle the different types of input, and what output should it produce? How should errors be handled? State the non-functional requirements, in terms of per-formance, portability, efficiency, reliability, usability, availability, and maintainability. Verify and validate each requirement:

o Has the requirement been designated an identifica-tion for traceability purposes?

o What is the justification and motivation for the re-quirement?

o How will the fulfillment of the requirement be veri-fied?

o What other requirements is the requirement related to?

o Is the requirement necessary? Will a deficiency exist if the requirement is deleted?

o Is the requirement minimal or can it be divided into separate requirements?

o Does the requirement state what is required, and not how it should be met?

o Is the requirement attainable? o Is the requirement complete or does it need further

amplification? o Is the requirement consistent with the other re-

quirements in such that it doesn’t contradict any

5 Artifacts


other requirement, isn’t a duplicate of any other re-quirement and uses the same terminology as the other requirements?

o Is the requirement unambiguous? o Is the requirement written without the use of vague

words, such as “flexible” and “user friendly”? o Is the requirement verifiable by inspection, analysis,

demonstration, or test and is a verification criterion stated for each requirement?

Prioritize the requirements. Describe what risks are connected with each requirement. Give implementation time estimates for each requirement. Verify and validate the total set of requirements:

o Is the total set of requirements complete, and doesn’t need any further amplification?

o Is the total set of requirements consistent, in such that no duplicate or contradictory requirements ex-ist, and the same terminology is used for all re-quirements?

State any installation requirements. State any requirements regarding manuals and documen-tation.

5.1.6 Maintenance plan

The purpose of the maintenance plan is to describe how the sup-port and maintenance organization will be working.

Contents checklist

Identify the members of the support and maintenance or-ganization, their respective responsibilities, and how they are organized. Describe any collaborations and interfaces with other groups and organizations.

5 Artifacts


Describe the resources allocated for maintenance activities in form of equipment, tools, material, documents, facili-ties, etc. Specify allocated funding and budget for maintenance ac-tivities. Determine maintenance scope and how the maintenance activities will be coordinated. Describe procedures for problem and maintenance report-ing. Describe how problem and modification analysis will be performed. Describe procedures for modification implementations. Describe how maintenance activities will be reviewed and accepted. State migration and software retirement procedures. State configuration management activities and proce-dures. State the continuation of requirements management ac-tivities and procedures. Specify what standards, methods, practices and conven-tions will be followed.

5.2 Solicitation During the solicitation phase requests for proposals are sent out to potential contractors, proposals are received and evaluated, and a contractor is selected with which a contract is negotiated and signed.

5.2.1 Request for proposal

The purpose of the request for proposal, RFP, is to communicate the scope of the work and the contractual conditions to poten-tial bidders.

5 Artifacts


Contents checklist

Describe the customer’s company. State the acquisition project’s goal and vision. Describe the target domain of the software product. List the deliverables of the project. Include the requirements specification (see page 44). State any issues regarding security, nondisclosure, and confidentiality. Specify any support, training, installation, and mainte-nance requirements. Give instructions for bidders regarding the proposal such as: date for reply, number of copies, contents and struc-ture of proposal, etc. See the Proposal on page 50. State any requirements regarding development practices, quality assurance procedures, standards compliance, con-figuration management, communication and reporting progress and problems, etc. Specify the contract type (see page 88). Cost and schedule estimates. Describe any investment issues, such as facilities and equipment. Specify the customer’s rights to inspect the bidder’s facili-ties to investigate and evaluate financial position, techni-cal capability, experience, quality practices, etc. State any legal issues regarding warranties, licenses, own-ership rights, copyright, patent, usage rights, and other intellectual property issues. Determine use and control of subcontractors and suppli-ers. Describe identified risks and issues that need to be dealt with. State any other significant contractual terms and condi-tions, such as constraints. See the Contract on page 51.

5 Artifacts


5.2.2 Contractor evaluation criteria

The purpose of the contractor evaluation criteria is to give a set of criteria against which the bidding contractors’ proposals will be compared in order to be able to select the most appropriate con-tractor. Be explicit about the minimum standard for each crite-rion. The set of criteria can cover the following issues:

Contents checklist

Technical approach, suggested solution, and understand-ing of the problem. Proposed mitigation and management of certain risks. Cost and schedule estimates. Company and owner history. Any bankruptcies or litiga-tions? How long has the company been in business? What previous customers has the company had? Contractor stability. Is the organization undergoing any major change, such as changing ownership or moving of-fice, etc? Outcome of contractor’s previous projects regarding cost, schedule, performance, and quality as well as staffing levels, error rates etc. See Project status metrics on page 100. Comments from customers to previous projects. Contractor’s financial status. Eventual independent finan-cial rating. The staff’s experience, competence, constitution, and per-formance. Technical capabilities. Management qualifications. Resources: equipment, tools, facilities, etc. Location. Adequacy of software development processes and quality practices. Previous experience with contractor.

5 Artifacts


Legal issues regarding warranties, licenses, intellectual property rights, etc. Adherence to eventual standards.

5.2.3 Proposal

The purpose of the proposal given by the bidding contractor(s) is to present their company, capabilities, suggested solution, esti-mates, advantages, etc.

Contents checklist

Company description. Its background and history. Previous and current customers. Outcome and data on previous projects. Financial position. Understanding of problem, technical approach, solution suggestion, description on how the different requirements will be met. Technical and managerial capabilities. Working methods and development processes. Quality practices. Resources: equipment, tools, facilities, etc. Compliance with standards (such as ISO 9000). Technical and domain experience. Cost and schedule estimates. Mitigation and management of risks. Organization and staffing. Key personnel. The staff’s ex-perience and constitution. Contact persons and addresses. Charges, prices, fees and payment issues. Use of subcontractors and suppliers. Needed investments in facilities and tools, etc. Legal issues regarding warranties, licenses, intellectual property rights, etc.

5 Artifacts


5.2.4 Contract

The purpose of the contract is to define the relationship obliga-tions and promises that the customer and contractor make to each other. The contents checklist is based upon Oskarsson (n.d. a, b), Marciniak and Reifer (1990), and the frameworks in appen-dix A.

Contents checklist

State the purpose of the contract. Define the terminology used. State the scope of work. Refer to the requirements specifica-tion (see page 44), which should be attached to the con-tract. Address software and documentation deliverables, support, installation, and training. Specify the acceptance procedures and criteria such as the following: acceptance test completed with a satisfactory result, installation completed, user training completed, all deliverables received, correction for errors found a de-termined period after delivery are corrected. Specify the quality measures by which the contractor’s work will be evaluated. Describe what constitutes a satisfactory per-formance of the contractor. Specify procedures and conditions for making modifica-tions or amendments to the scope of work. Specify who is authorized to make changes in the contract and answer questions from the contractor. Also specify contract change control procedures. Describe payment conditions, considerations, and amounts. Determine allowable costs, expenses, charges and their variation. Determine when payments are to be made linked to deliverables and milestones. Specify con-cerns regarding taxes, late payments, and interests. Specify use of incentives for earlier delivery, reduced costs, significant results and achievements, or for use of certain “best practices,” etc. See page 88 for more on in-centives.

5 Artifacts


Specify any penalties or payment reductions for delays or for not meeting certain requirements, etc. Ran (2000) says that you shouldn’t have penalties unless the risk is shared, i.e. both sides should lose something. Marciniak and Reifer (1990) state that penalties should only be used to correct a situation in the project or to prevent a similar situation from occurring in the future. Determine allowable royalty and license fees. State the timeframe of the contract and issues regarding a final delivery date if that is applicable. State conditions, provisions and obligations concerning contract termination. The contractor should deliver all material and deliverables that are associated with the con-tract if the contract is terminated and the customer should pay for the work performed so far. Describe responsibilities for maintenance undertakings. State ownership rights, copyright, patent, trade secrets, usage rights, licenses, and other intellectual property is-sues. Describe legal concerns on warranties, representation, in-fringement, limitation of liability, indemnification, non-disclosures, etc. Describe security policies, nondisclosure, access to infor-mation, and confidentiality issues. Specify how public relations issues will be handled. Specify the contractor’s rights and obligations to assign its duties, partition the work, and use subcontractors. Determine how and where disputes will be solved, and whether an arbitrator should be used. Specify the relationship between the parties in the con-tract such as joint venture, partnership, employer and employee relationship, etc. Describe the commitments by the involved parties and the division of responsibilities, obligations, and tasks. Determine the amount and conditions of customer in-volvement.

5 Artifacts


Specify relationships to other parties involved in the pro-ject, such as IV&V and SETA. Specify use and replacement of key personnel. Specify use of facilities and equipment. Also determine any investment issues regarding facilities and equipment. Establish means for monitoring progress, through WBS, short iterations, milestones, or other means. Specify the allowable requirements change rate (for fixed price contracts). Specify procedures for reporting problems and progress. Determine what status metrics should be reported by the contractor and how often. See the Project status metrics on page 100. Specify any requirements on the contractor’s software de-velopment, management and quality assurance processes. Specify any requirements on a software development plan for the project and a latest date to receive the plan from the contractor and what it should contain. Specify any requirements on the use of COTS or hard-ware or any restrictions on what suppliers to use. Determine the customer’s inspection rights and proce-dures for audits. Signatures.

5.3 Monitoring During the monitoring phase the progress and performance of the contractor is monitored and the software product is evalu-ated on a regular basis.

5.3.1 Artifacts from the contractor

During the monitoring phase some of the communication from the contractor can consist of the following artifacts, depending on the actual project:

5 Artifacts


• Project plans. • Test reports. • Problem reports. • Payment requests. • Progress and status reports. • Change requests. • Quality reports.

5.3.2 Product evaluation plan

The purpose of the product evaluation plan is to specify the objec-tives, scope, and activities of the different product evaluations. See Product evaluation approaches on page 109 for a list of differ-ent kinds of evaluations that can be performed.

Contents checklist

Specify who will participate in the evaluation and the re-sponsibilities of each participant. Describe the objectives and scope of the evaluation. Give instructions for preparing for, carrying out, and evaluating the results of the product evaluation. Give a schedule for the evaluation. Specify what resources are required regarding equipment, tools, materials, documents, facilities, etc. Determine the test configuration, technical environment, and other prerequisite conditions. Describe each test case in the evaluation. What require-ments are addressed during each test case? Specify input and expected results for each test case. Describe the coverage of the test cases in the evaluation. For instance, for an acceptance test it should be shown that all implemented requirements are covered by the test cases in the evaluation.

5 Artifacts


State criteria for evaluating and reporting evaluation re-sults. See the Product evaluation report on page 55.

5.3.3 Product evaluation report

The purpose of the product evaluation report is to document the outcomes of a performed product evaluation. See Product evaluation approaches on page 109 for a list of different kinds of evaluations that can be performed.

Contents checklist

Describe the objectives and scope of the evaluation. State the time for the evaluation, the participants and their responsibilities. Identify the test configuration, technical environment, and other prerequisite conditions. Describe any special considerations, simplifications, and assumptions. List the outcomes of each test case. Determine whether the coverage of the test cases in the evaluation was sufficient or not. Summarize all errors and problems encountered. Analyze the most common errors and identify trends. State how the errors and problems will be followed-up. Give recommendations for decisions, such as changes to requirements, acceptance, etc.

5.3.4 Change request

The purpose of the change request is to provide motivation and relevant information to enable an informed decision on whether to approve a change to requirements, contract or plans.

Contents checklist

State what should be changed. State who originated the request.

5 Artifacts


Give a justification for the proposed change. State the priority of the change request. Describe any assumptions and constraints that affect the change request. State the impact on cost, time, quality, scope, and risks.

5.3.5 Project status report

The purpose of the project status report is to summarize the status of the project so that it can be used as a basis for a deci-sion on whether to continue the project or cancel it.

Contents checklist

List the status of the risks in the risk list. Any changes? Any problems that need to be dealt with? Report results from product evaluations and the contrac-tor’s testing efforts. Analyze the technical feasibility of the contractor’s ap-proach. State any technical problems or changes in the technical approach. State any changes to the requirements. Describe the available resources regarding equipment, tools, materials, documents, facilities, etc. Are the origi-nally planned resources available? Are the resources ap-propriate? List measurements regarding schedule and budget, etc. See section 6.9.1, Project status metrics, on page 100. Identify and analyze trends regarding schedule and budget, etc. Is the project on schedule? Forecast delivery date. How much of the funding have been expended? Is the current funding appropriate? Will the project reach its budget goals? List the results achieved since the last report. What mile-stones have been accomplished? Describe ongoing activities.

5 Artifacts


What actions need to be taken within the project? What actions need to be taken by the sponsor or steering group? Does the benefits, short-term or long-term, outweigh the predicted remaining costs for acquiring the software? Give conclusions and recommendations and relate them to project objectives. Should the project continue or be canceled? Any changes to the project direction?

5.4 Finalization During the finalization phase the product and other deliverables are delivered to the support and maintenance organization and a post partum review is held.

5.4.1 Deliverables

The deliverables received from the contractor could consist of some of the following artifacts, depending on the actual project:

• Software executables. • Source code. • Technical documentation. • User training material. • Rest list – lists requirements not implemented and known

errors. • User manual. • Support publications.

5.4.2 Post partum plan

The purpose of the post partum plan is to describe how the post partum will be conducted. The plan is based upon Kerth (n.d.).

5 Artifacts


Contents checklist

Determine who should attend the post partum. Preferably representatives from all involved organizations and roles should participate. Determine when to hold the post partum. Preferably within 7-14 days after project completion. Select an appropriate location for the post partum. Pref-erably away from the office. Determine the length of the post partum. 2-4 hours may be sufficient for minimal projects while 3 days could be needed for robust projects. Determine the rules for the post partum in order to create a non-threatening environment.

5.4.3 Post partum

The purpose of the post partum is to document, analyze, and package experiences and wisdoms for reference in future acqui-sition projects.

Contents checklist

Give the background for the project and the time of events in the project. Document the customization factors values (see the Customization factors on page 21) for the project. Compare project goals to actual outcome regarding the deliverables, time schedule, costs, etc. Indicate deviations and the reasons for them. Document what processes and methods were used and how the project was organized. Document how the pro-ject was customized. Document special difficulties and measures taken and the effect of these measures. Document the performance of the contractor and other involved organizations – their strengths and weaknesses. If earned value was used – document SPI, CPI, etc.

5 Artifacts


Document what development methodology and working methods were used by the contractor. Document what risks occurred and what measures were taken regarding risks. Document experiences and observations. Document what activities/techniques were considered key to the success of the project. Propose improvements on the acquisition processes and other things.

60

6 Processes

“I have it anecdotally that the most commonly used methodology in the world is chaos.”

– Ron Jeffries

“A good process will tell you to do what a good manager and staff would do anyway.”

– Tom DeMarco

This chapter describes the different acquisition processes. The contents of the processes have been assembled and adapted from the different frameworks stated in appendix A unless oth-erwise stated.

6.1 Overview The different aspects of managing a software acquisition project have been divided into the following eleven processes:

• Project steering • Project management • Planning and organizing • Acquisition training • Requirements management • Risk management

6 Processes


• Contractor selection • Contract management • Product evaluation • Transition to support • Post partum

Each process contain suggestions on what activities to perform for the different project customization levels suggested in chap-ter 3, Customization, on page 20. The process template in the ta-ble below explains how to interpret the process descriptions. Keep in mind that the activities given in the different customi-zation levels are only suggestions. When planning for an acqui-sition project you can use the suggestions as a basis but you have to remain flexible and adapt to the specific circumstances of the actual project. For instance, if you have a project where all of the factors described in chapter 3, Customization, on page 20, are suggesting minimal processes except for requirements volatility, which suggest robust processes, then you have to consider having most processes minimal, but a higher customi-zation level for requirements management, and as a conse-quence a higher customization level on product evaluation, or other adaptations that fit your needs. And remember: following a process requires discipline.

Process name

Possible start condition for the process.

Possible end condition for the process.

Possible inputs to the process. Possible outputs from the process.

Process group

Roles that could be considered as participants in the process group. The roles that are in bold face are roles that should par-ticipate as a recommended minimum.

6 Processes


Minimal process activities

Activities suitable to perform in a project with minimal pro-ject characteristics.

Additional controlled process activities

Additional activities suitable to perform in a project with controlled project characteristics.

Additional robust process activities

Additional activities suitable to perform in a project with ro-bust project characteristics.

Table 3. Process template explaining the elements of the process de-scriptions.

As an illustration, Figure 4 below depicts at what point in time of the acquisition lifecycle the processes occur.

Figure 4. Timeline of the acquisition processes.

Figure 5 below shows how the processes are related to the dif-ferent roles and major artifacts. The numbers in the figure refer to the following example of a sequence of major activities from all process customization levels:

Project steeringProject management

Planning and organizingAcquisition training

Requirements managementRisk management

Contractor selectionContract management

Product evaluationTransition to support

Post partumInception Solicitation Monitoring Finalization

6 Processes


Figure 5. Overview of the software acquisition processes, major roles and artifacts.

Sponsor

Project steering

Project specification

Project management

Contractor selection

Contract management

Reports, requests, etc.

Project status report

Acquisition training

Product evaluation

Planning and organizing

Contractor

Risk management

Risk list

Requirements management

Requirements specification

Transition to support

Maintenance plan

Post partum

Post partum

RFP Proposal Contract

Reports and measurements

Acquisition manager

Maintenance & support staff

End user

Administrator

Legal expert

Acquisition expert

Translator

Domain expert

Technical expert

IV&V

Deliverables

Acquisition plan

Acquisition proposal

Contractorevaluation

criteria

Post partum plan

Evaluation plan

Evaluation report

Change request

1

2

3

4

6

7 8

9

11

12

5

13 14

15

16

17

18

19

20

10

6 Processes


1. The sponsor initiates the project steering process. The needs of the organization, and other project concerns are elicited, analyzed, and documented in the acquisition proposal, which serves as a decision basis.

2. If the sponsor decides to initiate the project, the project goal, funding and time bounds, etc. are specified in the project specification.

3. An acquisition manager is appointed.

4. The acquisition manager customizes, plans, and organizes the acquisition project based upon the project specification, possibly with the assistance of an acquisition expert and others. The plan is documented in the acquisition plan.

5. The maintenance and support organization is identified and the maintenance plan is drafted.

6. The acquisition team performs the necessary acquisition training activities.

7. The requirements management process is initiated. The product requirements are elicited with the help from end users, maintenance and support staff, translator, domain experts, and technical experts, etc. The requirements are documented in the requirements specification.

8. The risk management process is initiated. Identified risks are documented in the risk list by the acquisition manager and others.

9. The contractor selection process is initiated with the contrac-tor evaluation criteria being written followed by writing and sending out request for proposals to potential contractors. The acquisition manager is responsible for this with assis-tance from acquisition expert, legal expert, etc.

10. The proposals from the bidding contractors along with data gathered from audits and background checks are compared

6 Processes


with the contractor evaluation criteria and the most appro-priate contractor is selected.

11. The contract is negotiated, reviewed and signed between the acquisition manager and the manager on the contractor’s side, with the legal expert advising the proceedings.

12. A joint review and discussion of the requirements specifica-tion and each of the parties risk lists is performed prior to commencing development.

13. During the development of the product the contractor deliv-ers project plans, test reports, progress reports, payment re-quests etc. and the software acquisition team gives feedback and takes necessary action.

14. The contractor submits deliverables for evaluation by the software acquisition team on a regular basis. Feedback is given to the contractor through an evaluation report. Possi-bly the software is evaluated by and independent verifica-tion and validation organization.

15. Changes to requirements are handled and possibly the con-tract is renegotiated.

16. The acquisition team and the contractor collaborate to solve problems, and jointly reviews project status, plans, risk lists, etc. on a regular basis.

17. For lengthier projects it might be necessary to audit the con-tractor with the assistance of a technical expert, etc.

18. The acquisition manager regularly briefs the project spon-sors on the status of the project through a project status re-port. The sponsors use this as a basis for a decision to con-tinue or cancel the project.

19. After the final evaluation and acceptance of the software product it is transitioned to the maintenance and support

6 Processes


organization. The contract is settled, open items are resolved and final payment is made.

20. A post partum is planned and performed. Celebrate a pro-ject well done and document experiences and lessons learned. Remember not to make it a blame session.

A more detailed description of the different activities that can be performed in a software acquisition project are given in the process descriptions that follow.

6.2 Project steering The purpose of the project steering process is to give the project sponsors control over the project. It is a two-phase process, where the first phase consists of eliciting and analyzing the needs of the organization, set a goal for the project and take a decision on whether to initiate the project or not. The second phase consists of evaluating the status of the project on a regu-lar basis, at least for lengthier projects, in order to determine whether to continue the project or not.

Project steering process

Starts when the idea of the project is established.

Ends when the project is ter-minated.

Input: Organizational needs and objectives, Project status report

Output: Project specification

Project steering group

The participants to consider for this process should be those who are authorized to make decisions about the project’s exis-tence and those who can give advice on different project issues such as the following roles: sponsor, collaborating customer, product manager, administrator, acquisition expert, technical expert, domain expert, legal expert, translator, end user, main-

6 Processes


tenance and support staff, and system engineering and techni-cal assistance. Make sure to involve the necessary organizations as early as possible in the project.


Elicit the organizational needs. Review the organization’s objectives. Elicit, analyze, and conceptualize the needs and expectations of the organiza-tion. Determine the scope and qualities the software product must possess to achieve the organization’s objectives.

Analyze and select acquisition type. Analyze the advantages, drawbacks, risks, costs, and bene-fits for the following alternatives: COTS, MOTS, internal de-velopment, outsourced development, joint venture devel-opment, or combinations thereof, and enhancement of exist-ing software. Select the most suitable alternative. COTS should be the primary candidate, then MOTS, and only as a final solution fully developed software should be considered for most acquisition projects.

Investigate intellectual property rights. Determine if there are any conflicts regarding patents, copy-rights, trademarks, and registered names, etc.

Decide to initiate project or not. Is this the right time to acquire the product? Is the product really needed or is it just cementing the old way of working? Are there any foreseeable risks that are very likely to termi-nate the project? What are the risks for not acquiring the software? Does the benefits, short-term or long-term, out-weigh the costs for acquiring the software?

Determine project parameters. Determine the scope of work of the project. Who will pro-vide software support? Identify potential contractors or de-velop a list of capabilities that can be used for identifying po-tential contractors. Identify the extent of the responsibilities for the acquiring organization and contracting organization

6 Processes


respectively. Identify interfaces and collaborations with other projects and organizations. Define the expected out-come; roughly estimate cost limits, time schedule, etc. See the Project specification on page 40.

Establish and communicate a project goal. Define the purpose of the project and determine what is con-sidered a successful outcome of the project. Inform all in-volved parties.

Appoint an acquisition manager. Establish project responsibility, authority, and accountabil-ity.

Decide to cancel or continue the project based on the status of the project or external factors. For minimal, and thus short projects, this decision should be taken in the beginning of the development (about 15-20% into the project). For controlled and robust project this deci-sion needs to be taken on regular intervals. Remember that termination can be success. Document and communicate the decision. The decision should be well motivated and given in a timely manner.


Investigate constraints and consequences. What short-term and long-term consequences (advantages and drawbacks) of the acquisition are there for different in-dividuals and organizations? Identify external constraints on the project such as standards, regulations, interfacing sys-tems, technical constraints, etc.

Prepare an acquisition proposal. See the Acquisition proposal on page 39. Use it as a basis for the decision on whether to start the project or not.

Document the project parameters. Write a Project specification (see page 40).

6 Processes


Evaluate the status of the project on a regular basis. Review the Project status report (see page 56). Evaluate the feasibility to achieve project goals with available resources and constraints. Estimate the tasks and resources necessary to complete the project. Has the business objectives changed? Has the market changed? Has other external constraints changed? Is the project progressing satisfactory? Are there any delays or budget overruns? Are appropriate resources allocated? Use the evaluation as a basis for decisions on whether to continue the project or not.


Analyze the technical feasibility of the software. Identify technical and developmental risks. Investigate whether the necessary technical resources can be made available, etc.

6.3 Project management The purpose of the project management process is to perform the project according to plan, replan if necessary, and handle issues not planned for.

Project management process

Starts when the acquisition manager has been appointed.

Ends when the acquisition project is complete.

Input: Acquisition plan, Risk list, Measurements, Product evaluation report

Output: Project status report

Project management group

The participants to consider for this process should be those with the responsibility and authority to manage and administer

6 Processes


the project such as the following roles: acquisition manager, technical manager, contract manager, and administrator.


Customize, plan, and organize the acquisition project. Develop an acquisition plan. See the Planning and organizing process on page 72.

Train the acquisition staff. Make sure that the people involved in the project are trained in how to perform the different process activities. See the Acquisition training process on page 74.

Review project feasibility. Ensure the feasibility of the project regarding adequate and timely resources and that the goal is achievable with the given scope, funding, and schedule. Remember that you can have fixed scope, product quality, delivery date, or total cost but not all four. You may therefore need to make trade-offs to achieve the most important goal of the project. Discuss any feasibility issues with the project sponsors.

Regularly review the work of the acquisition process groups. Regularly review what has been done, and decide on next actions for the different process groups. Follow up on previ-ous decisions.

Regularly review the work of the contractor(s). Determine how measurements are to be assembled, reported and analyzed. Gather measurements on progress, resource usage, cost, quality, and schedule, etc. and compare to plans and specifications. Identify trends. Evaluate the status of the project and take necessary action. See the Contract manage-ment process on page 96.

Report project status. The acquisition manager should at agreed upon points in time assemble and report the status of the project to the

6 Processes


steering group. See the Project status report on page 56.

Perform team-building and social activities. Perform suitable project kick-off and other social or team-building activities. Recognize the efforts of the staff mem-bers. Make the project fun and exciting. Don’t over-use over-time.


Identify and coordinate activities with other projects and sign contracts with supporting organizations.

Establish problem resolution procedures. Any problems discovered during the project should be duly reported, investigated, and solved. The impact of any changes due to problems should be determined, controlled and monitored. Establish means for resolving the issues that will arise during the project, such as consensus building, ne-gotiating, voting, brainstorming, etc. Inform the involved parties on the problems status. Document and track status of problems until resolved. Resolve both problem and cause. Evaluate the problem resolutions: Were they correctly im-plemented? Were new problems introduced?

Establish quality assurance and configuration management procedures. Establish a configuration management methodology. Iden-tify the items that should be placed under configuration management such as plans, agreements and specifications. Control and track changes to those items. Identify and stan-dardize documents. Control the release and delivery of documents, correspondence, and other items in the project. Verify that the project documents are adequate, complete, and consistent by peer review or other means. Maintain plans and other documents current throughout the project as re-planning occurs, issues are resolved, requirements are changed, and risks are mitigated or discovered.

6 Processes



Perform in-depth problem analysis. Analyze, categorize, and prioritize problems and identify root causes. Detect trends. Develop a system for identifying, recording, and tracking problems that occur during the pro-ject. Update the risk list.

Document lessons learned. For long-term projects it might be of value to prepare inter-mediate reports that documents observations, experiences and lessons learned and gives suggestions on improvements for the rest of the project and future projects.

6.4 Planning and organizing The purpose of the planning and organizing process is to custom-ize the acquisition processes for the project and make plans for each process accordingly.

Planning and organizing process

Starts when the acquisition manager has been appointed.

Ends when the planning ac-tivities are finished.

Input: Project specification Output: Acquisition plan

Planning and organizing group

The participants to consider for this process should be those re-sponsible for performing the project and those who can give in-put to and assistance on planning and organizing issues such as the following roles: sponsor, collaborating customer, product manager, acquisition manager, technical manager, contract manager, administrator, acquisition expert, system engineering technical assistance, contractor, collaborating contractor, and subcontractor.

6 Processes



Customize the acquisition processes. Review the Project specification (See page 40). Get input from involved organizations. Analyze the characteristics of the project and select an appropriate customization level, see Customization on page 20. Review stored information on how previous acquisition projects were managed. Select suitable working methods and make adaptations to suit the actual project.

Form groups for all acquisition processes. Determine what roles are necessary for the project. Deter-mine the necessary skills for those roles. Employ the best and most competent staff as possible. Form groups for the differ-ent acquisition processes. Assign responsibility, authority, and accountability for the different project activities.

Develop an acquisition plan. Develop an appropriate Acquisition plan (see page 41). As a minimum the acquisition plan should cover the following:

o Define the acquisition life cycle, activities, and mile-stones to be used in the project. Address all acquisition processes as appropriate: project steering, project man-agement, planning and organizing, acquisition training, requirements management, risk management, contractor selection, contract management, product evaluation, transition to support, and post partum.

o Schedule and budget the project activities.

o Document how the project team is organized.

o Determine resource allocation. Resources to consider are: tools, equipment, facilities, materials, documents, etc.

Review the acquisition plan. All affected parties should review the plan.

6 Processes



Determine what type of help and services need to be ob-tained from external organizations besides the contractor.

Organize and establish commitments for all involved ex-ternal supporting organizations and individuals. Assign responsibility, authority, and accountability for the delegated project activities.

Develop an acquisition plan. A more elaborate and detailed plan should be made for con-trolled and robust processes, i.e. most of the items on the Acquisition plan checklist on page 41 should be covered.


None.

6.5 Acquisition training The purpose of the acquisition training process is to determine and develop the skills needed and the skills that the team members have in order to be able to perform their roles appro-priately.

Acquisition training process

Starts when the acquisition team has been assembled.

Ends when the training needs have been met.

Input: Output:

Acquisition training group

The participants to consider for this process should be those who can determine training needs of the staff and provide or

6 Processes


acquire the required training and those in need of training. Roles to consider for the acquisition training group are: acquisi-tion manager, technical manager, contract manager, administra-tor, acquisition expert, and all other roles in need of training.


Determine skills and knowledge needs. Determine the skills and knowledge that is required to per-form each role in the acquisition team.

Determine training needs. Determine the current skills and knowledge of the persons that will be designated to the different roles.

Determine and prioritize what training activities are essen-tial for the project. Develop individual training plans.

Determine who will provide the training and how the training will be performed.

Develop or acquire training material. Develop or acquire the necessary training material from within the organization or from external sources. Review the quality of the training courses and material.

Perform the training activities.

Determine if the performed training was sufficient or if more training is required.


Maintain training records. Maintain training records such as who has completed which training.


6 Processes


Spread knowledge. For long-term projects it might be of value to arrange semi-nars where different team members share their knowledge to the other team members.

6.6 Requirements management The purpose of the requirements management process is to elicit, review, and manage change of the product requirements. Apart from the frameworks in appendix A, this process has partially been based upon Christel and Kang (1992).

Requirements management process

Starts when the needs for the software have been identified.

Ends when the software has been transitioned to the sup-port and maintenance organi-zation.

Input: Project specification, Change requests

Output: Requirements specifi-cation

Requirements management group

The participants to consider for this process should be those whose needs are to be fulfilled by the software and those who will maintain and support the software. Other participants should be roles that can give advise and valuable input on how to formulate the requirements. Thus the following roles should be considered: sponsor, collaborating customer, product man-ager, acquisition manager, technical manager, contract man-ager, administrator, acquisition expert, technical expert, domain expert, legal expert, translator, end user, maintenance and support staff, independent verification and validation, system engineering and technical assistance, contractor, collaborating contractor, and subcontractor roles.

6 Processes



Elicit the requirements. Analyze the needs, conceptualizations, and requirements given in the project specification. Determine if the require-ments should be elicited mainly before or after contract award. Define and analyze the operational and problem con-text. Analyze similar systems. Define operational modes, goals, and scenarios. Get requirements wish lists from suit-able organizations, groups and individuals, especially end users and maintainers. Categorize and integrate the re-quirements. Consider possible future expansions of the sys-tem when eliciting the requirements.

Document the requirements. Document the requirements in a suitable form: use cases, user stories, or requirements statements. See the Requirements specification on page 44.

Prioritize the requirements. Identify the most critical requirements that have a strong in-fluence on usability, functionality, efficiency, reliability, portability, maintainability, other quality factors, perform-ance, cost, schedule, or risk and prioritize accordingly.

Establish evaluation criteria for each requirement.

Jointly review the requirements and evaluation criteria with the contractor prior to development start.

Review and approve change requests. Review and appraise all requirements change requests on what impact they will have on schedule, scope, cost, quality, and risks. Avoid creeping scope, goldplating, and other common problems. Document and inform all parties about all approved requirements changes.


Formally validate and verify the requirements prior to the

6 Processes


joint review with the contractor. Analyze the feasibility of the requirements and validate them against the needs of the organization. Verify that the requirements reflect end user needs, are verifiable, neces-sary, concise, implementation free, minimal, attainable, complete, consistent, and unambiguous. Verify especially that the requirements related to safety, security, and privacy protection are correct as shown by suitably rigorous meth-ods.

Measure the requirements change rate. Baseline the requirements before development start and measure the rate of changes to the requirements. The target value for total requirements growth in function points or equivalent is ≤1% per month according to Brown (1998).


Get external review on critical requirements. Let external experts review the requirements concerning safety, security, and privacy protection.

6.7 Risk management The purpose of the risk management process is to identify, man-age, mitigate, monitor, and handle the risks of the acquisition project.

Risk management process

Starts as soon as the need for acquiring the software is iden-tified.

Ends when the acquisition project is complete.

Input: Project specification Output: Risk list

6 Processes


Risk management group

The participants to consider for this process should be those who are in charge of managing the project and those who can give input on possible risks such as the following roles: spon-sor, collaborating customer, product manager, acquisition manager, technical manager, contract manager, administrator, acquisition expert, technical expert, domain expert, legal expert, end user, maintenance and support staff, independent verifica-tion and validation, system engineering and technical assis-tance, contractor, collaborating contractor, and subcontractor.


Encourage risk identification and management. Encourage and reward project-wide participation in the identification and mitigation of risks and foster a non-threatening environment in which the involved parties can discuss risks.

Identify risks and develop a project risk list. Identify the project risks, see section 6.7.1, Risk identification, on page 80. Develop a risk list that contains all identified risks, their estimated probability and impact. Identify “show-stoppers.” See the Risk list on page 43.

Analyze the developed risk list and decide which risks to take action on.

Conduct joint risk reviews. Review risks jointly with all affected parties. Establish a common view of risk scenarios and a mutual understanding and expectation of what can go wrong in the project.


Develop a more elaborate risk list. Add mitigation and contingency plans and other informa-tion as deemed necessary for all risks in the risk list. Most items in the Risk list check list on page 43 should be covered

6 Processes


for controlled and robust processes.

Continuously track risk status. Track risk status and update and maintain the risk list con-tinuously throughout the project. Track mitigation activities and reassess risks on the risk list on a regular basis. Analyze, track, and control risks until mitigated.

Establish means for reporting and communicating risk in-formation throughout the project team.


None.

6.7.1 Risk identification

A risk can be defined as a potential problem. The risks for a project can be identified via brainstorming, reviewing previous projects, reviewing checklists and taxonomies, etc. The factors to look for are those that are likely to impact the project objec-tives of scope, quality, time, and cost according to Wideman (1992). There are several taxonomies and checklists to choose from. The table below has been assembled and adapted from Carr, et al. (1993), D.I.R. (2000), and Wideman (1992). It can be used as a basis for risk identification and classification. You can also have a look at the problems listed in section 2.5, Common problems, on page 13. It is recommended to build a company risk database.

A. External risks

Risk category Problem examples Possible impact A.1 Natural hazards and accidents

Storm, flood, earth-quake, fire, etc.

Destruction of equip-ment and stored data.

A.2 Crime Vandalism, sabotage, theft, hacking, crack-ing, etc.

Destruction and theft of property and data.

A.3 Political unrest Revolution, war, etc. Cancellation of project.

6 Processes


Risk category Problem examples Possible impact A.4 Regulatory Unanticipated gov-

ernment intervention in product require-ments, pricing, export regulations, etc.

Increased total cost. Cancellation of project.

A.5 Economical factors Currency changes, in-flation, taxation changes, etc.

Increased total cost.

A.6 Indirect effects Unwanted social or environmental indirect effects from the use of the software.

Lawsuits, loss of com-pany credibility, etc.

B. Project and contract management risks

Risk category Problem examples Possible impact B.1 Project manage-ment

Insufficient support from corporate man-agement, poor com-munication, vague goals and direction.

Increased total cost. Increased delivery time. Staff morale is low-ered. Cancellation of project.

B.2 Acquisition staff Unavailability of suffi-ciently competent staff. Fragmentation.

Increased total cost. Increased delivery time. Cancellation of project.

B.3 Organizational needs

The product concept and needs of the or-ganization has not been sufficiently investigated.

Product gives no value to the end user. Benefit to develop product is less than costs.

B.4 Estimation Substantial underestimation of required time and budget.


B.5 Contractor selec-tion

Selection of a contrac-tor that is inexperi-enced, low-performing, non-cooperative, or in risk of bankruptcy.

Increased total cost. Increased delivery time. Poor quality. Cancellation of project.

B.6 Contract Inappropriate contract – not addressing neces-sary issues, or with too restricting or impeding clauses.

Friction between par-ties. Increased total cost. Increased delivery time.

6 Processes


Risk category Problem examples Possible impact B.7 Contract manage-ment

Over-administration. Increased total cost. Increased delivery time. Staff morale is low-ered.

B.8 Relationship be-tween parties

Friction, the parties do not communicate or work well together.

Increased total cost. Increased delivery time. Poor quality. Cancellation of project.

B.9 Coordination be-tween parties

Poor coordination be-tween parties regard-ing configuration changes, division of work, etc.

Rework. Increased total cost. Increased delivery time. Staff morale is low-ered.

B.10 Funding Unstable or insufficient funding.

Cancellation of project.

B.11 Schedule Time pressure – project is rushed.

Poor quality. Product doesn’t meet end user’s needs.

B.12 Project scale and complexity

Large-scale develop-ment of a complex sys-tem with many in-volved organizations.

Increased total cost. Increased delivery time.

B.13 Internal politics Conflicting political agendas between in-ternal groups and/or contracted groups.

Poor quality. Increased total cost. Increased delivery time. Cancellation of project.

B.14 Suppliers of hardware and software

Suppliers not deliver-ing as specified or not supporting critical sys-tem components in a timely manner.

Increased delivery time.

B.15 Legal risks Lawsuits. Force Ma-jeure.


C. Technical and product development risks

Risk category Problem examples Possible impact C.1 Requirements The requirements are

changing frequently due to incompleteness,

Rework Product doesn’t meet end user’s needs.

6 Processes


Risk category Problem examples Possible impact unclarity, or invalidity. Scope creep. Goldplat-ing.

Increased total cost. Increased delivery time. Underestimation of to-tal effort required.

C.2 Technical feasibil-ity

A requirement is im-possible to implement or a set of require-ments are conflicting and cannot be imple-mented simultane-ously. The design is more difficult to im-plement than antici-pated.

Product doesn’t meet end user’s needs. Increased total cost. Increased delivery time. Cancellation of project.

C.3 Product quality The delivered product is buggy and crashes frequently.

Damage to persons or expensive equipment. Lawsuits.

C.4 Project status indi-cators

Lack of indicators. In-adequate indicators. Reporting erroneous data. Reports are not given on a regular ba-sis.

Not knowing where project is heading. Di-minished ability to cor-rect the situation early on.

C.5 Development management

Inexperienced manag-ers. Inefficient organi-zation. Ineffective communications. No control over schedule and budget. Poor co-operation between de-velopment teams and with external organiza-tions such as other con-tractors.

Friction between par-ties. Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time.

C.6 Development staff Inexperienced staff. Low moral, enthusi-asm and productivity. No team spirit. High turnover. Difficult to recruit staff members. Bad attitude towards doing quality work and following stan-dards. Fragmentation.

Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time.

6 Processes


Risk category Problem examples Possible impact Loss of key personnel.

C.7 Development sys-tem

Inadequate develop-ment system. New sys-tem – training re-quired. Development system doesn’t match the user’s or the main-tainer’s system.

Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time.

C.8 Development process

Inefficient or unsuit-able process. New and unfamiliar process. Too complicated or administrative process. Process is not fol-lowed. Lack of control and coordination of process. Poor quality assurance and lack of configuration man-agement.

Rework. Poor quality. Increased total cost. Increased delivery time.

C.9 Design and coding Large parts of the product are unspeci-fied – developers have to guess. Harder to implement design than anticipated. Goldplat-ing.

Product doesn’t meet end user’s needs. Rework. Poor quality. Increased total cost. Increased delivery time.

C.10 Integration Integration of the soft-ware with COTS and hardware is harder than anticipated.

Rework. Increased total cost. Increased delivery time.

C.11 Testing and evaluation

Not sufficiently tested by developers. Not tested in a realistic en-vironment prior to de-ployment. Not ade-quately tested. Not evaluated by end user.

Rework. Product doesn’t meet end user’s needs. Poor quality. Increased total cost. Increased delivery time.

C.12 Maintenance Maintenance of prod-uct impaired by poorly designed product, lack of documentation, etc. Maintainer might go out of business.

Expensive to maintain. Increased total cost of ownership.

6 Processes


6.8 Contractor selection The purpose of the contractor selection process is to evaluate and select a suitable contractor for the acquisition project. For a more in-depth study of contractor selection, see Assmann (2000).

Contractor selection process

Starts when the initial set of requirements has been elicited.

Ends when the contract has been awarded.

Input: Project specification, Requirements specification, Risk list, Proposals

Output: Contract

Contractor selection group

The participants to consider for this process should be those who have the authority to sign a contract and those who can give advice on the selection of a suitable contractor and contrac-tual issues such as the following roles: sponsor, collaborating customer, product manager, acquisition manager, technical manager, contract manager, administrator, acquisition expert, technical expert, domain expert, legal expert, and translator.


Certify that the solicitation activities are conducted in ac-cordance with relevant laws, policies, and guidance.

Identify potential contractors. List contractors to send requests for proposals to. Well-known contractors should be primary candidates to minimize risk of getting unserious or over-optimistic bids according to Os-karsson (2000).

Make cost and time estimates. Use estimates by analogy or parametric models for minimal processes as they are the least time-consuming to do accord-

6 Processes


ing to Marciniak and Reifer (1990). Have the estimates inde-pendently reviewed.

Develop and send out a request for proposal to the poten-tial contractors. See the Request for proposal on page 47.

Select contractor. Select the winning contractor based upon all available data on the contractors such as the proposals (see page 50), even-tual audits, previous customer comments, previous data on contractor performance, etc. Remember that the lowest bid doesn’t necessarily mean the lowest total cost.

Develop and negotiate a contract. Select contract type (see page 88) and develop a suitable con-tract. Get legal assistance for this. Standard software con-tracts can be achieved from legal firms. Have legal counsel review all contracts. Determine how payment is to be made. Establish acquirer and contractor obligations. Incorporate the requirements and evaluation criteria in the contract. Es-tablish acceptance test procedures. See the Contract on page 51.

Jointly review the requirements. Review the requirements with the contractor prior to signing the contract to ensure a mutual understanding of the re-quirements.

Jointly review the contract. Review the contract prior to signing it to ensure that both parties have a common understanding of it. Especially ac-ceptance criteria and procedures, handling of changes to re-quirements, handling of problems after delivery, the obliga-tions of the acquirer, and legal issues regarding intellectual property, warranty, confidentiality, etc. should be reviewed.

Sign the contract.

6 Processes



Develop contractor evaluation criteria and evaluate the proposals against them. Develop Contractor evaluation criteria (see page 49) and evaluate the proposals against them and not against each other, according to Salwin (1998).

Audit the candidate contractors. Conduct audits at the candidate contractors’ sites; Evaluate their financial position, technical capability, experience, and quality practices. Review the experience of the contractor’s staff. Consider interviewing the staff members. Assess the stability of the contractor, is the organization undergoing change, such as changing ownership or moving office? In-terview previous customers. Check the background of the contractor and their staff. Evaluate the adequacy of the con-tractor’s development process. Evaluate the quality of previ-ous work; see code samples for instance. Evaluate the con-tractor’s overall capabilities and identify any limitations and liabilities in meeting the organization’s objectives.

Make cost and time estimates. Make bottom-up estimates based upon the work breakdown structure (WBS). Use the Delphi method, i.e. let various ex-perts reach a consensus on the most plausible estimates. Have the estimates independently reviewed.


Distribute proposal guidelines for review to potential bid-ders prior to issuing the formal request for proposal.

Consider using two-phase acquisitions. Consider diving the project into two separate phases with separate contracts. The initial phase is used to define the re-quirements, develop alternate design approaches, and proto-type implementations. The second phase consists of the full-scale development of the product. (Reifer, 1997)

6 Processes


Make cost and time estimates. Use several different estimation techniques, such as the Del-phi method, Monte Carlo-simulations, parametric estimates, etc. Have the estimates independently reviewed.

6.8.1 Contract types

Selecting contract type is a fundamental decision in the acquisi-tion process. There are two main types of contracts, fixed-price and cost-reimbursable, and a number of variants of them. In fixed-price contracts, the contractor assumes the main part of the cost risk. Because of this, the scope of the work needs to be well defined and attainable, and the work and any changes to the scope require strict management. In contrast, the customer assumes the main part of the cost risk for cost-reimbursable contracts. This requires an administrative overhead of keeping detailed and accurate cost and progress records. A benefit of cost-reimbursable contracts is that you can have a more flexible management of the work and changes to the scope, suitable for innovative projects or projects where the scope of the work is volatile. A drawback is that there is a lack of incentives to con-trol the cost and schedule, according to Urmi (2000).

According to Marciniak and Reifer (1990), there are two types of incentives: cost incentives and award incentives. Cost incen-tives can be used to motivate the contractor to shorten schedule or reduce cost. The fee is based upon how well the contractor meets cost or schedule objectives. However, a caveat should be given that if the target cost of the incentive fee structure is se-verely off target to the disadvantage of the contractor, the con-tractor might consider continuing to accrue labor hours in order to increase revenue and make up for the loss of fee. This possi-bility could be negated by having negative fees or penalties. The customer should also consider stopping the work of the contractor to protect the budget.

Award incentives can be used to motivate the contractor to meet certain success criteria regarding end user satisfaction and

6 Processes


software quality, or for using certain software development “best practices” such as peer reviews, configuration manage-ment, earned value, etc. Marciniak and Reifer (1990) argue that award fees need to be connected with the completion of a well-defined event. Brown, et al. (1998) presents some questions that should be of concern when using award incentives: How can the incentive targets be objectively evaluated? How will cases where factors beyond the control of the contractor impact the contractor’s achievement of the incentives criteria be dealt with? Are the additional administrative costs that will incur in managing the contract outweighed by the benefits achieved?

In the tables below the different variations on these two con-tract types and the time and materials contract type are pre-sented. Each table contains a brief description on how the con-tract works, when it is applicable, who takes the main cost risk – contractor or customer, and also rates its administration level on a subjective scale from very low to very high. The contract types presented are the most common, although there are other contracting arrangements, such as basic ordering agreements, etc. When choosing contract type you could consider dividing the project into phases, such as a requirements elicitation phase followed by a software development phase, and have different contracting vehicles for each phase.

The contents of the contract types presented below are based upon Brown et al. (1998), Marciniak and Reifer (1990), Reifer (1997), and Wideman (1992).

FFP – Firm fixed price

Description The customer pays a fixed price for the con-tractors work.

Applicability FFP contracts apply for projects where the cost estimates are fairly reliable, the scope of the work is well defined and non-volatile, and the attainability of the project is very likely.

6 Processes


Cost risk Customer ��

��

��

��

��

��

Contractor

Administration Very low – if a lump sum is paid at final de-livery. Low – if partial payments are used (e.g. monthly). High – if partial progress payments are used (e.g. milestone driven).

FPEPA – Fixed price with economic price adjustment

Description A fixed price contract with price adjustments for certain costs for material, labor, travel ex-penses, etc.

Applicability Similarly to FFP contracts, FPEPA contracts apply for projects where the scope of the work is well defined and non-volatile and the attainability of the project is very likely. The FPEPA contract is specifically applicable for lengthier contracts where both parties need to be protected from certain cost fluctuations, such as airfares, etc. This contract can be combined with FPR.

Cost risk Customer

��

��

��

��

��

��

Contractor

Administration Moderate – with lump sum payment. High – with partial payments.

FPR – Fixed price redetermination

Description An initial fixed price is negotiated. Certain factors are agreed upon à priori for adjusting the price at specific times such as require-

6 Processes


ments growth, changes in function points, etc.

Applicability The FPR contract applies to situations where the scope of the work is partially well defined and non-volatile and partially volatile.

Cost risk Customer

��

��

��

��

��

��

Contractor

Administration Moderate

FPIF – Fixed price plus incentive fee

Description In addition to paying a fixed price, a variable fee is determined by negotiating the follow-ing parameters:

• Target and ceiling cost • Minimum, maximum, and target fee • Adjustment formula (or share ratio)

The adjustment formula specifies how target cost underruns and overruns – without going over the ceiling cost – will be shared between the customer and contractor.

Applicability Similarly to FFP contracts, FPIF contracts ap-ply for projects where the cost estimates are fairly reliable, the scope of the work is well defined and non-volatile, and the attainability of the project is very likely. FPIF contracts can specifically be used as an instrument to moti-vate the contractor to increase efficiency, shorten schedule, and reduce cost. This con-tract can be combined with FPR.

Cost risk Customer

��

��

��

��

��

��

Contractor

6 Processes


Administration Moderate – with lump sum payment. High – with partial payments.

FPAF – Fixed price plus award fee

Description The contractor is paid a fixed price and a base fee. In addition, the contractor receives award fees when certain criteria are met. The base fee should be set low, around 3% of the target cost according to Marciniak and Reifer (1990), and the maximum fee could go up to 20%.

Applicability Similarly to FFP contracts, FPAF contracts apply for projects where the cost estimates are fairly reliable, the scope of the work is well defined and non-volatile, and the attain-ability of the project is very likely. FPAF con-tracts can specifically be used as an instru-ment to stimulate the contractor’s perform-ance in certain areas that are difficult to measure objectively such as end user satisfac-tion, software quality, use of “best practices,” etc.


��

��

��

��

��

Contractor

Administration High

CS – Cost and cost share

Description In a cost share contract the customer and con-tractor agree on the ratio by which they will share costs. In a cost contract the customer pays for all allowable costs. The contractor re-ceives no fee in either contract type.

6 Processes


Applicability Cost share contracts apply for joint develop-ment efforts where both parties share risk in order to share the future rewards from the product. Cost contracts apply for projects with nonprofit organizations.

Cost risk 100% by the customer for cost contracts. Shared, according to the agreed ratio, for cost share contracts.

Administration High

CPFF – Cost plus fixed fee

Description The contractor's allowable costs are reim-bursed and a fixed fee is paid upon delivery.

Applicability CPFF contracts apply for projects where the costs cannot be reasonably estimated such as innovative projects or projects where the scope of the work is volatile or difficult to de-fine.

Cost risk Customer

��

��

��

��

��

�� Contractor

Administration High

CPPF – Cost plus percentage fee

Description The contractor’s allowable costs are reim-bursed and a percentage of the costs is re-ceived as a fee.

Applicability CPPF contracts apply for projects where the costs cannot be reasonably estimated such as

6 Processes


innovative projects or projects where the scope of the work is difficult to define.


��

��

��

��

��

Contractor

Administration High

CPIF – Cost plus incentive fee

Description In addition to reimbursing the contractor’s al-lowable costs, a variable fee is determined by negotiating the following parameters:

• Target and ceiling cost • Minimum, maximum, and target fee • Adjustment formula (or share ratio)

The adjustment formula specifies how target cost underruns and overruns – without going over the ceiling cost – will be shared between the customer and contractor.

Applicability Similarly to CPFF contracts, CPIF contracts apply for projects where the costs cannot be reasonably estimated such as innovative pro-jects or projects where the scope of the work is difficult to define. CPIF contracts can spe-cifically be used as an instrument to motivate the contractor to increase efficiency, shorten schedule, and reduce cost.

Cost risk Customer

��

��

��

��

��

��

Contractor

Administration Very high

6 Processes


CPAF – Cost plus award fee

Description The contractor is reimbursed for allowable costs and receives a base fee. In addition, the contractor receives award fees when certain criteria are met. The base fee should be set low, around 3% of the target cost according to Marciniak and Reifer (1990), and the maxi-mum fee could go up to 20%.

Applicability Similarly to CPFF contracts, CPAF contracts apply for projects where the costs cannot be reasonably estimated such as innovative pro-jects or projects where the scope of the work is difficult to define. CPAF contracts can spe-cifically be used as an instrument to stimulate the contractor’s performance in certain areas that are difficult to measure objectively such as end user satisfaction, software quality, use of “best practices,” etc.

Cost risk Customer

��

��

��

��

��

��

Contractor

Administration Very high

TM/L – Time and materials / Labor hours

Description The contractor is paid an hourly rate for the time to perform the agreed upon work and is reimbursed for any necessary purchases.

Applicability TM/L contracts apply for projects where the extent and duration of the work cannot be reasonably estimated. The contract type suits projects where you want to be able to quickly change the direction of the work and can monitor the work closely. A ceiling price

6 Processes


should be determined.

Cost risk Customer

��

��

��

��

��

��

Contractor

Administration Low

The table below gives suggestions for what contract types to consider for the different customization levels:

Minimal FFP, FPR, FPIF, TM/L

Controlled FPEPA, FPR, FPIF, CS, CPFF, CPPF, CPIF

Robust FPEPA, FPR, FPIF, FPAF, CS, CPFF, CPPF, CPIF, CPAF

6.9 Contract management The purpose of the contract management process is to monitor the work of the contractor, gather data on project status, resolve problems, administer payments, and other contractual issues.

Contract management process

Starts when the contract has been signed.

Ends at the conclusion of the contract’s period of perform-ance.

Input: Contract Output: Measurements

Contract management group

The participants to consider for this process should be those with the ability and authority to administer the contract, resolve contractual problems and issues, and those who can give assis-

6 Processes


tance in monitoring and assessing the progress of the contractor such as the following roles: acquisition manager, contract man-ager, administrator, acquisition expert, technical expert, domain expert, translator, and legal expert.


Perform the acquirer’s contractual obligations. Perform the acquirer’s obligations as stated in the contract such as delivering software, hardware, data, or other deliv-erables essential for the development of the software product in a timely manner and with the agreed specifications.

Gather project status data. Determine what to monitor, for instance measurements on budget, schedule, progress, risk, scope, quality, etc. Establish routines for gathering the data. See Project status metrics on page 100. Require frequent status reports etc. from the con-tractor (see Artifacts from the contractor on page 53).

Review budget and schedule status. Track budget and schedule and identify trends. Compare to plan and what was accomplished. See Earned value on page 102. Watch out for delays, small delays accumulate. Force replanning if schedule slippage is greater than 10% accord-ing to Brown (1998). (Oskarsson, 2000)

Evaluate the software product regularly. See the Product evaluation process on page 106.

Report measurements and problems. Report measurements on the process and product, etc. on a regular basis to the acquisition manager for inclusion in the next Project status report (see page 56) to the steering group. Also give feedback to the contractor. Eventual problems should be reported immediately.

Communicate regularly and promote a supporting rela-tionship with the contractor. Create an effective, functional, supporting and collaborative

6 Processes


culture in the relationship with the contractor built on trust. Both parties should realize that they have a mutual respon-sibility for the success of the project. The involved parties should work as a team, jointly solve problems, and refrain from blaming each other. Conduct regular project meetings with all involved parties to give feedback and discuss pro-gress, requirements, budget, schedule, risks, problems, etc.

Manage contractual change. Coordinate all contractual changes with all affected groups and individuals. Ensure that all proposed changes to the contract are analyzed, documented, agreed upon or denied and communicated to all affected parties.

Administer partial payments, incentives, etc.

Settle the contract. On final delivery and acceptance of the product, the contract should be completed and settled, any open issues should be resolved. Final payment should be withheld until all is de-livered. Check that deliveries have been completed success-fully, ownership has been transferred, etc. See also the Transition to support process on page 110.


Review the contractor’s plans. Review the contractor’s plans regarding project manage-ment, risk management, quality control, configuration man-agement, integration, and testing and use them as a basis for overseeing the contractor’s development effort. The contrac-tor’s use of subcontractors, software/hardware suppliers, and tools should be reviewed and approved.

Reestimate total cost and delivery time when appropriate. Base the estimates on WBS and extrapolate from actuals.

Audit the contractor on a regular basis. Hold audits to determine project and product status. Audits can be performed both on predetermined occasions and on

6 Processes


an ad hoc basis. Audit procedures should be agreed upon by both parties. Analyze the outcome of the audit, identify trends, and adjust the acquisition approach to mitigate risks if necessary. The outcome regarding risks, problems and eventual change of project direction should be discussed with the contractor. The outcome of the audit and responsi-bilities for resolving found problems should be agreed upon by both parties and documented. Technical experts or ac-quisition experts can be of assistance during audits. Examples on what to review:

o That the working methods of the contractor regarding methodologies, processes, procedures, practices, qual-ity assurance, etc. are adhered to and conducted in an effective fashion.

o That the contractor has planned the project adequately and timely.

o That the project is staffed appropriately, that the staff is working on the actual project, and that the personnel is trained as required by the contract.

o That the allocated resources are appropriate.

o That documents and deliverables meet specifications and follows coding and documentation standards, etc.

o That the technical approach and working methods are appropriate.

o The configuration management practices.

o Risk mitigation activities.

o Interview managers and technicians on technical and managerial difficulties.

6 Processes



Use external auditors. For longer projects you could have an external auditor visit the contractor and review the status of the project, interview the managers and especially technicians about the project and eventual problems. The auditor can also look at random error reports and track the error as a test of the contractor’s configuration management routines, etc. (Oskarsson, 2000)

6.9.1 Project status metrics

It is important to choose metrics that motivates the right behav-ior, or in the words of Jim Coplien:

What gets measured gets done.

The following are examples of metrics that can be used to measure the status of the project:

• Requirements change rate. The target value for total re-quirements growth is ≤ 1% per month. (Brown, 1998)

• Contractor staff turnover rate. The target value for volun-tary staff turnover per year is 1-3%according to Brown (1998). High staff turnover indicates low staff moral.

• Contractor staff overtime rate. (Brown, 1998) • Contractor staffing rate. Actual staffing compared to

needs. (Marciniak and Reifer, 1990) • Test progress. Executed test cases compared to planned,

successful tests, tests with errors, etc. (Oskarsson, 2000) • Software maturity index, SMI. Indicates the stability of

the product. The product is stabilizing when SMI ap-proaches 1.0. (IEEE 982.1-1988) MT = number of modules in current version. Fc = number of changed modules in current version.

6 Processes


Fa = number of added modules in current version. Fd = number of deleted modules in current version.

MT – (Fa + Fc + Fd) SMI=

MT

• Developmental progress. Function points completed, re-quirements completed, incremental releases, modules completed, etc. (Marciniak and Reifer, 1990)

• Milestones completed. Actual vs. planned, see the figure below. (Oskarsson, 2000)

Figure 6. Milestone tracking and trend diagram, Oskarsson (2000).

• Problem reports. Open vs. closed. Marciniak and Reifer (1990) define problems as anomalies discovered in re-quirements, design, or software.

• Size growth. How the size of each software item grows as a function of time. Size is directly related to cost accord-ing to Marciniak and Reifer (1990).

• Error rate. The amount off discrepancies found during testing and evaluation. High error rates indicate product immaturity and low quality according to Marciniak and Reifer (1990).

• Earned value metrics. See Earned value, on page 102.

1 2 3 4 Time

Milestones

Time

Replan 2

Replan 1

6 Processes


6.9.2 Earned value

According to Fleming and Koppelman (1996) the concept of earned value has been around for at least a hundred years, first used on the factory floors in the late 1800s. The usage of earned value enables the prediction of final costs and final schedule re-sults, within a certain range, for the actual project. Since 1967, every new major systems project implemented by the United States Department of Defense (DoD) has been required to use the Cost/Schedule Control Systems Criteria, or C/SCSC for short, which is a set of 35 requirements embedding the earned value concept. Over 700 major DoD contracts since 1977, where C/SCSC (and thus earned value) has been used has shown that you can, as early as 15-20% into the project, predict the final costs and time requirements within a predictable range of val-ues.

Fleming and Koppelman (1996) suggests a simplified version of earned value to be used for virtually any project who relates planned costs to actual expenses and don’t have a clue about what they got for what they spent. I agree with them and sug-gest that you can use the concept for any size of project and with any type of development model, even iterative.

Earned value project requirements

The earned value concept requires the following:

1. An appropriate defined scope of the project (whole pro-ject or iteration), or work to be done, which enables you to tell what’s in the scope and what’s not. This is preferably made with a work breakdown structure, WBS, down to the necessary level of granularity. For cost-reimbursable contracts, the buyer defines the top two or three levels of the WBS. For fixed-price contracts the contractor defines all of the WBS. The work should be defined down to dis-crete work tasks that can be managed and easily deter-mined complete not complete.

6 Processes


2. The work segments defined in the previous step must be planned and scheduled. Typically, CPM1 is appropriate to accomplish this. If several contractors and subcontractors are used, their schedules must of course be in accordance with a master schedule.

3. Responsibility for each work segment should be assigned and resources will need to be estimated and budgeted for all work segments within the defined project scope. When estimating, refrain from adding reserve time. Do this later when all of the work segments have been estimated. Thus, the so called Cost Account Plan should contain the following:

a. Statement of work (scope).

b. Schedule (start/stop dates for each task).

c. Budget (in dollars or hours or units).

d. Responsible person.

e. Responsible department.

f. Type of effort (recurring or non-recurring).

g. Division into discrete work packages.

h. Method used to measure earned value performance (milestone, formula, percent complete, standards, apportioned).

4. Establish a baseline of the estimated, scheduled and budgeted work segments in the previous step, not includ-ing any reserves, from which project or iteration perform-ance can be measured. The baseline should be under con-

1 Critical Path Method

6 Processes


figuration management to enable traceability back to the original baseline. All changes to the baseline have to be reviewed, controlled, and approved or rejected. The base-line is needed to be able to tell how much of the project is finished, to be able to compare completed work against actual work and actual costs.

5. Monitor iteration/project performance against the base-line and forecast final results. Also manage baseline changes to maintain the baseline current.

An example

Assume you have a one-year $100,000 project scheduled with an expenditure rate of $25,000 per quarter. At the end of the first quarter the actual expenditures are $22,000 and thus $3,000 under the planned effort. From these values you can’t really tell whether the project is behind schedule or underrunning costs.

By adding the earned value dimension, which measures the value of the actual work accomplished, we can tell much more about the actual status of the project. Let’s say that the work segments completed at the end of the first quarter had an earned value of $20,000. Then we can immediately tell that we

are $5,000 behind the planned work of $25,000. Since the actual expenditures to accom-plish the work are $22,000 we can also tell that we have a cost overrun of 10%. So, with earned value we are able to tell that the actual project is running late and overspending. See the figure to the left.

Figure 7. Earned value example.

$

1Q

25 000 Planned

22 000 Spent20 000 EV

6 Processes


Testable requirements

Peter B. Wilson (2000) suggests the use of testable requirements with earned value. A testable requirement is defined as a re-quirement that someone is able to write one or more test cases for that would validate whether the requirement has or has not been implemented correctly.

As an example you plan to implement 80 testable requirements in 500 labor hours during the first month of the project. When the month has passed 475 labor hours has been expended and 72 of the requirements have been implemented correctly. This gives you an earned value of (72/80) * 500 = 450 labor hours. So in this case you overspent 25 labor hours.

Schedule and cost performance indices

Schedule performance index, or SPI, is determined by the follow-ing formula:

Earned Value SPI =

Planned Value

This value tells you how much of the work planned to be ac-complished actually was accomplished. A SPI < 1.0 is a warning signal. The SPI can be a valuable tool for use in conjunction with critical path analysis to forecast the expected completion date for the project.

Cost performance index, or CPI, is determined by the following formula:

Earned Value CPI =

Actual Costs

This value tells you how much work was accomplished for every project dollar spent. A CPI < 1.0 is a warning signal. The CPI can be used to forecast a statistical range of estimated final costs to complete the project.

6 Processes


Forecasting final cost and schedule

A realistic bottoms-up estimate of the remaining tasks is the most desirable forecasting method, but CPI and SPI can be used to produce statistical forecasts of final required funds to com-plete the project.

Statistical range of cost-to-complete forecasts:

Total budgeted funds Low end forecast = (Best case) CPI

Total budgeted funds High end forecast = (Most likely) CPI * SPI

The To-Complete Performance Index, or TCPI, can be used to de-termine what performance level it will take to accomplish all remaining effort to achieve some specified management objec-tive.

Earned Value of work remaining TCPI =

Funds remaining

6.10 Product evaluation The purpose of the product evaluation process is to evaluate the deliverables from the customer to see if they meet the specified requirements and quality objectives. This includes evaluation of partial or incremental deliveries and the final evaluation, i.e. ac-ceptance test.

6 Processes


Product evaluation process

Starts when the requirements are being developed.

Ends when the product and other deliverables have been finally accepted and delivered as agreed.

Input: Requirements specifica-tion

Output: Product evaluation reports, Change requests

Product evaluation group

The participants to consider for this process should be those whose needs are to be fulfilled by the product, those who can give assistance and advice on the evaluation of the deliverables, and those who have the authority to accept the deliverables such as the following roles: sponsor, collaborating customer, product manager, acquisition manager, technical manager, administrator, technical expert, domain expert, legal expert, translator, end user, maintenance and support staff, independ-ent verification and validation, and system engineering and technical assistance.


Determine what testing approaches to use. Determine the appropriate testing approaches and the test-ing effort required. See Product evaluation approaches on page 109.

Identify the artifacts to be evaluated.

Write test cases up-front. Make sure that evaluation criteria have been established for all the requirements with instructions on how to perform the evaluation for each requirement. Establish evaluation proce-dures and criteria for all chosen testing approaches.

Conduct periodical evaluations. Conduct periodical evaluations throughout the acquisition

6 Processes


project to minimize the risk of getting a product that doesn’t live up to the expectations of the end user. Write plans for the different evaluations if that is deemed necessary (see the Product evaluation plan on page 54).

Analyze and take action on evaluation results. Certify that the coverage of the tests is sufficient. Analyze the test results regarding requirements and quality fulfill-ment and determine what action needs to be taken for the found deficiencies. Determine if any changes need to be made to the requirements. Make appropriate documentation of the test results (see the Product evaluation report on page 55).

Review the evaluation results with the contractor. Discuss the found deficiencies, non-conformances, remedies, and eventual requirements changes with the contractor. Don’t forget to give some positive feedback too to maintain good relations with the contractor.

Conduct acceptance testing. Conduct the procedures for final evaluation and acceptance of the product as agreed upon in the contract. Use all evalua-tion results as a basis for accepting the product.


Document requirements change requests. See the Change request on page 55.

Track found deficiencies. Document and follow-up on the deficiencies that were found during the evaluations.

Overview the contractor’s testing efforts. Review the results of the contractor’s testing activities.


Identify common errors.

6 Processes


Collect statistics on common errors. Suggest improvements and fault preventions.

Consider using IV&V for products that can affect people’s health or huge amounts of money should they malfunc-tion.

6.10.1 Product evaluation approaches

The following is an overview of some different approaches for evaluating a software product. Several of these tests should be conducted throughout the project and during the final accep-tance testing. It is important to remember that tests cannot show that a product is 100% error-free. I’ve not included tests that are primarily performed by the contractor such as unit and integration tests, but limited the list to tests in which the cus-tomer takes part:

Alpha tests are conducted on pre-release versions of the soft-ware product by the customer in a controlled environment at the contractor’s site with developers present. (Pressman, 1997)

Beta tests are conducted on pre-release versions of the software product by end users at one or more customer sites. The con-tractor is normally not present during beta tests. (Ibid.)

Functional tests demonstrate that the functional requirements have been met. These tests can be conducted by working through use cases or performing several tests defined for each requirement or other means. Not only correct but also errone-ous inputs should be given to the system.

Installation tests should be conducted to ensure that the prod-uct can be installed correctly.

Support tests should be conducted to ensure that the needs of the support and maintenance organization are met.

6 Processes


Recovery tests forces the software to fail and verifies that the system recovers properly. (Ibid.)

Security tests should be performed to verify that the protection mechanisms built into a system will in fact protect it from im-proper penetration. (Ibid.)

Stress tests executes a system in a manner that demands re-sources in an abnormal quantity, frequency, or volume in an at-tempt to break the program. (Ibid.)

Performance tests are important for real-time and embedded systems to see if they fulfill the performance requirements. (Ibid.)

User interface tests are important tests performed by the end users to evaluate the user interface’s appropriateness.

6.11 Transition to support The purpose of the Transition to support process is to prepare for a smooth transition of the deliverables to the support and main-tenance organization and to verify that all deliverables and other services, such as user training and installation, has been received as agreed upon.

Transition to support process

Starts when the software re-quirements are being devel-oped.

Ends when the software and other deliverables have been turned over to the support or-ganization.

Input: Deliverables Output: Maintenance plan

Transition to support group

The participants to consider for this process should be those in charge of approving the delivery of the product and those who

6 Processes


are to install, maintain and support the product such as the fol-lowing roles: acquisition manager, technical manager, contract manager, administrator, technical expert, domain expert, legal expert, maintenance and support staff, and contractor, collabo-rating contractor, and subcontractor.


Identify the support and maintenance organization. Identify and provide a budget for the support and mainte-nance organization early in the project, before the request for proposal is issued. Will maintenance be performed by the ac-quirer organization or will it be outsourced?

Identify the deliverables to be transferred. Make a list of the deliverables that are to be transferred to the support organization, such as maintenance and support documentation, software, training materials, configuration management systems, etc.

Plan the installation of the software product. Establish installation procedures regarding schedule, access to facilities, personnel, availability and access to systems and equipment, approval of installation, etc.

Review the results of the user training activities. Make sure that the agreed upon user training has been per-formed.

Verify that all deliverables has been received as specified.


Prepare the support and maintenance organization. Transfer and customize necessary processes from the devel-opment organization to the support organization. A mainte-nance plan should be developed. See the Maintenance plan on page 46.

Review the deliverables and preparations prior to transfer.

6 Processes


Review the usability of the software, its readiness for instal-lation, status of user training, user manuals, status of instal-lation preparations and activities. Review the readiness of the software for transition, the maintenance plans and manuals, the status of transition preparation activities. Re-view copyright and licensing concerns addressed and agreed to. Review concerns regarding back-up copies of the soft-ware.


Test the support capabilities of the support and mainte-nance organization. Before transferring responsibility for the software, the sup-port organization needs to demonstrate its capability to sup-port and maintain the software:

o Does it have the appropriate hardware, software, physical, fiscal, and personnel resources?

o Does it have a maintenance plan and other documenta-tion, processes and procedures?

o Does it have a configuration management system ca-pable of supporting the software?

o Is the personnel trained?

Maintain configuration and change management through-out the transition.

6 Processes


6.12 Post partum The purpose of the post partum1 process is not about passing judgment, but to learn from experience. What can be done bet-ter on future acquisition projects? It also includes documenting data on the project effort and on the contractor’s performance for future reference. Apart from the frameworks mentioned in appendix A, this process is based upon Kerth (n.d.).

Post partum process

Starts when the project has been finalized.

Ends when the project has been sufficiently reviewed and documented for future refer-ence.

Input: Project data Output: Post partum

Post partum group

Preferably representatives from all involved organizations and roles should participate in the post partum.


Form a post partum group. Also assign responsibility, authority, and accountability for the post partum activities. Make sure that the people in-volved in the process are trained in how to perform the process activities.

Allocate adequate resources for post partum activities. Resources to consider are: funding, people, tools, equipment, facilities, etc.

1 Post partum means “after birth” in Latin. I find this to be a more appro-priate name for the process instead of the more common term post mortem, which means “after death.”

6 Processes


Set a goal for the post partum. There can be many different purposes for holding a post par-tum:

o To capture effort data in order to understand the exist-ing process, or to improve the existing process, or to provide data for future acquisitions, or to identify training needs, etc.

o To document what happened and why. To capture the collective wisdom.

o To repair damage to the team.

o To enjoy the accomplishment.

Plan the post partum. See the Post partum plan on page 57.

Prepare for the post partum. Inform the participants that the post partum is a learning experience and not a blame session. Establish rules that cre-ate a non-threatening atmosphere for the post partum. Gather the data that will be analyzed during the post partum such as data on efforts, resources, product quality, etc. Each participant should refresh their memory and individually review what happened.

Conduct the post partum review. Review what happened during the acquisition project and identify good and bad acquisition practices. What went wrong and what went right? Review the efforts expended regarding budget, staff, schedule, etc. Identify deviations and reasons to them. What was accomplished? What was the quality of the delivered product? Review any special diffi-culties and how they were handled. Review the contractor’s performance and compliance with requirements. If the con-tractor doesn’t participate in the post partum review, a post partum from the contractor should be reviewed. Give im-provement suggestions. What should be done different the

6 Processes


next time?

Document data and lessons learned. Identify and document the software acquisition lessons learned, project data, customization data, and data on the contractor’s performance. See the Post partum on page 58.


None.


Get a skilled facilitator. A suitable facilitator is preferably an outsider to the organi-zations involved in the project, though knowledgeable about software projects and with good people interaction skills.

116

7 Results and summary

“Caution: Cape does not enable user to fly.”

– Batman costume warning label

This chapter summarizes the guidelines and gives an outlook on possible future research on software acquisition manage-ment.

7.1 Guidelines summary The objective of this master’s thesis is to provide guidelines from the viewpoint of the customer for managing an acquisition project concerning an outsourced development of a software product customized for the specific needs of the customer. The objective is further broken down into seeking answers to the following problems:

• What problems are common?

• What activities should be performed?

• What roles are conceivable?

• What artifacts are conceivable?

• How can the activities, roles, and artifacts be customized for different kinds of projects?



The guidelines presented in this thesis can be used as a basis for planning an acquisition project or be read as an overview over the many issues that concern software acquisitions.

The following summarizes the answers to each of the above stated questions briefly:

What problems are common?

The most common problems with software acquisition projects are the following:

• Development costs exceeds budget. • Time schedule is overrun. • Outcome does not meet expectations.

According to the Standish Group (1995) Chaos report, which surveyed over 8 000 software development projects, only 16% of the examined projects were completed on time, within budget, and with all features and functions as initially speci-fied.

What activities should be performed?

There are many activities that can and should be performed for software acquisition projects. In this thesis they have been col-lected into the following eleven processes:

• Project steering • Project management • Planning and organizing • Acquisition training • Requirements management • Risk management • Contractor selection • Contract management • Product evaluation



• Transition to support • Post partum

The activities and guidelines presented in the processes can be summarized into the following high-level activities and guide-lines:

1. Define and communicate a goal and vision for the pro-ject.

2. Appoint an acquisition manager that has the ultimate re-sponsibility for the success of the project. The role of the project sponsor is to set the outer limits for the project, provide stable and adequate funding and to support the project enthusiastically or cancel it.

3. Adapt and customize the acquisition approach and strat-egy according to the characteristics of the project.

4. Avoid having too much administration, but as a mini-mum: have written requirements and agreements, docu-ment important decisions and why they were taken, also measure progress, software quality, and requirements changes quantitatively.

5. Know what you want. Have as stable, complete, feasible, and validated requirements as possible. It is important to have a defined scope so that you and the contractor know when you are done and when the contract needs to be changed.

6. Define how the fulfillment of each requirement will be evaluated.

7. Ensure end user involvement in the definition of the product requirements and in the evaluation of the prod-uct.

8. Tell the contractor what you want and not how to do it. Jointly review the requirements before the development starts to sort out misunderstandings and ambiguities.

9. Choose contractor carefully. The contractor with the low-est bid or most optimistic schedule and budget estimates



isn’t always the best choice. No management practices can make up for having a bad contractor.

10. Create an effective, functional, supporting, and collabora-tive culture in the relationship with the contractor based on trust and mutual benefit. Both parties should realize that they have a mutual responsibility for the success of the project. The involved parties should work as a team, jointly solve problems, and refrain from blaming each other.

11. Establish effective communication channels and break down barriers between people and departments in the organizations involved in the project. Ensure that you understand each other.

12. Don’t take advantage of the contractor. Create a win-win situation. Have a mutually beneficial contract that you would be comfortable signing as either party.

13. Expect the best, but be proactive and prepare for all even-tualities. Jointly discuss possible risks and how they should be mitigated with the contractor.

14. Employ the best and most competent people that is pos-sible, train them, plan and organize their work. Provide necessary assistance and resources.

15. Plan for maintenance and identify the maintenance and support organization before issuing request for propos-als.

16. Evaluate the evolving software product early and often prior to final acceptance evaluation to reduce risk of get-ting a software product that won’t meet the needs of the end user.

17. Perform reality checks periodically. Take necessary ac-tions if the answers to the following questions aren’t sat-isfactory: o Are we acquiring the right software? o Are we acquiring the software right? o Is the contractor building the right software? o Is the contractor building the software right?



18. Be realistic and reassess your expectations as you learn more about the problem domain and the technical feasi-bility of the program as the project progresses. The initial estimates and specifications of the following four pa-rameters are likely to change to some degree during the course of the project: o Delivery time o Development costs o Product scope o Product quality

You may therefore need to make trade-offs to achieve the most important goal of the project.

What roles are conceivable?

The roles that are conceivable for a software acquisition project can be divided into four categories:

The sponsor roles provide the funding for the project and have the power to cancel the project. The sponsor roles presented in this thesis are the following: sponsor, collaborating customer, and product manager.

The managing roles plans, manages, monitors, and administers the project. The managing roles presented in this thesis are the following: acquisition manager, technical manager, contract manager, and administrator.

The assisting roles are played by different experts and support contractors that can assist and give advise to the managing, steering and executing roles. The assisting roles presented in this thesis are the following: acquisition expert, technical expert, domain expert, legal expert, translator, end user, maintenance and support staff, independent verification and validation, and system engineering and technical assistance.

The executing roles are played by representatives from the con-tractors side, i.e. those developing the actual software product.



The executing roles presented in this thesis are the following: contractor, collaborating contractor, and subcontractor.

What artifacts are conceivable?

There are many artifacts that are conceivable for a software ac-quisition project. The artifacts in this thesis are presented ac-cording to which phase of the project they are most likely to appear for the first time:

During the inception phase the needs of the software are iden-tified, the requirements are elicited, risks are analyzed, and the acquisition processes are planned. The artifacts presented in this thesis for the inception phase are the following: acquisition proposal, project specification, acquisition plan, risk list, re-quirements specification, and maintenance plan.

During the solicitation phase requests for proposals are sent out to potential contractors, proposals are received and evalu-ated, and a contractor is selected with which a contract is nego-tiated and signed. The artifacts presented in this thesis for the solicitation phase are the following: request for proposal, con-tractor evaluation criteria, proposal, and contract.

During the monitoring phase the progress and performance of the contractor is monitored and the software product is evalu-ated on a regular basis. The artifacts presented in this thesis for the monitoring phase are the following: project management plans, test reports, problem reports, payment requests, progress reports, change requests, quality reports, software demonstra-tions and incremental releases, product evaluation plan, prod-uct evaluation report, change request, and project status report.

During the finalization phase the product and other deliver-ables are delivered to the support and maintenance organiza-tion and a post partum review is held. The artifacts presented in this thesis for the finalization phase are the following: software executables, source code, technical documentation, user train-ing material, rest list, user manual, support publications, post partum plan, and post partum.



How can the activities, roles, and artifacts be customized for different kinds of projects?

To answer that question we must first find a way to differenti-ate projects. This thesis examines a number of customization fac-tors that are used to suggest an appropriate formality level of the acquisition processes based upon the project’s complexity and risk.

By examining these factors you can determine which customiza-tion level is appropriate for your project. Three different cus-tomization levels are given in the thesis:

Minimal processes: suitable for small projects with stable, well-defined and attainable requirements, developed by a small, ef-ficient team from a well-known contractor, etc.

Controlled processes: suitable for somewhat lengthier me-dium-sized projects with somewhat volatile requirements, de-veloped by an average contractor, etc.

Robust processes: suitable for large, long-term projects, with innovative, critical, and volatile requirements, and with a com-plex set of contractors and subcontractors involved in the de-velopment effort, etc.

The thesis suggests what activities are appropriate for the dif-ferent customization levels, and also gives some suggestions for customization of roles and artifacts.

7.2 Outlook Software acquisition management involves a wide variety of topics ranging from general management theory to special is-sues regarding software testing, contract types, risk manage-ment, etc. It isn’t possible for a single document to cover any of these topics completely. Therefore, the reader probably needs to study texts that cover these topics in depth to gain a better un-derstanding on how to carry out a successful software acquisi-tion project.



However, the following are some suggestions for future re-search on software acquisition management that could be based upon this thesis:

• It could be further investigated how to customize and adapt the different processes and artifacts. Perhaps two customization levels are sufficient?

• Are there more customization factors? Which factors are important for which process?

• Similar guidelines for the acquisition of COTS software could be made.

• Can an expert system or other tools be made for assisting software acquisition management?

• How can the guidelines be adapted to match common software development methodologies?

124

8 Bibliography

Assmann, Danilo (2000), MASS – a Method for Assessing Soft-ware Subcontractors, Kaiserslautern, Fraunhofer IESE

Beck, Kent (1999), Embracing Change with eXtreme Programming, Computer, Vol. 32, No. 10, Oct 1999, pp.70-77, IEEE 0018-9126/99

Breu, Michael (1995), A Concise Guide to Euromethod, European Software Institute, ESI-1995-CG004.01

Brooks, Frederick P., Jr (1995), The Mythical Man-Month: essays on software engineering, Anniversary ed., Reading, Addi-son-Wesley, ISBN 0-201-83595-9

Brown, Norm et al. (1998), The Program Manager’s Guide to Soft-ware Acquisition Best Practices version 2.2, Arlington, Soft-ware Program Managers Network

Brown, Norm et al. (1999), The Guidebook of Software Acquisition Questions version 1.0, Arlington, Software Program Manag-ers Network

Carr, Marvin J. et al. (1993), Taxonomy-Based Risk Identification, Pittsburgh, Software Engineering Institute, CMU/SEI-93-TR-6

Christel, Michael G. and Kang, Kyo C. (1992), Issues in Re-quirements Elicitation, Pittsburgh, Software Engineering In-stitute, CMU/SEI-92-TR-12

8 Bibliography


CMMI (2000), Capability Maturity Model Integration, http://www.sei.cmu.edu/cmmi/ (15 September 2000)

Cockburn, Alistair (n.d.), Balancing Lightness with Sufficiency, http://members.aol.com/acockburn/papers/barelysufficient.htm (2 August 2000)

DAD (2000), Defense Acquisition Deskbook, http://web2.deskbook.osd.mil/default.asp (1 September, 2000)

D.I.R. (2000), Tailoring the Guidelines, http://www.dir.texas.gov/eod/qa/tailor.htm (1 August 2000)

Duncan, William R. (1996), A Guide to the Project Management Body of Knowledge, Newtown Square, Project Management Institute, ISBN 1-880410-12-5

Euromethod (1996), Euromethod version 1, Euromethod Project

FAA-iCMM (1997), The Federal Aviation Administration Inte-grated Capability Maturity Model (FAA-iCMM) Version 1.0, U.S. Federal Aviation Administration

Fleming, Quentin W. and Koppelman, Joel M. (1996), Earned value project management, Newtown Square, Project Man-agement Institute, ISBN 1-880410-38-9

Gallivan, Michael J. (1999), Analyzing IT Outsourcing Relation-ships as Alliances among Multiple Clients and Vendors, Pro-ceedings of the 32nd Hawaii International Conference on System Sciences

Ganssle, Jack G. (1998), The Seven Habits of Highly Defective Pro-grammers, Embedded Systems Programming – July 1998

8 Bibliography


Haimes, Yacov Y. et al. (1997), A Holistic Management Framework for Software Acquisition, Acquisition Review Quarterly – Winter 1997, Defense Systems Management College, pp. 55-86

IEEE 982.1 (1988), IEEE Standard Dictionary of Measures to Pro-duce Reliable Software, New York, The Institute of Electrical and Electronics Engineers, Inc.

IEEE 1062 (1998), IEEE Recommended Practice for Software Acqui-sition, New York, The Institute of Electrical and Electronics Engineers, Inc., ISBN 0-7381-1514-2

IEEE/EIA 12207.0 (1996), Standard for information technology – Software life cycle processes, New York, The Institute of Elec-trical and Electronics Engineers, Inc., ISBN 1-55037-077-4

ISO/IEC 9126 (1991), Information technology – Software product evaluation – Quality characteristics and guidelines for their use, ISO/IEC

ISO/IEC 15504 (1998), Information technology – Software process assessment, ISO/IEC

Kar, Pradip and Bailey, Michelle (1996), Characteristics of Good Requirements, 1996 INCOSE Symposium

Kerth, Norman L. (n.d.), An Approach to Post Morta, Post Parta, and Post Project Reviews, Beaverton, Elite Systems

Marciniak, John J. and Reifer, Donald J. (1990), Software Acqui-sition Management, New York, John Wiley & Sons, Inc., ISBN 0-471-50643-5

Mosemann, II, Lloyd K. (1996), Guidelines for Successful Acquisi-tion and Management of Software-Intensive Systems, U.S. De-partment of the Air Force, Software Technology Support Center

8 Bibliography


Oskarsson, Östen (2000), interviewed by author at Östen Oskarsson’s home, Linköping, 21 April 2000

Oskarsson, Östen (n.d. a), Tutorial – Acquisition and Supply of Software Development, http://www.oskarsson.se/useful_info/acqtutorial.html (4 April 2000)

Oskarsson, Östen (n.d. b), Example of Software Contract Require-ments, http://www.oskarsson.se/useful_info/Contract.html (3 May 2000)

Oskarsson, Östen and Glass, Robert L. (1996), An ISO 9000 Approach to Building Quality Software, Upper Saddle River, Prentice-Hall, Inc., ISBN 0-13-228925-3

Paulk, Mark C. et al. (1993), Capability Maturity Model for Soft-ware, Version 1.1, Pittsburgh, Software Engineering Insti-tute, CMU/SEI-93-TR-24

Ran, Mats (2000), interviewed by author at Q-Labs AB, Linköping, 3 May 2000

Reeves, Jack W. (1992), What is Software Design?, C++ Journal Vol. 2, Nr. 2 1992

Reifer, Donald J. (1997), Software management, New York, The Institute of Electrical and Electronics Engineers, Inc., ISBN 0-8186-8001-6

Salwin, Arthur E. (1998), The Road to Successful ITS Software Ac-quisition, Washington, U.S. Department of Transportation, Report No. FHWA-JPO-98-035

SS-ISO 9000-3 (1992), Kvalitetssystemstandarder – Del 3: Riktlinjer för tillämpning av SS-9001 vid utveckling, leverans och under-håll av programvara, SIS – Standardiseringskommissionen i Sverige

8 Bibliography


The Standish Group (1995), The Standish Group Report Chaos, http://www.scs.carleton.ca/~beau/PM/Standish-Report.html (28 August 2000)

Urmi, Jaak (2000), interviewed by author at Ericsson AB, Kista, 14 April 2000

Wideman, R. Max (1992), Project and program risk management: a guide to managing project risks and opportunities, Project Management Institute, ISBN 1-880410-06-0

Wilson, Peter B. (2000), Sizing Software with Testable Require-ments, Systems Development Management, Auerbach Pub-lications

129

A Frameworks coverage

This chapter shows that the processes in the guidelines covers most issues in the main frameworks that have comparable processes or sections.

A.1 SA-CMM Guidelines SA-CMM

Project steering Implicitly covered in all key process areas by the “verifying implementation” section.

Requirements management Requirements development and management

Risk management Acquisition risk management

Project management Project management, Project performance management

Planning and organizing Software acquisition planning, Project performance manage-ment

Acquisition training Training program

Contractor selection Solicitation



Contract management Contract tracking and over-sight, Project performance management, Contract per-formance management

Product evaluation Evaluation, Contract perform-ance management

Transition to support Transition to support

Post partum Project performance manage-ment

Not covered Process definition and mainte-nance, Quantitative process management, Quantitative ac-quisition management, Con-tinuous process improvement, Acquisition innovation man-agement

A.2 FAA-iCMM Guidelines FAA-iCMM

Project steering Needs, Coordination

Requirements management Needs, Requirements

Risk management Risk management

Project management Project management, Configu-ration management, Coordina-tion

Planning and organizing Project management

Acquisition training Training



Contractor selection Outsourcing

Contract management Contract management, Project management, Measurement

Product evaluation System test and evaluation

Transition to support Transition

Post partum Not covered

Not covered Integration, Software devel-opment and maintenance, Ar-chitecture, Alternatives, Qual-ity assurance and manage-ment, Peer review, Prevention, Organization process defini-tion, Organization process im-provement, Innovation, Prod-uct evolution

A.3 IEEE 1062 Guidelines IEEE 1062

Project steering Planning organizational strat-egy

Requirements management Defining the software re-quirements

Risk management Defining the software re-quirements

Project management Implementing organization’s process

Planning and organizing Planning organizational strat-egy, Implementing organiza-



tion’s process

Acquisition training Implementing organization’s process

Contractor selection Implementing organization’s process, Identifying potential suppliers, Preparing contract requirements, Evaluating pro-posals and selecting supplier

Contract management Managing for supplier per-formance

Product evaluation Accepting the software

Transition to support Not covered

Post partum Using the software

A.4 ISO 9000-3 Guidelines ISO 9000-3

Project steering Management responsibility

Requirements management Purchaser’s requirements specification

Risk management Not covered

Project management Not covered

Planning and organizing Not covered

Acquisition training Not covered

Contractor selection Contract review



Contract management Measurement, Purchasing, In-cluded software product, Management responsibility

Product evaluation Testing and validation

Transition to support Replication, delivery and in-stallation, Maintenance


Not covered Quality System, Internal qual-ity system audits, Corrective action, Development planning, Quality planning, Design and implementation, Configura-tion management, Document control, Quality records, Rules, practices and conventions, Tools and techniques, Training

A.5 ISO 12207 Guidelines ISO 12207

Project steering Acquisition, Joint review, Management

Requirements management Acquisition, Development, Verification

Risk management Management

Project management Management, Configuration management

Planning and organizing Management, Tailoring, Veri-fication



Acquisition training Training

Contractor selection Acquisition, Verification

Contract management Acquisition, Quality assur-ance, Joint review, Audit, Problem resolution, Verifica-tion, Validation

Product evaluation Acquisition, Verification, Vali-dation



Not covered Supply, Operation, Mainte-nance, Documentation, Infra-structure, Improvement

A.6 ISO 15504 Guidelines ISO 15504

Project steering Acquisition preparation, Man-agement, Project management

Requirements management Acquisition preparation, Re-quirements elicitation

Risk management Risk management

Project management Configuration management, Documentation, Problem reso-lution, Management, Project management, Measurement

Planning and organizing Management



Acquisition training Human resource management

Contractor selection Supplier selection

Contract management Supplier monitoring, Verifica-tion, Validation, Joint review, Audit, Measurement

Product evaluation Customer acceptance, Verifica-tion, Validation



Not covered Operation, Customer support, Development, System and software maintenance, Quality assurance, Quality manage-ment, Organizational align-ment, Improvement, Infra-structure, Reuse

A.7 Euromethod Guidelines Euromethod

Project steering Acquisition goal definition, Acquisition planning

Requirements management Acquisition goal definition, Tendering

Risk management Acquisition planning, Tender-ing,

Project management Acquisition planning, Tender-ing, Contract monitoring, Con-



tract completion

Planning and organizing Acquisition planning

Acquisition training Not covered

Contractor selection Tendering

Contract management Contract monitoring, Contract completion

Product evaluation Contract monitoring

Transition to support Contract completion

Post partum Contract completion

137

Index

A Acquisition expert......................... 33 Acquisition manager..................... 30 Acquisition plan............................ 41 Acquisition proposal..................... 39 Acquisition training ...................... 74 Administrative overload............. 15 Administrator................................ 32 Alpha tests ................................. 109 Approach......................................... 3 Artifacts ........................................ 38 Assisting roles............................... 33 Assmann ........................... 2, 3, 4, 85 Audit....................................... 87, 98 auditor ......................................... 100 award incentives ........................... 88

B baseline ....................................... 103 best practices................................. 89 Beta tests .................................... 109

C C/SCSC....................................... 102 Capability Maturity Model ........... 17 Capability Maturity Model

Integration ................................ 17 Change request ............................. 55 Chaos” report ................................ 13 characteristics of software .............. 6

CMMI ...........................................17 code and fix .....................................8 Collaborating contractor................37 Collaborating customer .................30 Commercial-off-the-shelf..............7 Common problems ........................13 communications ...........................16 configuration management.........71 Contract .........................................51 Contract management....................96 Contract manager ..........................32 Contract types................................88 Contracting environment...............26 contractor ... i, 2, 3, 9, 10, 11, 18, 33,

34, 35, 36, 47, 50, 56, 57, 58, 59, 64, 65, 73, 74, 81, 94, 97, 109

Contractor development team characteristics ............................27

Contractor development team size 25 Contractor evaluation criteria........49 Contractor selection.......................85 Controlled processes ...................21 co-sourcing ....................................12 Cost and cost share.......................92 cost incentives ...............................88 Cost performance index ..............105 Cost plus award fee.......................95 Cost plus fixed fee.........................93 Cost plus incentive fee ..................94 Cost plus percentage fee...............93 cost-reimbursable ..........................88

Index


COTS ................ 7, 18, 37, 40, 53, 84 CPAF............................................ 95 CPFF ............................................ 93 CPI .............................................. 105 CPIF ............................................. 94 CPPF ............................................ 93 Creeping scope ............................ 15 Criticality ...................................... 23 CS ................................................. 92 customer i, 2, 7, 8, 10, 11, 12, 34, 35,

109 Customization factors ................... 21 Customization levels..................... 20 Customize .............................. 70, 73

D D.I.R. ............................................ 21 Deliverables .................................. 57 Delivery time ................................ 24 discipline ...................................... 16 DOD-STD-2167A ........................ 18 DOD-STD-2168 ........................... 18 Domain expert .............................. 33 dyadic............................................ 11

E Earned value ............................... 102 Effort............................................. 24 end user........... 10, 15, 81, 82, 83, 84 End user ........................................ 34 Error rate .................................. 101 estimates................................. 85, 87 Euromethod................................. 19 executive support ........................ 16 Experience with contractor........... 27 experts........................................... 33 External risks .............................. 80 eXtreme Programming ................... 9

F FAA-iCMM ................................. 17 feasibility...................................... 69 Federal Aviation Administration .. 17

FFP................................................89 Finalization....................................57 Firm fixed price ............................89 Fixed price plus award fee ...........92 Fixed price plus incentive fee ......91 Fixed price redetermination.........90 Fixed price with economic price

adjustment.................................90 fixed-price .....................................88 forecast ........................................106 FPAF .............................................92 FPEPA ..........................................90 FPIF ..............................................91 FPR ...............................................90 Fragmentation .............................15 Frameworks ...................................16 Fraunhofer Institute ........................ ii Frederick P. Brooks.........................1 Friction .........................................16 Fully developed software ..............8 Functional tests ..........................109

G Geographic dispersion...................28 Goldplating...................................15 groups ...........................................73

I IEEE 1062 ......................... i, 4, 7, 18 IEEE/EIA 12207............................18 incentives.......................................88 Inception........................................39 Independent verification and

validation ...................................35 Innovation......................................23 Installation tests.........................109 intellectual property rights.........67 ISO 12207 .....................................18 ISO 15504 .....................................19 ISO 9000 .......................................18 ISO 9000-3 ....................................18 IV&V.....................................35, 108

Index


L Labor hours.................................. 95 Laissez-faire................................. 14 Legal expert .................................. 34 Linköping University.................. 1, ii

M Maintenance and support staff...... 34 Maintenance plan.......................... 46 Milestones .................................. 101 Minimal processes....................... 21 Modified-off-the-shelf................... 7 Monitoring .................................... 53 MOTS ............................. 7, 8, 18, 40 multi-vendor.................................. 12

N nature of software ........................... 6

O Objective......................................... 2 objectives ..................................... 16 Organizations involved................. 26 Outline ............................................ 4 Outsourcing..................................... 9 Overpromising ............................ 15 overtime ..................................... 100

P participants.................................... 10 Performance tests ..................... 110 phase ............................................. 38 Planning and organizing ............... 72 Post partum ........................... 58, 113 Post partum plan ........................... 57 problem analysis ......................... 72 Problem reports ........................ 101 problem resolution...................... 71 Process group .............................. 61 Processes....................................... 60 Product evaluation ...................... 106 Product evaluation approaches ... 109

Product evaluation plan.................54 Product evaluation report ..............55 Program size ..................................22 Project and contract management

risks...........................................81 Project management ......................69 project parameters ......................67 Project specification ......................40 Project status metrics...................100 Project status report .......................56 Project steering ..............................66 Proposal .........................................50

Q Q-Labs ........................................ 1, ii quality assurance .........................71

R reality checks.................................31 Recovery tests ............................110 Request for proposal......................47 requirements change rate ...........77 Requirements change rate ........100 Requirements management ...........76 Requirements specification ...........44 Requirements volatility .................22 resources.......................................16 RFP................................................47 Risk identification .........................80 Risk list..........................................43 Risk management ..........................78 Robust processes..........................21 Roles..............................................29 root cause.......................................14

S SA-CMM ......................................17 Schedule performance index .......105 Scope ...............................................3 SE-CMM .......................................17 Security tests ..............................110 SETA .............................................35 sites ................................................28

Index


SMI............................................. 100 Software development .................... 8 Software maturity index .......... 100 Solicitation.................................... 47 SPI............................................... 105 Sponsor ......................................... 29 staffing ....................................... 100 Standard........................................ 18 Stress tests.................................. 110 Subcontractor................................ 37 Supplier......................................... 37 support and maintenance

organization........................... 111 support contractors ....................... 33 Support tests.............................. 109 SW-CMM .................................... 17 System engineering and technical

assistance .................................. 35

T Target audience............................... 2 taxonomy .......................... 11, 12, 16 TCPI............................................ 106 team-building .............................. 70 Technical and product

development risks ................... 82

Technical expert ............................33 Technical manager ........................32 test cases .....................................107 Testable requirements .................105 The Standish Group.......................13 Time and materials .......................95 Timeline.........................................62 TM/L.............................................95 To-Complete Performance Index 106 Total cost .......................................25 Transformational impact ...............24 Transition to support ...................110 Translator.......................................34 turnover .........................................27 turnover rate ..............................100

U use cases ........................................44 User interface tests ....................110 user stories.....................................44

W waterfall...........................................9 WBS ............................................102 work breakdown structure...........102

Software Acquisition Management Guidelines

Documents

Transcript of Software Acquisition Management Guidelines