Software Project Management Tools Draft 5.0 - July 18, 2008 ...
-
Upload
samuel90 -
Category
Technology
-
view
904 -
download
0
Transcript of Software Project Management Tools Draft 5.0 - July 18, 2008 ...
1
Software Project Management Tools
Draft 5.0 - July 18, 2008
Capers Jones
Keywords: Software project management, software sizing, software cost estimating, software
schedule planning, software quality estimating, software risk analysis, software progress
tracking, software litigation.
Abstract
Software projects have been notoriously difficult to manage well. Cost overruns, schedule
delays, and outright cancellations of software projects have been common occurrences for more
than 50 years. Poor quality when delivered remains a chronic issue. Although software project
management tools are not a panacea, when used by capable managers they do simplify the
complexity of software project management and raise the odds of successful completions of
large software projects. Software project management tools have many divisions and
subcategories. However they can be broadly classified into two major areas of focus: 1) Tools
for planning and estimating software projects before they begin; 2) Tools for tracking and
reporting software project status and costs while projects are underway. Beneath this division
are 12 topical areas and scores of project management activities.
Capers Jones
Chief Scientist Emeritus, Software Productivity Research LLC
Web: http://www.spr.com
Email: [email protected]
Copyright © 2008 by Capers Jones. All rights reserved.
2
Software Project Management Tools
The Tasks of Software Project Managers
This article discusses the tasks of project management and the available tools that support those
tasks. From the frequent failure of large software projects and from analysis of the depositions
and testimony of litigation for software failures, it is apparent that software project managers are
often not well trained in managerial tasks, and often fail to utilize appropriate tools to support
critical tasks such as estimating and progress tracking.
Before discussing the tools of software project management, it is appropriate to consider the
many tasks that software project mangers are likely to confront in carrying out their jobs. The
two major divisions of software project management include planning and estimating before
projects start, and tracking status and expenses while software projects are underway. Beneath
this broad division there are a dozen major functional areas and scores of individual tasks, as
shown below:
1) Software Project Sizing
a. Size in function points
b. Size in lines of code
c. Size in data base contents
d. Numbers of screens
e. Numbers of reports
f. Numbers of interfaces
g. Numbers of files
h. Prediction of requirements creep
i. Prediction of requirements churn
2) Software Project Schedule Planning
a. Project schedules
b. Activity schedules
3
c. Task schedules
d. Critical paths
e. Team assignments
f. Individual assignments
g. Requirements creep
h. Requirements churn
i. Requirements slip
3) Software Project Cost Estimating
a. Estimating method selection
b. Estimating tool selection
c. Project cost estimating
d. Activity cost estimating
e. Task cost estimating
f. Project earned value calculation
g. Creeping requirements cost prediction
h. Churned requirements cost prediction
i. Slipped requirements cost prediction
j. Complexity estimating
4) Software Project Risk Estimating
a. Schedule delays
b. Cost overruns
c. Cancellation
d. Poor quality
e. Poor customer satisfaction
4
f. Litigation for breach of contract
g. Loss of key personnel
h. Potential security problems
5) Software Project Quality Estimating
a. Defect potentials
b. Defect removal efficiency
c. Bad fix injections
d. Error-prone modules
e. Bad test cases
f. Cyclomatic and essential complexity
g. Number of defect removal activities
h. Number of test stages
i. Number of test cases per stage
6) Software Project Outsource Contract Analysis
a. Requirements scope in contract
b. Requirements changes in contract
c. Quality terms in contract
d. Incentives for early completion
e. Penalties for late completion
f. Penalties for poor quality
g. Required tracking and reporting
7) Software Project Commercial Off-the-Shelf (COTS) Acquisition
a. COTS Acquisition Analysis
b. Due diligence prior to major acquisitions
5
c. Warranty provisions
d. Installation schedules
e. Installation defects
f. Learning curves
g. Loss of performance during learning
8) Software Personnel Management
a. Specialist selection
b. Hiring of new employees
c. Appraisals of existing employees
d. Promotions
e. Demotions
f. Terminations
g. Awards
9) Software Project Measurement, Tracking, and Control
a. Financial tracking
b. Earned value tracking
c. Balanced score card tracking
d. Goal-question metric tracking
e. Milestone tracking
f. Issue and “red flag” tracking
g. Quality and defect tracking
h. Change request tracking
i. Requirements creep tracking
j. Requirements churn tracking
6
k. Complexity measurement
l. Benchmark productivity data
m. Benchmark quality data
n. Coordination with other managers
o. Coordination with supply chain managers
10) Software Technology Acquisition
a. Project planning tools
b. Project estimating tools
c. Project tracking tools
d. Defect tracking tools
e. Complexity analysis tools
f. Development methodologies
g. Maintenance and enhancement methodologies
h. Quality control methodologies
i. Methodology management tools
j. Development workbenches
k. Maintenance workbenches
l. Reverse engineering tools
m. Reengineering tools
n. Programming languages
11) Software Mergers and Acquisitions
a. Due diligence for purchaser
b. Analysis of tools in company to be acquired
c. Technology conversion after merger
7
d. Organization planning
e. Reorganization after mergers
12) Software Litigation Support
a. Breach of contract litigation
b. Intellectual property and patent litigation
c. Employee confidentiality litigation
d. Preparation of materials on plaintiff side
e. Preparation of materials on defendant side
f. Depositions if required
g. Testimony if required
As of 2008 automated tools are available for many but not all of the tasks that software project
managers are likely to perform. For common tasks such as schedule planning and cost
estimating there are dozens of available tools. For less common tasks such as litigation support
or due diligence prior to a merger or acquisition, few tools are available.
Project Management Knowledge Acquisition
Academic training in the area of software project management is not as sophisticated as
academic training in basic software engineering or even in other forms of engineering
management. For example there is a shortage of academic training even for important topics
such as quality estimation, risk analysis, and milestone tracking. For more unusual topics, such
as litigation support or commercial off the shelf software (COTS) acquisition academic training
hardly exists at all.
The lack of academic training for software project managers is compensated for in some degree
by training and “bodies of knowledge” or BOK provided by a number of professional
associations. Among the large associations that provide useful information to software project
managers can be found:
Academy of Management (AOM) www.aomonline.org
American Management Association (AMA) www.amanet.org
American Society for Quality (ASQ) www.asq.org
IEEE Computer Society www.computer.org
8
Information Technology Metrics and Productivity Institute (ITMPI) www.itmpi.org
International Function Point Users Group (IFPUG) www.ifpug.org
International Project Management Association (IPMA) www.ipma.org
International Organization for Standards (ISO) www.iso.org
Project Management Association of Japan www.pmaj.or.jp
International Software Benchmark Standards Group (ISBSG) www.isbsg.org
Project Management Institute (PMI) www.pmi.org
Software Engineering Institute (SEI) www.sei.org
Society of Project Management (SPM) www.spm-japan.jp
There are also scores of national and local organizations that also provide information and
seminars on software project management topics. Examples include the Australian Institute of
Project Management & Australian Software Metrics Association & Software Quality Assurance
(ASMA-SQA), the Brazilian Association for Project Management, the Netherlands Software
Metrics Association (NESMA), and dozens of local chapters of the Software Process
Improvement Network (SPIN) in the United States, Europe, South America, and Asia.
Several web sites contain useful links to the project management literature. Among the
commercial links can be found the previously mentioned Information Technology Metrics and
Productivity Institute ITMPI) portal (www.ITMPI.org). Among academic links one of the most
complete is that of Dave W. Farthing of the University of Glamorgan in the United Kingdom
(www.comp.glam.ac.uk). This interesting portal has links to dozens of project management sites
and the publishers of scores of project management books.
From discussions with project managers in large corporations, in-house training of managers in
companies such as IBM, Microsoft, AT&T, Google, and CISCO is actually the most effective
source of information for software project managers. Topics such as quality estimation,
measurement, and risk analysis are often better covered via in-house training than by
universities.
However this form of training is only available for project managers in rather large corporations.
Small companies must depend upon commercial training or upon professional associations, plus
a few universities that do have curricula in software management topics.
Training in specific project management tools is also available from the vendors themselves. For
example fairly sophisticated training in software cost estimating is available from the vendors of
9
commercial estimating tools such as KnowledgePlan, Price-S, SLIM, and SEER. Computer Aid
Inc. and its subsidiary the Information Systems Metrics and Productivity Institute (ITMPI) also
provide training in a number of software management topics.
Overall, training of software project managers is a topic that needs to be improved. Quality
control, economic analysis, estimation, and measurement are all topics where additional training
is needed for entry-level managers just starting their careers.
The History of Software Project Management Tools
Software project management is of course only one of many different forms of project
management, such as home construction projects, electrical engineering projects, or civil
engineering projects such as building roads and bridges. Software project management does
have some distinguishing attributes that seem to differ from other and more established forms of
project management:
1. Because software is abstract and invisible to the eye, sizing software projects is more
difficult than for many other kinds of project.
2. Because software engineering is not highly automated, manual effort by skilled
knowledge workers plays a more important role in software projects than in many other
kinds of project.
3. Because of the large volume of bugs or defects in all software deliverables, quality
estimation and quality control are more important for software projects than for many
other kinds of project.
4. For large software projects in the 10,000 function point range or larger, outright failure,
major cost overruns, and major schedule delays occur more frequently than for many
other kinds of project.
5. Although changing requirements can occur for any kind of project, they occur with
unusual frequency for software projects. Sometimes more than 50% of the delivered
features of software projects are not identified until the project is already in process.
Software requirements changes have been measured to occur at a rate of between 1% and
2% per calendar month.
Long before the computer era and long before software itself, project management was starting
to be studied as a technology that needed formalization. Among the pioneers who dealt with
project management issues include Henry Gantt whose famous Gantt charts circa 1917 facilitated
schedule planning. Later in the 1950’s Willard Fazar’s “program evaluation and review
technique” (PERT) added rigor to isolating and identifying the critical paths which complex
projects needed to follow. On the quality side, W. Edwards Deming revolutionized the statistical
10
analysis of quality. Deming’s work was initially adopted in Japan circa 1950 and later came to
the United States when it became obvious that quality was a major factor in successful
competition. Other pioneers in related topics include Walter Shewart (one of Deming’s
teachers), Phil Crosby of ITT, Joseph Juran, and Frederick Taylor.
In terms of software project management tools as opposed to general-purpose project
management tools, some of the pioneers include A.J. Albrecht and the invention of function
point metrics circa 1973; Dr. Barry Boehm and the development of the famous “constructive
cost model” or COCOMO circa 1981; Watts Humphrey and the development of the capability
maturity model (CMM) circa 1983; and Peter Hill and the foundation of the International
Software Benchmarking Standards Group (ISBSG) circa 1997. A more recent pioneer is Tony
Salvaggio of Computer Aid, Inc. This company is pioneering a flexible form of project
management tool that can add “cartridges” at will for specific project management functions.
The phrase “project management tools” has been applied to a large family of tools whose
primary purpose is sophisticated scheduling for projects with hundreds or even thousands of
overlapping and partially interdependent tasks. These tools are able to drop down to very
detailed task levels, and can even handle the schedules of individual workers. However for
software, there are many other critical areas that also need the support of tools: quality
prediction and measurement; requirements changes, and the earned-value of ongoing projects.
Software projects require additional capabilities to be under full management control.
There are a host of managerial functions that standard project management tools don’t deal with
in depth, such as sizing with function point metrics, contract management, personnel
management, assessments, quality estimating, defect tracking, and the like.
The family of project management tools originated in the 1950’s with the U.S. Navy and Air
Force. Weapons systems and other complex projects were becoming so large that manual
planning and control were no longer feasible. Since computers were starting to become more
powerful, it was natural to want to use computer power as an aid for project managers.
Project management tools are an automated form of several management aids developed by the
Navy for controlling large and complex weapons systems: the “program evaluation and review
technique” (PERT), critical path analysis, resource leveling, and the classic Gantt charts. Such
general-purpose tools can of course be used for software projects, but in addition specialized
tools are needed that support the unique requirements of software projects.
Project management tools did not originate for software, but rather for handling very complex
scheduling situations where hundreds or even thousands of tasks need to be determined and
sequenced, and where dependencies such as the completion of a task might affect the start of
subsequent tasks. Project management tools are general-purpose in nature, and can be applied to
any kind of project: building an aircraft, constructing an office building, or a software project.
11
General-purpose project management tools have no built-in expertise regarding software, as do
specialized software cost estimating tools. For example, if you wish to explore the quality and
cost impact of an object-oriented programming language such as Objective C a standard project
management tool is not the right choice. By contrast, many software cost estimating tools have
built-in tables of programming languages and will automatically adjust the estimate based on
which language is selected for the application.
Since software cost estimating tools originated about 10 years after commercial project
management tools, the developers of software cost estimating tools seldom tried to replicate
project management functions such as construction of detailed PERT diagrams or critical path
analysis. Instead, the cost estimation tools would export data so that the final schedule could be
adjusted via the project management tool. Thus bi-directional interfaces between software cost
estimating tools and generic project management tools are now standard features in the
commercial estimation market.
The latest versions of software project management tools integrate cost and schedule estimation,
quality estimation, risk analysis, and many other features. The Computer Aid “application
insight” tool is an example of a general-purpose tool with the ability to be extended in many
directions via plug-in “cartridges.”
Usage Patterns of Software Project Management Tools
Observations made from depositions and testimony during software litigation for breach of
contract reveals that software projects that end up in court for outright failure or major overruns
usually do not utilize many project management tools. Estimates were often informal and
tracking seriously deficient. By contrast, data gathered while performing assessments and
benchmarks of successful software projects that combined high productivity and high quality
indicate that many specialized project management tools were utilized for both predictions
before the projects started and for tracking and measurement while they were underway.
Table 1 shows the approximate sizes of project management tool features measured in function
points. The table also illustrates the patterns of tool usage noted on “lagging,” “average,” and
“leading” projects. Information on the lagging projects is taken from information discovered
during litigation. Information on the average and leading project is taken from benchmark and
assessment studies by the author.
12
Table 1: Patterns of Project Management Tool Usage
(Tool size expressed in terms of function point metrics)
Predictive Tools Lagging Average Leading
Project sizing (function points) 750 750
Project sizing (lines of code) 250 250 250
Project sizing (requirements changes) 250
Project schedule estimating 1,000 1,500 3,000
Project cost estimating 250 500 3,000
Project quality estimating 2,000
Project risk estimating 2,000
Project value estimating 1,500
Departmental budget planning 2,000 2,000 2,000
SUBTOTAL 3,500 5,000 14,750
Measurement and Tracking Tools
Lagging Average Leading
Earned Value Tracking 2,000 3,000
Project cost tracking 2,000 2,000 2,000
Project milestone tracking 1,000 1,500
Project defect tracking 1,500 2,000
Resource tracking 1,000 1,000 2,000
Cost variance reporting 1,000 2,000 2,000
Productivity measurements 1,000
13
Project benchmark reporting 1,000
Project quality measurement 1,000
SUBTOTAL 4,000 9,500 15,500
TOTAL 7,500 14,500 30,250
Table 1 illustrates two important aspects of software project management tool usage: 1) Leading
or sophisticated project managers use more kinds of tools than do managers on lagging and
average projects; 2) Leading or sophisticated project managers use more of the features or select
more powerful tools than do project managers on lagging or average projects.
The very significant use of project management tools on leading projects results in one
overwhelming advantage: “No surprises.” The number of on-time projects in the leading set is
far greater than in the lagging set, and all measurement attributes (quality, schedules,
productivity, etc.) are also significantly better.
Software project management tools are far from being a panacea, but advanced knowledge of
software project size, potential numbers of defects or bugs, and schedules derived from similar
historical projects can go a long way towards yielding a successful conclusion and eliminating
the rather high odds of failure associated with large software projects.
For large software projects in large corporations, the project will typically be supported by a
“Project Office.” IBM was a pioneer in the utilization of project offices and first deployed them
on versions of the main IBM operating system in the early 1970’s.
Typically a project office will be formed for a major software application in the 100,000 function
point size range. Applications of this large size have hundreds of development personnel and
dozens of subordinate project managers under at least three higher levels of management. A
typical project office for such a large system would contain perhaps half a dozen staff to as many
as 10 personnel equipped with a variety of estimating and planning tools, plus historical data
from similar projects. The project office would assist the individual project managers in cost
estimating, quality estimating, schedule estimating and other predictive tasks. They would also
monitor rates of progress, bug reports, earned value, costs, and other status data as the project
was underway.
Project offices tend to raise the percentage of successful large projects, but failures can still
occur. For example one major cancelled project occurred primarily because the head of the
project office and the head of the development group were political rivals and therefore did not
cooperate well. Incidentally, in large companies and large projects, personal disputes and
dislikes are frequent contributors to software disasters. This topic is difficult to study because
none of the managers involved are likely to be candid.
14
Recent Evolution of Software Project Management Tools
Project management tools, like other forms of commercial software, continue to evolve and add
new features. The following paragraphs summarize some of the recent observations of project
management tool capabilities noted in 2008.
Software Project Sizing: Sizing software projects has been a difficult activity for project
managers. Sizing in terms of “lines of code” primarily involved either comparison to similar
projects or guess work. Adding to the difficulty, there are more than 700 programming
languages in existence and some of these lack any rules for counting code. Further many
software applications utilize several programming languages at the same time. More important,
for large software projects the costs and schedules associated with producing paper documents
and finding and repairing defects costs more than the code itself and takes more time during the
schedule. These non-coding tasks cannot be estimated or measured using lines of code metrics.
Function point metrics have proven to be effective for sizing non-coding work and also
programming, so function points are a good choice for life-cycle economic analysis of software
projects. Sizing in terms of function point metrics can be done with accuracy from requirements,
but by the time requirements are available several iterations of cost estimates are normally
required. Also, function point analysis has been somewhat slow and expensive. Certified
function point analysts count function points only at a rate of perhaps 400 function points per
day. Thus the long schedules and high costs of counting function point metrics have been a
barrier
As of 2008 several new forms of high-speed function point tools have been announced.
Relativity Technologies has announced a tool for sizing legacy applications called “Function
Point Analyzer” or FPA by parsing source code in selected languages. The assertion is that this
new tool provides counts with an accuracy equivalent to normal function point analysis, but is
able to accomplish this in a matter of minutes rather than a matter of days or weeks.
Several other forms of high-speed function point sizing are also under development. Other high-
speed function point methods include: 1) pattern matching against historical projects; 2)
Automated prediction of function points from requirements and specifications; 3) data mining of
legacy source code to recreate synthetic specifications; 4) automated source code analysis to
improve mathematical conversion from source code statements to equivalent function point
metrics.
The last method, of mathematical conversion between source code and function points has
actually existed since 1975. The common name for this method is “backfiring.” The first ratios
of source code to function points were developed by A.J. Albrecht and his colleagues at IBM
during the development of the original function point metric. As of 2008 published ratios are
available for more than 700 programming languages and dialects. However the accuracy of
15
backfiring is low due to variations in code counting methods and variations in individual
programming styles. The backfire method is most effective when based on automated counts of
logical source code statements, as opposed to counts of physical lines.
A problem with function point sizing as of 2008 is the fact that there are several different kinds
of function point metrics. The function point metric defined by the International Function Point
Users Group (IFPUG) is the most widely used. But there are also COSMIC function points,
Mark II function Points, web-object points, Story points, and several national variants from the
Netherlands and Australia, plus usage of mathematical conversion from lines of code to function
points called “backfiring.” All of these variants produce different size predications, and as of
2008 there are no proven conversion rules from one variation to another.
Another new sizing topic getting underway in 2008 deals with the impact of technical and
architectural factors on application size. While user-requested functions are measured with
function point metrics, part of the size of applications is also due to architectural and technical
factors and the “layers” that the application will contain.
Other sizing topics are waiting more research and more empirical data: sizing the volume data
bases and sizing value other than financial value (i.e. customer satisfaction; enterprise prestige)
are two topics that need more research.
Software Project Schedule Planning: There are dozens of commercial tools, in-house tools,
and even open-source freeware tools that can perform critical path analysis of software projects,
as well as other kinds of projects. Examples of such tools include Microsoft Project, Artemis
Views, and TimeLine. However for software projects schedule planning can also be performed
by specialized software estimating tools such as COCOMO, KnowledgePlan, Price-S, SEER,
and SLIM in alphabetical order. There are even newer forms of tools about to be released circa
2008 that can handle schedules. For example the Application Insight tool (AI) by Computer Aid
Inc. integrates schedule planning, risk analysis, cost estimates, and tracking of resources, defects,
and costs. This tool also supports interfaces with other tools, and even allows plugging in
“cartridges” of specialized functions such as quality prediction or function point analysis.
Overall a detailed work-breakdown structure that reaches the level of specific tasks and
individual assignments is the most effective. This level of granularity can only be provided by
automated tools: manual methods are too labor-intensive and also to difficult to update.
As of 2008, schedule estimating remains a difficult challenge for software project managers.
Above 10,000 function points in size, about two thirds of all software projects run late, exceed
their budgets, or are cancelled because the delays and overruns have degraded the return on
investment to a negative number.
16
Software Cost Estimating: Software cost estimation can be traced back to the first IBM
software cost estimating tool in 1973 and also to the first commercial estimating tool which was
the Price-S tool originally marketed by the RCA corporation. These pioneering tools were soon
followed by other commercial software cost estimating tools such as Checkpoint, COCOMO,
Estimacs, ISBSG, SEER, SLIM, SoftCost, SPQR/20 and dozens of others.
These specialized software cost estimating tools were often superior to manual cost estimates,
and they also included schedule and quality estimates as well. Because the software estimating
tools appeared about 10 years after general-purpose project management tools, they support bi-
directional data transfer with general purpose tools such as Microsoft Project or Artemis Views.
Another advantage of the commercial software cost estimating tools is that they contain built-in
knowledge of topics that affect software projects, which are lacking in general-purpose tools.
Examples of these factors include the specific programming languages to be used, the CMM
level of the development organization, the methodology to be used such as Agile development or
Extreme programming, and the structure or complexity of the application. The commercial
software estimating tools also have built-in knowledge of key topics such as the defect removal
efficiency levels of various kinds of inspection and testing. The first commercial estimating tool
that could predict defect quantities and defect removal efficiency levels was SPQR/20, which
entered the market in 1985. This was also the first commercial estimating tool based on function
point metrics, although it predicted lines of code as well.
Large software projects often spend more money on the production of paper documents such as
requirements, specifications, plans, user manuals, etc. than they do on the production of the
source code itself. Estimating these costs lagged for many years because lines of code metrics
could neither measure nor predict paperwork. However modern commercial software estimating
tools can predict the sizes and costs for more than 50 kinds of paper document, which
collectively can accumulate more than 35% of the total development cost of a large software
project. A few software cost estimating tools can even predict the costs of Nationalization, or
converting some documents into foreign languages for international sales.
Software Project Risk Estimating: The major forms of risk for software projects include
outright cancellation due to failure to reach operational stability, cost overruns, schedule
overruns, poor quality, security vulnerabilities, difficult human interfaces, and sluggish
performance. Since software project failures are common, it can be stated that software risk
assessment and risk predictions are either inadequate or not performed with rigor.
As of 2008 there are some risk prediction features in commercial estimating tools such as
KnowledgePlan, SLIM, and SEER. There are also some stand-alone risk models as well as risk
estimation in integrated tools such as the Application Insight tool (AI) by Computer Aid Inc..
Automated risk estimation is becoming very sophisticated circa 2008, and is also very quick.
17
Currently risk analysis by trained risk experts was the traditional and most effective form of risk
abatement through 2008, but automated risk analysis is now about equal in effectiveness and
much faster in terms of speed.. While risks associated with cost overruns and schedule delays
can be predicted early and with good accuracy, risks associated with security flaws are more
difficult. In part this is due to the fact that security vulnerabilities are not completely understood
circa 2008, coupled with the fact that hackers are constantly adding new forms of viruses and
new forms of spyware.
Another aspect of risk management is that of “governance” or ensuring the integrity of software
applications from hacking, security flaws, and possible misuse for fraudulent purposes. The
Sarbanes-Oxley act of 2002 has elevated governance to a very important activity, because it
threatens harsh penalties including criminal charges for software executives and boards of
directors who are lax in governance.
Make no mistake about it, large software projects in the 10,000 function point size range are
among the most risky business endeavors in human history. The major risks for large software
projects include:
1. Outright cancellation due to excessive cost and schedule overruns 2. Inadequate or careless estimating 3. Failure to compare estimates to validated historical data 4. Cost overruns in excess of 50% compared to initial estimates 5. Schedule overruns in excess of 12 months compared to initial estimates 6. Quality control so inept that the software does not work effectively 7. Requirements changes in excess of 2% per calendar month 8. Executive or client interference, so that the project is disrupted 9. Failure of clients to review requirements and plans effectively 10. Security flaws and vulnerabilities 11. Potential misuse or tampering with outputs for fraudulent purposes 12. Performance or speed too slow to be effective 13. Loss of key personnel from the project during development 14. The presence of “error-prone modules” in legacy applications 15. Patent violations or theft of intellectual property 16. External risks (fire, earthquakes, hurricanes, etc.)
From analysis of depositions and court documents in breach of contract litigation, most failing
projects did not even perform a formal risk analysis. Worse, project tracking was so inept that
major problems were concealed rather than being dealt with as they occurred.
Research and prototypes of risk analysis tools are a sign that future kinds of risk analysis may
improve the situation. In particular, security vulnerabilities need better predictive capabilities in
the modern world where viruses, spyware, and denial of service attacks are daily events that
18
affect thousands of companies and millions of software applications. As outsourcing becomes
more and more common, litigation avoidance is also a risk that needs more attention than it has
received to date.
Software Quality Estimating: Quality estimation for software was not well done before the
invention of function point metrics because lines of code could not be used for either measuring
or predicting defects in requirements, design, user documents and other non-code items. Once
function point metrics were applied to the task of quality prediction, it was soon discovered that
errors in requirements and design outnumbered coding errors. Commercial software estimating
tools that could predict the full spectrum of software defects from all major sources
(requirements, design, code, user documents, and “bad fixes” or secondary defects) only became
available in 1985 with the SPQR/20 commercial estimation tool. As of 2008 most of the
commercial software estimation tools can also predict quantities of defects.
Another important aspect of quality estimation is that of defect removal efficiency, or predicting
the percentages of defects that will be found by various kinds of review, inspection, and test
stage. Historical data gathered since the 1970’s indicate that formal design and code inspections
have the highest defect removal efficiency and alone can top 85%. Most forms of testing such as
unit test, new function test, regression test, etc. are less than 35% efficient or only find about one
bug out of three. A sequence of formal inspections followed by six to eight different testing
stages can top 95% in cumulative defect removal efficiency. As of 2008 several commercial
estimating tools can predict defect potentials and defect removal efficiency levels.
Some commercial estimating tools can also predict the specific sequence of inspections and test
stages that should be used to achieve optimal levels of defect removal efficiency. Numbers of
test cases and test coverage can also be predicted for many forms of testing such as unit test
(manual or automated), regression test, new function test, performance test, system test, etc.
Since most forms of testing are less than 40% efficient in finding bugs, it is obvious that a multi-
stage series of inspections and tests will be needed to achieve high levels of defect removal
efficiency. The good news about this fact is that projects that approximate 95% in cumulative
defect removal efficiency have shorter schedules and lower costs than “average” projects with
only about 85% cumulative defect removal efficiency. Maintenance costs are also reduced. Phil
Crosby’s aphorism that “Quality is Free” has not only been proven, but also measured on many
successful software projects. An automated testing company, Agitar, has developed a testing
dashboard that provides interesting data on test results, test coverage, and other relevant topics.
Because the costs and schedules associated with finding and fixing bugs is the most significant
cost element of large software projects and the primary factor leading to cancellation and delay,
quality prediction is a critical element of successful cost and schedule prediction. Poor quality is
a frequent cause of litigation too.
19
Many specific software quality methods such as six-sigma for software, quality function
deployment (QFD), formal design and code inspections, automated testing, error-prone module
removal, and joint application design (JAD) can now be modeled and predicted using automated
tools. The most interesting results come from side-by-side comparisons that show a specific
approach such as six-sigma compared against an identical project that did not use the same
approach.
Software Project Outsource Contract Analysis: Since many outsource contracts are cancelled
and about 5% end up in court for breach of contract, it can be stated that the available tools for
predicting the results of software outsource contracts need improvement. There are some
specialized and proprietary tools available from consulting companies that assist clients in
dealing with outsource vendors. Also, normal commercial software cost estimating tools can be
useful in predicting topics such as schedules, costs, and quality levels for specific projects.
However contract analysis as of 2008 does not have much in the way of sophisticated project
management tools.
What often occurs in litigation involving cancelled projects are side-by-side comparisons of the
project as it was actually performed against an alternate versions that would have used better
methods of quality control and dealing with requirement changes. This feature of producing
side-by-side comparisons of alternate approaches should become a standard part of predictive
tools uses for cost, quality, and schedule planning purposes.
Software contracts often are incomplete or ambiguous in dealing with requirements changes and
also with quality control. All contracts need language and sections that define how changes in
requirements will be funded and what kind of schedule relief will be provided. Contracts also
should deal specifically with quality, such as including clauses that require at least 95% defect
removal efficiency levels on the part of the developing company.
Software Project Commercial Off-the-Shelf (COTS) Acquisition: Most corporations and
government agencies lease or purchase more than half of the software they use on a daily basis.
Operating systems, office suites, enterprise resource planning (ERP) packages, financial and
accounting packages, human resource packages, and a host of others comprise a large portion of
total corporate portfolios. There are very few tools available that can assist in making good
selections of commercial software.
Software vendors are reluctant to publish data on defects in their commercial packages. They are
also reluctant to publish size data in either function point or lines of code form. As a result, it is
hard to ascertain basic information such as after-market defect levels, installation problems, and
other issue. There are some specialized tools available for specific tasks such as estimating the
deployment of ERP packages, but these are usually proprietary tools owned by consulting
20
companies that specialize in ERP deployment. Overall, this is a major gap in project
management tools circa 2008.
Historical data on the numbers of bugs and installation problems in major commercial software
packages such as Microsoft Vista, of the ERP packages from Oracle, SAP, and other vendors is
not available to ordinary consumers and clients, event though the vendors actually have such
data available internally.
As of 2008 there are prototype tools available that can predict defects and deployment costs of
large commercial applications such as Vista and ERP packages. Some of the data comes from
litigation filed against these vendors, so the predictions also include learning curves by users,
installation defects, and even predications of consequential damages or the harm which bugs in
commercial packages can cause to client companies. For example a serious bug in a commercial
financial package caused a client company to restate past year earnings and therefore lose credit
with its bank.
Software Personnel Management: Strictly speaking personnel management is not the same as
project management. However in day to day work, software project managers are also software
personnel managers so they need to be involved in both kinds of activities. As of 2008 there are
dozens of commercial tools for dealing with personnel issues such as hiring, skills inventories,
appraisals, compensation, and other human resource issues. This is one of the most extensively
automated areas of the entire spectrum of management work.
Software Project Measurement, Tracking, and Control: There are scores of commercial and
in-house tools for collecting resource and cost data for software projects. Often these cost and
resource tracking tools are part of corporate-wide tool suites, so they operate for other kinds of
projects besides software projects. However there are some common gaps and omissions in
resource and cost tracking. The most common omission is measuring unpaid overtime.
Software personnel tend to have rather energetic work habits, and often work on weekends or at
home. However accurate measurement of unpaid overtime is rare. Failure to record unpaid
overtime tends to artificially inflate productivity rates.
Another gap in resource tracking concerns the work of part-time specialists such as quality
assurance, data base administration, and technical writers. Since these workers may charge time
to several projects or even be classed as overhead, sometimes their effort on specific projects is
missing or inaccurate.
Yet another kind of gap is the work performed by users when they participate in requirements
analysis or are assigned to project teams as may occur when Agile development is used.
From on-site studies where software project personnel were interviewed and asked to compare
their actual effort to the results of project tracking systems, it was noted that the “leakage” from
21
many corporate resource tracking systems was in excess of 25%. That is more than one fourth of
the actual effort was not recorded. Unpaid overtime and the work of users and specialists were
the most common omissions. However, some companies also failed to track managerial effort
and only reported technical effort by the programming team or software engineers. Leakage of
more than 50% was noted in some cases.
An interesting form of project tracking has been developed by the Shoulders Corporation for
keeping track of object-oriented projects. This method uses a 3-Dimensional model of software
objects and classes using Styrofoam balls of various sizes that are connected by dowels to create
a kind of mobile. The overall structure is kept in a visible location viewable by as many team
members as possible. The mobile makes the status instantly visible to all viewers. Color coded
ribbons indicate status of each component, with different colors indicated design complete, code
complete, documentation complete, and testing complete (gold). There are also ribbons for
possible problems or delays. This method provides almost instantaneous visibility of overall
project status. The same method has been automated using a 3-D modeling package, but the
physical structures are easier to see and have proven more useful on actual projects. The
Shoulders Corporation method condenses a great deal of important information into a single
visual representation that non-technical staff can readily understand.
Software Technology Acquisition: A weak link in the chain of project management tasks is
that of technology acquisition. With more than 700 programming languages available, more
than 40 named design methods, and more than 30 forms of development method is not easy to be
sure that the languages, tools, and methods selected for any given project are in fact the best
available.
There are many important questions that are not easy to answer: when should Agile methods be
used, and when are Agile methods inappropriate? Is level 3 on the capability maturity model
(CMM) sufficient for a specific project, or would level 5 give better results? What kind of
benefits will occur from using the Team Software Process (TSP) and Personal Software Projects
(PSP)? What value does lean Six Sigma provide to specific projects? Will object-oriented
programming languages be suitable for a specific project, or should some other language be
used? Will automated testing be beneficial or not? If formal inspections are used will they
lower testing costs? Are Use Cases more effective than other forms of specification? These are
samples of questions that need accurate empirical data.
There are few if any tools available for technology selection that match specific projects with
the best choices for methodologies, languages, or design techniques. The growing volume of
benchmark data collected by the International Software Benchmark Standards Group (ISBSG) is
starting to provide better data based on historical projects. However more data and more
measured projects are still needed.
22
Some of the commercial software estimation tools provide a side-by-side capability that allows
the same project to be estimated with varying assumptions, such as changing programming
languages, changing design methods, or changing CMM levels. These side-by-side estimates
can be helpful, but not every language or method is supported. However the use of side-by-side
modeling to predict the results of alternate scenarios is definitely a feature that should be added
to all software project predictive tools.
Software Mergers and Acquisitions: Some software managers can spend an entire career
without being involved with mergers and acquisitions. Other managers may be involved as often
as several times a year. Some of the issues with mergers and acquisitions are those of joining
the portfolios of the two companies; with merging the software organizations; and with selecting
the software tools and methods that will be used after the merger is complete, assuming the two
companies use different approaches for development and maintenance. There are no standard
tools available for supporting mergers and acquisitions, although some commercial estimating
tools and personnel management tools can be helpful.
Because the technologies deployed by merging partners are often different, here too side-by-side
comparisons of one set of methods against another would be useful.
Software Litigation Support: Software litigation comes in many varieties such as breach of
contract, patent violations, tax litigation, theft of intellectual property, and violations of non-
competition agreements. For breach of contract litigation expert witnesses are often asked to
predict what a failing project would have looked like if state of the art methods had been used;
i.e. what would the schedule and costs have been had the project been done well instead of done
poorly. Questions about the cost and quality impacts of requirements changes often occurs in
litigation too. For tax litigation, reconstructing the original development cost of software at a
given point in time is a topic that occurs often, as well as questions about what it would cost to
build a new version today.
A combination of software cost estimating tools, software quality estimation tools, and general-
purpose critical path tools are often used by expert witnesses in software litigation. For cases
involving patent violations or theft of intellectual property, specialized tools that analyze the
structure of software applications and produce graphs of that structure may be used. If two
different software applications have common structures, it can be assumed that one version may
have been copied from another version.
For many kinds of litigation it is important to compare what actually took place against an
alternate scenario of what might have taken place had more suitable methods been deployed.
Therefore side-by-side comparisons are an important aspect of litigation support.
23
Because many lawsuits are filed due to poor quality or the presence of major bugs that prevented
the applications from being used successfully, quality prediction and quality measurement are
important aspects of litigation support.
Litigation is a topic that is seldom taught by universities to computer science students, software
engineering students, or even MBA candidates. However litigation is common enough in the
software world so that information on litigation costs, probable duration, and disruption of staff
time needs to be included in software project management tools.
As of 2008 there is a prototype tool that can predict litigation costs and schedules for breach of
contract suits for both the plaintiff and defendant. But the prototype does not deal with patent
violations, theft of intellectual property, criminal hacking, violations of employment agreements,
or other forms of litigation.
The Costs and Value of Software Project Management Tools
The cost of software project management tools ranges from free (for open-source tools or in-
house tools) to more than $5,000 per seat for some commercial software cost estimating tools.
Assuming a normal mix of in-house tools, open-source tools, and commercial tools the
acquisition cost of a suite of software project management tools would cost about $0.50 per
function point using the size of the tools in function points as the base. Thus to equip an
individual software project manager with a suite of tools totaling 30,000 function points would
cost about $15,000 for one manager. Because of discounts and multi-seat licenses, the effective
cost per manager in a large company or a large project would probably be about $5000 for a suite
of project management tools that encompass estimating, planning, risk analysis, quality control,
and tracking of costs and progress.
The average cost to build a successful software application in the 10,000 function point size
range is about $1000 per function point or $10,000,000. However, unsuccessful projects that are
cancelled usually accrue costs of more than $1,500 per function point and are about 12 months
late when they are cancelled. In other words, cancelled software projects in the 10,000 function
point range cost about $15,000,000 and obviously are a write-off without any positive value.
If the failing project becomes subject to breach of contract litigation, the damage costs can be
much higher than the cost of the software itself. But litigation damages are difficult to predict
ahead of time.
A 10,000 function point project would normally have about eight project managers and perhaps
64 development personnel. The costs of fully equipping the software project managers on
10,000 function point projects with state of the art software cost and quality estimating tools is
not very expensive: at an average cost of $5,000 per manager and eight managers, the total cost
would be perhaps $40,000.
24
Thus an investment of $40,000 in state of the art project management tools and quality control
tools has the potential to keep the project within bounds and hence eliminate a $15,000,000
failure. If you subtract the $10,000,000 cost of a successful project from the potential cost of a
cancelled project, the savings due to effective project management with a powerful tool suite
would be about $5,000,000. Thus an investment of $40,000 in software project management
tools could yield savings of $5,000,000 which is a return on investment of $125 for every dollar
expended. This prediction is only an approximation, but the basic principles are probably valid.
The presence of a suite of project management tools is not, by itself, the main differentiating
factor between successful and unsuccessful software projects. The primary reason for the
differences noted between failing and successful projects is that the project managers who utilize
a full suite of management tools are usually better trained and have a firmer grasp of the
intricacies of software development than the managers who lack adequate management tools.
Bringing a large software project to a successful conclusion is a very difficult task which is filled
with complexity. The managers who can deal with this complexity recognize that some of the
cost and resource scheduling calculations exceed the ability of manual methods. They also
recognize that state of the art quality control is a necessary prerequisite for success.
Managers on failing projects, on the other hand, tend to have a naïve belief that project planning
and estimating are simple enough to be done using rough rules of thumb and manual methods.
Further, when projects began to show stress and experience problems, the managers on failing
projects often do not know how to take corrective actions, and may just ignore the problems until
far too late to solve them. Last, managers on failing projects (as noted in litigation) do not
understand the economic value of quality control and hence encounter massive delays and cost
overruns when testing begins.
The Future of Software Project Management Tools
Software project management circa 2008 is facing a number of future challenges even before
successfully solving historical challenges. For the past 50 years the average size of software
projects has been increasing. Since risk of failure and risk of poor quality correlate directly with
size, larger projects require more care in predicting quality and controlling quality than small
projects. This has long been a weakness of both software engineering and software management.
New forms of software such as the service-oriented architecture (SOA) running on new kinds of
platforms such as “cloud computing” are rapidly approaching. New development methods such
as Agile development, Extreme Programming (XP), and Team Software Programming (TSP)
need specialized estimating and planning methods. Security vulnerabilities have long been
troublesome, and in the future will become even more troublesome than in the past. Thus project
planning and estimating are becoming more complex as the industry evolves.
25
Although hundreds of commercial project management tools are available in 2008, many of
these deal only with a small subset of overall management tasks. For example a software project
management tool that can deal with schedule assignments, but which cannot deal with quality, is
not an effective solution because bugs or defects are the primary source of schedule slippage.
Further, current project management tools often do not interface well or share data well with
other tools, so that it is difficult to create a consolidated picture of every important topic. What
would benefit software project management tools for the next generation would be better ease of
sharing data, and an improved ability to link together sets of specialized functions. In other
words, apply the principles of service-oriented architecture to the domain of project management
tools.
From examining scores of cancelled projects and hundreds of costs and schedule overruns, it is
obvious that a few basic topics need to be estimated and controlled in the future much better than
they have been in the past:
Quality control is the prime weakness of software projects. Excessive numbers of bugs and
inadequate methods of defect removal are associated with every project failure and all litigation
involving breach of contract.
Requirements changes are endemic in the software world. It is not possible to stop requirements
from changing, so the solution is to deal with changing requirements in the most effective and
efficient manner. Cancelled projects or those with major overruns tended to be swamped by
unexpected changes for which no schedule or cost relief had been envisioned.
Tracking of progress on software projects has been a weak link. Project tracking omissions or
outright concealment of true status tends to occur in breach of contract lawsuits. When problems
are noted, they should be highlighted and addressed, not concealed and ignored.
Furthermore, gaps in historical cost data are common. Unpaid overtime, managerial costs, and
the work of part-time specialists such as quality assurance and data base administration are often
missing or incomplete in software project cost tracking systems. This is especially true for in-
house projects, but some omissions such as unpaid overtime even occur for outsourced
applications.
Accurate plans and estimates are needed before projects start. But it often happens that even
accurate plans and estimates are rejected by clients or top executives and replaced by impossible
schedule demands. Accurate estimation by itself is not a total solution, but accurate estimation
supported by accurate historical data is likely to be believed. Therefore accurate benchmarks of
historical projects are needed in larger volumes than have been available in the past.
26
The shared goals of the software project management community should be to improve the
professionalism of software project management. We need to be able to develop large software
projects with lower defect potentials, higher levels of defect removal efficiency, lower costs, and
shorter schedules than has been accomplished in the past. The technologies for achieving these
goals are starting to become available, but improvements in academic preparation for software
project managers and improvements in selecting the best methods are still needed.
27
Suggested Readings on Software Project Management
Baird, Linda M. & Brennan, M. Carol; Software Measurement and Estimation: A Practical
Approach; IEEE Computer Society Press, Los Alamitos, CA; John Wiley & Sons, Hoboken NJ;
ISBN 0-471-67622-5; 2006; 257 pages.
Boehm, Barry Dr.; Software Engineering Economics; Prentice Hall, Englewood Cliffs, NJ; 1981;
900 pages.
Brooks, Fred: The Mythical Man-Month, Addison-Wesley, Reading, Mass., 1974, rev. 1995.
Crosby, Phil; Quality is Free; New American Library, Mentor Books; New York, NY; 1979, 270 pages.
DeMarco, Tom; Why Does Software Cost So Much?; Dorset House, New York, NY; ISBN 0-9932633-34-X; 1995; 237 pages.
Fleming, Quentin W. & Koppelman, Joel M.; Earned Value Project Management; 2nd edition; Project Management Institute, NY; ISBN 10 1880410273; 2000; 212 pages.
Galorath, Daniel D. & Evans, Michael W.; Software Sizing, Estimation, and Risk Management: When Performance is Measured Performance Improves; Auerbach, Philadelphia, AP; ISBN 10-0849335930; 2006; 576 pages.
Garmus, David & Herron, David; Function Point Analysis; Addison Wesley, Boston, MA; ISBN 0-201069944-3; 363 pages; 2001.
Garmus, David & Herron, David; Measuring the Software Process: A Practical Guide to
Functional Measurement; Prentice Hall, Englewood Cliffs, NJ; 1995.
Jones, Capers; Program Quality and Programmer Productivity; IBM Technical Report TR
02.764, IBM San Jose, CA; January 1977.
Jones, Capers; Sizing Up Software; Scientific American Magazine; New York NY; Dec. 1998,
Vol. 279 No. 6; December 1998; pp 104-109.
Jones, Capers; Applied Software Measurement; McGraw Hill, 3rd edition 2008; ISBN 978-0-07-
150244-3; 575 pages; 3rd edition (March 2008).
Jones, Capers; Software Assessments, Benchmarks, and Best Practices; Addison Wesley
Longman, Boston, MA, 2000; 659 pages.
Jones, Capers; Conflict and Litigation Between Software Clients and Developers; Version 6;
Software Productivity Research, Burlington, MA; June 2006; 54 pages.
Jones, Capers; Estimating Software Costs; McGraw Hill, New York; 2nd edition, 2007; 644
pages; ISBN13: 978- 0-07-148300-1.
28
Kan, Stephen H.; Metrics and Models in Software Quality Engineering, 2nd edition; Addison
Wesley Longman, Boston, MA; ISBN 0-201-72915-6; 2003; 528 pages.
Kaplan, Robert S & Norton, David B.; The Balanced Scorecard; Harvard University Press,
Boston, MA; ISBN 1591391342; 2004.
Love, Tom; Object Lessons – Lessons Learned in Object-Oriented Development Projects; SIG
Books Inc., New York NY; ISBN 0-9627477-3-4; 1993; 266 pages.
McConnell, Steve; Software Estimation – Demystifying the Black Art; Microsoft Press,
Redmond, Wa; ISBN 10: 0-7356-0535-1; 2006.
Parthasarathy, M.A.; Practical Software Estimation – Function Point Methods for Insourced and
Outsourced Projects; Addison Wesley, Boston, MA; ISBN 0-321-43910-4; 2007; 388
pages.
Putnam, Lawrence H.; Measures for Excellence – Reliable Software On-Time Within Budget;
Yourdon Press, Prentice Hall, Englewood Cliffs, NJ; ISBN 0-13-567694-0; 1992; 336
pages.
Putnam, Lawrence & Myers, Ware; Industrial Strength Software – Effective Management Using
Measurement; IEEE Press, Los Alamitos CA; ISBN 0-8186-7532-2; 1997; 320 pages.
Strassmann, Paul; The Squandered Computer; Information Economics Press, Stamford, CT;
1997.
Stutzke, Richard D.; Estimating Software-Intensive Systems – Projects, Products, and Processes;
Addison Wesley, Boston, MA; ISBN 0-301-70312-2; 2005; 917 pages.
Yourdon, Ed; Outsource – Competing in the Global Productivity Race; Prentice Hall PTR,
Upper Saddle River, NJ; ISBN 0-13-147571-1; 2004; 251 pages.
Yourdon, Ed: Death March—The Complete Software Developer’s Guide to Surviving “Mission
Impossible” Projects, Prentice Hall PTR, Upper Saddle River, N.J., ISBN 0-13-748310-4,
1997.
29