Software Maintenance

“It doesn't take a lot of skill to get a program to work. The skill comes in when you have to keep it working.”

--Robert Martin

What is software maintenance?

• Changes made to software after delivery. Software may include source code, documentation, and operating procedures.

Motivation• Any software system that is used will almost certainly

require maintenance.

• Surveys show that 40-70% of the total life cycle cost of a software system is spent on maintenance.

• Software maintenance is more than just continued development after release.

• While there are many similarities between maintenance work and new development work, there are some important differences as well.– Maintenance must be done in the context of an existing system

which imposes additional constraints.

– Home builders and programmers alike prefer green field development.

Why software maintenance is needed

• Software providers have an obligation to fix defects that significantly impair the functionality of their software.

• Software systems that get used tend to generate requests for enhancements and new functionality. Heavy users of a software system often discover ways of enhancing existing functionality or opportunities for new features.

• When the environment around software changes (operating system, government regulation, business rules), the software must be updated, otherwise it becomes less useful.

• Changes that improve the maintainability of software without changing its functionality are sometimes helpful. Such changes make it easier to complete functional changes on a short schedule.

Types of software maintenance• The four types of software maintenance are:

– Corrective. Despite your best efforts, any non-trivial program will likely have defects. Corrective changes are changes to fix defects. This includes defects discovered by end users and those found through internal testing and other means.

– Preventive. Software can have flaws that don’t rise to the level of defects. For example, a programmer might carelessly use single character variable names or neglect design. Neither causes a direct failure in the program but both make the program harder to maintain in the future. Preventive changes are changes that improve the maintainability of a program or reduce the potential for future failures.

– Adaptive. Deployed software operates in an environment comprised of hardware and other software. Changes in this operating environment may compel changes in hosted programs. For example, if your Internet Service Provider (ISP) moves to a newer version of PHP, you may have to make changes to your PHP scripts in order for them to continue to work in the new environment. Adaptive changes are changes needed to cope with changes in the operating environment. The changes don’t bring any new functionality, they simply keep existing functionality working in the new environment.

– Perfective. Perfective maintenance are changes that add new features or capabilities in response to changes in requirements. While most of the changes in this category are likely to be for new functional requirements , it also includes changes made to implement new non-functional requirements such as usability.

Summary of Maintenance Types

• Correction– Corrective Maintenance – changes made to correct

defects– Preventive Maintenance – changes made to improve

maintainability or prevent problems from occurring

• Enhancement– Adaptive Maintenance – changes made to adapt the

software to changes in its technical environment– Perfective Maintenance – changes made to add new

features or capabilities

Terminology Warning• While the above categories are probably the most common, they aren’t the

only way of categorizing the different types of software maintenance.

• The different reasons or motivations for making a change are generally recognized but different literature sources use different taxonomies for grouping the reasons. For example, some sources don’t recognize a separate category for preventive changes. Without a category for preventive changes, refactoring would be considered new functionality since it addresses the non-functional requirement maintainability.

• Even more confusing are instances when the same category label is interpreted differently. For example, I (and others) define preventive changes in such a way to include changes made that improve maintainability. IEEE standard 14764, Standard for Software Life Cycle Processes — Maintenance, considers changes that improve maintainability to be perfective changes. The IEEE standard 14764 defines a preventive change as ones that is “made to detect and correct latent faults in the software product before they become operational faults”.

Example

Question: Assume you rewrite an algorithm to run at O(n log n) vs O (n2)? What type of change is it?

Answer: It depends. If there is a non-functional performance requirement that isn’t being met and the change brings the software into compliance with this non-functional requirement it would be considered a corrective change. If the software was in danger of violating this requirement it would be considered a preventive change. If the change was in response to a new non-functional performance requirement, it would be considered a new capability.

Effort Distribution

• On average, about 80% of the maintenance effort goes toward non-corrective changes [Lientz and Swanson 1980, via Grubb 2003]

Metaphors for understanding and communicating subtle maintenance concepts

• Two useful metaphors for understanding and communicating subtle concepts in software maintenance are:– Software Entropy, and– Technical debt

Entropy• Entropy is a measure of “disorder” in a closed system. The more

disorganized something is, the higher its entropy.

• According to the Second Law of Thermodynamics, a system free of external influences will tend to become more disordered with time. Left alone, cars rust, gardens become overgrown with weeds, and houses fall into a state of disrepair.

• It takes extra effort from outside a closed system in order to reverse the tendency toward disorder within a closed system.

• Just to preserve the status quo, gardeners must periodically pull weeds and homeowners must perform routine maintenance.

High Entropy

Software Entropy• Software isn’t a thermodynamic system but the concept of increasing

entropy or disorder applies just the same.

• As software is modified or extended its internal structure tends to degrade unless extra effort is devoted to making sure changes not only work but also leave the resulting code no harder to understand and modify. If extra effort isn’t devoted to controlling system entropy, future changes will be harder.

• Entropy in software takes the form of:– Duplicate code. It’s often easier in the short run to copy-paste-modify code rather

than factor out commonalities and represent separately only what is unique.

– Comments that explain “what” rather than “why” and comments out-of-sync with code.

– Classes that expose (don’t encapsulate) important design decisions and implementation details.

• Software entropy increases when principles of good design and construction aren’t followed. System entropy can be avoided or reversed by following the principles of good design discussed in chapter x and by applying systematic refactorings to recognized code smells as discussed later in this chapter.

Technical Debt• Teams are often under pressure to cut the time and cost of

enhancements. By compromising internal code quality and taking other shortcuts it is possible to rush features into production but not without consequences going forward. Technical debt is a metaphor for communicating these consequences to non-technical stakeholders.

• Software changes can be accelerated in the short-term by:– Taking less time for design. Design time can be shortened by neglecting non-

functional requirements like maintainability and testability.

– Writing and running fewer tests. Development time can be shortened by writing fewer tests and/or performing less regression testing. Also, some initial time might be saved by executing manual tests rather than taking the time to write automated tests that can be reran.

– Postponing needed changes to design documents.

• None of the above are time savers; they all time shifters. Skimping on design might help get a feature out the door but it will also increase complexity and the cost of making future changes. Missing test cases have to eventually be written or stakeholders have to accept a higher risk of errors. If documentation isn’t keep up to date, it will take more time for new staff members to come up to speed on the software.

Servicing Technical Debt

• Taking these shortcuts amounts to borrowing from the future. The consequence for taking shortcuts during development is growing technical debt that must be serviced in the future.

• Certain types of technical debt have ongoing interest payments in the form of lower productivity / velocity going forward. Shortcuts that increase software entropy make it harder to understand and modify the software in the future. The extra effort needed to modify the software caused by increased system entropy can be thought of as interest payments on the technical debt. The greater the technical debt; the greater the interest payments.

• Don’t like spending (wasting?) money/time on interest payments? The alternative is to pay down the principle on the debt. Paying down principle on technical debt amounts to redirecting effort from new feature development to rework and/or catching up on work postponed (e.g. updating design documents, writing automated test cases, etc.)

Strategic Debt• Not all debt is bad. Businesses regularly borrow money to

purchase raw materials which are processed and converted into useful products. The products are sold at a profit relative to the cost of the raw materials and labor input. Voila! Wealth creation.

• While there is a range of personal views on debt, most people recognize good and bad (strategic and non-strategic) debt. Borrowing money to purchase a house or an education: good. Both have the potential to appreciate in value. Borrowing to finance a vacation to Tahiti: bad. Assuming a delayed vacation would be just as enjoyable (and maybe more enjoyable because you wouldn’t be worrying about how you were going to pay for it) you are better off to wait and pay cash.

Strategic Debt [Cont]

• Debt (for an individual for a nation for a software system) is a beneficial if:– The debt is small relative to income (or assets) (GDP for a

nation, staff hours for a project)

– The debt is put to productive use. In other words, the benefits of time shifting purchasing power are worth the costs. In financial terms: there is an adequate return on the investment.

• Valid reasons you might take on technical debt:– In order to meet a deadline or catch up with a competitor or beat

the competition to market.

– You simply might not have the resources to finance “the right approach.” This is especially likely for cash-poor startup companies.

– You plan to retire the system in the near future. As long as the current changes work, who cares how unstructured the code is?

Non-Strategic Debt

• Unproductive reasons for taking on technical debt:– It’s just easier to throw together a solution rather than think it through.

– If management presses for features to be completed 20% of the estimated schedule, developers will work more efficiently.

– Anyone unfamiliar with the concept of system entropy and technical debt could inadvertently take on technical debt by not considering the consequences.

Strategic Debt [Cont]• Sometimes it pays to take on technical debt. A team might

hack some features in order to meet a deadline with the understanding that any shortcuts taken will be reworked or paid for in the future. (See notes below.)

Retiring Principle• When software entropy is a significant component of technical debt,

you can get back to normal development rates by retiring principle.

Understanding debt levels

• National debt clocks—physical and virtual—are used to draw attention to the growing national debt in the US and around the world. As of 6/10/2010:

• How nice it would be if all software systems came with a technical debt clock that clearly showed the amount of technical debt in the system.

• Project managers could use it in planning. “Hum…looks like technical debt increased 10% since the last iteration. I had better make allowances for lower productivity during the next iteration.”

Understanding debt levels [Cont]

• Unfortunately, technical debt, like many other forces of software development, is intangible and hard to measure.

• Some ways of assessing the level of technical debt being carried by a software system:– Tracking project velocity/productivity. (Admittedly a

lagging indicator.) A drop in productivity could signal a rise in technical debt.

– Track rework. What percentage of time is being spent on rework (corrective maintenance) as apposed to extending existing capabilities.

Lehman’s Laws of Software Evolution

1. The Law of Continuing Change – The environments around most software systems undergo continuous change. Systems in a changing environment must be continuously adapted or they become progressively less useful. Just as biological organisms must adapt to a changing environment (or become extinct), software systems must adapt or become progressively less useful and possibly extinct (retired). Continuous change in the environment is almost guaranteed because, in kind of a feedback loop, the addition of the software system is a change itself.

2. The Law of Increasing Complexity – Recall from the discussion above that entropy is a measure of the disorder or randomness in a closed system. According to the Law of increasing Complexity, as a software system evolves its complexity (disorder) increases unless work is done to maintain or reduce it. (See notes below for a corollary to this and other laws.)

Lehman’s Laws of Software Evolution [Cont]

3. The Law of Self-Regulation – A self-regulated system tends to maintain a stable, constant condition. For example, warm-blooded animals maintain a roughly constant body temperature regardless of the ambient temperature. The program evolution process is self-regulating. Product and process measures such as number of reported errors and time between releases is invariant across releases.

4. The Law of Conservation of Organizational Stability – The average output or production during maintenance is invariant over the product life time. The number of people devoted to system maintenance might increase or decrease but the effective output or production tends to remain the same.

Lehman’s Laws of Software Evolution [Cont]

5. The Law of Conservation of Familiarity – Users and other stakeholders must maintain a certain level of familiarity with the software from release to release in order to make effective use of the software. There is a limit to stakeholders’ capacity to assimilate changes. Consequently, the content of successive releases is statistically invariant. Stakeholders’ ability to assimilate new information bounds the amount of change allowed.

6. The Law of Continuing Growth – Both the first law and this law say that E-Type systems are destine to grow. The first law says they will grow through adaptive maintenance in response to changes in the environment. This law says they will grow through perfective maintenance in order to maintain user satisfaction.

Lehman’s Laws of Software Evolution [Cont]

7. The Law of Declining Quality – The perceived quality of a software system will appear to decline unless it is rigorously maintained and adapted to changes in its operational environment. (see notes below for justification)

8. The Feedback System Law – Evolution processes are complex multi-loop, multi-level, multi-agent feedback systems. This feedback must be taken into account when planning and following software evolution processes. This law generalizes the idea of feedback present in the other laws.

Reengineering Legacy Systems

• Reverse Engineering

• Restructuring

• Forward Engineering

• Reengineering

The lure of the complete rewrite

• Legacy systems are expensive to maintain.• At times it may seem like the answer is a

total rewrite.• A complete rewrite is (almost) never the

best way to deal with the headaches of maintenance.

• The problems and cost of ongoing maintenance—as unpleasant as they might be—usually pale in comparison to the risks and cost of rewriting a system from scratch.

Things to consider before making the decision to rewrite a system

• Cost. Maintenance is a relatively small incremental cost spread over time. Rewriting the software takes a large upfront investment. Consider the time value of money.

• How to handle the transition. Time spent creating a new system is time not spent maintaining the existing system. During the transition you will likely need to devote some time to maintaining the old system.

• The existing system embodies years of accumulated knowledge. Extracting or reengineering this knowledge can be difficult.

• The existing system has been tested for years. It will take several years before the new code can reach the same level of reliability. New code might be better structured and easier to understand but it is not likely to have fewer defects.

Maintenance Process Models

• Quick Fix Model – When a problem occurs, fix it and release the fix as quickly as possible. Skip making a detailed analysis of the long-term effects and don’t worry about degrading code structure. Using the quick fix model during maintenance is analogous to using code-and-fix during development.

• Might work for small, non-critical systems, maintained by a single person. Does deliver fixes quickly and cheaply. (It only delivers change economically if it doesn’t cause more expensive problems later.)

Maintenance Process Models [Cont]

• Alternative to quick fix model: Adapt one of the more modern development models (spiral, staged, evolutionary prototyping, evolutionary delivery) to the maintenance environment.

• Maintenance process models have two stages not found on new development models:– Evaluate need for the change. Not all defects have to be

fixed. Not all requests for new features are economically justifiable.

– Understand the current system. You have to understand the current system before you can safely change it.

Generic Maintenance Process Model

1. A request for change is received (aka Modification Request or MR)

2. Categorize change request (Corrective, Perfective, Preventive or Adaptive) and estimate priority.

3. Analyze change request. Do impact analysis to determine ramifications of change. Is it feasible? Needed? Economical? Change control board (CDB) decides whether or not to accept request.

4. Understand existing program and how to effect change. (Initial investigation and review might begin during step 3.)

5. Design / Implement and Unit test change.6. System test and regression test7. Acceptance test8. Delivery

Defect Triage

Maintenance compared to new development

• Maintenance work is done within the parameters and constraints of an existing system

• It’s like the difference between waterproofing a basement during construction vs. after construction.

Program Understanding

• Maintenance has the added challenge of understanding the existing system as a whole and the specific parts that will be changed in more detail.

• About 50% of the total effort spent maintaining software is spent understanding the program to be changed.

Factors that affect program understanding

• Expertise – The programmer’s knowledge of the domain and implementation technologies has a significant impact on program comprehension.

• Implementation issues – coding style, quality and quantity of comments, variable names, design.

• Documentation – is there external documentation? Is it up-to-date? External documentation is especially important when the original author is not available.

• Availability of program comprehension tools – static and dynamic analysis tools.

Refactoring

• Code refactoring is the process of changing a computer program's internal structure without modifying its external functional behavior.

• Refactoring is a preventive change. It doesn’t change the programs functionality but it does make it easier to understand, modify and extend.

• Refactoring is a prescribed step in some methodologies. With TDD, developers refactor after getting a test to pass.

• Refactoring is a technique for dealing with software entropy. (Entropy = tendency for the structure or design of a software system to deteriorate over time as it undergoes changes.)

Refactoring is more than just “cleaning up code”

• The term refactoring may be used to refer to any general cleanup activity. What is being discussed here are systematic changes to code that address reoccurring or identifiable problems.

• What makes systematic refactoring different from “cleaning up code”? Systematic, disciplined changes. Code transformations that don’t change the functionality of the program. Solutions to reoccurring or identifiable problems in code.

• There are catalogues of refactorings which serve as handbooks for software engineers. (Engineers always favor routine solutions over original solutions.)

Many programming devolvement

environments offer refactoring tools • Eclipse (right)

• Visual Studio (below)

Example Refactorings

• Encapsulate field – Make a public field private and add accessors.

• Extract method – Turn a sequence of statements into a method with a descriptive name.

• Inline method – (Opposite of extract method.)• Hide delegate – Add methods to hide a delegate.• Rename method – Change the name of a method

to a more descriptive name.• Etc.

Does the following code violate any principles of design?

Hide Delegate

1. Add a delegating method on the service for each method which the client is calling indirectly.

2. Update client to call new delegating methods.

3. Compile and test after adding each method.

4. If no other clients need to access to the delegate object, remove the accessor method on the service for the delegate object.

5. Compile and test.

Refactored Code

Maintenance Measures

• Measurement supports decision making.

• Example measures:– Size (LOC, FP)– Complexity

• McCabe’s Cyclomatic Complexity

• Halstead’s Measures

– Understandability– Maintainability

Code Metrics in Visual Studio

• LOC

• Class Coupling

• Depth of Inheritance

• Cyclomatic Complexity• Maintainability Index =

MAX(0, (171 - 5.2 * ln(Halstead Volume)

- 0.23 * Cyclomatic Complexity

- 16.2 * ln(Lines of Code)

) * 100 / 171)

Cyclomatic Complexity—Defined

• Cyclomatic complexity (CC) measures the number of decisions or branch points in a unit of code.

• Proposed by McCabe in the 1970’s.• CC is a complexity metric that gives some idea of the

maintainability of a routine. For example, a company might require formal review of all modules with routines that have a cyclomatic complexity > 10.

• CC is also used during testing. The CC of a routine is also the number of linearly independent paths through the routine’s source code.

Calculating Cyclomatic Complexity

• One way of calculating the cyclomatic complexity of a routine is to:1. Create the control flow graph for the routine2. The cyclomatic complexity of the routine is:

V(G) = E – N +2Where

E = the number of edges in the control flow graphN = the number of nodes in the control flow graph

Example-1

Example-2

Assumptions about control flow graph

• Single entry and single exit

• It is a single connected component (i.e. one routine). The generalized formula for multiple connected components is:

V(G) = E – N + 2 * PWhere E = Edges N = Nodes P = Connected components

Cyclomatic Complexity—Example 1

Cyclomatic Complexity—Example 2