University of Waterloo On software maintenance and evolution Michael W. Godfrey Software...

38
University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo

Transcript of University of Waterloo On software maintenance and evolution Michael W. Godfrey Software...

Page 1: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

University of Waterloo

On software maintenance and evolution

Michael W. Godfrey

Software Architecture Group (SWAG)

University of Waterloo

Page 2: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 2

Overview

• Software aging

• Software “maintenance”

• Software “evolution”

• A case study of growth and evolution

Page 3: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 3

Software Aging

Page 4: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 4

Parnas on “Software Aging”

• “Bit rot” isn’t the problem!

• Rather:– Our expectations change, new demands, new

environments, emergent properties, …

• e.g., MS-Word generates HTML, OLE, vmware, network drives, XML as a RDBMS, …

Page 5: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 5

Parnas on “Software Aging”

BUT– Large software systems are complex

• Difficult and expensive to change!

• “Legacy” systems are expensive assets!

– “Ignorant surgery” leads to brittle, hard-to-change systems

• Business pressures lead to “JIT maintenance”

• Accidental complexity accrues over time

Page 6: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 6

Parnas on “Software Aging”

• Parnas’ advice: – Design for change!

• Isolate likely-to-change hotspots and virtualize

• Keep documentation up-to-date

• Perform design reviews

• Perform redesigns

• But we know this already, right?

Page 7: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 7

Software “maintenance”

Page 8: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 8

A thought experiment …

• Ignore software for a moment …

– What does the term “maintenance” connote?

• Keep it running, replace worn out bits, upgrades sometimes, …

– Is this what software maintainers do?• Mostly not!

Page 9: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 9

Kinds of software maintenance

• What are the common categories of software maintenance?

• Corrective• Adaptive• Perfective• Preventative

• Alas, there is much confusion about what these terms (ought to) mean!

Page 10: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 10

Swanson’s original view [ICSE 1976]

1. Corrective: [Fix program so that it meets its specs]– Processing failure

• wrong answer or program aborts

– Performance failure • doesn’t satisfy non-functional reqs, esp. real time performance

– Impl. failure • programming standards ignored, impl. inconsistent/incomplete,

reqs not implemented correctly

2. Adaptive: [Fix to adjust to new underlying env./formats]– Change in data format – Change in processing environment

Page 11: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 11

Swanson’s original view [ICSE 1976]

3. Perfective: [It works fine, but let’s make it even better]– Processing inefficiency

• Even in performance is within specified params, may still want to make it faster

– Performance enhancement• Make the output more readable

(NOT related to response time)

– Maintainability • Better use of comments, documentation

(NOT refactoring/restructuring/redesigning)

Page 12: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 12

Swanson’s original view [ICSE 1976]

1. Corrective: [Fix program so that it meets its specs]

2. Adaptive: [Fix to adjust to new underlying env./formats]

3. Perfective: [It works fine, but let’s make it even better]

BUT• Where does preventative maintenance fit in?• Where does “make it faster” fit in?

Page 13: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 13

Swanson/Leintz [CACM 1978]

Refined model, most commonly quoted:

1. Corrective: • fixing bugs

2. Adaptive: • adapting to new environment

– but no outward semantic change, no new features

3. Perfective: • any other kind of change intended to make the

system “better”• e.g, performance tuning, adding new features,

“preventative maintenance” (redesign/refactoring)

Page 14: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 14

IEEE Std on Software Engineering Terminology [1990]• Software maintenance

– “The process of modifying a software system or component after delivery to correct faults, improve performance, or adapt to a changed environment”

• Corrective maintenance– “Maintenance performed to correct faults in hardware or software.”

• Adaptive maintenance– “Software maintenance performed to make a computer program usable in a

changed environment.”• Perfective maintenance

– “Software maintenance performed to improve the performance, maintainability, or other attributes of a computer program.”

• Preventative maintenance– “Maintenance performed for the purpose of preventing problems before they

occur.”

Q: Adding new features?

Page 15: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 15

IEEE Std on Software Maintenance Terminology [1998]• Software maintenance

– “Modification of a software product after delivery to correct faults, improve performance or other attributes, or adapt to a modified environment”

• Corrective maintenance– “Reactive modification of a software product performed after delivery to correct

discovered faults.”• Adaptive maintenance

– “Modification of a software product performed after delivery to keep a computer program usable in a changed or changing environment.”

• Perfective maintenance– “Modification of a software product after delivery to improve performance or

maintainability.”• Preventative maintenance … is MIA

Q: Adding new features?• Part of the “environment”?• Not a maintenance activity per se?

Page 16: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 16

The current view

There’s no standard taxonomy … but here’s a reasonable view:

• Corrective maintenance– Fix bugs

• Adaptive maintenance– Add support for new deployment environments– May include “adding new features”

• Perfective maintenance– Improve non-functional requirements (performance, security) apart

from portability– May include “adding new features”– May include preventative maintenance

• Preventative maintenance– Improve internal design w/o changing external semantics

Page 17: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 17

Software maintenance vs. evolution

Page 18: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 18

Software maintenance vs. evolution

• So far, we’ve treated initial delivery as the major milestone in a product’s lifespan.

– What about changes during “initial” development?– Is it reasonable to make such a sharp distinction between

pre- and post-initial delivery?– Are there other important phases in a software systems?– What are the forces at play as a system evolves?– Is “evolution” a better term than “maintenance”?

Page 19: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 19

Waterfall-like process model

Requirementsengineering

Design and implementation

Testing

Page 20: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 20

Staged model of maintenance and evolution [Bennet/Rajlich 2000]

• The staged model recognizes that there are fundamentally different life phases of a software system!

– At each stage you are typically performing mini-waterfall development cycles … differently!

– The problems, tensions, solutions, processes are very different!• Need to understand what is appropriate when.

– Lots of research needs to be done here!• What is an evolvable architecture?• What are the likely effects of different kinds of change?… etc

Page 21: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 21

Staged model of maintenance and evolution [Bennet/Rajlich 2000]

• Initial development – First version developed, already

possess the architecture and the fundamental system knowledge.

• [Active] evolution – Takes place only when the initial

development was successful, the goal is to adapt the application to the ever-changing user requirements.

• Servicing– Once the architecture or the

knowledge disappears, product is no longer evolvable (i.e., enters the service stage)

• Phase out– no more servicing.

Page 22: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 22

Staged model of maintenance and evolution [Bennet/Rajlich 2000]

• Initial development– System architecture is devised

• Often create a runnable infrastructure as soon as possible, gradually fill in missing/fudged bits [aka staged delivery]

– Design for change!• Consider strategically where you expect changes to occur and design your

architecture accordingly• Ideal: Cost of making a future change is proportional to size/complexity of

change, not size/complexity of system

– Change is the rule, not the exception.• Developers are aware of this.• QA artifacts developed

– Knowledge about “the system” is fresh, accurate; experts can be found easily

Page 23: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 23

Staged model of maintenance and evolution [Bennet/Rajlich 2000]

• Active evolution– Knowledge about system is good; some personnel turnover

since initial development.– Simple changes and adaptation to new

requirements/environments are mostly straightforward– System architecture is flexible enough to allow for more

wide-ranging changes to be planned and executed successfully

• Architecture can be redesigned if necessary, and still be coherent and reasonably faithful to the original system

Page 24: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 24

Staged model of maintenance and evolution [Bennet/Rajlich 2000]

• Servicing– Simple changes and adaptation to new

requirements/environments can still be done• Architectural change is harder and risky!

– Knowledge about system is disappearing• JIT maintenance starts to occur• Effects of change (e.g., on other parts of system) become harder to

predict– Start to treat system more like a caged animal than an

intellectual creation• May perform preventative maintenance (program comprehension,

reverse engineering) tasks to recover design, stave off death– In practice, it’s very hard to swim upstream back to “evolution” stage

• Legacy wrapping, adapters, regression testing, impact analysis, …

Page 25: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 25

Staged model of maintenance and evolution [Bennet/Rajlich 2000]

• Phase out– System is getting too risky and expensive ($$$,

performance, time/effort) to keep.• A new solution is sought

• An exit strategy must be devised

– Common approaches:• Create new adapter-style interface for clients/rest of system

• Perform data recovery and migration

• Deploy new solution to fit adapter interface, integrate with recovered data

• Shut down old system

Page 26: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 26

Maintenance vs. evolution

Page 27: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 27

Software maintenance vs. evolution

• Maintenance connotes – Fixing, rather than intellectually enhancing– Short-term, not long-term goals

• Many prefer the term evolution:– Fundamental change and adaptation– Short- and long-term change– Planned and unplanned phenomena [my two cents]

Page 28: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 28

What is evolution?

• My answer:

– Essential change in some identifiable thing (or family) over time.

• Warning: Some very personal views follow.– They are far from widely accepted.

Page 29: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 29

Evolution: Software vs. biology

• Biology has the hard scientific theory and empirical results, but evolution is a generally occurring phenomenon!

– Frogs, economies, ideas, music all evolve… and software does too!

– Although the phenomenon of evolution is apparent widely, it is hard to be truly scientific in its study outside of biology.

Page 30: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 30

Three fundamentals ideas of evolution

1. Mechanisms that increase variation / act as an agent for essential change.

2. Mechanisms that decrease variation / act as an inhibitor for essential change.

3. It’s all relative, man.– Change occurs within an environment, which also

evolves.– That is, what succeeds today may not succeed tomorrow

(but may succeed the day after).

Page 31: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 31

Biological evolution in a nutshell

“[Biological] evolution is change in a gene pool of a population over time.” [talk.origins FAQ]

• Genes/alleles are the essential encoding that is inherited by living creatures; they comprise the genotype of the individual.– A gene is the dedicated “slot” or memory location– Alleles are the set of abstract values at those genes– DNA are the bits that encode the allele messages (ACGT)

• The phenotype is the totality of what an individual becomes.– The phenotype is determined by the genotype PLUS interaction with

the environment.– Non-genotypic change is not inheritable! [Lamarck was wrong!]

Page 32: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 32

Biological evolution in a nutshell

• Consider:– Biston betularia, an English moth

• 2% dark in 1848, 95% dark in 1898 [Manchester]

– Hair colour, tattoos, height• Morphological change evolution• Evolution morphological change

– Sickle cell anemia– Evolution is not progress!

• Biological evolution is a greedy algorithm

– Individuals are embedded in and alter their environments!• They consume resource, alter the competitive balances, and

modify their environment to suit themselves as they are able

Page 33: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 33

Biological evolution in a nutshell

• Mechanisms that increase genetic variation– Mutation (new values)

– Recombination (new combinations)

– Gene flow

– Genetic drift (chance increase in existing allele freq.)

• Mechanisms that decrease genetic variation– Natural selection (!!)

– Sexual selection

– Genetic drift (chance decrease in existing allele freq.)

Page 34: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 34

Biological vs. software evolution

• How hard should we push the analogy of biology into software?– OK, genotype == source code, but what is a

software phenotype?– What is a software individual/population?

I won’t try to answer this …

Page 35: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 35

Biological vs. software evolution

• One important fact:– Unlike biology, the prime agent for new

values/traits of software is not chance, it’s marketing!

• Rate of change is much, much faster!

• Software evolution is (arguably) Lamarckian!

Page 36: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 36

Biological vs. software evolution

• One important observation:– Not all of the effects of software change are

planned or foreseen.• Software reacts with its environment (development,

deployment, political) and forms a complex feedback system that influences the future evolution of the software [Lehman]

• Software systems exhibit interesting emergent phenomena

Page 37: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 37

Summary:Biological vs. software evolution

• If you take away nothing else from this, at least remember this:

– “We shape our tools, then our tools shape us.”[McLuhan]

– We can plan and execute changes …• … but we cannot understand in advance all of the effects of

the changes, both on the environment and feeding back into the development of the software system.

– Understanding how software evolves requires studying both the planned and unplanned phenomena of software change.

Page 38: University of Waterloo On software maintenance and evolution Michael W. Godfrey Software Architecture Group (SWAG) University of Waterloo.

CS446 Michael W. Godfrey 38

Further reading

• Evolutionary biology:– The talk.origins archives

• http://www.talkorigins.org

– The Selfish Gene, Richard Dawkins

• “Other” evolution:– The Meme Machine, Susan Blackmore– The Nature of Economies, Jane Jacobs