University of Waterloo How does your software grow? Evolution and architectural change in open...

University of Waterloo

How does your software grow?Evolution and architectural change in open source software

Michael Godfrey

Software Architecture Group (SWAG)

University of Waterloo

Michael W. Godfrey Open source evolution — How does your software grow?

2

What is software evolution?

“Evolution is what happens while you’re busy making other plans.”

• We distinguish between maintenance and evolution:– Maintenance is the planned set of tasks to effect changes.– Evolution is what actually happens to the software.

• All I want to know is:How and why does software evolve?


3

Why should we care?

• Much of the commercial software world operates in perpetual crisis mode.– “Fix it, don’t try to understand it.”

– Just-in-time program comprehension [Lethbridge]

• … but … large software systems are major assets of many businesses– Getting it right more important than getting it done fast.

– Budget and time for preventive maintenance, navel gazing.

• Relatively little research on trying to understand how and why programs evolve.


4

Lehman’s Laws of Software Evolution

• Based on measurement of a few (commercially-developed) systems, most notably IBM’s OS 360– Originally three laws, now there are eight.

• Controversial as “laws”– Has been criticized for strong claims based on limited data.

– However, it’s pioneering work on software evolution and software engineering.


5


1. Continuing change — An E-type program that is used must be continually adapted else it becomes progressively less satisfactory.

2. Increasing complexity — As a program is evolved, its complexity increases unless work is done to maintain or reduce it.

3. Self regulation — The program evolution process is self-regulating with close to normal distribution of measures of product and process attributes.

4. Invariant work rate — The average effective global activity rate on an evolving system is invariant over the product lifetime.


6


5. Conservation of familiarity — During the active life of an evolving program, the content of successive releases is statistically invariant.

6. Continuing growth — Functional content of a program must be continually increased to maintain user satisfaction over its lifetime.

7. Declining quality — E-type programs will be perceived as of declining quality unless rigorously maintained and adapted to a changing operation environment.

8. Feedback system — E-type programming processes constitute multi-loop, multi-level feedback systems and must be treated as such to be successfully modified or improved.


7

Lehman’s Laws in a nutshell

• Observations:– (Most) useful software must evolve or die.– As a software system gets bigger, its resulting complexity tends to

limit its ability to grow.– Development progress/effort is (more or less) constant;

growth is at best constant.Lehman/Turski’s model: y’= y + E/y2 ~ (3Ex)1/3

where y= # of modules, x = release number

• Advice: – Need to manage complexity. – Do periodic redesigns.– Treat software and its development process as a feedback system

(and not as a passive theorem).


8

Lehman’s examples


9

The S curve

time

size


10

A case study in evolution:The Linux OS kernel [ICSM-00]


11

A case study in evolution:The Linux OS kernel [ICSM-00]

Evolution in Open Source Software: A Case Study” [Godfrey and Tu, ICSM 2000]

• It’s Linux!– Large system, very stable, many releases over several

years, many developers– Growing mainstream adoption (e.g., IBM S390 port)

– Commonly used within networked systems

• Open source development model– Interesting phenomenon in itself– Easy to track, can publish results, many experts – Not much previous study


12

Evolution of Linux: Questions

• How has Linux evolved over time?– Does it obey Lehman’s laws?

– What is the best way to characterize growth?

• How has its (open source) process model affected its development?

• How has the (high-level) architecture– changed over time?

– affected the system’s evolution?


13

Open source development

• Open source development vs. open source softwareGNU, Linux, Apache, vim, gcc, FreeBSD

vs.Mozilla, JDK, Jikes, NetBeans

• “The Cathedral and the Bazaar” [Raymond]

– Usual goal: scratching an interesting itch, not filling a commercial void.– Anyone may contribute, tho owner(s) have final say.

• Usually, developers work part-time and for free.– Motivation is peer recognition and personal satisfaction, not money.– However, industrial participation also increasing

(e.g., Cygnus, IBM)


14

Open source development

• Largely immune from time-to-market pressures– Can release when it’s really ready

• Can be hard to control/direct developers– Big egos, can’t be “fired”– What’s cool vs. what’s needed– Less “sexy” development tasks often suffer

e.g., planned testing, preventive maintenance

• Code quality varies widely– Some projects have coding standards– Unstable/experimental code common (and even encouraged)– Quality maintained via “massively parallel debugging”, not rigorous

testing.


15

Linux background

• Linux kernel v1.0 released March 1994– 487 source files, 165 KLOC, i386 only

• Linux kernel v2.3.39 released January 2000– 4854 source files, 2.2 MLOC, 10 hardware

architectures supported, over 300 developers credited

• Maintained along two parallel paths: – development and stable


16

Methodology

• Examined 96 versions of Linux kernel – 34 of the 67 stable releases– 62 of the 369 development releases

• All measures considered only .c/.h files contained in tarball– Counted LOC using “wc –l” and an awk script that ignored comments

and blank lines– Counted # of fcns/vars/macros using ctags– Architectural model (SSs hierarchy) based on default directory structure

• We plotted growth against calendar time– Lehman suggests plotting growth against release number


17

Software architecture of Linux

[IWPC-00]


18

Growth of # of source files

0

1000

2000

3000

4000

5000

6000

Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

# o

f so

urc

e co

de

file

s (*

.[ch

] )

Development releases (1.1, 1.3, 2.1, 2.3)

Stable releases (1.0, 1.2, 2.0, 2.2)


19

Growth of # of global fcns, variables, and macros

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000


# o

f g

lob

al f

cns,

var

iab

les,

an

d m

acro

s Development releases (1.1, 1.3, 2.1, 2.3)

Stable releases (1.0, 1.2, 2.0, 2.2)


21

Growth of Lines of Code (LOC)

0

500,000

1,000,000

1,500,000

2,000,000

2,500,000


Tota

l LO

C

Total LOC ("wc -l") -- development releases

Total LOC ("wc -l") -- stable releases

Total LOC uncommented -- development releases

Total LOC uncommented -- stable releases

y = .21*x2 + 252*x + 90,055 r2=.997


23

Average/median .h file size

0

20

40

60

80

100

120

140


Un

com

men

ted

LO

C

Average .h file size -- dev. releasesAverage .h file size -- stable releasesMedian .h file size -- dev. releasesMedian .h file size -- stable releases


25

SS LOC as percentage of total system

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0


Per

cen

tag

e o

f to

tal

sy

ste

m u

nc

om

men

ted

LO

C

driversarchincludenetfskernelmmipclibinit


26

SS LOC as percentage of total system (ignoring drivers)

0.0

5.0

10.0

15.0

20.0

25.0

30.0


Per

ce

nta

ge

of

tota

l s

ys

tem

un

co

mm

en

ted

LO

C

archincludenetfskernelmmipclibinit


29

Observations and hypotheses

• Growth along devel. path is super-linear!

y = .21*x2 + 252*x + 90,055 r2=.997y = size in LOC x = days since v1.0 r2 is “coefficient of determination” using least squares

Lehman/Turski’s model: y’= y + E/y2 ~ (3Ex)1/3

where y= # of modules, x= release number

– Linux’s strong growth is continuing.– This is stronger growth at MLOC level than observed by others

(Lehman, Gall), even for other OSs.


30

Growth of fetchmail [Raymond]


33

Growth of gcc/g++/egcs

0

100

200

300

400

500

600

700

800

900

1000

Aug-87 Dec-88 May-90 Sep-91 Jan-93 Jun-94 Oct-95 Mar-97 Jul-98 Dec-99 Apr-01

# o

f m

od

ule

s g++

gcc

egcs


34

Growth of vim (text editor)

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

May 1990 Sep 1991 Jan 1993 Jun 1994 Oct 1995 Mar 1997 Jul 1998 Dec 1999 Apr 2001

Tota

l L

OC

Total LOC ("wc -l")

Total LOC (ignoring comments and blank lines)


35

vim avg % comments and blank lines per file

25.0

26.0

27.0

28.0

29.0

30.0

31.0


Ave

rag

e p

erce

nt

com

men

ts +

bla

nk

lin

es


36

vim avg/median file size

0

100

200

300

400

500

600

700

800

900

1000


Un

com

men

ted

LO

C

Average uncommented LOC per source fileMedian uncommented LOC per source file


37

vim’s architecture


38

Some open questions

• Philosophical:– Does software evolve in the same way as frogs and social structures?

• The Selfish Gene, by Richard Dawkins• The Nature of Economies, by Jane Jacobs

– What are the recurring patterns and compelling metaphors of software evolution?

• Methodological:– How to measure size?

• How to correlate size and quality?– How to measure change?

• How to model architectural change?– What is the predictive power of such models?

• Do the “other phenomena” dominate?


39

Some open questions

• Practical:– What information do developers need to know about how a software

system has evolved?

– What kinds of tools would be useful:• to the front-line developer?

• to the manager?

– How best to deal with:• Large data sets (large_system many_versions)

• Visualization and navigation


40

Change patterns and evolutionary narratives• “Band-aid evolution” (just add a layer)

– quick way to add new functionality, esp. if system is not well understoode.g., Y2K fixing, adding portability, new features

• “Vestigial features”– design artifact persists after rationale dies

e.g., whale fin bone structure resembles hand

• “Adaptive radiation” [Lehman]– when conditions permit, encourage wild variation for a while. – later, evaluate and let “best” ideas live on.

e.g., Linux kernel evolution

• “Convergent evolution”– compare similar systems to reference arch. (or to each other)

e.g., everyone grows an XML generator in response to market pressure


41

Change patterns and evolutionary narratives

• Cathedral style [Raymond]– careful control and management– debugging done before committing code– evolution is slow, planned, rarely undone

• Bazaar style (OSD)– lots of low-level changes, frequent fixes– lots of “building around” rather than wholesale changing, occasional

redesigns– creeping feature-itis, “complete” dependency graph


44

Change patterns and evolutionary narratives• Phenomena observed in Linux evolution

– Careful control of core code; more flexibility on contributed drivers, experimental features

• Linus has many lieutenants

– “Aunt Tillie” effect• Simplicity and scrutability of code, development processes,

approval process, etc.

– “Mostly parallel” enables sustained growth• “Hard interfaces” make good neighbours.• Loadable modules makes feature development easier

– “Clone and hack” makes sense!


45

Change patterns and evolutionary narratives• Phenomena observed in Linux evolution

– Amazing social phenomenon of OSD• “You can try this at home”

– and they did!

• Anti-MS sentiments, – “We can build it ourselves!”– Enlightened self-interest for many large computer industry companies

» “If we can’t own the standard, no one should.”

• Bandwagon effect (both OS developers and industry)– Support for Linux as deployed OS by IBM, Dell, Sun, …– Lots of contributed production-quality third party code from industry

(IBM S/390, drivers)


46

An observed evolutionary phenomenon

• Code cloning!– Usually regarded as a bad sign– Usual solution:

• Abstract commonality into a single place, remove duplication

• In an OO setting, can use inheritance

– But as observed in Linux, it seems less problematic than one might think!


47

Case study: Cloning in Linux SCSI drivers

• Nice, controlled experiment:– Large body of code, multiple versions, well used system,

open source

– SCSI drivers all do similar tasks

– Source comments shows cloning has occurred!

• Approx. 500 releases of Linux since 1994.

• Kernel v2.3.39: (released Jan 2000)

– 5000 source files, 2.2 MLOC, 10 hardware architectures

– drivers/scsi has 212 source files, 166 KLOC,


48

Goals of case study

• Examine “real world” cloning:– How common is it?– Why is it done?– What do the “cloning patterns” look like?

• Examine parallel evolution:– What kinds of changes are common?– Do developers (need to) change clone relatives too?

• Is there a better design structure lurking?• Compare against existing clone detection tools

– Are detections tools looking for the right indications of cloning?


49

SCSI Subsystem - Size (rel. 2.2.16)

• Number of source files: 211

• Number of functions: 2512

• Number of lines of code: 254,953

• % of comments: 38

• Number of low-level drivers: 80

• File size: – on average ~3000 lines

– large multi-card drivers ~15,000 lines


50

SCSI Subsystem - Architecture

• Upper Layer – Uniform way of handling devices

– Hard Disk, CD-ROM Disk, Tape, Generic

• Middle Layer– “bridge” between Upper Layer and Low-Level Devices

• Low-Level Device Drivers– low-level driver functionality and management


51

Clones Expected?

• Why did we expect to find clones?– Every driver must implement uniform interface

– Design of subsystem does not support other forms of reuse

– Driver logic is relatively simple (!)

– Devices from same family more cloning

– Completely different hardware less or no cloning

– Open source anyone can reuse code

– Easier and more efficient to reuse existing code

– Reused code already tested, so probably better quality than if we build it from scratch


52

Clones - Manual Inspection

• From source code comments, we have found:

esp.[ch]

jazz_esp.[ch]

dec_esp.[ch]

cyberstorm.[ch]

cyberstormII.[ch] mca_53c9x.[ch] blz2060.[ch] fastlane.[ch]

qlogicisp.[ch]

qlogicpti.[ch]

fdomain.[ch]

fd_mcs.[ch]

sd.[ch]

sr.[ch]

t128.[ch]

pas16.[ch]


53

Types of Changes Detected

• Names of variables

• Initialization parameters and constants

• Driver specific initialization logic removed/added

• Small change in supporting functions

• Small changes in driver management code

• Comments are updated

• Code changed is highly embedded into other code, which makes extraction of that code hard


54

Conclusion (Cloning)

• Unclear that current clone detection tools “do the right thing”– Combination of different approaches should give the best detection

results

• Theory developed on clone management, detection, and removal is not universally applicable to all types of applications, languages, and designs– Need more qualitative analysis of “cloning in the real world”

• As practised, code cloning in the Linux SCSI subsystem seems like a reasonable approach!


55

The past, present, and future of open source software● Past

➢ An outgrowth of the Unix sysadmin tradition➢ Bug fixes and evolution!

● Present➢ Trendy ... but large companies see more

Apache is used by 63% of web servers [Aug 02 Netcraft survey]

BIND used by vast majority of DNS servers Sendmail is most widely used email transport

...


56

The past, present, and future of open source software● Future

➢ Corporate prisoner's dilemma: Enforced open-ness allows companies to breathe easier, can

concentrate on core strengths and real innovations➢ Governments are beginning to require open source and open

standards when available The German government and IBM recently announced a "far-

reaching co-operation agreement" [NYTimes]➢ Concern over Microsoft, .NET, and monopolistic practices

Mono, an open source implementation of the .NET framework, being developed.


57

Open source infrastructure

• Pros● Reliability, trust, confidence by users

➢ Many users/readers aids in debugging● Can usually “fork” projects for specialized needs ● Interoperability when component vendors conform to standards

● Cons● Some open source licences can cause headaches

e.g., the GPL “virus”● Not in everyone's strategic best interests● Non-compliance is a recurring problem


58

Summary: Evolution and open source software

• Open source software seems to break some of the “rules” of how successful software is built and evolved:

– Motivation for developers is fun, pride, professionalism, politics, … rather than money– It violates some of Lehman’s laws

• … yet this software is often of high quality and in wide use.– esp. common infrastructure-type systems – e.g., Linux/MacOS kernel, apache, ldap, samba, imap, …

University of Waterloo How does your software grow? Evolution and architectural change in open...

Documents

Transcript of University of Waterloo How does your software grow? Evolution and architectural change in open...