Software Systems as Cities

Software Systems as CitiesRichard WettelREVEAL@Faculty of informaticsUniversità della Svizzera italiana

Dissertation committeeProf. Matthias HauswirthProf. Cesare PautassoProf. Rainer KoschkeProf. André van der Hoek

Research advisor Prof. Michele Lanza

cost of a software system


maintenance

90%

60%[Zelkowitz et al., 1979]

[Lientz & Swanson., 1981]

[McKee, 1984]

[Erlikh, 2000]


maintenance

90%

60%

program comprehension

[Zelkowitz et al., 1979]

[Lientz & Swanson., 1981]

[McKee, 1984]

[Erlikh, 2000]

[Corbi, 1989]

Understanding software is

expensive

What makes software hard to understand?


1 size


21 size

complexity


21

3

size

complexity

evolution

Software is

intangible

seeingis understanding

Visualization to the rescue

[Lanza & Ducasse, TSE 2003]

number oflines of code

number of attributes

number of m

ethods

class

inheritance

Visuallanguage

Visualization to the rescue

[Lanza & Ducasse, TSE 2003]

met•a•phor |ˈmetəˌfôr; -fər|nouna figure of speech in which a word or

phrase is applied to an object or action to which it is not literally applicable

City metaphor in software visualization

City metaphor in software visualization

Figure 1. Architectural Program Visualization of a C++ Program.

ferent views [27, 28, 32]. In our current research, we are

particularly interested in whether these difficulties can be

overcome by judicious rendering within a single view.

To help answer this question, we have implemented a

single-view visualization tool (Section 4). We use a

metaphor based on cities that provides users with an in-

tuitive physical interpretation of the system. In addition,

we use a layout algorithm that permits the system to be

rendered consistently each time the visualization is per-

formed. That is, the layout is designed to be insensitive to

small structural changes, thereby helping users remember

and navigate in the visualization. Our specific implemen-

tation supports C and C++ programs, extending our earlier

work for Java programs [20, 34].

Our overall approach allows multiple stakeholders to see

and communicate about the same view, which facilitates the

collective understanding and discussion of the system’s ar-

chitecture (Section 5).

2. Visualizing Architectures

There are roughly three levels of program visualization,

based on the level of abstraction [15, 26]: source code level,middle level, and architecture level. This section explains

what we mean by architecture level, and describes our spe-

cific model of the program and analyses in detail.

On the source code level, typical visualization tools in-

clude program and aspect editors. Advanced tools may in-

tegrate program editors with online debuggers and profilers.

Source-level visualizations are very “low-level,” as they re-

late directly to the underlying software artifact, and are pri-

marily of interest to developers and maintainers.

On the middle level, visualizations are problem-specific.

Developers and code maintainers have specific problems to

solve, and they usually apply tailored algorithms and vi-

sualizations to the program to better understand both the

problem and the program. Typical middle level visualiza-

tions include sequence diagrams, abstract syntax tree (AST)

representations, dominance tree visualizations, concept lat-

tices, control and data flow graphs.

The aim of architecture-level visualization is to rapidly

summarize and communicate the architecture and design

decisions of the overall software system. Architectural visu-

alization is naturally more abstract than source- or middle-

level visualizations, and therefore better suited to visualiza-

tions in the large. Abstract visualizations of software ar-

chitectures combined with metrics can help different stake-

holders to answer many questions about a software system.

Authorized licensed use limited to: Universia della Svizzera Italiana. Downloaded on January 30, 2009 at 12:57 from IEEE Xplore. Restrictions apply.

Software world

The second, FileVis, showed a view of the software system with the code files represented individually as floating platforms around a central point which represented the connectivity of the source code files. A view of these visualisations can be seen in Figure 2. This visualisation combines both in order to show two aspects of the C code at the same time.

Fig. 3. View over a software district showing

many, possibly complex, methods

Further advancing the three-dimensional space aspect of the visualisations, work by Knight and Munro [1, 20, 21] moved to considering the use of virtual reality environments for software visualisation. Software World was created to show that three-dimensions (in this case also showing the viability of real world metaphors) could be used to create automatable and scalable software visualisations. Buildings, cities, and also at the highest level atlas views, were used to represent Java source code. An example view of this visualisation can be seen in Figure 3.

In order to show some of the method level views that

can be created with this visualisation over 17,000 lines of Java code (which together composed a package) written by others was parsed, and district visualisations automatically generated. The code was split across 70 classes, which provided different district views.

The use of environments allowed extensions of this

work to consider the benefits and challenges of creating virtually inhabitable spaces where the visualisation provides a view of the data under consideration and facilitates communication and collaboration within that data by those working on the same tasks or having similar knowledge. Some of these further issues can be found in [25].

2.4 Summary In his paper No Silver Bullet [13], Brooks wrote

``Software is invisible and unvisualizable. … …software is very difficult to visualize. Whether one diagrams control flow, variable-scope nesting, variable cross-references, dataflow, hierarchical data structures, or whatever, one feels only one dimension of the intricately interlocked software elephant. If one superimposes all the diagrams generated by the many relevant views, it is difficult to extract any global overview.” At the time Brooks wrote this, visualising software

meant displaying some information about (or some aspect of) the software in a graph structure. From what can be seen in Figure 1, he has a point. This need not now be the case with the advances in computer hardware and graphics technology.

For an overview of the software visualisation field

Software Visualisation; Programming as a multimedia experience [26] shows how much of the prior focus has been on the use of algorithm animation techniques. The need to visualise the complexities of dynamic execution are covered, as well as some of the cognitive aspects of visualisation of software. There is very little reference made to the future work in the field, and the benefits that visualisation can bring if it is applied to software through the consideration of software as a complex artefact that can be decomposed into other complex artefacts. Visualising program code, and the various static and dyanamic analysis that program comprehension often employs have not yet found any universal solution and therefore these still require research, but there is also a need to consider software as information and therefore the sorts of visualisations of subsets of that information that would be appropriate for which users for which situations.

Two-dimensional techniques have shortcomings, but this is not to trivialise the issues that still remain with the nodes and arcs techniques. Layout, for example, is a hard problem and one that is not appreciated by many. The information visualisation field has shown that there is no longer the need for a reliance on nodes and arcs and solely investigating layout algorithms and clustering is not necessarily the only way forward, even with two-dimensions. This work is a stepping stone to moving onto more mature software visualisations.

Balzer et al. / Software Landscapes

Figure 4: Closeup view of the software system ’SystemX’containing 52 packages, 546 classes, 4856 methods and2588 attributes

work, it was first attempted to solve the problem in thetwo-dimensional space. Based on diverse criteria the rela-tions were summarized, occlusions were avoided and over-lappings were reduced. The results for a few objects and re-lations were satisfactory, but a large number of participantentities and relations resulted in unsatisfactory representa-tions. The conclusion was that for a clear representation ofthe relations, the third dimension has to be used.

We propose a solution called Hierarchical Net. Therebythe relations are represented not as direct connections be-tween the involved objects, but routed according to the hi-erarchy levels of the software entities. For example, if a re-lation exists between a class X in package A and a class Yin package B, and furthermore, the packages A and B arecontained in package C, then the connection is routed fromclass X to package A, to package C, then to package B, andlastly to class Y . For this purpose a point is defined aboveevery object, where the connections of the objects of thelower hierarchy levels are combined and forwarded to thenext level. This point always rests within a fixed relative dis-tance above the center of the considered object. Since theobjects at higher hierarchy levels are larger, this results in athree-dimensional tree of connections, as shown in Figure 5.

The type and direction of the relations is shown by thecolor and brightness gradient of the corresponding lines.Connections of the same type and direction, and with thesame start and end points, are combined to one connection.Thereby their quantity is mapped to the width of the newline, so that thicker lines stand for a larger quantity of rep-resented connections. Because only lines of the same typeand direction are combined, occlusion still appears, makingthinner lines sometimes hard to recognize. In order to avoidthis, lines are sorted by width and rendered in descendingsequence. Thus it is possible to differentiate between the dif-ferent line types and line directions despite mutual covering.

Figure 5: Visualization of 2574 relations between 1269 en-tities within the software system ’CrocoCosmos’ using a Hi-erarchical Net

In order to analyze the relations the user can control thevisualization. The first possibility is to select only specifiedtypes of relations, e.g. the user can solely view all inheri-tances. The second is to choose an entity, whereby a list withall connected relations is presented, and additionally onlyrelations connected to this entity are drawn in the visualiza-tion. These two alternatives enable the better traceability ofthe relations.

6. Conclusions and Future Work

This work only represents a first step for the visualizationof the structure of large software systems with the land-scape metaphor. It introduced a layout technique for con-tainment hierarchies, dynamic transparencies to reduce thevisual complexity, and Hierarchical Nets to clearly representthe relations in large software systems.

In future works, more possibilities of the landscapemetaphor in the context of software visualization are to beexamined. One direction will be the examination of layoutmethods which are not only based on the hierarchy of thesoftware system, but also involve the relations in the layoutgeneration process. The information density in the visualiza-tions can be further improved by mapping the values of soft-ware metrics on landscape objects. For example, the heightof the objects that represent methods could be proportionalto the size of the methods, as measured by the number oflines of code.

References

[Cha93] CHALMERS M.: Using a landscape metaphorto represent a corpus of documents. In Pro-ceedings of the International Conference onSpatial Information Theory (COSIT) (1993),LNCS 716, Springer-Verlag, pp. 377–390.

c� The Eurographics Association 2004.

Figure 5: (Top) Modified Treemap technique and (bottom) modified Sunburst technique as displayed in ourframework. They both represent PCGEN, a tool for character generation in RPG (1129 classes).

Poly cylinders

[Marcus et al., IWCP 2003]

Software landscapes

[Balzer et al., VisSym 2004]

3D boxes

[Langelier et al., ASE 2005]

City metaphor

[Panas et al., ICECCS 2007]

UML city

[Lange & Chaudron, ICPC 2007]

Figure 3. MetricView: Combining UML andMetrics Visualization

Figure 4. UML-City: Combining the MetaViewand MetricView

quality and of the maturity or completeness of a model.

UML-City View

Figure 4 shows an example UML-City. This view combinesthe concepts of the MetaView and MetricView. As metricvisualization the ‘3D-heightbar’ is used, this visualizationshows a box on top of the model element where the heightand the color of the box indicate the value of the metric.Low metric values are depicted by flat green boxes whilehigh values are depicted by tall red boxes. Applying this tothe MetaView results in a view for which city is a metaphor.Note that in the example in Figure 4 both color and hight areused do represent the same metric. The dimensions can beused to visualize two metrics simultaneously.

Search and Highlight Functionality

The aforementioned views are complemented with searchand highlight functionality. The results of search actionsare visualized by highlighting the relevant diagrams and di-agram elements with a distinct color (e.g. yellow). Addi-tionally relations between elements in the set of result andother elements in the model are drawn.

3. Experimental Design

3.1. Purpose and Hypotheses

We have proposed new views for analyzing UML modelsas described in Section 2. The purpose of this experiment isto validate the views. We summarize the purpose of the ex-periment according to the Goal-Question-Metric paradigm(GQM) [1] as follows:Analyze Comprehension of UML viewsfor the purpose of evaluationwith respect to correctness and effortfrom the point of view of the researcherin the context of Master’s students at the TU Eindhoven.

More specifically, we want to compare the usefulness ofthe views with the traditional views, which are used as abaseline in this experiment. Comprehension plays an im-portant role in the development and maintenance of soft-ware systems. In particular two concepts are of interest forthe evaluation of understanding techniques: correctness andeffort. Correctness is essential for understanding. Incorrectunderstanding of a system can lead to wrong actions intro-ducing faults and communication overhead. Effort is rele-vant from an economical point of view.

To evaluate the view we are interested whether they dif-fer from the traditional UML views. This leads us to thefollowing hypotheses:

• H10: There is no significant difference between pro-posed and traditional views in terms of correctness ofunderstanding the model.

• H20: There is no significant difference between pro-posed and traditional views in terms of effort neededfor understanding the model.

The alternative hypotheses are the negations of the null-hypotheses.

In addition to testing the described hypotheses, we areinterested in information describing how the subjects expe-rience the use of the different views. We expect this addi-tional information to be useful to explain the results, to findopportunities for improvements and as an indicator for thelikelihood of engineers adopting the technique in practice.

3.2. Task, Objects and Treatment

The task in the reported experiment was to answer acomprehension questionnaire about a UML model. In orderto answer the questionnaire, the subjects had to analyze agiven UML model using either traditional tools or the Met-ricView Evolution tool. The experiment was carried out intwo runs. In each of the two experimental runs, a differentUML model had to be analyzed.

[Knight & Munro, IV 2000]

Drawbacks of existing approaches


1 limited coverage of the manyfacets of software


2

1 limited coverage of the manyfacets of software

poor tool support


2

1

3

limited coverage of the manyfacets of software

lack of empirical evaluation

poor tool support

Thesis

Depicting software systems as cities is a versatile metaphor which enables the creation of efficient software visualizations to support reverse engineering.

Prologue

Approac

h

Evalua

tion

Epilogue

Software Systems as Cities[VISSOFT 2007]

Our city metaphor

Our city metaphor

class building

package district

Our city metaphor

class building

package district

number of methods (NOM) heightheight

number of attributes (NOA) base sizebase size

number of lines of code (LOC) color

Our city metaphor

class building

package district

number of methods (NOM) heightheight

number of attributes (NOA) base sizebase size

number of lines of code (LOC) color

nesting level color

Code city visualization

ArgoUMLArgoUMLlines 136,325

Code city visualizationskyscraper



office building



parking lot

office building



parking lot

office building

house


Layout

Layout

org

org.argouml

org.argouml.uml

org.argouml.uml.reveng

org.argouml.uml.reveng.java

Fine-grained representation


class

methods


class

methods

packages

Vertical layouts

Vertical layouts

JmolJmollines 84,984

Vertical layouts

JmolJmollines 84,984

class ViewerNOM 750

Reflections

structural overview

Reflections

reveals outliers

structural overview

Reflections

reveals outliers

two granularities

structural overview

Reflections

Application contexts

programcomprehension



softwareevolution analysis



softwareevolution analysis

software design quality assessment


Program Comprehension[ICPC 2007]

Finding hotspotsin ArgoUML

Antenna-like skyscrapers

Antenna-like skyscrapersFacadeFacadeFacade

modelmodelmodelLOC: 0NOM: 337NOA: 1

FacadeMDRImplFacadeMDRImplFacadeMDRImplmodelmodelmodelLOC: 3,413NOM: 349NOA: 3

An interface with asingle implementation?

FacadeFacadeFacademodelmodelmodel

LOC 0NOM 337NOA 1

FacadeMDRImplFacadeMDRImplFacadeMDRImplmodelmodelmodelLOC 3,413NOM 349NOA 3

Relying on Facade...FacadeFacadeFacade

modelmodelmodelLOC 0NOM 337NOA 1

Relying on Facade...FacadeFacadeFacade

modelmodelmodelLOC 0NOM 337NOA 1

1classes

3classes

Office buildings& parking lots


JavaRecognizerJavaRecognizerJavaRecognizerreveng.javareveng.javareveng.javaLOC: 5,654NOM: 176NOA: 79

CPPParserCPPParserCPPParserreveng.classfilereveng.classfilereveng.classfileLOC: 9,111NOM: 204NOA: 85




STDCTokenTypesSTDCTokenTypesSTDCTokenTypesreveng.classfilereveng.classfilereveng.classfileLOC: 0NOM: 0NOA: 152

JavaTokenTypesJavaTokenTypesJavaTokenTypesreveng.javareveng.javareveng.java

LOC: 0NOM: 0NOA: 173




JavaTokenTypesJavaTokenTypesJavaTokenTypeslanguage.java.generatorlanguage.java.generatorlanguage.java.generator


STDCTokenTypesSTDCTokenTypesSTDCTokenTypesreveng.classfilereveng.classfilereveng.classfileLOC: 0NOM: 0NOA: 152



JavaRecognizerJavaRecognizerJavaRecognizerlanguage.java.generatorlanguage.java.generatorlanguage.java.generator

LOC: 3,406NOM: 91NOA: 24

System overviews

System Language NOP NOC kLOC

ScummVM C++ 141 3,117 305


ScummVM C++ 141 3,117 305

CodeCity Smalltalk 34 173 29


ScummVM C++ 141 3,117 305


iTextSharp C# 22 485 58


ScummVM C++ 141 3,117 305



iText Java 36 566 59


ScummVM C++ 141 3,117 305




Jmol Java 50 558 85


ScummVM C++ 141 3,117 305




Jmol Java 50 558 85

jEdit Java 59 966 98


ScummVM C++ 141 3,117 305




Jmol Java 50 558 85


ArgoUML Java 88 1,817 144


ScummVM C++ 141 3,117 305




Jmol Java 50 558 85



GWT Java 302 4,372 212


ScummVM C++ 141 3,117 305




Jmol Java 50 558 85



GWT Java 302 4,372 212

JBoss Java 1,507 7,881 435


ScummVM C++ 141 3,117 305




Jmol Java 50 558 85



GWT Java 302 4,372 212

JBoss Java 1,507 7,881 435

JDK 1.5 Java 664 12,888 1,085


ScummVM C++ 141 3,117 305




Jmol Java 50 558 85



GWT Java 302 4,372 212

JBoss Java 1,507 7,881 435

JDK 1.5 Java 664 12,888 1,085

Eclipse Java 1,800 27,900 2,871

CodeCityCodeCity

SmalltalkSmalltalk

29 kLOC29 kLOC

ArgoUMLArgoUML

JavaJava

144 kLOC144 kLOC

JBossJBoss

JavaJava

435 kLOC435 kLOC

JDKJDK

JavaJava

1,085 kLOC1,085 kLOC

EclipseEclipse

JavaJava

2,871 kLOC2,871 kLOC

GWTGWT

JavaJava

212 kLOC212 kLOC

JmolJmol

JavaJava

85 kLOC85 kLOC

jEditjEdit

JavaJava

98 kLOC98 kLOC

iTextiText

C#C#

58 kLOC58 kLOC

iTextiText

JavaJava

59 kLOC59 kLOC

ScummVMScummVM

C++C++

305 kLOC305 kLOC

Reflections

first impression

Reflections

interesting insights

first impression

Reflections

interesting insights

limited to one version

first impression

Reflections

Software Evolution Analysis[WCRE 2008]

Age mapTime travel

Timeline

Time travelTimeline

Age map

Coarse-grained age map

1 2 3 4 5 6 7 8


1 2 3 4 5 6 7 8


AGE: 8NOM: 0NOA: 146




1 2 3 4 5 6 7 8










1 2 3 4 5 6 7 8

CPPParserCPPParserCPPParserreveng.classfilereveng.classfilereveng.classfileAGE: 3NOM: 200NOA: 85









STDCTokenTypesSTDCTokenTypesSTDCTokenTypesreveng.classfilereveng.classfilereveng.classfileAGE: 3NOM: 0NOA: 152

Fine-grained age map

1 2 3 4 5 6 7 8

Age mapTime travel

Timeline

Coarse-grained time travel

2002 2003 2004 2005 2006 2007

0.10.1 0.12 0.14 0.16 0.18.1 0.20 0.22 0.24

ArgoUMLArgoUMLyears 6major releases 8

2002 2003 2004 2005 2006 2007

0.10.1 0.12 0.14 0.16 0.18.1 0.20 0.22 0.24

2002 2003 2004 2005 2006 2007

0.10.1 0.12 0.14 0.16 0.18.1 0.20 0.22 0.24

ModelFacadeModelFacadeModelFacademodelmodelmodel

LOC: 1,280NOM: 183NOA: 60

2002 2003 2004 2005 2006 2007

0.10.1 0.12 0.14 0.16 0.18.1 0.20 0.22 0.24

ModelFacadeModelFacadeModelFacademodelmodelmodel

LOC: 3,275NOM: 426NOA: 108

2002 2003 2004 2005 2006 2007

0.10.1 0.12 0.14 0.16 0.18.1 0.20 0.22 0.24

NSUMLModelFacadeNSUMLModelFacadeNSUMLModelFacademodelmodelmodel

LOC: 2,102NOM: 316NOA: 2

FacadeFacadeFacademodelmodelmodelLOC: 0NOM: 306NOA: 1

2002 2003 2004 2005 2006 2007

0.10.1 0.12 0.14 0.16 0.18.1 0.20 0.22 0.24

NSUMLModelFacadeNSUMLModelFacadeNSUMLModelFacade

FacadeFacadeFacadeFacadeMDRImplFacadeMDRImplFacadeMDRImplmodelmodelmodel

LOC: 2,709NOM: 329NOA: 2

2002 2003 2004 2005 2006 2007

0.10.1 0.12 0.14 0.16 0.18.1 0.20 0.22 0.24

FacadeFacadeFacadeFacadeMDRImplFacadeMDRImplFacadeMDRImpl

Fine-grained time travel with age map

JHotDrawJHotDrawyears 5revisions 8

2000 2001 2002 2003 2004

2000 2001 2002 2003 2004

1

2000 2001 2002 2003 2004

2

1

2000 2001 2002 2003 2004

3

2

1

2000 2001 2002 2003 2004

4

3

2

1

2000 2001 2002 2003 2004

5

4

3

2

1

2000 2001 2002 2003 2004

6

5

4

3

2

1

2000 2001 2002 2003 2004

7

6

5

4

3

2

1

2000 2001 2002 2003 2004

8

7

6

5

4

3

2

1

Age mapTime travel

Timeline

Evolution overview

1887 Sep. 1887 Dec. 1888 Mar. 1888 Jun. 1888 Sep. 1888 Dec. 1889 Mar.

Timeline of a class

time

Timeline of a class

v1

timem1 m2m3

m5m6

m7

Timeline of a class

v1

timev2

m9 m10

m11 m12

m1 m2m3

m5m6

m7

Timeline of a class

m1 m2m3

m5m6

m7

m9 m10

m11 m12m13 m14

m15

m17 m18

v1

timev2

v3

Class timelineclass StandardDrawingViewsystem JHotDrawNOMH 105

Package timelinepackage StandardDrawingViewsystem JHotDraw

Patterns

Patterns

Class NOMHJmolViewer 177TransformerManager 220Eval 432Viewer 1,029

Reality check

“Your hypothesis is probably correct. We found some major problems, and diagnostic was too difficult, so we reverted to a stable version and tried to apply patches in small batch.”

Jmol Developer

Reflections

age map, time travel, and timeline

Reflections

exclusive insights, complementary to the previous context


Reflections

exclusive insights, complementary to the previous context

evolutionary patterns


Reflections

Design Quality Assessment[SoftVis 2008]

Detecting design flaws

[Marinescu, ICSM 2004]

[Lanza & Marinescu, Object-Oriented Metrics in Practice, 2006]

Textual data presentation

Moose [Nierstrasz et al., ESEC/FSE 2005]

Disharmony maps show design problems in context

JDK*JDK* 1,9661,966

God ClassGod Class 81

Design flaws visualization

God Class

Brain Class

Data Class

Feature Envy

Shotgun Surgery

Class-level design flaws

God Class

Brain Class

Data Class

Feature Envy

Shotgun Surgery

JDK*JDK* 1,9661,966

Brain ClassBrain Class 12


God + Brain ClassGod + Brain Class 16

Data ClassData Class 109

JDK*JDK* 1,9661,966





ComponentComponentComponentjava.awtjava.awtjava.awt

NOM 280NOA 88

CalendarCalendarCalendarjava.utilsjava.utilsjava.utilsNOM 71NOA 81

JDK*JDK* 1,9661,966






NOM 280NOA 88


SecuritySecuritySecurityjava.securityjava.securityjava.securityNOM 30NOA 3

JDK*JDK* 1,9661,966






NOM 280NOA 88


InputEventInputEventInputEventjava.awt.eventjava.awt.eventjava.awt.event

NOM 14NOA 21

KeyEventKeyEventKeyEventjava.awt.eventjava.awt.eventjava.awt.eventNOM 18NOA 205

SecuritySecuritySecurityjava.securityjava.securityjava.securityNOM 30NOA 3

iTextiText 711711





iTextiText 711711





PDFWriterPDFWriterPDFWritertext.pdftext.pdftext.pdf

NOM 11NOA 39

iTextiText 711711





PDFWriterPDFWriterPDFWritertext.pdftext.pdftext.pdf

NOM 11NOA 39

SpecialSymbolSpecialSymbolSpecialSymboltexttexttextNOM 3NOA 0

JmolJmol 558558





JmolJmol 558558





ViewerViewerViewerorg.jmol.viewerorg.jmol.viewerorg.jmol.viewer

NOM 750NOA 57

Graphics3DGraphics3DGraphics3Dorg.jmol.g3dorg.jmol.g3dorg.jmol.g3dNOM 158NOA 95

ArgoUMLArgoUML 1,7761,776










FacadeMDRImplFacadeMDRImplFacadeMDRImpluml.model.mdruml.model.mdruml.model.mdrNOM 349NOA 3







CPPParserCPPParserCPPParserreveng.classfilereveng.classfilereveng.classfile

NOM 204NOA 85








NOM 204NOA 85

JavaRecognizerJavaRecognizerJavaRecognizerreveng.javareveng.javareveng.javaNOM 176NOA 79








NOM 204NOA 85


FigNodeModelElementFigNodeModelElementFigNodeModelElementuml.diagram.uiuml.diagram.uiuml.diagram.uiNOM 98NOA 39







FigEdgeModelElementFigEdgeModelElementFigEdgeModelElementuml.diagram.uiuml.diagram.uiuml.diagram.ui

NOM 73NOA 13


NOM 204NOA 85


FigNodeModelElementFigNodeModelElementFigNodeModelElementuml.diagram.uiuml.diagram.uiuml.diagram.uiNOM 98NOA 39

Method-level design flaws

God Class

Brain Class

Data Class

Feature Envy

Shotgun Surgery

ArgoUML*ArgoUML*ArgoUML*

Shotgun surgery 210

JmolJmol 5,9685,968

Feature envyFeature envy 1,555

Reflections

provides a design overview

Reflections

places design problems in context


Reflections

places design problems in context

complements the metric-based analysis


Reflections

Tool Support[ICSE 2008 tool demo, FAMOOSR 2008, WASDeTT 2008]

CodeCity

On the shoulders of others


FAMIX[Demeyer et al., Technical Report, University of Bern, 2001]

Moose[Nierstrasz et al., ESEC/FSE 2005]

Hismo[Gîrba, Ph.D. Thesis, University of Bern, 2005]




iPlasma[Marinescu et al., ICSM 2005]

PMCS[Daka, Master Thesis, University of Lugano, 2009]





iPlasma[Marinescu et al., ICSM 2005]

PMCS[Daka, Master Thesis, University of Lugano, 2009]


CodeCity supports Java, C++, C#, Smalltalk

http://codecity.inf.usi.ch



downloads30 months


5,000+



Social impact

Social impact

testimonials“[...] I loaded my music composition application into CodeCity. See the attached picture. Interestingly, the resulting image looks like expected. To a large extent, it matches the image I always had in my head.”

Social impact

testimonials

community recognition

“[...] I loaded my music composition application into CodeCity. See the attached picture. Interestingly, the resulting image looks like expected. To a large extent, it matches the image I always had in my head.”

ated-ICT Ticino recognition prize for 2008

Reflections

advantage through the use of existing technology

Reflections

access to valuable user feedback


Reflections

access to valuable user feedback

necessary for empirical evaluation


Reflections

Prologue

Approac

h

Evalua

tion

Epilogue

Empirical Evaluation[Technical Report USI, 2010]

State-of-the-art s

urvey

State-of-the-art s

urvey

State-of-the-art s

urvey

50+publications

State-of-the-art s

urvey

1 Avoid comparing using a technique against not using it.

2 Involve participants from the industry.

3 Provide a not-so-short tutorial of the experimental tool to the participants.

4 Avoid, whenever possible, giving the tutorial right before the experiment.

5 Use the tutorial to cover both the research behind the approach and the tool.

6 Find a set of relevant tasks.

7 Choose real object systems that are relevant for the tasks.

8 Include more than one object system in the design.

9 Provide the same data to all participants.

10 Limit the amount of time allowed for solving each task.

11 Provide all the details needed to make the experiment replicable.

12 Report results on individual tasks.

13 Include tasks on which the expected result is not always to the advantage of the tool being evaluated.

14 Take into account the possible wide range of experience level of the participants.

Design desiderata

Main research questions


Does the use of CodeCity increase the correctness of the solutions to program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size?1


Does the use of CodeCity increase the correctness of the solutions to program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size?

Does the use of CodeCity reduce the time needed to solve program comprehension tasks, compared to non-visual exploration tools, regardless of the object system size?2

1

Variables of the experiment


dependentcorrectnesscorrectnesscorrectnesscorrectnesscorrectness

dependentcompletion timecompletion timecompletion timecompletion timecompletion time

independenttool CodeCityCodeCityCodeCityCodeCity

independenttool Eclipse + ExcelEclipse + ExcelEclipse + ExcelEclipse + Excelindependentobject system sizeobject system sizeobject system sizeobject system size mediumindependentobject system sizeobject system sizeobject system sizeobject system size large

controlledexperience levelexperience levelexperience level beginnerbeginner

controlledexperience levelexperience levelexperience level advancedadvancedcontrolledbackgroundbackground academiaacademiaacademiacontrolledbackgroundbackground industryindustryindustry








FindBugs

1,320 classes

93,310 LOC

Azureus

4,656 classes

454,387 LOC

TasksA1 Identity the convention used in the system to organize unit tests.

A2.1&A2.2

What is the spread of term T in the name of the classes, their attributes and methods?

A3 Evaluate the change impact of class C, in terms of intensity and dispersion.

A4.1 Find the three classes with the highest number of methods.

A4.2Find the three classes with the highest average number of lines of code per method.

B1.1 Identify the package with the highest percentage of god classes.

B1.2 Identify the god class with the largest number of methods.

B2.1Identify the dominant (affecting the highest number of classes) class-level design problem.

B2.2 Write an overview of the class-level design problems in the system.

TasksA1 Identity the convention used in the system to organize unit tests.

A2.1&A2.2









program comprehension

design quality assessment 4

6

Tasksquantitative

qualitative 1

A1 Identity the convention used in the system to organize unit tests.

A2.1&A2.2









9

Operation timeline

1 1 1 1 13

2 1 6

5

4 1

1 1 1 1 3

1 1 1 1

6

6

20092009 2010201020102010

November December January February ... April

18 24 25 2 9 21 28 5 8 14 28 18 22 24 25 14

R

R R

RBern

Bologna

Antwerpen

Lugano

nm

R

Training session

followed by an experimental session with

n subjects using CodeCity (experimental)

m subjects using Eclipse+Excel (control)

remote, self-performed session

Pilot Experiment

Subjects

Subjects

21 20 industryacademia

Background

Subjects


developersoftware engineersystem analystsoftware architectproject managerIT leadCTOconsultantprofessorPh.D. studentMaster student

Background

Subjects



Background

Block

industry-advanced

academy-advanced

academy-beginner

Age

5 5

50

45

40

35

30

25

20

EXAMINE VARIABLES=Age BY Syssize BY Tool /PLOT=BOXPLOT /STATISTICS=NONE /NOTOTAL.

Explore

Page 22

Subjects

7

1218

ItalyBelgiumSwitzerandArgentinaCanadaGermanyRomania



Country of origin

Background

Block

industry-advanced

academy-advanced

academy-beginner

Age

5 5

50

45

40

35

30

25

20

EXAMINE VARIABLES=Age BY Syssize BY Tool /PLOT=BOXPLOT /STATISTICS=NONE /NOTOTAL.

Explore

Page 22

Replicability

Replicability

Pre-experimentquestionnaire

Replicability

Introduction

The aim of this experiment is to compare tool efficiency in supporting software

practitioners analyzing medium to large-scale software systems.You will use CodeCity to analyze Azureus, a BitTorrent client written in Java.You are given maximum 100 minutes for solving 10 tasks (10 minutes per task). You are asked:• not to consult any other participant during the experiment;• to perform the tasks in the specified order;• to write down the current time each time before starting to read a task and once

after completing all the tasks;• to announce the experimenter that you are moving on to another task, in order

to reset your 10-minutes-per-task allocated timer;• not to return to earlier tasks because it affects the timing;• for each task, to fill in the required information. In the case of multiple choices

check the most appropriate answer and provide additional information, if

requested.

The experiment is concluded with a short debriefing questionnaire.

Thank you for participating in this experiment!! ! ! ! Richard Wettel, Michele Lanza, Romain Robbes

CodeCity Experiment

Participant:

T1

Introduction

The aim of this experiment is to compare tool efficiency in supporting software practitioners analyzing medium to large-scale software systems.You will use CodeCity to analyze Azureus, a BitTorrent client written in Java.You are given maximum 100 minutes for solving 10 tasks (10 minutes per task). You are asked:

• not to consult any other participant during the experiment;• to perform the tasks in the specified order;• to write down the current time each time before starting to read a task and once after completing all the tasks;• to announce the experimenter that you are moving on to another task, in order to reset your 10-minutes-per-task allocated timer;• not to return to earlier tasks because it affects the timing;• for each task, to fill in the required information. In the case of multiple choices check the most appropriate answer and provide additional information, if requested.



CodeCity Experiment

Participant:

T1

Introduction

The aim of this experiment is to compare tool efficiency in supporting software practitioners analyzing medium to large-scale software systems.

You will use CodeCity to analyze Azureus, a BitTorrent client written in Java.

You are given maximum 100 minutes for solving 10 tasks (10 minutes per task).

You are asked:• not to consult any other participant during the experiment;• to perform the tasks in the specified order;• to write down the current time each time before starting to read a task and once



check the most appropriate answer and provide additional information, if requested.


Thank you for participating in this experiment!

! ! ! ! Richard Wettel, Michele Lanza, Romain Robbes

CodeCity Experiment

Participant:

T1

Introduction











CodeCity Experiment

Participant:

T1


Experimenthandouts

Replicability

174

A.4 Task Solution OraclesA.4.1 T1: Azureus, analyzed with CodeCityA1

EitherThere are no unit tests in the system [1pt],orCentralized in a single package hierarchy whose root is in org.gudy.azureus2.ui.console.multiuser

[1pt]. Since there is only one test class (i.e., TestUserManager), if they don’t give the full correct

answer, the answer is completely wrong.A2.1

Dispersed [0pts otherwise]in the following (max. 5) packages [0.2pts for each]:• com.aelitis.azureus.core• com.aelitis.azureus.core.content• com.aelitis.azureus.core.download• com.aelitis.azureus.core.impl• com.aelitis.azureus.core.lws• com.aelitis.azureus.core.peermanager.peerdb• com.aelitis.azureus.core.stats

• com.aelitis.azureus.core.torrent• com.aelitis.azureus.plugins.net.buddy• com.aelitis.azureus.plugins.net.buddy.swt• com.aelitis.azureus.plugins.net.buddy.tracker• com.aelitis.azureus.plugins.removerules• com.aelitis.azureus.plugins.sharing.hoster• com.aelitis.azureus.plugins.startstoprules.defaultplugin• com.aelitis.azureus.plugins.tracker.dht

• com.aelitis.azureus.plugins.tracker.local• com.aelitis.azureus.plugins.tracker.peerauth• com.aelitis.azureus.ui.swt.content.columns• com.aelitis.azureus.ui.swt.shells.main• com.aelitis.azureus.util

174



[1pt]. Since there is only one test class (i.e., TestUserManager), if they don’t give the full correctanswer, the answer is completely wrong.

A2.1

Dispersed [0pts otherwise]in the following (max. 5) packages [0.2pts for each]:• com.aelitis.azureus.core• com.aelitis.azureus.core.content• com.aelitis.azureus.core.download• com.aelitis.azureus.core.impl• com.aelitis.azureus.core.lws• com.aelitis.azureus.core.peermanager.peerdb• com.aelitis.azureus.core.stats• com.aelitis.azureus.core.torrent• com.aelitis.azureus.plugins.net.buddy• com.aelitis.azureus.plugins.net.buddy.swt• com.aelitis.azureus.plugins.net.buddy.tracker• com.aelitis.azureus.plugins.removerules• com.aelitis.azureus.plugins.sharing.hoster• com.aelitis.azureus.plugins.startstoprules.defaultplugin• com.aelitis.azureus.plugins.tracker.dht


174 A.4 Task Solution Oracles

A.4.1 T1: Azureus, analyzed with CodeCityA1

EitherThere are no unit tests in the system [1pt],orCentralized in a single package hierarchy whose root is in org.gudy.azureus2.ui.console.multiuser[1pt]. Since there is only one test class (i.e., TestUserManager), if they don’t give the full correctanswer, the answer is completely wrong.

A2.1

Dispersed [0pts otherwise]in the following (max. 5) packages [0.2pts for each]:

• com.aelitis.azureus.core

• com.aelitis.azureus.core.content

• com.aelitis.azureus.core.download

• com.aelitis.azureus.core.impl

• com.aelitis.azureus.core.lws

• com.aelitis.azureus.core.peermanager.peerdb

• com.aelitis.azureus.core.stats

• com.aelitis.azureus.core.torrent

• com.aelitis.azureus.plugins.net.buddy

• com.aelitis.azureus.plugins.net.buddy.swt

• com.aelitis.azureus.plugins.net.buddy.tracker

• com.aelitis.azureus.plugins.removerules

• com.aelitis.azureus.plugins.sharing.hoster

• com.aelitis.azureus.plugins.startstoprules.defaultplugin

• com.aelitis.azureus.plugins.tracker.dht

• com.aelitis.azureus.plugins.tracker.local

• com.aelitis.azureus.plugins.tracker.peerauth

• com.aelitis.azureus.ui.swt.content.columns

• com.aelitis.azureus.ui.swt.shells.main

• com.aelitis.azureus.util

Introduction






requested.



CodeCity Experiment

Participant:

T1

Introduction





CodeCity Experiment

Participant:

T1

Introduction











CodeCity Experiment

Participant:

T1

Introduction











CodeCity Experiment

Participant:

T1


Experimenthandouts


A.4.1 T1: Azureus, analyzed with CodeCity

A1


A2.1






















Task solutions &grading schemes

Replicability193

A.5 Data

Code

Treatment

Blocking Criteria

Number ToolSystem size Background Experience

IA011 CodeCity

largeindustry

advanced

IA021 CodeCity

largeindustry

advanced

IA031 CodeCity

largeindustry

advanced

IA041 CodeCity

largeindustry

advanced

AB011 CodeCity

largeacademy

beginner

AB021 CodeCity

largeacademy

beginner

IA051 CodeCity

largeindustry

advanced

AA011 CodeCity

largeacademy

advanced

AA021 CodeCity

largeacademy

advanced

IA061 CodeCity

largeindustry

advanced

IA072 CodeCity

mediumindustry

advanced

IA082 CodeCity

mediumindustry

advanced

IA092 CodeCity

mediumindustry

advanced

AB032 CodeCity

mediumacademy

beginner

IA102 CodeCity

mediumindustry

advanced

IA112 CodeCity

mediumindustry

advanced

IA122 CodeCity

mediumindustry

advanced

AB042 CodeCity

mediumacademy

beginner

AA032 CodeCity

mediumacademy

advanced

AB052 CodeCity

mediumacademy

beginner

AA042 CodeCity

mediumacademy

advanced

IA132 CodeCity

mediumindustry

advanced

IA143 Ecl+Excl large

industryadvanced

AB063 Ecl+Excl large

academybeginner

AB073 Ecl+Excl large

academybeginner

AA053 Ecl+Excl large

academyadvanced


academyadvanced


academyadvanced


industryadvanced


industryadvanced

IA014 Ecl+Excl medium

industryadvanced


industryadvanced


industryadvanced

AB084 Ecl+Excl medium

academybeginner

AB094 Ecl+Excl medium

academybeginner

AA104 Ecl+Excl medium

academyadvanced


academyadvanced


academyadvanced


academyadvanced


academyadvanced


industryadvanced

Table A.2. The assignment of the subjects to treatments and blocks

174



[1pt]. Since there is only one test class (i.e., TestUserManager), if they don’t give the full correct

answer, the answer is completely wrong.A2.1

Dispersed [0pts otherwise]in the following (max. 5) packages [0.2pts for each]:• com.aelitis.azureus.core• com.aelitis.azureus.core.content• com.aelitis.azureus.core.download• com.aelitis.azureus.core.impl• com.aelitis.azureus.core.lws• com.aelitis.azureus.core.peermanager.peerdb• com.aelitis.azureus.core.stats

• com.aelitis.azureus.core.torrent• com.aelitis.azureus.plugins.net.buddy• com.aelitis.azureus.plugins.net.buddy.swt• com.aelitis.azureus.plugins.net.buddy.tracker• com.aelitis.azureus.plugins.removerules• com.aelitis.azureus.plugins.sharing.hoster• com.aelitis.azureus.plugins.startstoprules.defaultplugin• com.aelitis.azureus.plugins.tracker.dht


174



[1pt]. Since there is only one test class (i.e., TestUserManager), if they don’t give the full correctanswer, the answer is completely wrong.

A2.1

Dispersed [0pts otherwise]in the following (max. 5) packages [0.2pts for each]:• com.aelitis.azureus.core• com.aelitis.azureus.core.content• com.aelitis.azureus.core.download• com.aelitis.azureus.core.impl• com.aelitis.azureus.core.lws• com.aelitis.azureus.core.peermanager.peerdb• com.aelitis.azureus.core.stats• com.aelitis.azureus.core.torrent• com.aelitis.azureus.plugins.net.buddy• com.aelitis.azureus.plugins.net.buddy.swt• com.aelitis.azureus.plugins.net.buddy.tracker• com.aelitis.azureus.plugins.removerules• com.aelitis.azureus.plugins.sharing.hoster• com.aelitis.azureus.plugins.startstoprules.defaultplugin• com.aelitis.azureus.plugins.tracker.dht



A.4.1 T1: Azureus, analyzed with CodeCityA1


A2.1






















Introduction






requested.



CodeCity Experiment

Participant:

T1

Introduction





CodeCity Experiment

Participant:

T1

Introduction











CodeCity Experiment

Participant:

T1

Introduction











CodeCity Experiment

Participant:

T1


Experimenthandouts


A.4.1 T1: Azureus, analyzed with CodeCity

A1


A2.1






















Task solutions &grading schemes

Data set

196

A.5Data

Code

Diffi

culty

Leve

l Per

Task

Tim

ePr

essu

re

A1

A2.1

A2.2

A3

A4.1

A4.2

B1.1

B1.2

B2.1

B2.2

IA01

diffi

cult

inte

rmed

iate

inte

rmed

iate

simpl

e

simpl

e

impo

ssib

le

diffi

cult

trivi

al

trivi

al

inte

rmed

iate

fair

amou

nt

IA02

fair

amou

nt

IA03

diffi

cult

inte

rmed

iate

inte

rmed

iate

diffi

cult

inte

rmed

iate

diffi

cult

simpl

e

simpl

e

simpl

e

impo

ssib

le

fair

amou

nt

IA04

simpl

e

simpl

e

simpl

e

simpl

e

inte

rmed

iate

impo

ssib

le

diffi

cult

simpl

e

trivi

al

diffi

cult

too

muc

h

AB01

trivi

al

trivi

al

trivi

al

simpl

e

simpl

e

inte

rmed

iate

diffi

cult

simpl

e

simpl

e

inte

rmed

iate

very

little

AB02

trivi

al

trivi

al

trivi

al

trivi

al

simpl

e

diffi

cult

impo

ssib

le

simpl

e

trivi

al

simpl

e

fair

amou

nt

IA05

simpl

e

simpl

e

simpl

e

simpl

e

inte

rmed

iate

impo

ssib

le

diffi

cult

diffi

cult

trivi

al

simpl

e

not s

om

uch

AA01

inte

rmed

iate

trivi

al

simpl

e

inte

rmed

iate

trivi

al

impo

ssib

le

impo

ssib

le

trivi

al

simpl

e

trivi

al

very

little

AA02

trivi

al

trivi

al

trivi

al

simpl

e

simpl

e

impo

ssib

le

diffi

cult

simpl

e

trivi

al

inte

rmed

iate

not s

om

uch

IA06

inte

rmed

iate

simpl

e

simpl

e

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

simpl

e

simpl

e

diffi

cult

fair

amou

nt

IA07

diffi

cult

inte

rmed

iate

inte

rmed

iate

simpl

e

simpl

e

diffi

cult

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

not s

om

uch

IA08

trivi

al

trivi

al

simpl

e

inte

rmed

iate

inte

rmed

iate

impo

ssib

le

diffi

cult

inte

rmed

iate

simpl

e

inte

rmed

iate

very

little

IA09

diffi

cult

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

diffi

cult

inte

rmed

iate

simpl

e

trivi

al

diffi

cult

not s

om

uch

AB03

simpl

e

simpl

e

simpl

e

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

simpl

e

simpl

e

inte

rmed

iate

very

little

IA10

simpl

e

simpl

e

simpl

e

simpl

e

diffi

cult

impo

ssib

le

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

diffi

cult

not s

om

uch

IA11

simpl

e

simpl

e

simpl

e

diffi

cult

inte

rmed

iate

diffi

cult

diffi

cult

simpl

e

simpl

e

diffi

cult

fair

amou

nt

IA12

simpl

e

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

diffi

cult

trivi

al

simpl

e

simpl

e

diffi

cult

none

AB04

trivi

al

trivi

al

trivi

al

simpl

e

inte

rmed

iate

diffi

cult

trivi

al

trivi

al

simpl

e

diffi

cult

very

little

AA03

trivi

al

trivi

al

trivi

al

diffi

cult

trivi

al

impo

ssib

le

diffi

cult

trivi

al

trivi

al

simpl

e

not s

om

uch

AB05

simpl

e

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

impo

ssib

le

diffi

cult

inte

rmed

iate

trivi

al

simpl

e

not s

om

uch

AA04

inte

rmed

iate

inte

rmed

iate

diffi

cult

diffi

cult

inte

rmed

iate

impo

ssib

le

inte

rmed

iate

inte

rmed

iate

diffi

cult

impo

ssib

le

fair

amou

nt

IA13

trivi

al

simpl

e

simpl

e

simpl

e

inte

rmed

iate

diffi

cult

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

diffi

cult

none

IA14

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

fair

amou

nt

AB06

simpl

e

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

diffi

cult

impo

ssib

le

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

fair

amou

nt

AB07

inte

rmed

iate

simpl

e

simpl

e

impo

ssib

le

trivi

al

diffi

cult

simpl

e

inte

rmed

iate

inte

rmed

iate

diffi

cult

too

muc

h

AA05

simpl

e

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

diffi

cult

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

fair

amou

nt

AA06

inte

rmed

iate

simpl

e

simpl

e

diffi

cult

trivi

al

inte

rmed

iate

simpl

e

simpl

e

inte

rmed

iate

diffi

cult

very

little

AA07

trivi

al

inte

rmed

iate

inte

rmed

iate

trivi

al

trivi

al

simpl

e

inte

rmed

iate

trivi

al

simpl

e

inte

rmed

iate

none

IA15

diffi

cult

inte

rmed

iate

inte

rmed

iate

diffi

cult

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

impo

ssib

le

impo

ssib

le

fair

amou

nt

IA16

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

impo

ssib

le

simpl

e

simpl

e

inte

rmed

iate

simpl

e

inte

rmed

iate

diffi

cult

fair

amou

nt

IA01

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

diffi

cult

trivi

al

simpl

e

impo

ssib

le

trivi

al

simpl

e

diffi

cult

not s

om

uch

IA18

inte

rmed

iate

simpl

e

inte

rmed

iate

diffi

cult

simpl

e

simpl

e

diffi

cult

inte

rmed

iate

simpl

e

diffi

cult

fair

amou

nt

IA19

simpl

e

simpl

e

inte

rmed

iate

simpl

e

trivi

al

simpl

e

simpl

e

simpl

e

inte

rmed

iate

impo

ssib

le

fair

amou

nt

AB08

inte

rmed

iate

inte

rmed

iate

simpl

e

inte

rmed

iate

trivi

al

trivi

al

diffi

cult

trivi

al

simpl

e

inte

rmed

iate

not s

om

uch

AB09

trivi

al

simpl

e

simpl

e

impo

ssib

le

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

simpl

e

diffi

cult

fair

amou

nt

AA10

simpl

e

inte

rmed

iate

inte

rmed

iate

inte

rmed

iate

trivi

al

trivi

al

simpl

e

simpl

e

inte

rmed

iate

impo

ssib

le

very

little

AA11

diffi

cult

simpl

e

inte

rmed

iate

inte

rmed

iate

simpl

e

simpl

e

simpl

e

inte

rmed

iate

inte

rmed

iate

impo

ssib

le

not s

om

uch

AA12

simpl

e

simpl

e

simpl

e

simpl

e

simpl

e

inte

rmed

iate

simpl

e

simpl

e

inte

rmed

iate

diffi

cult

not s

om

uch

AA13

simpl

e

inte

rmed

iate

inte

rmed

iate

diffi

cult

simpl

e

simpl

e

simpl

e

simpl

e

simpl

e

inte

rmed

iate

not s

om

uch

AA14

trivi

al

trivi

al

trivi

al

impo

ssib

le

trivi

al

trivi

al

diffi

cult

trivi

al

diffi

cult

impo

ssib

le

not s

om

uch

IA20

inte

rmed

iate

simpl

e

simpl

e

inte

rmed

iate

trivi

al

trivi

al

trivi

al

trivi

al

trivi

al

diffi

cult

not s

om

uch

Tabl

eA.

5.Th

esu

bjec

ts’pe

rcei

ved

time

pres

sure

and

task

diffi

culty

192

A.5Data

Code

Age

Job

Posit

ion

Expe

rienc

e Leve

l

Numbe

r ofYe

ars

OOP

Java

Eclip

se

Rev.E

ng.

OOP

Java

Eclip

seRe

v.Eng

.

IA01

30

Develo

per

know

ledge

able

adva

nced

know

ledge

able

begin

ner

7–10

7–10

4–6

1–3

IA02

34

Develo

per

adva

nced

adva

nced

know

ledge

able

know

ledge

able

7–10

4–6

1–3

4–6

IA03

42

CTO, D

evelo

per

expe

rt

know

ledge

able

begin

ner

know

ledge

able

>10

1–3

1–3

>10

IA04

37

Develo

per

adva

nced

adva

nced

know

ledge

able

begin

ner

7–10

7–10

4–6

1–3

AB01

21

Mas

terSt

uden

t

adva

nced

adva

nced

adva

nced

begin

ner

4–6

4–6

4–6

<1

AB02

21

Mas

terSt

uden

t

adva

nced

adva

nced

adva

nced

begin

ner

1–3

1–3

1–3

<1

IA05

29

Consu

ltant

, Ph.D

. Stu

dent

expe

rt

begin

ner

begin

ner

know

ledge

able

7–10

7–10

4–6

4–6

AA01

26

Ph.D

. Stu

dent

expe

rt

expe

rt

know

ledge

able

begin

ner

>10

>10

1–3

<1

AA02

26

Ph.D

. Stu

dent

expe

rt

adva

nced

know

ledge

able

know

ledge

able

4–6

1–3

1–3

1–3

IA06

35

Head of

IT

expe

rt

expe

rt

adva

nced

adva

nced

>10

>10

4–6

7–10

IA07

27

Softw

are En

ginee

r

know

ledge

able

know

ledge

able

begin

ner

know

ledge

able

7–10

7–10

1–3

4–6

IA08

25

Softw

are En

ginee

r

know

ledge

able

adva

nced

know

ledge

able

begin

ner

4–6

4–6

1–3

<1

IA09

32

Develo

pmen

t Lead

er,Re

sear

cher

adva

nced

begin

ner

none

adva

nced

7–10

7–10

<1

4–6

AB03

28

Stud

ent

know

ledge

able

know

ledge

able

begin

ner

begin

ner

4–6

1–3

1–3

1–3

IA10

39

Proje

ctM

anag

er

expe

rt

adva

nced

know

ledge

able

know

ledge

able

>10

7–10

7–10

4–6

IA11

38

Consu

ltant

, Sys

temM

anag

er/A

nalys

t

know

ledge

able

begin

ner

begin

ner

adva

nced

7–10

7–10

1–3

7–10

IA12

34

Senio

r Java

Archite

ct

expe

rt

expe

rt

adva

nced

adva

nced

>10

>10

>10

>10

AB04

22

Mas

terSt

uden

t

adva

nced

adva

nced

adva

nced

know

ledge

able

4–6

1–3

1–3

<1

AA03

22

Mas

terSt

uden

t

adva

nced

adva

nced

adva

nced

begin

ner

7–10

4–6

4–6

<1

AB05

22

Mas

terSt

uden

t

adva

nced

adva

nced

know

ledge

able

begin

ner

4–6

1–3

1–3

1–3

AA04

29

Ph.D

. Stu

dent

adva

nced

adva

nced

know

ledge

able

begin

ner

4–6

4–6

1–3

<1

IA13

32

Consu

ltant

expe

rt

know

ledge

able

know

ledge

able

expe

rt

7–10

4–6

1–3

7–10

IA14

31

Softw

are Arch

itect

adva

nced

know

ledge

able

know

ledge

able

begin

ner

7–10

7–10

1–3

1–3

AB06

23

Mas

terSt

uden

t

adva

nced

adva

nced

adva

nced

begin

ner

4–6

1–3

1–3

<1

AB07

23

Mas

terSt

uden

t

adva

nced

adva

nced

adva

nced

begin

ner

4–6

1–3

1–3

<1

AA05

30

Ph.D

. Stu

dent

adva

nced

adva

nced

adva

nced

know

ledge

able

7–10

7–10

7–10

4–6

AA06

26

Ph.D

. Stu

dent

expe

rt

know

ledge

able

adva

nced

expe

rt

7–10

7–10

4–6

4–6

AA07

30

Ph.D

. Stu

dent

adva

nced

adva

nced

know

ledge

able

know

ledge

able

7–10

7–10

1–3

1–3

IA15

40

Proje

ctM

anag

er

expe

rt

expe

rt

adva

nced

adva

nced

>10

>10

7–10

4–6

IA16

39

Softw

are Arch

itect

adva

nced

adva

nced

know

ledge

able

know

ledge

able

4–6

4–6

4–6

1–3

IA01

30

Develo

per

know

ledge

able

adva

nced

know

ledge

able

begin

ner

7–10

7–10

4–6

1–3

IA17

27

Softw

are En

ginee

r

know

ledge

able

adva

nced

know

ledge

able

begin

ner

4–6

4–6

4–6

<1

IA19

39

Consu

ltant

, Pro

ject M

anag

er,Arch

itect

expe

rt

expe

rt

know

ledge

able

adva

nced

>10

7–10

7–10

4–6

AB08

21

Mas

terSt

uden

t

adva

nced

adva

nced

adva

nced

begin

ner

1–3

1–3

1–3

<1

AB09

23

Ph.D

. Stu

dent

adva

nced

adva

nced

adva

nced

know

ledge

able

4–6

1–3

1–3

1–3

AA10

24

Ph.D

. Stu

dent

adva

nced

adva

nced

adva

nced

adva

nced

4–6

4–6

4–6

1–3

AA11

23

Ph.D

. Stu

dent

adva

nced

adva

nced

adva

nced

know

ledge

able

4–6

4–6

4–6

1–3

AA12

52

Profe

ssor

expe

rt

adva

nced

know

ledge

able

adva

nced

>10

>10

4–6

>10

AA13

28

Ph.D

. Stu

dent

adva

nced

adva

nced

know

ledge

able

know

ledge

able

4–6

4–6

4–6

1–3

AA14

24

Mas

terSt

uden

t

expe

rt

expe

rt

expe

rt

know

ledge

able

4–6

4–6

4–6

1–3

IA20

36

Develo

per

adva

nced

expe

rt

adva

nced

begin

ner

>10

7–10

7–10

1–3

Table

A.1.Th

e subje

cts’ p

erson

alinf

ormati

on, c

luster

edby

treatm

ent c

ombin

ation

s

195

A.5 Data

Code

Completion Time Per Task TotalCompl. Time

A1 A2.1 A2.2 A3 A4.1 A4.2 B1.1 B1.2 B2.1 (excl. A4.2)

IA01 8.80 4.32 5.38 10.00 7.28 4.42 9.85 1.90 1.02 52.97 48.55

IA02 2.42 6.08 1.75 8.58 5.33 10.00 5.25 2.92 2.33 44.67 34.67

IA03 6.25 6.92 2.42 9.43 5.62 9.37 2.55 2.95 1.83 47.33 37.97

IA04 9.08 6.00 4.00 6.08 7.92 10.00 10.00 1.33 1.08 55.50 45.50

AB01 1.33 4.83 3.92 4.75 6.25 10.00 6.92 2.25 2.58 42.83 32.83

AB02 10.00 5.58 1.67 4.25 8.08 10.00 9.42 2.58 1.67 53.25 43.25

IA05 6.33 3.67 1.33 4.75 3.50 10.00 3.75 1.58 2.17 37.08 27.08

AA01 9.67 4.42 2.08 8.75 3.17 10.00 5.17 1.33 1.42 46.00 36.00

AA02 8.33 7.17 2.50 10.00 5.00 9.75 8.33 3.25 1.33 55.67 45.92

IA06 10.00 5.92 4.42 10.00 4.17 10.00 7.33 1.33 1.50 54.67 44.67

IA07 7.25 5.67 6.25 6.33 4.58 10.00 2.42 1.42 2.33 46.25 36.25

IA08 2.67 2.25 2.95 4.55 3.75 10.00 3.33 4.00 1.17 34.67 24.67

IA09 10.00 3.67 3.00 10.00 5.25 7.83 3.67 2.50 1.42 47.33 39.50

AB03 6.33 2.00 9.33 5.00 5.00 8.50 7.17 1.50 1.25 46.08 37.58

IA10 3.25 3.67 7.33 6.83 3.83 7.75 3.08 1.75 8.08 45.58 37.83

IA11 3.83 2.40 5.92 7.17 4.00 6.92 7.42 2.83 1.75 42.23 35.32

IA12 3.75 2.67 4.17 5.58 5.17 10.00 5.75 2.00 1.25 40.33 30.33

AB04 2.67 3.92 2.58 3.67 5.58 9.75 2.00 0.50 3.83 34.50 24.75

AA03 2.50 3.17 3.75 10.00 3.17 6.33 4.08 1.25 2.58 36.83 30.50

AB05 5.50 4.67 4.33 5.58 5.83 10.00 9.58 1.83 1.67 49.00 39.00

AA04 7.08 3.67 5.17 6.33 3.50 7.83 6.00 1.25 2.83 43.67 35.83

IA13 3.00 4.67 2.33 4.75 3.25 10.00 4.83 1.67 2.08 36.58 26.58

IA14 6.67 9.00 2.42 10.00 4.50 5.83 5.33 1.83 4.17 49.75 43.92

AB06 5.67 5.75 2.05 6.95 10.00 10.00 10.00 3.85 2.00 56.27 46.27

AB07 8.75 9.33 4.67 10.00 6.83 10.00 10.00 3.67 2.67 65.92 55.92

AA05 7.00 6.08 3.75 8.42 5.25 4.42 10.00 3.17 6.67 54.75 50.33

AA06 6.83 2.00 3.67 6.58 3.25 6.75 5.33 1.58 1.58 37.58 30.83

AA07 3.67 3.67 2.50 2.17 2.92 5.17 2.75 3.33 1.83 28.00 22.83

IA15 9.75 8.83 5.08 10.00 4.33 10.00 5.08 1.75 7.83 62.67 52.67

IA16 10.00 7.42 4.33 9.50 4.00 6.50 8.50 2.00 4.50 56.75 50.25

IA01 2.55 3.90 4.38 9.20 4.03 4.30 9.92 2.10 2.93 43.32 39.02

IA18 5.28 5.13 4.58 9.82 4.72 3.98 9.77 4.13 3.12 50.53 46.55

IA19 3.33 3.83 4.50 3.50 3.83 5.08 6.58 1.92 5.58 38.17 33.08

AB08 6.08 1.08 10.00 8.83 3.50 5.92 9.42 3.58 1.92 50.33 44.42

AB09 5.83 4.83 4.33 10.00 3.42 6.00 2.00 1.92 1.50 39.83 33.83

AA10 3.17 6.08 5.08 7.33 8.00 3.33 4.83 1.17 2.33 41.33 38.00

AA11 6.17 4.08 6.08 3.83 3.50 5.17 3.67 1.67 2.92 37.08 31.92

AA12 6.75 4.75 3.92 4.75 5.25 3.33 4.42 1.42 3.00 37.58 34.25

AA13 7.00 7.00 10.00 10.00 5.92 5.67 6.33 1.42 5.42 58.75 53.08

AA14 6.33 1.50 3.33 10.00 2.83 3.25 9.17 2.42 4.50 43.33 40.08

IA20 6.83 5.58 4.75 8.00 2.33 3.42 2.42 1.42 1.33 36.08 32.67

Table A.4. The subjects’ task completion time, in minutes

194 A.5 Data

CodeCorrectness Per Task

TotalCorrectness

A1 A2.1 A2.2 A3 A4.1 A4.2 B1.1 B1.2 B2.1 (excl. A4.2)IA01 0.00 1.00 1.00 1.00 1.00 0.00 0.00 1.00 1.00 6.00 6.00IA02 1.00 0.80 0.50 1.00 1.00 0.00 0.00 1.00 1.00 6.30 6.30IA03 0.00 0.00 1.00 0.00 1.00 0.00 0.00 1.00 0.00 3.00 3.00IA04 0.00 1.00 0.00 1.00 1.00 0.00 0.00 1.00 1.00 5.00 5.00AB01 0.00 1.00 1.00 0.80 1.00 0.00 0.00 1.00 1.00 5.80 5.80AB02 0.00 1.00 1.00 0.70 1.00 0.00 0.00 0.00 1.00 4.70 4.70IA05 0.00 1.00 1.00 0.20 1.00 0.00 0.00 1.00 1.00 5.20 5.20AA01 0.00 0.80 1.00 0.40 1.00 0.00 0.00 1.00 0.00 4.20 4.20AA02 0.00 1.00 1.00 0.00 1.00 0.00 1.00 1.00 1.00 6.00 6.00IA06 0.00 1.00 0.00 0.30 1.00 0.00 0.00 1.00 1.00 4.30 4.30IA07 1.00 1.00 1.00 0.17 1.00 0.00 0.00 1.00 1.00 6.17 6.17IA08 1.00 0.50 1.00 0.83 1.00 0.00 1.00 1.00 1.00 7.33 7.33IA09 1.00 1.00 1.00 1.00 1.00 0.00 1.00 1.00 1.00 8.00 8.00AB03 1.00 1.00 1.00 1.00 1.00 0.00 1.00 1.00 1.00 8.00 8.00IA10 1.00 1.00 1.00 0.17 1.00 0.00 0.00 1.00 1.00 6.17 6.17IA11 1.00 1.00 1.00 0.00 1.00 0.16 0.00 0.00 1.00 5.16 5.00IA12 1.00 1.00 1.00 1.00 1.00 0.00 1.00 1.00 1.00 8.00 8.00AB04 1.00 1.00 1.00 0.67 1.00 0.00 0.00 1.00 1.00 6.67 6.67AA03 1.00 1.00 0.80 0.33 1.00 0.16 0.00 1.00 1.00 6.29 6.13AB05 1.00 1.00 1.00 1.00 1.00 0.00 0.00 1.00 1.00 7.00 7.00AA04 1.00 1.00 1.00 0.50 1.00 0.00 0.00 1.00 1.00 6.50 6.50IA13 1.00 1.00 1.00 0.83 1.00 1.00 0.00 0.00 1.00 6.83 5.83IA14 0.00 0.80 0.33 0.30 1.00 0.67 0.00 1.00 1.00 5.10 4.43AB06 1.00 1.00 1.00 0.00 0.00 0.00 0.00 1.00 0.00 4.00 4.00AB07 0.00 0.60 0.67 0.00 0.00 0.00 0.00 0.00 1.00 2.27 2.27AA05 1.00 1.00 1.00 0.00 1.00 1.00 0.00 1.00 1.00 7.00 6.00AA06 0.00 1.00 0.00 0.00 1.00 1.00 0.00 0.00 1.00 4.00 3.00AA07 1.00 1.00 0.00 0.00 1.00 1.00 0.00 1.00 0.00 5.00 4.00IA15 1.00 0.80 0.00 0.00 1.00 1.00 0.00 1.00 0.00 4.80 3.80IA16 0.00 0.00 0.67 0.00 1.00 0.67 0.00 1.00 1.00 4.34 3.67IA01 1.00 1.00 1.00 0.50 1.00 1.00 0.00 1.00 1.00 7.50 6.50IA18 1.00 1.00 1.00 0.50 1.00 1.00 0.00 1.00 1.00 7.50 6.50IA19 0.00 1.00 0.60 0.00 1.00 1.00 0.00 1.00 0.00 4.60 3.60AB08 0.00 1.00 0.40 0.38 1.00 1.00 1.00 1.00 1.00 6.78 5.78AB09 1.00 1.00 0.60 0.50 1.00 1.00 0.00 1.00 1.00 7.10 6.10AA10 1.00 1.00 0.80 0.38 1.00 1.00 0.00 1.00 1.00 7.18 6.18AA11 1.00 1.00 0.80 0.00 1.00 1.00 0.00 1.00 1.00 6.80 5.80AA12 0.00 0.50 0.00 0.00 1.00 1.00 0.00 1.00 1.00 4.50 3.50AA13 0.00 1.00 0.00 0.25 1.00 1.00 0.00 1.00 1.00 5.25 4.25AA14 1.00 1.00 0.60 0.00 1.00 1.00 0.90 1.00 1.00 7.50 6.50IA20 0.00 1.00 1.00 0.38 1.00 1.00 0.00 1.00 1.00 6.38 5.38

Table A.3. The correctness of the subjects’ solutions to the tasks

Results

Correctness

2

3

4

5

6

7

8

medium large

Eclipse + Excel

CodeCity

24.26%more correct with CodeCity

statistically significant

95% confidence interval

large effect size (d=0.89)

Completion time

12.01%faster with CodeCity

statistically significant

95% confidence interval20

30

40

50

60

medium large

Eclipse + Excel

CodeCity

moderate effect size (d=0.63)

Reflections

design desiderata

Reflections

extensive, replicable controlled experiment

design desiderata

Reflections

extensive, replicable controlled experiment

+24% correctness, -12% completion time

design desiderata

Reflections

Prologue

Approac

h

Evalua

tion

Epilogue

Thesis

Depicting software systems as cities is a versatile metaphor which enables the creation of efficient software visualizations to support reverse engineering.

Contributions

Contributions

1The definition of a city metaphor and its application in three context (program comprehension, software evolution analysis, and design quality assessment), which demonstrated the versatility of the metaphor.

Contributions

21

The definition of a city metaphor and its application in three context (program comprehension, software evolution analysis, and design quality assessment), which demonstrated the versatility of the metaphor.

The implementation of CodeCity, a scalable tool which supports our approach based on the city metaphor.

Contributions

21

3

The definition of a city metaphor and its application in three context (program comprehension, software evolution analysis, and design quality assessment), which demonstrated the versatility of the metaphor.

The implementation of CodeCity, a scalable tool which supports our approach based on the city metaphor.

An empirical validation, by means of an extensive controlled experiment, whose results provided evidence that the visualizations built on top of the city metaphor are efficient.

+24.26% correctness

-12.01% time

Future work

Future work

1 Researching representations for dependencies more appropriate for the city metaphor.

Future work

21 Researching representations for

dependencies more appropriate for the city metaphor.

Evaluation of the support for software evolution assessment.

Future work

21

3

Researching representations for dependencies more appropriate for the city metaphor.

Evaluation of the support for software evolution assessment.

Integration with IDEs and operational workflows.

Publications1. Visual Exploration of Large-Scale System Evolution

Richard Wettel and Michele Lanza Proc. of WCRE 2008 (15th Working Conference on Reverse Engineering), pp. 219 - 228, IEEE CS Press, 2008.

2. Visually Localizing Design Problems with Disharmony MapsRichard Wettel and Michele Lanza Proc. of SoftVis 2008 (4th International ACM Symposium on Software Visualization), pp. 155 - 164, ACM Press, 2008.

3. Program Comprehension through Software HabitabilityRichard Wettel and Michele Lanza Proc. of ICPC 2007 (15th IEEE International Conference on Program Comprehension), pp. 231 - 240, IEEE CS Press, 2007.

4. Evospaces - multi-dimensional navigation spaces for software evolutionSazzadul Alam, Sandro Boccuzzo, Richard Wettel, Philippe Dugerdil, Harald Gall, Michele Lanza In “Human Machine Interaction”, pp. 167 - 192, LNCS Series. Springer, 2009.

7. Visualizing Software Systems as CitiesRichard Wettel and Michele Lanza Proc. of VISSOFT 2007 (4th IEEE International Workshop on Visualizing Software For Understanding and Analysis), pp. 92 - 99, IEEE CS Press, 2007.

6. CodeCityRichard Wettel and Michele Lanza Proc. of WASDeTT 2008 (1st International Workshop on Advanced Software Development Tools and Techniques), 2008.

5. Scripting 3D Visualizations with CodeCityRichard Wettel Proc. of FAMOOSr 2008 (2nd Workshop on FAMIX and Moose in Reengineering), 2008.

8. Visual Exploration of Large-Scale Evolving SoftwareRichard Wettel Companion Proc. of ICSE 2009 (31st International Conference on Software Engineering), Doctoral Symposium, pp. 391 - 394, ACM Press, 2009.

9. CodeCity: 3D Visualization of Large-Scale SoftwareRichard Wettel and Michele Lanza Companion Proc. of ICSE 2008 (30th International Conference on Software Engineering), Research Demonstration Track, pp. 921 - 922, ACM Press, 2008.

10. Empirical Validation of CodeCity: A Controlled ExperimentRichard Wettel, Michele Lanza, Romain Robbes Technical Report 2010/05, University of Lugano, 2010.

conferences

book chapter

peer-reviewedworkshops

doctoral symposium

tool demonstration

technical report

Richard Wettelhttp://www.inf.usi.ch/phd/wettel

http://creativecommons.org/licenses/by/3.0/

http://www.inf.unisi.ch/phd/wettel

http://www.inf.unisi.ch/phd/wettel



Software Systems as Cities

Technology

Transcript of Software Systems as Cities