Toward an Environment for Comprehending Distributed Systems

Reverse Engineering Reverse Engineering (Dynamic Analysis)(Dynamic Analysis) © SERG

Toward an Environment for Comprehending Distributed

Systems


Outline

• Motivation

• Software comprehension environment

• Data gathering

• Data repository

• Preliminary results

• Conclusions and future work


Background

• Project– Comprehending distributed component-based

systems– Phase-I

• Development of a core toolset

– Phase-II (Iterative)• Modeling and visualization

• Evaluation


Characteristics of modern software systems

• Large and complex

• Distributed

• Written in multiple programming languages

• Developed using component-models


Software Comprehension Challenges

• Unavailability of source code– Vigder-2001, Weyuker-2001

• Physical distribution of components– Bruegge-1993, Mendonca-1999

• Use of pre-built component– Stroermer-2002, Vigder-2001, Weyuker-2001

• Multiple programming languages– Hassan-2002, Chen-1995,– Koschke-1998, Stroermer-2002– Demeyer-1999, CDIF-1994, XML/XMI-2002


Approach

• Software Comprehension Environment– Technical issues

• Distributed profilers

• Single data repository

• Data management

• Extensible architecture Data gathering and Repository subsystems

– Modeling and visualization subsystem

– Empirical evaluation• Survey/questionnaire

• Industrial-strength case studies – Cigna Corporation


Software Comprehension Environment

Program Data(Static &Dynamic)

Data ModelMeta-data

Model

Universal Schema

Meta-Data

Data Gathering Subsystem

Analysis and Visualization Subsystem

Dynamic Analyzers(Debugger/Profiler)

Static AnalyzersSoftware

Program(s) Visualization(Presentation)

ConceptualModels

NavigationHuman-Driven/Feature-Driven

Sourcecode

Execution Traces

Repository Subsystem


Data Gathering subsystem

DataCollectionAdapter

GDBProfiler


JVMPIJVMDIProfiler


Java StaticAnalyzer


Win32C/C++/VBProfiler

GNU C/C++Run-time

Java Run-time

Java Source-codeor Byte-code

Win32 C/C++ and VBRuntime

Data Collection Manager

Logical Time Server

Data Collection ServerTo DataRepository

XML Documents

Import

/Exp

ort



• Distributed Profiler– Local Profiler

– Logical Time Server

– Data collection manager

• Remote Interactions– Network interceptor

– Communication Endpoint

• Program Facts/Data– Static: Entities and Relations

– Dynamic: Entities, Relations and Events



• Endpoint entity– Local and Remote host– Local and Remote port numbers– Time-stamp

• Connects RelationLocal Host Remote Host Local Port Remote Port Time

Local Host Remote Host Local Port Remote Port Time+Delta

Delta < Threshold

Endpoint-A

Endpoint-B


Example: Sequence Diagram

Java Program C/C++ Win32 Program

UserClassjava.netSokect-

OutputStreamUserModule

(Winsock)Ws2_32.DLL

write()

recv()

send()

EP(A)

EP(B)

Logi

cal T

ime

EPTime(A)

EPTime(B)

Java.net.Sokect-

InputStream

read()



• Supported data collectors– WDBG Win32 Profiler– JVPROF Java Profiler– JSA Java static analyzer– Others, provided the conversion to XML

• Network-call interceptors – WDBG and JVPROF


Data Repository

• Logical Models– Language definitions– Program Data– Used-defined views

• Implementation– Relational database

• Query/Manipulation Language– SQL– SMQL


Language Definitions

• Model(General) = Graph(Eg , Rg)Eg: Set of entity types

Rg: Set of relation types

• Java, example:• Model(Java) = Graph(Ejava , Rjava)

Ejava Eg : Entity types supported by Java

Rjava Rg : Relation types supported by Java

Such that the source and destination entity Ejava

• C++, definingModel(C++) = Graph(Ec++, Rc++)

Ec++ = Ejava {template, struct, typedef, function}

- {interface, package}


Java Language Example

method

object interface

package

class

field

creates

reads

writes

contains

defines

declares

instanceof

declares

End-point

connects

implements

invokes


SMQL

• SMQL– Software Modeling Query Language

• Set-based:– Typed Set

• Entity Set

• Relation Set

• Event Set

– Generic Set

• Extensible using Java


SMQL

• Operations– Union, intersection, Difference

• Functions– Closure

Closure(Source, Target, RelationTypes)

Returns a Relation-Set

– Composition• Builds a hierarchical model from flat set of entities into a nested

set of entities

– Output: Saves a typed-set as an XML document– Output_dot: Saves a relation set as DOT graph


Example: Deployment Architecture

Environment

Windows 2000IP Address:192.92.200.3

LinuxIP Address:192.92.200.2

Jetty WebServer(Java JVM 1.4.2)

jvprofData Collection

Tear.exe

wdbgData collection

Data CollectionManager SCDR

SQL Server 2000

SQ

L

SM

QL Analysis

Jetty Source code

jsaData collection

Dyn

am

ic

Sta

tic


SMQL: Defining Entity Set

Entity Identifier

{

[caption = String ;]

type = StringSet ;

(EntityFirstOrderFilter)+

}

EntityFirstOrderFilter ::

include ( EntityAttrib ) = StringSet ;

| exclude ( EntityAttrib ) = StringSet ;


SMQL: Inheritance example

// Get Inheritance tree rooted at org.mortbay.http.HttpListener

// OR org.mortbay.http.HttpHandler

EntitySet Http

{

caption = "HttpListener" ;

type = {“interface”, “class”} ;

include (name) = { "org.mortbay.http.HttpListener", "org.mortbay.http.HttpHandler" } ;

include (project) = { "jetty" } ;

}

Inh_Http = closure(null, Http, { “implement”, “subclass” } ) ;

Output (“file.xml”, Inh_Http, Http) ;

Output_dot (“graph.dot”, Inh_Http) ;


SMQL: Inheritance example


SMQL: Events Example// Events from Tear & Jetty

import java:serg.sc.filter.CallGraph as CallGraph ;

EventSet Events{caption = "Event (Jetty & Tear) " ;type = { "method-entry",

"method-exit", "endpoint", "thread-start", "thread-end", "module-load","process-start","process-end"} ;

include (project) = {"jetty-rt", "Tear"} ;}

// Convert events trace into a CallGraph (Relation Set)

GC = CallGraph( events ) ;

// Save as resultsoutput ("tear-jetty.xml", events);

Output_dot (“GC.dot”, GC) ;


SMQL: Events Example


SMQL: User defined functions

• User-defined functions– IFilter interfaceImport java:Qualified-Java-Class-Name as FunctionName ;

• ExampleImport java:serg.sc.MyClass as MyFunction

EntitySet E

{

caption = "HttpListener" ;

type = {“interface”, “class”} ;

include (name) = { "org.mortbay.http.HttpListener” } ;

include (project) = { "jetty" } ;

}

X = MyFunction(E) ;


Modeling and visualization subsystem

SMQL Analysis Visualization

Vis

ua

l Sty

le

Na

vig

atio

na

lS

tyle

Software ViewModelRevisedModel

ProgramData


Progress

• Repository implementation– SQL Server & Hypersonic DB

• Data gathering– WDBG, JVPROF, and JSA

• Tools– SMQL– Dbmgr - Database and SMQL front-end– dbImport, dbexport


Conclusions and Future Work

• Software Comprehension Environment• Dynamic analysis of distributed systems• Multi-language systems• Common repository• Distributed profiler• SMQL

• Future work– Phase I

• Performance• Support for Microsoft.Net, VB and COM+

– Phase II (Early stage)• Construction and visualization of views• Empirical evaluation in an industrial setting (at Cigna)


ReferencesVigder-2001] M. Vigder. The evolution, maintenance, and management of

component-based systems. In G. Heineman and W. Councill, editors, Component-Based Software Engineering: Putting the Pieces Together, chapter 29, pages 527. 539. Addison-Wesley, 2001.

[Weyuker-2001] E. J. Weyuker. The trouble with testing components. In G. Heineman and W. Councill, editors, Component-Based Software Engineering: Putting the Pieces Together, chapter 28, pages 499.512. Addison-Wesley, 2001.

[Stroermer-2002] C. Stroermer, L. O'Brien, and C. Verhoef. Practice patterns for architecture reconstruction. In Proceedings of the 9th Working Conference on Reverse Engineering (WCRE), Richmond, VA, USA, October 2002.

[Hassan-2002] A. E. Hassan and R. C. Holt. Architecture recovery of web applications. In 24th International Conference on Software Engineering (ICSE), Orlando, Florida, USA, May 2002.

[Chen-1995] Y.-F. R. Chen, G. S. Fowler, E. Koutsoos, and R. S. Wallach. Ciao: A graphical navigator for software and document repositories. In Proceedings of International Conference on Software Maintenance (ICSM), Nice, France, October 1995. IEEE.


References[Demeyer-1999] S. Demeyer, S. Tichelaar, and P. Steyaert. Famix 2.0 – the

famoos information exchange model. Technical report, University of Berne, August 1999.

[CDIF-1994] Electronic Industries Alliance and International Standards Organization (EIA/ISO). CDIF - Framework for Modeling and Extensibility, 1994.

[XML/XMI-2002] Object Management Group. XML Metadata Interchange (XMI), version 1.2 edition, January 2002.

[Koschke-1998] R. Koschke, J.-F. Girard, and M.Wurthner. An intermediate representation for reverse engineering analysis. In Proceedings of the 5th Working Conference on Reverse Engineering (WCRE), Honolulu, HI, USA, October 1998.

[Bruegge-1993] B. Bruegge, T. Gottschalk, and B. Luo. A framework for dynamic program analysis. In Conference on Object Oriented Programming Systems, Languages, and Applications (OOSLA93), Washington, USA, September 1993.

[Mendonca-1999] N. C. Mendonca. Software Architecture Recovery for Distributed Systems. PhD thesis, Department of University of London, November 1999.

Toward an Environment for Comprehending Distributed Systems

Documents

Transcript of Toward an Environment for Comprehending Distributed Systems