Architectural Styles of HEP Experiments Presented by RD Schaffer CERN/EP and LAL Orsay LCB Workshop...

Architectural Styles Architectural Styles of HEP Experimentsof HEP Experiments

Presented by RD Schaffer

CERN/EP and LAL Orsay

LCB Workshop

Marseille, September 29 1999

29 September 1999 Architecture Session – Rapporteur talk

2

Architectural Styles of HEP Architectural Styles of HEP ExperimentsExperiments

HEP experiments have been moving their software to Object-oriented systems for a few years

Each LHC experiment needs to develop a framework to be used in their event data processing applications physics/detector simulation, high level triggers,

reconstruction, analysis, visualization, etc.

These experiment frameworks will incorporate external frameworks/toolkits: e.g. persistency, simulation, GUI


3

Architectural Styles of HEP Architectural Styles of HEP ExperimentsExperiments (2) (2)

In this talk we would like to begin to

Distill abstract views of the architecture of our software systems High level view is not completely accurate

Any real architecture is a hybrid of styles

This should help us reason and to discuss about the different architectural designs and choices

Goal: help us to define simple and clear architectures so that the software systems we build are easy to understand and to interact with


4

Software ArchitectureSoftware Architecture

Outline of talk Architecture: what is it?

Definitions of the terminology/vocabulary

Architecture styles: examples from the literature

Architectural styles in HEP An example architecture: LHCb’s GAUDI Variations of algorithm/event data styles Architectural issues of persistency Application Meta Information User interface (UI) and visualization

Implementation and physical design


5

BibliographyBibliography

Notation: […] is used as cross-reference in slides.

Architecture

[G&S]: D. Garlan and M. Shaw. "An Introduction to Software Architecture," 1994

http://www.cs.cmu.edu/afs/cs/project/vit/ftp/pdf/intro_softarch.pdf

[S&C]: Shaw and Clements, “Preliminary classification of architectural styles for software systems”, 1996

http://www.cs.cmu.edu/afs/cs.cmu.edu/project/vit/www/paper_abstracts/Boxology.html

[USDP]: I. Jacobson, et al. “The Unified Software Development Process”, Addison Wesley 1999

[Booch]: G. Booch, “Object Solutions”, Addison Wesley 1996

http://www.cs.cmu.edu/afs/cs/project/vit/ftp/pdf/intro_softarch.pdf




6

BibliographyBibliography (2) (2)

Frameworks

[Gamma]: E. Gamma, et al., “Design Patterns”, Addison-Wesley 1995

[IBM1]: Building Object Oriented Frameworks (html) http://www.ibm.com/java/education/oobuilding/index.html

[IBM2]: Leveraging Object Oriented Frameworks (html) http://www.ibm.com/java/education/ooleveraging/index.html

Physical design

[Martin]: R. Martin, “Designing O-O C++ Applications using the Booch Method”, Prentice Hall 1995

[Lakos]: J. Lakos, “Large Scale C++ Software Design”, Addison-Wesley 1998

http://www.ibm.com/java/education/oobuilding/index.html

http://www.ibm.com/java/education/ooleveraging/index.html


7

Architecture: What Is It?Architecture: What Is It?

Definitions architecture [USDP]

Software architecture is also concerned functionality, performance, resilience, reuse, comprehensibility, economic and technology constraints and trade-offs, and aesthetic concerns.

Set of significant decisions about the organization of the software system

Selection of the structural elements and their interfaces which compose the system

Their behavior — collaboration among the structural elements

Composition of these structural and behavioral elements into progressively larger subsystems

The architectural style that guides this organization


8

Architecture: DefinitionsArchitecture: Definitions (2) (2)

toolkit [Gamma]

Examples C++ I/O stream library, containers/iterators/algorithms library, CLHEP, GEANT4 (can be toolkit or framework continuous spectrum)

Comments They do not impose a particular design on one’s application They provide functionality to help one’s application do it’s job

set of related and reusable classes designed to provide useful, general-purpose functionality


9


framework [Booch] + [Gamma]

A kind of micro-architecture that codifies a particular domain

Provides the suitable knobs, slots and tabs that permit clients to use and adapt to specific applications within a given range of behavior

A framework is generally composed of two elements a set of classes that capture the vocabulary of a particular domain a control policy that orchestrates the instances of those classes

A framework realizes an architecture

A large O-O system is constructed from severalcooperating frameworks


10


interface [USDP]

A physical or replaceable part of a system that conforms to and provides the realization of one or more interfaces

component [USDP]

A collection of operations that are used to specify a service of a class or component


11


design pattern [Gamma]

Description of communicating objects and classes that are customized to solve a general design problem in a particular context

Is more abstract than a framework a framework itself can be embodied in code a pattern can only have examples embodied in code

Is a smaller architectural unit than a framework Is less specialized than a framework

can apply to >1 application domains

Architectural abstractions can be described as patterns


12

Architecture Styles: OutlineArchitecture Styles: Outline

Part I General categorization of systems [Booch]

user-centric data-centric computation-centric

Part II Further classification of architectural styles [C&S]

Constituent parts: components and connectors Examples of architectural patterns

data abstraction (object-oriented) pipe-and-filter systems data-centered repository implicit invocation


13

Architecture Styles: Part IArchitecture Styles: Part I

General categorization of systems [Booch]

user-centric focus on the direct visualizationand manipulation of the

objectsthat define a certain domain

data-centric focus upon preserving theintegrity of the persistentobjects in a system

computation-centric focus is on the transformationof objects that are interestingto the system

Our applications have elements with all three.

The interesting question is which one dominates?


14

Architecture Styles: Part I (2)Architecture Styles: Part I (2)

Booch describes three dominant layers for each system type user-centric

classes that provide the system’s look-and-feel classes that map the GUI layer to the domain model classes that denote the domain model

data-centric classes that access and manipulate the domain model classes that denote the domain model classes that provide persistence for the domain model


15

Architecture Styles: Part I (3)Architecture Styles: Part I (3)

computation-centric

classes whose objects act as agents responsible for carrying out algorithms that involve the collaboration of several other objects

classes that model the objects transformed by the system

classes to which higher level objects delegate certain more primitive responsibilities, so that common behaviors can be localized and thus reused

In my opinion it is the computation-centric architectural style that is at the heart of the reconstruction, analysis, and high-level triggers


16

Architecture Styles: Part IIArchitecture Styles: Part II

Further classification of architectural styles [C&S]

Constituent parts – the building blocks of architecture components – a functional unit of software

e.g. objects, processes, filters

connectors – mechanisms that mediate communication, coordination, or cooperation of components

e.g. shared representations, data streams, or data format converters

To uniquely identify a style, one must also specify the control discipline, the data organization, and the interaction of control and data


17

Architecture Styles: Part IIArchitecture Styles: Part II (2) (2)

Control issues How control passes among components

control topology linear or acyclic, hierarchical/tree-like, star/hub-and-spoke, or

arbitrary

How components work together in time synchronicity lockstep (sequential or parallel),

synchronous, or asynchronous

Data issues How data moves around a system

Data topology (as for control) Continuity continuous/sporadic, high/low volume Mode passed, shared, or broadcast


18


Control/data interaction issues Are control-flow and data-flow topologies isomorphic?

If isomorphic, is the direction the same or opposite?

Useful examples of architectural patterns Call-and-return styles data abstraction

(object-oriented)

Data flow styles pipe-and-filter systems

Data-centered repository styles blackboard

Interacting process styles implicit invocation

Note: few systems are purely any one of these!


19


Call-and-return styles: data abstraction Localized state maintenance (encapsulation) components: managers connectors: method invocation control: decentralized, usually single thread data: passed disadvantage: interface coupling Same topology

and directiontopology: arbitrary

method calls

managers


20


Data flow styles: pipe-and-filter systemsFilters transform input into output components: filters - computational i.e. retain minimal

state connectors: data streams control: asynchronous data: passed

pipefilter

Same topology and directiontopologies: arbitrary, acyclic (no feedback), fanout, pipeline (linear)


21


Data-centered repository styles: blackboard or DBCentralized data, usually structured components: central data store, many computational objects connectors: computational objects interact with central

store directly or via method invocation control: may be external, predetermined or

internal data: shared or passed topology: star

computational objects

memory

Blackboard (shared data)


22


Interacting process styles: implicit invocationIndependent reactive objects (or processes) components: objects that register interest in “events” and

objects that “signal events” connectors: automatic method invocation control: decentralized, de-coupling of sender and

receiver data: passed with event, may also require a shared

repository topology: arbitrary

“event” signals?!

?!

!

?

method invocation


23

Architectural Styles in HEP: OutlineArchitectural Styles in HEP: Outline

Global structure

Foundation libraries

An example architecture: LHCb’s GAUDI

Variations of algorithm/event data styles

Application Meta Information

Architectural issues of persistency

User interface (UI) and visualization


24

Architectural Styles of HEP Architectural Styles of HEP ExperimentsExperiments

I believe that most people would agree with a global software structure as expressed by

FrameworksToolkits

Reco

nst

ruct

ion

Sim

ula

tion

Analy

sis

Foundation LibrariesH

igh level tr

iggers

One main framework

Various specialized frameworks: visualization, persistency, interactivity, simulation, etc.

A series of basic libraries widely used: STL, CLHEP, etc.

Applications built on top of frameworks Implements the required physics algorithms.


25

Foundation LibrariesFoundation Libraries

These form a basic vocabulary which is used throughout the code

Most experiments today propose to use C++ Standard library (STD)

collections, iterators, algorithms, stings, streams, numerics

For missing or HEP-specific pieces (e.g. via LHC++) CLHEP - random number generators, physics vectors,

geometry and linear algebra

ROOT libraries: containers, vectors, etc.

open questions: more linear algebra (Blitz++ or SL++), units - G4 or Fermilab’s SIunits package, pieces missing - error logger, exception handling


26

An Example Architecture: LHCb’s An Example Architecture: LHCb’s GAUDIGAUDI

LHCb has specified the high level view of their software system’s architecture

The specification consists of Scenarios and requirements

Overall system design set of major design criteria identification of the major components and their interactions physical design of their system (i.e. packaging)

More detailed specification of the individual components e.g. purpose, interface, dependencies

Currently GAUDI is being implemented and deployed


27

GAUDI ArchitectureGAUDI Architecture (2) (2)

Major design criteria Clear separation between data and algorithms

Three basic types of data: event, detector, statistics

Clear separation between persistent and transient data

Data store -centered architectural style

User code encapsulated in few specific places algorithms and converters

All components with well defined interfaces and as generic as possible


28

GAUDI ArchitectureGAUDI Architecture (3) (3)

TObjTObj

Obj2

AppManager

PersistencySvc

Algorithm1Algorithm1

Algorithm1

JobOptionsSvc

TObj

TObjContainerTObjContainer

ObjContainer

Obj3

MessageSvc

TObj1Obj1

DetDataSrv

TDetElem1TDetElem1

TDetElem1

PObjectPObjectPDetElem

EventDataSvc

AlgFactory

AnotherPercySvc

Transient Event Store

PObjPObj

PObj

PObjPObj

PObj

DetPerstySvc

Alg Properties

T Detector Store

T Histogram Store

HistogramSvc

Hist1Hist1

Hist1

HistPerstySvc PHistPHist

ConverterConverter

Converter ConverterConverter

Converter

EventSelector

Converter

TObjObj1

uses

creates

navigability

User code

User code

User code


29

Architectural Styles: Use of Architectural Styles: Use of InterfacesInterfaces

GAUDI’s top level components are interacting

objects Application Manager, Algorithms and

Services

To limit the coupling between objects, they make use of Java-like interfaces

any component implements one or more interfaces

clients of a component hold references to interfaces, not to the concrete componentIAlgorithm

IProperty

IDataProvider

IDataProvider

IHistogramSvc

IMessageSvc

Client

EventDataService

DetectorDataService

HistogramService

MessageService

ConcreteAlgorithm


30


Call-and-return styles: data abstraction Localized state maintenance (encapsulation) components: managers connectors: method invocation control: decentralized, usually single thread data: passed

method calls

managers Same topology and directiontopology: arbitrary


31

GAUDI AlgorithmsGAUDI Algorithms

AlgorithmA

AlgorithmB

AlgorithmC

Transient EventData Store

Data T1

Data T2, T3

Data T2

Data T3, T4

Data T4

Data T5

T1

T5

Real data flow Apparent data flow

• Each algorithm only knows what data (type and name) is expecting as input and creating as output

• The only coupling is through the data

• Scheduling of sub-algorithms is responsibility of the parent algorithm

A

C

B

Parent

T2

T4

T3

Control flow


32

GAUDI’s Algorithm/Event Data GAUDI’s Algorithm/Event Data Style Style

Algorithm and event data logical view – follows pipe-and-filter style

algorithms – filters event data – data stream

data flow is implemented as a data-centered repository

this implies that data is effectively shared between all filters issues: policy on shared data needed, e.g. who can modify what?

control flow follows the logical data flow no general mechanism for control (who calls whom) control is left up to the application manager and each “parent”

algorithm

data and control flow: in same direction, but different paths


33


Data flow styles: pipe-and-filter systemsFilters transform input into output components: filters - computational i.e. retain minimal

state connectors: data streams control: asynchronous data: passed

pipefilter

Same topology and directiontopologies: arbitrary, acyclic (no feedback), fanout, pipeline (linear)


34


Data-centered repository styles: blackboard or DBCentralized data, usually structured components: central data store, many computational objects connectors: computational objects interact with central

store directly or via method invocation control: may be external, predetermined or

internal data: shared or passed topology: star

computational objects

memory

Blackboard (shared data)


35

Variations of Algorithm/Event Data Variations of Algorithm/Event Data StylesStyles

Many of today’s HEP experiments use a similar style BaBar and CDF share AC++ framework

framework is a state machine: loops over modules at transitions

module interface: Init, BeginRun, Event, EndRun, EndJob, TalkTo

follows pipe-and-filter style allow parallel data flow paths, filters can stop flow data flow is the event: central repository control flow (done by framework) is separate but parallel

input modul

e

output modul

eeven

t

control framework

app modul

e

app modul

e

app modul

e

app modul

e

control

input/output modules do

event selection


36

CMS CARF: Reconstruction via notification andaction-on-demand

follows implicit invocation and central repository styles

No central ordering of actions, no explicit control of data flow: only implicit dependencies

External dependencies managed through an Event Driven Notification to “subscribers”

Internal dependencies through an Action on Demand mechanism

how does it work? reconstruction algorithms register themselves saying what

they produce and their name a client asks the event for something, e.g. a list of tracks, and

may specify which algorithm is to be used iterating over the list induces the algorithms to be run

Variations of Algorithm/Event Data Variations of Algorithm/Event Data StylesStyles (2) (2)


37

CMS CARF: example

EventEvent

Rec T2Rec T2

Rec T1Rec T1

AnalysisAnalysis

HitsHits

T1T1

T2T2

DetectorDetectorElementElementRec HitsRec Hits



38


Interacting process styles: implicit invocationIndependent reactive objects (or processes) components: objects that register interest in “events” and

objects that “signal events” connectors: automatic method invocation control: decentralized, de-coupling of sender and

receiver data: passed with event, may also require a shared

repository topology: arbitrary

“event” signals?!

?!

!

?

method invocation


39

Object Networks under consideration by ATLAS follows both pipe-and-filter and implicit invocation

styles: has general mechanism to pass by reference any object: event,

tracks, clusters - no central repository control flows in same direction as data flow (done by framework) module execution is triggered by an input which has changed

(uses observer pattern - implicit invocation) filters can stop flow, as for CDF/BaBar

input modul

e

output modul

eeven

t

module

module

module

module

module

module

module

event

intermediate data objects



40


Clustermanager

Data

AliRoot base classes

Raw datamanager

Data

Trackmanager

Data

TPC

Hits Pars

MUON

Hits Pars

Run Manager

ITS

Hits Pars

GlobalReconstruction

control

dataDetector modules

Global data managersblackboards

Global algorithms

ROOT IO

IOIO

ALICE’s reconstruction: data abstraction and data-centered repository


41

Architectural Issues of PersistencyArchitectural Issues of Persistency

The questions of persistency or I/O are primarily treated in another session

Which issues are pertinent to software architecture? What is the desired coupling of the system to the

persistency mechanism?

How does one design an architecture which allows access to the “advantages” of a desired I/O system, e.g.

ODBMS? access to different types of I/O systems? migration to new technologies?

Are there performance implications?

What are the implications of wide-area distribution of the data and the users?


42

Architectural Issues of PersistencyArchitectural Issues of Persistency (2)(2)

From the application’s point of view, clients (e.g. algorithms) want to see objects as if they were transient. They

do not want to “know” about persistency details

just want dereferencing semantics: ptr->object

want to be able to traverse relationships

Track HitsOnTrack* Hit*

Where is the knowledge about what is in memory or on disk, tape or WAN?

the I/O implications when one follows an association or dereferences a pointer?

TrackSet


43


Example architecture 1)

algo 2

algo 1converted shapepersistent

shapetransient

shapedisk

to memory

ODBMS knowledge

Forced conversion when bringing objects in - do not let application see “persistent object”

Advantages: application insulation, gain control over the persistency interfaces, future migration

Disadvantages: potential loss of ODBMS benefits - may need to implement on

transient side number of classes or converters

optimized for a specific purpose


44


Example architecture 2)ODBMS

knowledge

Allow application to access persistent objects directly

Reduces to architecture 1) if Conversions are used everywhere Paged-in from disk with non-trivial conversion/streamer

Advantages: ODBMS benefits – avoids reimplementing object manager, avoids extra classes where unnecessary

Disadvantages: must take care not to contaminate application with ODBMS knowledge, future migration

optimized for a specific purpose

algo 2

algo 1converted shapepersistent

shapedisk

to memory no transient

shape


45


ROOT and Objectivity can be used with either architecture

RDBMS & legacy systems require architecture 1)– needs transient shape

Who chose what? Arch 1 Arch 2 BaBar CDF (non-trivial streamers) D0 (done by D0om layer) ALICE ATLAS ? CMS LHCb (use converters)


46


What was the motivation?

What price was paid? (see the persistency session)


47

Application Meta InformationApplication Meta Information

This is run-time knowledge about classes

What is it? Can take a number of forms

ROOT dictionary – CINT & RTTI Database schema & indexes CORBA interface repositories Java reflection Automation or scripting by SWIG

Degree & ease of access to this information varies

What is the role of scripting? Complete scriptability requires rich meta-information


48

Application Meta InformationApplication Meta Information (2) (2)

Architects need to answer some questions What do you want to do with it?

What role does it play in the architecture?

How many separate repositories?

How complete should the information be?

Example from ROOT: the meta-information core is the key component of the architecture, used for I/O,

interpreter,

context-sensitive GUI, browsers,

class documentation


49

Application Application Meta Meta InformationInformation (3) (3)


50

User Interface (UI) and User Interface (UI) and VisualizationVisualization

It is important to be able to visualize and interact easily with objects in the application

One also must be able to run in batch mode

Are visualization and UI important? Yes!

Should they dominate the architecture? No

Questions Where should the knowledge about the UI be?

Can the architecture be designed with a detachable UI?

If the UI is detachable, what should be the back-bone communication channel? Decoupled in design? Scripting?


51

Implementation and Physical Implementation and Physical DesignDesign

Large software systems need to be decomposed into small and manageable components

A decomposition has significant implementation-related consequences Compile-time coupling, link-time dependencies, size of

executables, etc.

Physical design focuses on grouping classes together into packages Minimizing the number of package dependencies is a key

issue

Because of the consequences, system architects should be involved in the physical design


52

Implementation and Physical Implementation and Physical DesignDesign (2) (2)

Many HEP experiments find “guidance” in[Lakos] and [Martin]

Examples that help packaging succeed Abstract interfaces

Factory and strategy patterns [Gamma]

Dynamic loading

Component technology, e.g. CORBA or DCOM

Splitting into packages according to abstractness vs. instability

Should fall on line from abstract and stable/independent to concrete and unstable/dependent

Architectural Styles of HEP Experiments Presented by RD Schaffer CERN/EP and LAL Orsay LCB Workshop...

Documents

Transcript of Architectural Styles of HEP Experiments Presented by RD Schaffer CERN/EP and LAL Orsay LCB Workshop...