Summary of Software Components and Libraries Track Computing in High Energy and Nuclear Physics...

Summary of Software Components

and Libraries Track

Computing in High Energy and Nuclear Physics

13-17 February 2006, T.I.F.R. Mumbai, India

P. Hristov, L. Moneta

CERN

P. Hristov, L. Moneta 2

Summary✦ Review of LCG Application area software✦ Data Management

✦ Historical overview (J. Shiers)✦ ROOT I/O✦ Relation Databases (4 talks)✦ Condition Databases (3 talks)

✦ Reflection in C++ (2 talks) ✦ Math and Statistical libraries and tools (5 talks)✦ Geometry representation (3 talks) ✦ Graphics and Visualization (6 talks)✦ Visualization for quantitative analysis in nuclear physics (S.

Niranjani)✦ MC Generators services (2 talks)✦ Integrated development environment (Eclipse)✦ Total of 32 talks (2 missing, no-show )


LCG Application Software

Common Application software for the experiments (P. Mato)

Core

PluginMgr Dictionary

MathLibs I/O

Interpreter

GUI 2D Graphics

Geometry Histograms Fitters

Simulation

Foundation Utilities

Engines

Generators

Data Management

Persistency

FileCatalogFramework

DataBase

Grid Services

Batch

Interactive

OS binding

3D Graphics

NTuple Physics

Collections

Conditions

Experiment Frameworks

Simulation Program Reconstruction Program Analysis Program

Event Detector Calibration Algorithms

Software Domain DecompositionGoalsMinimize duplicationsSoftware re-useSimplify use of external librariesintegration elements (dictionaries, scripting, plug-in management )LCG AA Projects: SPI, ROOT, POOL, SIMU

seek confirmation that all products developed are used by the experiments crucial to keep strong contacts with experiments

3

4P. Hristov, L. Moneta,

ROOT I/ORecent developments of ROOT I/O and Trees (P. Canal)exhaustive list of new features introduced recently in ROOT

Main focus: Consolidation (Thread Safety)Generic Object Reference support

User defined reference objects supported byUser defined reference handlers (proxies)

I/O Improvements

n STL collectionsn Data compression using reduced precisionn Alternatives to default constructorsn Other I/O improvements

TTree extensions

n New Features : Circular TTree and ASCII file

n Fast Mergingn Indexing of TChainsn TTree Interface enhancementsn TRef and pool::Referencen Browsing

Important features requested by the experiments are implemented


Relational DataBases✦ CORAL (I. Papadopoulos)

✦ a software system for vendor-neutral access to relational databases

coral::ISchema& schema = session.nominalSchema();coral::TableDescription tableDescription ;tableDescription.setName( “ ” );tableDescription.insertColumn( “I”, “long long” );tableDescription.insertColumn( “X”, “double” );schema.createTable ( tableDescription);

( I BIGINT,X DOUBLE PRECISION)

CREATE TABLE “ ” ( I NUMBER(20),

Example 1: Table creation

Oracle MySQL

C++, SQL-free API (I)

Component Architecture

RDBMS Implementation(oracle)

RDBMS Implementation(sqlite)

RDBMS Implementation(frontier)

RDBMS Implementation(mysql)

Authentication Service(xml)

Authentication Service(environment)

Lookup Service(xml)

Lookup Service(lfc)

Relational Serviceimplementation

Monitoring Serviceimplementation

Connection Serviceimplementation

CORAL Interfaces(C++ abstract classes

user-level API)

CORAL C++ types(Row buffers, Blob,Date, TimeStamp,...)

Plug-in libraries, loaded at run-time, interacting only through the interfaces

Client Software

CommonImplementation

developer-levelinterfaces

POOL (G. Govi)✦CORAL integrated in the software of LHC experiments (CMS, ATLAS and LHCb)✦directly (i.e. on-line applications)✦indirectly (COOL, POOL)


Relational DataBases (Experiments View)

✦ ATLAS✦ Distributed DataBase Services Client library (A. Vaniachine)

✦ contribution to CORAL ✦ A Flexible distributed event level metadata system (D. Mallon)

✦ tag database for rapid and efficient event selection✦ based on POOL collections

✦ implemented in ROOT and relation database back-end✦ only user of POOL of that functionality

Rome Workshop Event Tag Production

AOD Files(1000 evts

each)

Root Collections(1 Per AOD File)

Dataset n

Dataset 2

Dataset 1

Table 1

Table 2

Table n

Event Tag Database(Tier 0)

Replica Databases(Tier 1)

RelationalDatabase

Site 1

Site 2

Site n

Collection vs. Direct File Access

Analysis

Result

UserSingle Large File

User

Collection

Analysis

Result

Database

Multiple Files

File Catalog


Conditions DataBase✦ DataBases to store time varying

data✦ COOL : Development and

Deployments (A. Valassi)✦ holds condition data for

reconstruction and analysis✦ access data from PVSS, local file

catalog (LFC) and bookeeping✦ implementation in ORACLE, MySQL

and SQLiteC++ Relational Access (CORAL)

OracleAccess

MySQLAccess

SQLiteAccess

OracleOCI

MyODBCAPI

SQLiteAPI

OracleDB

MySQLDB

SQLiteDB

Relational Database Deployment

and Data Distribution (3D)

Time-varyingmulti-version

data (~offline)

Time-varyingsingle-versiondata (~online)

Conditions DB Access (COOL)

Experiment conditions datacommon software and conventions

Experiment frameworkSub

detector#N

Subdetector

#2

Subdetector

#1

✦Now in deployment phase (ATLAS and LHCb)✦fully integrated in experiment frameworks✦Benefits from other LCG projects ✦CORAL, SEAL and 3D project✦Performance studies (poster 338 )


Conditions DataBase (2)✦ Multiple persistency technologies for the BABAR Condition

DataBase✦ migration is in progress to move away from Objectivity

✦Conditions DataBase of LHCb (M. Clemencic)✦conditions DB conversion service of Gaudi✦based on COOL


C++ Reflection✦ Reflex (S. Roiser)

✦an important step in the consolidation of C++ Reflection Software

✦JIL (D. Lawrence)✦tool to make object persistent (serializer)✦support for legacy file formats and allow metadata✦satisfy only specific requirements (GlueX experiment)

10


✦ New Developments of ROOT Mathematical Libraries

✦ new Vector package (3D and 4D)

✦ SMatrix (for small matrices with fixed size)

Mathematical and Statistical Libraries

✦Fitting and Minimization ✦Minuit2 (C++)✦Linear Fitter✦Robust Fitter✦SPlot (unfolding)


1

✦ Repository for Physics Statistical Software ✦ many statistical code available on individual base

✦Loose control from moderators✦Web site has been established (with policies for submitting)✦Next: classification/validation, extending the scope (adding extensions, enhance portability, etc..) ➡Added value is from users (submit code, provide comments and feedback)

Mathematical and Statistical Tools (1)

12


✦ Power of Goodness of Fit tests (M.G. Pia)✦ test statistical power of the

various tests available in the toolkit

✦ classify tests as a function of distribution characteristics: ✦ skewness and tail-weight

✦ presented results for un-binned distributions ✦ no clear winner✦ interesting results

✦ recommendation: ✦ first classify distributions✦ choose most appropriate tests

✦ publication with results in progress


➡valuable tool for the HEP community


3

✦ StatPatternRecognition (I. Narsky)✦ C++ package for multivariate classification of event data

✦ used in BaBar data analysis

✦Boost Trees give significant improvements with respect to other methods (Neural network)✦it would be very useful for LHC data analysis✦package has some external dependencies (on some BaBar software, CLHEP, CERNLIB, ROOT) ✦would be nice if could be used easily from ROOT (like RooFit)


14


✦ Operation Research in HEP (A. De Min)✦ Examples of application of

Linear Programming techniques in high energy physics ✦ Linear Programming:

✦ minimize linear function subject to some constraints

✦ used a lot in logistic and schedule optimizations (airlines)

✦ possible application in HEP: ✦ patter recognition and

outliers removal✦ detector alignment✦ analysis optimization

✦ various tools (library) available✦ commercially and as open

source packages



5

PAX ✦ PAX Toolkit (Physics Analysis Expert) (S. Kappler)

✦valuable the possibility to define set of competitive physics processes (signal and background)✦probably missing some functionality (kinematic constraints)

15


6

Geometry Modeling and Description (1)

✦ ROOT Geometry✦ New features in ROOT geometrical modeler for representing

non ideal geometries (A. Gheata)

Misalignment for Misalignment for TGeoTGeo

•• TGeo TGeo = ROOT geometrical modeller= ROOT geometrical modeller– Provides a CSG model based on 20 primitive solid types (basic

shapes), allowing Boolean combinations– Provides navigation functionality used during particle transport– Provides several tools for geometry checking and visualization

•• The model has a hierarchical structureThe model has a hierarchical structure– An object (volume) is also a container, having other volumes

inside positioned in a relative way (nodes).• We want to be able to move some objects w.r.t their ideal position• Moving some volume will naturally move all its content

•• Components of the model are generally replicatedComponents of the model are generally replicated– Changing the position of a node will change the position of all

replicas …• Not really what we want…• An object that has to be moved needs to be decoupled

add misalignments in perturbative approach

Alignment data

DetectorBuilder(ideal)

geom.rootDetectorBuilder

(alignment aware)

geom.root

Tracking,fitting

TGeoVolume TGeoVolumeAssembly

B

A

A

C

Overlap !

Volume assembliesVolume assemblies

✦provided support for complex mis-alignment of geometrical elements✦access to same geometry in simulation and reconstruction✦benefits also from the TGeo visualization capabilities ✦necessary to check for overlaps and extrusions (TGeo functionality)

17


✦can be used as primary geometry description✦proved to be very useful to have an application-independent geometry description language ✦geometry can be moved between applications

✦ GDML : Geometry Description Markup Language (W. Pokorski)

✦ describe GDML and its

✦ Using XML in the Virtual MC framework (M. Potekhin)✦ used to describe STAR geometry✦ based on a different description

(AGDD) 17

~19000 physicalvolumes

CMS detector: G4->GDML->ROOT

Geometry Modeling and Description (2)

18


Graphics and Visualization (1)

✦ ROOT 3D Graphics (R. Brun)✦ use of OpenGL for displaying

3D objects✦ profit of GL features:

✦ rendering ✦ 3 draw style, vector output,

support for composite shapes)

✦ cameras ✦ perspective and orthographic

cameras✦ manipulators, guides, clipping

✦ GL viewer✦ GL in a ROOT Pad✦ good performances

✦ can take advantage of hardware acceleration

✦ next step is to have animations ✦ follow track/shower for event

display➡significant progress in last years

19


Graphics and Visualization (2)

✦ V-Atlas (V. Tsulaia)✦ event visualization based on

OpenInventor✦ multifunctional event display✦ extensively used as debugging tool✦ going to be used as Event Display

at ATLAS Point 1➡ complementary to Atlantis

✦ AliEVE : Alice Event Visualization Environment (M. Tadel)✦ after using VSD and GLED use now

CINT scripts and ROOT native GUI✦ visualize using ROOT GL viewer➡ completed prototype cycle

(investigation/analysis/development) ✦ GLED (M. Tadel) http://www.gled.org

✦ A ROOT based framework for dynamic visualization and distributed computing

✦ used also for visualization of ALICE distributed analysis

http://www.gled.org/

20


Physics Work Environment

✦ Eclipse (W. Lavrijsen)✦ physicist work environment✦ useful tool for multi-language

environment as ATLAS✦ help in tools integration

✦ can easily interface to external tools (i.e. cmt for building)

➡Some concerns on heavy resource usage


1

Conclusions✦ Significant number of presentations from LCG projects

✦ 7 of the 32 presentations

✦ consolidation of the common software

✦ projects are moving from active development to deployment

✦ Adoption of common software from LHC experiments

✦ success will be defined by their validation and usage

✦ Large use of relational databases through different interfaces

✦ being integrated with existing software systems

✦ Various talks on mathematical and statistical libraries and tools

✦ Some interesting new developments (statistical tools for event classification)

✦ Substantial progress in graphics and visualization software


2

Acknowledgments

✦ Thanks to all the speakers✦ very interesting content ✦ high quality presentations

✦ Many thanks to the conference organizer

Extra Slides

24

P. Hristov, L. Moneta,

Data ManagementEvolution of DataBase in HEP (Jamie Shiers) Evolution of Data Base in HEP from the 92 CHEP conference until now

CHEP 92 – the Birth of OO in HEP?

ß Wide-ranging discussions on the future of s/w development in HEP

ß A number of proposals presenting – including ZOO(I preferred “OOZE”) –

leading to:

ß RD41 – MOOSE

® The applicability of OO to offline particle physics code

ß RD44 – GEANT4

® Produce a global object-oriented analysis and design of animproved GEANT simulation toolkit for HEP

ß RD45 – A Persistent Object Manager for HEP

ÿ (and later also LHC++ (subsequently ANAPHE))

ß ROOT

Those Questions again…

1. Should we buy or build database systems for our calibration and book-keeping needs?

ß It now seems to be accepted that we build our calibration & book-keeping systems on topof a database system.

ß Both commercial and open-source databases are supported.

2. Will database technology advance sufficiently in the next 8 to 10 years to be able toprovide byte-level access to petabytes of SSC/LHC data?

ß We (HEP) have run production database services up to the PB level. The issues related tolicensing, and – perhaps more importantly – support, to cover the full range of institutesparticipating in an LHC experiment, remain.

ß Risk analysis suggests a more cautious – and conservative – approach, such as that currentlyadopted.(Who are today the concrete alternatives to the market leader?)

25


Algorithms(2)✦ Kalman filter for track based

alignment (E. Widl) ✦ restrict matrix to detector units

which have significant correlations with the ones in the current track

✦ propose algorithm to control list of detector units with a threshold to avoid too large values (tune threshold on real track)

26


MC Generators for LHC✦ MonteCarlo Generator Services for LHC (M. Kirsanov)

✦ guarantee the generator support for LHC✦ GENSER (Generator Service Library)

✦ provide various level of testing✦ portability to new compilers (gcc4) ✦ collaborate with MC authors to provide support

✦ standardize on a common MC truth format (HEPMC)✦ MCDB (data base) keep track of full generation chain, storage for

shared event ✦ CEDAR (B.M. Waugh)

✦ archive of HEPDATA✦ tools for validating and tuning models (PDF’s)✦ XML data format

✦ HZTool✦ includes data from various expts✦ takes data from HEPEVT✦ have utilities

✦ RIVET ( OO to replace HZTool)✦ make use of existing libraries✦ newer interfaces HEPMC or AIDA✦ move to use GENSER

Summary of Software Components and Libraries Track Computing in High Energy and Nuclear Physics...

Documents

Transcript of Summary of Software Components and Libraries Track Computing in High Energy and Nuclear Physics...