Post on 27-Dec-2015
HENP software architecture and software engineering
• In our category, we have discussed mainly about tools and/or mechanisms to deal with– Large number of detector components– Large number of people– Multiple choices– Integrationat active development stages
• There are two different scales– One person trying to help 10 software developers– 10 people trying to organize development work by
100 people
Goals• For users, tools should just work; they should be
– Simple to use– Easy to understand– Error prone– Stable and reliable– Supported well
• Toward generic, non-experiment specific tools?– Standardization
• Common “data” format
– Coordination among experiments/laboratories– Good tools become being used (slowly) by more and
more people• Goodness itself doesn’t always make it happen, though
Talk/poster statistics
• LHC– ATLAS(6)– CMS(3)– LHCb(2)– ALICE(2)
• Other exp.– CDF(2)– Icecube, GLAST(2), CLEO-III(2), LC
• Accelerator/control– DESY(2), BaBar/PEP-II
• Non-exp. specific (Root, freeHEP)• LCG projects (SEAL, SCRAM)
Subcategories
• Event model, data definition(4)
• Package/project building, project management and release tool (7)
• Geometry/detector definition(3)
• Framework and library (8)
• Others (6)
Event model/data definition
• Storegate: a Data Model for the Atlas Software Architecture (ATLAS)
• Event Data definition in LHCb• Athena Data Dictionary and Description Langua
ge (ATLAS) (as I understand it), we are waiting for LCG versi
on of it• Event as an Object-Relational Database; Avoidin
g the Dependency Nightmare (CLEO-III)– Dependencies kept in one “library”
Object Relational Approach
Track:1Track:1 EM Shower:1EM Shower:1
Track:2Track:2
Hit:1Hit:1
Hit:2Hit:2
Hit:3Hit:3
Hit:4Hit:4
Hit:5Hit:5
Hit:6Hit:6
Hit:7Hit:7
Hit:8Hit:8
EM Shower:2EM Shower:2
EM Shower:3EM Shower:3
1:data:1
2:data:1
4:data:1
7:data:1
3:data:2
5:data:2
6:data:2
8:data:2
1:data:1,2
2:data:2
Describe N-M relations
Package/project building, project management, release tool
• OVAL: CMS testing tool• User-friendly tools for managing projects
(VCMT, GLAST)• Athena Startup Kit (ASK, ATLAS)
– User need to know only few commands when jumping into analysis
• GLAST Nightly Package Builds and Code Release System Tests
• BOA: framework for automated builds (CMS)• NICOS system of nightly build for distributed
development (ATLAS, LCG)• SCRAM (poster, LCG)
General Description
• Original goal : help to detect automatically the changes in the physics performances of the CMS software used for data production.
• Current features of the tool : compile the provided test programs, run them in specified runtime conditions (shell variables + auxiliary files), compare the output with references.
• User interface is CVS-like, configuration files are XML-like. It is written in PERL and can be configured for any build system (scram, make).
From OVAL
Configuration : OvalFile
<diffline expr="^OVAL:"><diffnumber expr="^OVAL NUM: .* is (.*)$" tolerance="5%">
<file name= ".orcarc">MaxEvents = 500Random:Seeds = 0 3</file>
<var name= "FEDERATION" value= "cmsuf01…">
<environment name= "pt15"> <var name= "DATASET" value= "eg_ele_pt15"> < program name= "Clusters.cpp"> < program name= "Electrons.cpp" args="-geo detailed"></environment>
<environment name= "flow"> < var name= "DATASET" value= "jm_minbias"> < program name= "EnergyFlow.cpp"></environment>
From OVAL
GLAST CHEP 03 March 24-28 2003
K.Young 12/22
Web Interface(Carrot, JAS,…,
Scripts)
Architecture of System TestsArchitecture of System Tests
Executive Script(Perl)
Suite of SimulationsRun on
Supported Platforms
Oracle DB
Root Files,Stdio Dumps Root Scripts
Root HistogramFiles
Detector/geometry definition
• Detector Description Framework in LHCb• GraXML – Modular Geometric Modeler (A
LICE)Waiting for LCG decision• Use of a generic identification scheme con
necting events and detector description in the ATLAS experiment– IDs for 100 million channels in the ATLAS det
ector!
14/18CHEP 2003 - 3/21/03Sébastien Ponce
Data Diagram
Structure
MaterialGeometry
DetElem GeometryInfo
CalibrationConditionAlignmentConditionReadoutCondition
MuonStation
EcalClusterCondition
Conditions
EcalClusterCondition
MuonStationAlignment
VeloReadout
Lvolume
Pvolume
Solid
Box
Sphere
*
Points to
Inherits fromResolved on demand Material
Isotope Mixture
Element
15/18CHEP 2003 - 3/21/03Sébastien Ponce
Transient Store
● Tree-like structure● Items identified by a logical name
● Load/update on demand● Automatic update when new event
Geometry
LvolumeLvolumeLvolumeLvolume GeometryDb
DetectorData Service
PersistencyService
Algorithm
Ask forObject
Retrievepointer
Checkpresence LoadAsk creation
Cnv
16/18CHEP 2003 - 3/21/03Sébastien Ponce
CondDB Implementation
Geometry
LvolumeLvolumeLvolumeLvolume
DetectorData Service
PersistencyService
Algorithm
Ask forObject
Retrievepointer
Checkpresence Ask creation
Cnv
ConditionDb
ConditionService
Ask for data Retrieve themfrom Database
Abstract interface
● Additional service of the framework● Independent of the type of data contained (XML)● Based on an abstract interface with several
possible backends (Objectivity, ORACLE, MySQL)
Framework/library
• Root: Status and Future (See slides by yourself)• The FreeHEP Java Library• SEAL: Common core libraries and services for LHC appli
cations• The AliRoot framework (ALICE) (See slides by yourself)• Dynamically configurable system for fast simulation and r
econstruction for CMS (FAMOS)• Java Physics Generator and Analysis Modules• The Virtual Monte Carlo (ALICE)• Reconstruction and Analysis on Demand: A Success Sto
ry (CLEO-III)
Mark Dönszelmann, SLAC CHEP, La Jolla, 24-28 March 2003
FreeHEPJava Library
HEP Specific Components
Physics package 3-vector, 4-vector’s and utilities Jet Finding, Event Shape routines Diagnostic Event Generator
HEP IO Access to a number of common HEP
formats: StdHEP MCFIO HBook Root
HEP3D Extensions to Java3D specific for Ge
ant4YaPPI – Yet another Particle Property Interface
XML Particle Property Database
AIDA - Abstract Interfaces for Data Analysis
Interfaces and JAIDA - Java reference implementation
AIDAJNI – C++ wrapper around AIDA More on this in Max Turri’s talk on T
uesdayHepRep - HEP Representables for Event Display
Interfaces and JHepRep - Java reference implementation
HepRepJNI – C++ wrapper around HepRep
see several other talks and postersFuture
Plot Widget (JAS)
Mark Dönszelmann, SLAC CHEP, La Jolla, 24-28 March 2003
FreeHEPJava Library
General Components
Application Framework General framework for Java applicati
ons using plugin and service technology.
AID – Abstract Interface Definition Compiler to make C++ headers and
Java Interfaces from aid files.Graphics3D
Non-Java3D implementation of 3D graphics, for WIRED.
FreeHEP IO Extensions to the JavaIO system to h
andle specific encodings, ASCII85, …
PSViewer PostScript Viewer to embed viewing
of EPS in applications, or for previews.
Swing ExtensionsFreeHEP Tools
JNeeds – java package dependency Ant – extensions to the build system
VectorGraphics Low level Input and Output of binary
vector graphics formats (CGM, EMF, PDF, SVG)
Vector output files, extension to Java2D
XML XMLIO – classes to store/restore st
ate of an applicationFuture
RecordLoop/EventLoop (IceCube)and more…
Domain Coverage
EventGeneration
Core Services
Dictionary
Whiteboard
Foundation and Utility Libraries
DetectorSimulation
Engine
Persistency
StoreMgr
Reconstruction
Algorithms
Geometry Event Model
GridServices
I nteractiveServices
Modeler
GUIAnalysis
EvtGen
Calibration
Scheduler
Fitter
PluginMgr
Monitor
NTuple
Scripting
FileCatalog
ROOT GEANT4 DataGrid Python Qt
Monitor
. . .MySQLFLUKA
EventGeneration
Core Services
Dictionary
Whiteboard
Foundation and Utility Libraries
DetectorSimulation
Engine
Persistency
StoreMgr
Reconstruction
Algorithms
Geometry Event Model
GridServices
I nteractiveServices
Modeler
GUIAnalysis
EvtGen
Calibration
Scheduler
Fitter
PluginMgr
Monitor
NTuple
Scripting
FileCatalog
ROOT GEANT4 DataGrid Python Qt
Monitor
. . .MySQLFLUKA
SEAL
Simulation/Reconstruction Chain
MC 4-vector
SimHit
RecHit / Digi
Analysis object
FAMOS/ORCA
Reconstruction: ORCA
FAMOS/ORCA
Simulation: OSCAR
FAMOS/OSCAR Start with fastest modeAIM: 1000 times faster
MC 4-vector
Analysis object
100-200 sec
1-10 sec
10-100 sec
DetectorSimulationEvent Generators
Interoperability
Pythia
Pandora
Whizard
EventSelection
LCD FastMC
TESLASimDet
JLCQuickSim
AnalysisCode
Java
Fortran
C, C++
VMC Concept
UserCode
VMC
Geant4 VMC
ParticlesHits
Geant4
Geant3
Output
Fluka VMC
Fluka
Input
Geant3 VMC
Other
• Revision control in the Grid Era – the unmet challenge (CDF)
• Commissioning the CDF Offline• Concrete uses of XML in software development
and data analysis (IceCube)• Improving the Security and Performance of the B
aBar Detector Controls System (poster)• Concepts for Integrating Information systems (po
ster, DESY)• UML and explorative prototyping for developing i
nteractive applications (DESY)
Ominous clouds in the LHC era
• The detector components, computer systems, software, and people are– Complex– Inter dependent– Rapidly changing– Globally distributed
• How can we be dealing with the situation?• How can we integrate all software to work
together?
Trends
• XML is coming (+ RDMS is back)– Describe, describe, describe
• Plug-ins– Handle multiple choices (how lucky!)
• Nightly builds– Just to see if anything has broken yesterday
• On demand (just in time?) reconstruction• Ever expanding roots?
– VMC, AliEn, Geometry ,(PROOF, AliEnFS on other sessions)
KEKB upgrade strategy
Present KEKB L=1034
2002
03 04 05 080706 09 10 11
L=103
5
L~1036
dt =500fb1~1PB/year
One year shutdown to: replace vacuum chambers double RF power upgrade inj. linac g C-band
larger beam currentsmaller y*long bunch optioncrab crossing
ILER=1.5A2.6A
ILER=9.4A
ILER=20A
Constraint:8GeV x 3.5GeVwall plug pwr.<100MWcrossing angle<30mrad
dt =3000fb1
~10PB/year beforeLHC!!
My personal favorites• For Belle, when I go home, I would investigate m
ore about– OVAL, GraXML, FreeHEP Java library– Root and related works
• For Super KEKB planning and work,– Virtual Monte Carlo and other MC frameworks– Definition tools– Nightly build tools– On demand reconstruction, analysis processing
• I am eager to see progresses of the LCG projects and adopt