StudyGuidesPDF PMstudy Project Management Framework Framework
FairRoot framework
description
Transcript of FairRoot framework
FairRoot framework
Mohammad Al-Turany(GSI-Scientific Computing)
This Talk:
• Introduction
• Database redesign
• Building and testing system
• Online reconstruction and analysis o Transport data using message queuing technology
• Test beams supporto Support direct communication with DAQ Systems
• Summary
5/18/13 M. Al-Turany, ACAT 2013, Beijing 2
What is FairRoot Framework? And why it is needed?
5/18/13 M. Al-Turany, ACAT 2013, Beijing 3
http://fairroot.gsi.de
• Simulation-, Reconstruction-, and Analysis-Framework (not only) for the FAIR experiments
• 2003 started as 2 person project for the CBM experiment at FAIR• Long list of base and/or ready to use modules and base classes of
needed by the particle experiments
Initially designed and implemented for CBM collaboration at GSI with following goals :(2003)
• Decouple the detector simulation description from the simulation
engine o Use different simulation engines (Geant3, Geant4, …) with the same user
code (Virtual Monte Carlo)
• Reuse existing software and tools, use standards as much as
possible
• Code should run on all platforms
• The Framework should be o Easy to install
o Easy to use
o Should allow fast development cycles
o Flexible to easily change experimental setup
o Extensible for new developments5/18/13 M. Al-Turany, ACAT 2013, Beijing 4
Start testing the VMC concept for CBM
First Release of CbmRoot
MPD (NICA) start also using FairRoot
ASYEOS joined(ASYEOSRoot)
GEM-TPC seperated from PANDA branch (FOPIRoot)
Panda decided to join->FairRoot: same Base package for different experiments
R3B joinedEIC (Electron Ion Collider BNL)EICRoot
2011201020062004
FairRoot : Timeline
2012
SOFIA (Studies On Fission with Aladin)
5/18/13 M. Al-Turany, ACAT 2013, Beijing 5
ENSAR-ROOTCollection of modules used by structural nuclear phsyics exp.
2013
Design
5/18/13 M. Al-Turany, ACAT 2013, Beijing 613.03.13
Florian Uhlig ROOT Users Workshop, Saas Fee
Root
TE
ve
RO
OT IO
TG
eo
TV
irtu
alM
C
Cin
t
TTr
ee …
Pro
of
Geant3
Geant4
Genat4
_VM
C
Libraries
…
VG
M
FairRoo
t
…
Run
Manager
IO
Manager
Runti
me
DB
DB
In
terf
ace
Even
t D
ispla
y
MC
A
pplic
ati
on
Module
Dete
ctor
Task
Magneti
c Fi
eld
…
Even
t G
enera
tor
CbmRoot
PandaRoot AsyEosRoot
R3BRoot SofiaRoot MPDRoot
FopiRoot EICRoot
R3BRoot was presented today in Track 2:http://indico.ihep.ac.cn/contributionDisplay.py?contribId=43&confId=2813
long list of people
who have
contributed pieces of
code to FairRoot
since the project
started end of 2003
5/18/13 M. Al-Turany, ACAT 2013, Beijing 7
Core Team:• Mohammad Al-Turany SC• Denis Bertini SC• Florian Uhlig CBM
/ SC• Radek Karabowicz
PANDA / SC• Dmytro Kresan R3B/
SC• Tobias Stockmanns
PANDAPeople participated to major features:Ilse König HADESVolker Friese CBMOlaf Hartman PANDA
FairRoot Developers:
Student:Dennis Klein (finished 02.2013)Alexey Rybalchenko (EE)
Database Re-Design
5/18/13 M. Al-Turany, ACAT 2013, Beijing 8
FairRoot DB Design (Old)
9
FairRoot
Run Manager
RunTime Database
ASCII FileConfigurationparameters.
IO Manager
Root FileMC-pointsDigits, etc
Root FileConfigurationparameters.
Oracle
5/18/13 M. Al-Turany, ACAT 2013, Beijing
Oracle only solution base on server-side PL/SQL code
Generic parameter container
parameter object
different versions
nametype: Int_t, Float_t, Double_t, Char_t, Text_t, UChar_t, class type
stored as byte array (RAW or BLOB)number of values (single value or array)class versionstreamer info, root version for ROOT classesown version management
any class derived from TObject decoded in the analysis interface by ROOT streamer
5/18/13 M. Al-Turany, ACAT 2013, Beijing 10
FairRoot DB Implementation (Old)
HADES
FairRoot DB Design (Old)
11
FairRoot
Run Manager
RunTime Database
ASCII FileConfigurationparameters.
IO Manager
Root FileMC-pointsDigits, etc
Root FileConfigurationparameters.
Oracle
5/18/13 M. Al-Turany, ACAT 2013, Beijing
FairRoot DB extended
12
FairRoot
Run Manager
RunTime Database
ASCII FileConfigurationparameters.
IO Manager
Root FileMC-pointsDigits, etc
Root FileConfigurationparameters.
TSQLServerOracle
Postgresql
MySQL
DB Interface
5/18/13 M. Al-Turany, ACAT 2013, Beijing
Re-design Database interface based on ROOT Database Connectivity (RDBC) API which provides uniform interface to Oracle, MySQL, PgSQL • Database Interface in FairRoot using TSQLServer
– (MySQL, Oracle, PostGre,... )
• Allows multiple connections to Dbs at runtime• Adds Version Management
• Data type: Real and/or MC• Detector type• Date and Time Range
• Reduces SQL coding• Simple Predefined Table• Only Simple SQL used• Ultimately Generic Container
• Handles Write/Read access
5/18/13 M. Al-Turany, ACAT 2013, Beijing 13
FairDB interface Concept
GUI Report Queries
Offline Analysis Client
Monte Carlo Client
OnlineTrigger
RunControl
• Connect• SQL I/O• Versions• Caching• Rollback
LVL1
LVL2
LVL3DBMSDBMSDBMSDBMS
5/18/13 M. Al-Turany, ACAT 2013, Beijing
TSqlServer
C
++
, R
OO
T
SQ
L
FairDB Interface
D. Bertini
14
Version Management
Detector
Time
Version
Validity time range (UTC)
STS CALMVD CAL
MVD TEMP
RunID t
D. Bertini
155/18/13 M. Al-Turany, ACAT 2013, Beijing
Time
Version Management The Query process1. Context ( Timestamp,Detector,Version) is the primary key2. Context converted to unique SeqNo3. SeqNo used as keys to access all rows in main table4. System gives user access of all such rows
SEQNoContextmatched
SeqNO Col 1 …
Col nValidity Frame
900001020900001020
900001020
900001020
Bigtable a Distributed Storage System forStructured Data, Google inc. OSDI 2006
Auxiliary validity table
D. Bertini
165/18/13 M. Al-Turany, ACAT 2013, Beijing
Building & Testing system
5/18/13 M. Al-Turany, ACAT 2013, Beijing 17
Testing and building system
• CMake– Creates Makefiles (and/or project files) for different
platforms.
– Test support.
– Large user base assures support.
• CDash to handle data created with CMake– PHP framework
– MySQL database
• Both tools are open source.
5/18/13 M. Al-Turany, ACAT 2013, Beijing 18
If someone experiments with new features in his local working copy and wants to test them (experimental build)
5/18/13 M. Al-Turany, ACAT 2013, Beijing 19
Central SVN repository
1. Update (optional)
2. Configure, build and test
on local machine
3. Send results automatically
to central web page
5. Developer check results
4. Dashboard prepares and display results
If new code enters the central code base (continuous build)
5/18/13 M. Al-Turany, ACAT 2013, Beijing 20
Central SVN repository
1. Developer
commit code
4. Configure, build and test
on local machine
5. Send results automatically
to central web page
Dedicated test server
3. SVN triggers
test server
7. In case of problems Dashboard sends an E-mail to Developer and Administrator
6. Dashboard prepares and display results
2. Basic checks:
Style, etc Pass
Fail
Reject commit
From time to time a full check on all supported platforms should be done (nightly build)
5/18/13 M. Al-Turany, ACAT 2013, Beijing 21
Central SVN repository
1. Update
2. Send results automatically
to central web page
3. Dashboard prepares and display results
4. In case of problems Dashboard sends an E-mail to Developer and Administrator
5. In the morning Developers and Administrators check their mails and the dashboard. And the development cycle starts again
New Data transfer layer for FairRoot
5/18/13 M. Al-Turany, ACAT 2013, Beijing 22
Data transfer layer for FairRoot
• FairRoot must be extended to support the continuous
pipeline-processing scenario of the online analysis
• Tasks have to be put on different compute nodes
• Transport data using message queuing technology
• Scheme would also allow to enable concurrency in
FairRoot for offline analysis
• The long term plan is to have the same framework for
online and offline
235/18/13 M. Al-Turany, ACAT 2013, Beijing
Flexible data transport for the online reconstruction of FAIR experimentshttp://indico.ihep.ac.cn/contributionDisplay.py?contribId=72&sessionId=3&confId=2813
ZeroMQRoot (Event loop)
5/18/13 M. Al-Turany, ACAT 2013, Beijing 24
FairRootManager
FairRunAna
FairTasks
Init()Re-Init()Exec()
Finish()
FairMQProcessorTask
Init()Re-Init()Exec()
Finish()
ROOT Files, Lmd Files, Remote event server, …
Where we are going with FairRoot?
5/18/13 M. Al-Turany, ACAT 2013, Beijing 25
Online (Test beams) support
FairRoot AnalysisData Source
New Data sources and histogram server for online monitoring
5/18/13 M. Al-Turany, ACAT 2013, Beijing 26
DAQ(MBS, …)
Remote event server
Remote event server client
MBS client
Online monito
r
Online monito
r
Online monito
r
Online monito
r
Online monito
r
Histogram server
….. ....
Syste
m D
irecto
ry
5/18/13 M. Al-Turany, ACAT 2013, Beijing 27
New Data Source for FairRoot
• A client for the GSI remote event server is implemented
• Incoming data can be saved to root objects/files processed on the fly with offline code
Summary
• In the last year we moved from event by event simulation
to time based (pile up simulation)
• The database interface has been re-designed
• With the new source design, the offline code can run
online and can be used for online monitoring
• We are moving slowly toward massage passing concept.
5/18/13 M. Al-Turany, ACAT 2013, Beijing 28
Thanks!