Shine Offline Framework Basics
description
Transcript of Shine Offline Framework Basics
![Page 1: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/1.jpg)
Darko Veberic NA61 meeting, CERN june 2011 1
Shine Offline FrameworkBasicsDarko Veberič
Laboratory for Astroparticle Physics
University of Nova Gorica, Slovenia
![Page 2: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/2.jpg)
Darko Veberic NA61 meeting, CERN june 2011 2
Evolution of Design Concepts
• Past:Client-Server: DSPACK – NA49 clients (shm) (historical parallel in OS: X11 – window applications [network transport])
• Present:Single-process – shared libraries: Shine Offline Framework (easier debugging, SMP memory saving)
• Future:Multi-core parallelism (OpenMP, GPU: CUDA, OpenCL [some astroparticle experiments, future CERN])
![Page 3: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/3.jpg)
Darko Veberic NA61 meeting, CERN june 2011 3
Shine: Organization• Frameworkevent (object persistence), detector & mag. field classes, central configuration, coordinate system registry, data-source managers
• Event I/Oraw event (BOS, DS), SHOE, NDST (export)
• Modulesreading/writing, calibration, reconstruction, simulation
• NA49clients, interfaces
• EventBrowser
• Validation(of event structures, of modules)
• Utilitiesmisc. helpers, STL (containers), Math, Geometry
• Documentation(I/O, modules, reconstruction, physics)
• Build systemCMakeModules, CMakeLists
![Page 4: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/4.jpg)
Darko Veberic NA61 meeting, CERN june 2011 4
Shine: Current Status
Lines of code:
118k NA49 (C 73%, F 19%, C++ 3%)23k Utilities12k Framework7k EventBrowser3k EventIO0.5k Modules
Total:
C 53%C++ 29%fortran 14%
![Page 5: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/5.jpg)
Darko Veberic NA61 meeting, CERN june 2011 5
Shine vs. Auger
Lines of code:
Shine Auger
118k NA4923k Utilities 28k12k Framework 51k7k EventBrowser 38k (ADST)3k EventIO 20k0.5k Modules 97k
163k Total 234k
![Page 6: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/6.jpg)
Darko Veberic NA61 meeting, CERN june 2011 6
Overview
Run Control
Central ConfigManager Registry
NA61:
TPC
TOF
Trigger
PSD
NA61:
TPC
TOF
Trigger
PSD
Mag. Field
![Page 7: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/7.jpg)
Darko Veberic NA61 meeting, CERN june 2011 7
Framework• Event structures
• Detector description, mostly lazy update of leafs
• singly-linked treesinitially with pointers, later with ShadowPtr (cleanup, propagation of constness, deep copy; compiler-generated ctor, dtor, copy-ctor, assign)
• strict separation of framework classes and (physics) code
• one class per header file
• non-deletableprocessing only adds things, Has/Make/Get interfaces
• prevent accidental processing of a copyprivate ctor/dtor
• data-source Managersconfig defined queues of Managers; easy source replacementwrappers for MySQL, SQLite, Xerces
• central configuration: Detector, Managers, Modules
• Registry of Random number streams, seeding conf
![Page 8: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/8.jpg)
Darko Veberic NA61 meeting, CERN june 2011 8
Framework: Data Managers
SQLiteMySQL Xerces
TPC
VTPC
MTPC
GTPC
TOF
BPD
Trigger
PSD
Magnetic field
Drift velocity
etc.
![Page 9: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/9.jpg)
Darko Veberic NA61 meeting, CERN june 2011 9
Framework: Configuration
• XML
• run-time XML Schema validation: XSD
• automatic conversion of physical quantities into internal units
<energy unit=”GeV”> 137.23 </energy><position unit=”cm”> 14.3 </position>
• default configuration check-summed, modifications reported in run logs
• full configuration saved with data
![Page 10: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/10.jpg)
Darko Veberic NA61 meeting, CERN june 2011 10
Event I/O
Currently supports:
• raw BOS reading
• DS reading
• SHOE read/write (ROOT persistent Event)
![Page 11: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/11.jpg)
Darko Veberic NA61 meeting, CERN june 2011 11
Simulation
(Geant4)
(interaction models)
...
missing for now
![Page 12: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/12.jpg)
Darko Veberic NA61 meeting, CERN june 2011 12
“NDST”
• summary files (ROOT) will have Event objects (no persistent/transient separation)
• needed for off-offline analysis
• GUIevent/detector browser, run configuration inspector, simple selection tools
• mega/mini/micro NDSTachieved by switching off increasing level of Event details (branches)reconstruction results only; simulation intermediaries included
![Page 13: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/13.jpg)
Darko Veberic NA61 meeting, CERN june 2011 13
Modules
• By detector: BPD, Trigger, TPCs, TOF, PSD etc.
• calibration, reconstruction, simulation (Geant4)
• readers/writers: native event/detector, NDST
• modules replaceableimplementing several alternative or competing approaches is easy
• (if needed) Modules communicate (only) through the Event structure
![Page 14: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/14.jpg)
Darko Veberic NA61 meeting, CERN june 2011 14
Modules: Run Control
• rudimentary control-language implemented in XML
<sequenceFile>
<enableTiming/>
<moduleControl>
<loop numTimes="unbounded">
<module> EventFileReaderSG </module>
<module> EventCheckerSG </module>
<module> BPDCalibratorSG </module>
<module> BPDEventSelectorSG </module>
<module> TPCCalibratorSG </module>
<module> TOFCalibratorSG </module>
<module> TPCClusterFinderFG </module>
<module> TPCTrackFinderWG </module>
<module> TOFTimeFinderDG </module>
<module> TPCTrackPlotterSG </module>
<module> ShineFileExporterSG </module>
</loop>
</moduleControl>
</sequenceFile>
intermediate Event states can be streamed at any time by inserting Writers
![Page 15: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/15.jpg)
Darko Veberic NA61 meeting, CERN june 2011 15
Modules: Interface
class ModuleName : public fwk::VModule {
public:
fwk::VModule::EResultFlag Init();
fwk::VModule::EResultFlag Process(evt::Event& event,
const utl::AttributeMap& attr);
fwk::VModule::EResultFlag Finish();
REGISTER_MODULE(”ModuleName”, Modulename, ”$Id$”);
};
![Page 16: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/16.jpg)
Darko Veberic NA61 meeting, CERN june 2011 16
NA49 Clients as Modules<sequenceFile>
<enableTiming/>
<moduleControl>
<loop numTimes="unbounded">
<module> EventFileReaderSG </module>
<module> ClientInitializerSG </module>
<module config="many"> MiniModuleSG </module>
<module config="vdinit"> MiniModuleSG </module>
<module> VdriftCorrModuleSG </module>
<module config="VTPCs"> Dipt256NewModuleSG </module>
<module config="MTPCs"> Dipt256NewModuleSG </module>
<module config="VT1"> EdistoModuleSG </module>
<module config="VT1"> VtNcalcModuleSG </module>
<module config="VT2"> EdistoModuleSG </module>
<module config="VT2"> VtNcalcModuleSG </module>
<module config="MTL"> EdistoModuleSG </module>
<module config="MTR"> EdistoModuleSG </module>
<module config="ALL"> ResCorrModuleSG </module>
<module config="TrckM1"> MtracModuleSG </module>
<module config="I2RTrck"> MtracModuleSG </module>
<module config="MTPCs"> MpatModuleSG </module>
<module config="VT-2"> MpatModuleSG </module>
<module> NA49ToShineConverterSG </module>
<module> ShineFileExporterSG </module>
</loop>
</moduleControl>
</sequenceFile>
current module sequence with wrapped clients
DSPACK assimilated into the same shared library as DSHACK, runs in local mode (still needs shm)
![Page 17: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/17.jpg)
Darko Veberic NA61 meeting, CERN june 2011 17
Client-Module Configuration
<ResCorrModuleSG>
<commandLine config="ALL">
-vt1_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vt1.corr
-vt2_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vt2.corr
-mtl_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/mtl.corr
-mtr_chris [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/mtr.corr
-p [DOCUMENTATION_PATH]/SampleNonEventData/RES_CORR_ZG/vdrift_2007.txt
</commandLine>
</ResCorrModuleSG>
example: XML configuration of RES_CORR_ZG client,run as Shine module
![Page 18: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/18.jpg)
Darko Veberic NA61 meeting, CERN june 2011 18
Utilities
• XML Reader (Xerces library wrapper)
• SI units, internal units definition
• Mathematics, Statistics & Physics functions
• Traces, trace algorithms
• Linear Algebra, Geometry
• ODE Runge-Kutta Integrator, Mag. Field Tracker
• MD5 for configuration checksums
• custom exceptions
• logging
• special template libraryspecialized event & detector containers, shadow pointer, Meyers singleton, counted objects, safe bool cast etc.
![Page 19: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/19.jpg)
Darko Veberic NA61 meeting, CERN june 2011 19
Utilities: Geometryone preferred coordinate system: NA61
Point(1.7*m, 2*cm, 3*cm, csMTPCR);
Vector(10*cm, 1*cm, 3.3*mm);
Local detector-aligned coordinate system
but also:
![Page 20: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/20.jpg)
Darko Veberic NA61 meeting, CERN june 2011 20
Build System
• CMakeseems to be preferred by large projects, multiplatform
• Shapefor external dependencies and production releasessmall dependency resolution engine; python
• buildbotsautomated compile, test, validation after each commitseveral linux flavors, email blame notifications
![Page 21: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/21.jpg)
Darko Veberic NA61 meeting, CERN june 2011 21
Validation• Unit testing: CppUnit
• I/O validation (test ROOT schema evolution)
• detect changes in reconstructed quantities on example datasets
![Page 22: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/22.jpg)
Darko Veberic NA61 meeting, CERN june 2011 22
Main Design Principles
• Abstraction of external dependencieswrappers for geometry (ROOT), XML reader (Xerces); exceptions: STL, boost, Modules (authors responsible for compatibility with new versions)
• const-correctness
• prefer static over dynamic polymorphism (templates)
• no pointers in interfaces and strongly discouraged elsewhere (references only)
• no code duplication
• modified Google+Taligent coding styleCamelCase, naming, strict enforcement (everywhere except Modules)written down in twiki with extensive examples
• Doxygen documentation
• bug tracking with savane (previously known as savannah)
![Page 23: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/23.jpg)
Darko Veberic NA61 meeting, CERN june 2011 23
End
![Page 24: Shine Offline Framework Basics](https://reader036.fdocuments.in/reader036/viewer/2022062803/568148e0550346895db5fa3b/html5/thumbnails/24.jpg)
Darko Veberic NA61 meeting, CERN june 2011 24