Muon dataflow

Edward Moyse

Introduction

• Turning the raw data / detector response into Nobel prizes, is a very complicated process, involving a lot of software

• … which of course means lots of potential bugs!

• In this talk, I will try to give an overview of the dataflow, so that you know roughly where to look in the case of problems

• I’ll also explain a little bit about some of the details of the sub-processes…

• Too much to cover everything!

• One thing to note: underlined words are links - this talk is supposed to be a starting point…

�2

DATAFLOW

Overview

�3

Bytestream

PRD

Roads

Segments MS Track SA Track

CB Track

ID Track

• Bytestream - raw bits and bytes from the detector readout

• RDO - Raw Data Object (object representation of the byte stream)

• SDO - Simulated Data Object

• PRD - Prepared Raw Data (after some calibration)

Sim Hits

RDO

SDO

Glossary

Digits

Simulation

(Usually) real data

Muon track Muon track extrapolated to the IP

DATAFLOW

Overview - links to (some) EDM packages

�4

Bytestream

PRD

Roads

Segments MS Track SA Track

CB Track

ID TrackSim Hits

RDO

SDO DigitsMuonDigitContainerMuonSimData

MuonSimEvent

MuonRDO

MuonPrepRawData

MuonSegment TrkTrack

https://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonDigitContainer/trunk/MuonDigitContainerhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimData/trunk/MuonSimData/?#ada9ea6600c4c4e225be09a813a68677bhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimEvent/trunk/MuonSimEvent/?#ac8b825360c824143e6e97bf4f4d64e1dhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonRDO/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonPrepRawData/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonSegment/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/Tracking/TrkEvent/TrkTrack/trunk

DATAFLOW

Overview - links to (some) EDM packages

�5

Bytestream

PRD

Roads

Segments MS Track SA Track

CB Track

ID TrackSim Hits

RDO

SDO DigitsMuonDigitContainerMuonSimData

MuonSimEvent

MuonRDO

MuonPrepRawData

MuonSegment TrkTrack

https://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonDigitContainer/trunk/MuonDigitContainerhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimData/trunk/MuonSimData/?#ada9ea6600c4c4e225be09a813a68677bhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimEvent/trunk/MuonSimEvent/?#ac8b825360c824143e6e97bf4f4d64e1dhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonRDO/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonPrepRawData/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonSegment/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/Tracking/TrkEvent/TrkTrack/trunk

Overview

�6

`

DATAFLOW

Bytestream

PRD

Roads

Segments MS Track SA Track

CB Track

ID TrackSim Hits

RDO

SDO Digits

RECONSTRUCTIONDIGITIZATION

BYTESTREAM DECODING

DATA PREP See Niel’s talk

Persistency

Configuration

• The main configuration (as is usual for Muon SW) is handled in MuonRecExample/MuonRec_jobOptions.py

• MuonRecFlags contains switches to turn off and on various parts of Reco/Dataflow.

!

• For users, parts are controlled in Reco_trf automatically by inputs and outputs

• i.e. doing Reco_trf inputRDOFile=myRDO.pool.root outputESDFile=myESD.pool.root will run normal muon reco

• From MuonRec_jobOptions.py, many other jobOpts files are called:

!

• For dataflow in particular, many live inside MuonCnvExample:

�7

https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonReconstruction/MuonRecExample/trunk/share/MuonRec_jobOptions.pyhttp://atlas-computing.web.cern.ch/atlas-computing/links/nightlyDocDirectory/MuonRecExample/html/namespacepython_1_1MuonRecFlags.html#a8c13e425d4973b479d425591fb1815fhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonReconstruction/MuonRecExample/trunk/share/MuonRec_jobOptions.pyhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonCnvExample/trunk/?#a54ce1fe31c8be632c9361454db65cb22

Digitisation

• Most packages are found under top level SVN directory MuonDigitization

• One (inexplicable?) exception: MuonDigitContainer, which is in the top level MuonSpectrometer directory.

• ‘Digitizers’ Algorithms (one per technology) call a IMuonDigitizationTool.

• Example: MDT_Digitizer calls MDT_DigitizationTool (see next slide)

!!

• It’s here that: wire sag, missing chambers, pileup etc etc are handled.

• Digits are stored in “IdentifiableContainers” - basically maps of collections of digits, where the key is the IdentifierHash of the collection

• Definition of IdentifierHash, can be found on MuonEventDataModel twiki, but for e.g. the CSC, the IdentifiableContainer has a collection (and IdentifierHash) per chamber - reading in a CSC collection, reads in an entire chamber’s worth of data.

�8

http://MuonDigitizationhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonDigitContainerhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonDigitization/MDT_Digitization/trunk/src/MDT_Digitizer.cxxhttps://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/MuonEventDataModel#IdentifierHash_table

MDT digitization

�9

MDT_DigitizerAlgorithm

MdtDigitizationTool

AlgTool, PileupTool

RandomNumberGenerators

MdtCalibrationDbSvc

IMDTConditionsSvc

• MDT_Digitizer calls an AlgTool (MdtDigitizationTool), which in turn calls a bunch of tools and services

• CalibrationSvc - which provides the t0 for MDT tubes

• ConditionsSvc - can provide a list of dead/masked chambers (also can be provided via jobOptions).

• IMDT_DigitizationTool - R/T relations

• MdtDigitizationTool reads in SimHits, and outputs Digits (MdtDigitContainer) and SDOs (MuonSimDataCollection)

• DigitsToRDO:

• Algorithms live in MuonByteStreamCnvTest(!) and have form, XXXDigitToMdtRDO.

IMDT_DigitizationTool

SDO!DigitsSimHits

MuonDetectorManager

DATAFLOW

https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonByteStreamCnvTest/?#a0a7378f95c480945c4457fdbdb91fcdf

Bytestream conversion

• Main package responsible is MuonByteStream

• Has several Algorithms responsible for conversations BS->RDO and RDO->BS

• BS->RDO: XXXRawDataProvider (XXX=Mdt/Csc/Rpc/Tgc)

• RDO->BS: XXXContByteStreamCnv (XXX=CscRdo/MdtCsm/RpcPad/TgcRdo)

• Both sets of algorithms call tools to do actual conversions:

• BS->RDO: Call tools implementing the IMuonRawDataProviderTool interface

• RDO->BS: Call tools implementing specific interfaces e.g. ICSC_RDOtoByteStreamTool

• Actual Tools are found in packages called MuonXXX_CnvTools (XXX=Mdt/Csc/Rpc/Tgc)

• These tools in turn call some extra tools - very technology dependent design at this point. Not much commonality

• One other important point - it’s here that the cabling services are called. These provide conversions from the online / offline identifiers.

• Found under MuonCablings - too detailed for this general overview talk

�10

https://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCnv/MuonByteStream/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCnv/MuonCnvToolInterfaces/trunk/MuonCnvToolInterfaces/IMuonRawDataProviderTool.hhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCnv/MuonCnvToolInterfaces/trunk/MuonCnvToolInterfaces/ICSC_RDOtoByteStreamTool.hhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCablings

Muon RDO to Prep Data

• Algorithms are of form XXXRdoToPrepData, and found in MuonRdoToPrepData

• Again, most of the work is done using a convertor tool with interface IMuonRdoToPrepDataTool (in MuonCnvToolInterfaces) and concrete implementations in one of four packages, MuonXXX_CnvTools, and called XXXRdoToPrepDataTool

• e.g. MdtRdoToMdtPrepData calls MdtRdoToPrepDataTool, and which in turn calls a bunch of further tools:

• IMDT_RDO_Decoder, MdtIdHelper

• One final comment: the interface for the tool is designed for use offline AND in the trigger

• UNSEEDED MODE: Offline we decode retrieve the RDOs from Storegate and convert them all into PrepRawData

• SEEDED MODE: In the Trigger, the tool can DIRECTLY call the byte stream conversion for a passed collection of IdentifierHashes, i.e. we can decode the data within a ‘Region Of Interest’ (ROI)

�11

https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonRdoToPrepData/?#ac6e3b8ec7d85e96190d1689b8ddbf005https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonCnvToolInterfaces/trunk/MuonCnvToolInterfaces/IMuonRdoToPrepDataTool.h

Truth

• We already mentioned SimHits (MuonSimEvent), and SDOs (MuonSimData), but we also store some other objects which link the reconstructed objects to the truth (HepMCParticleLink)

• Won’t go into details here, but you can see them in MuonRecOutputItemList_jobOptions

!!!!!!!

• Also store Muon(Entry/Exit)Layer, which store the HepMCParticleLinks at entrance/exit of spectrometer

�12

https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimEvent/trunk/MuonSimEvent/?#ac8b825360c824143e6e97bf4f4d64e1dhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimData/trunk/MuonSimData/?#ada9ea6600c4c4e225be09a813a68677bhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecExample/tags/MuonRecExample-02-05-16/share/MuonRecOutputItemList_jobOptions.py

EDM/AOD Persistency

• Six (excluding truth convertors) main packages involved in Muon persistency

• Pool convertors (think of them as the interface to Athena/Storegate)

• MuonEventAthenaPool, TrkEventAthenaPool, TrackParticleAthenaPool

• Main documentation: https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/WriteReadDataViaPool

• T/P convertors, called by Pool convertors and turn transient objects into something more ROOT-compatible, and handle schema evolution

• MuonEventTPCnv, TrkEventTPCnv, TrackParticleTPCnv

• Main documentation: https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/TransientPersistentSeparation

• The list of objects written to ESD/AOD is defined in MuonRecExample/MuonRecOutputItemList_jobOptions.py

• Consists of lines like: MuonESDList+=["TrackCollection#MuonSpectrometerTracks"]

�13

Type Name

https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonEventAthenaPool/?#a1f668c6977af618c00894b8891ee01c2https://svnweb.cern.ch/cern/wsvn/atlasoff/Tracking/TrkEventCnv/TrkEventAthenaPool/?#af4d963e8ae61c9fd6ffa9c792e7d7690https://svnweb.cern.ch/cern/wsvn/atlasoff/Reconstruction/TrackParticleAthenaPool/?#ab68c5245d9e5720a9dc772d780ea5837https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonEventTPCnv/?#a58711ea705062b0258676b6a5bd44841https://svnweb.cern.ch/cern/wsvn/atlasoff/Tracking/TrkEventCnv/TrkEventTPCnv/?#a2af1c40766c57795eaa08d69c55e2ff6https://svnweb.cern.ch/cern/wsvn/atlasoff/Reconstruction/TrackParticleTPCnvhttps://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/TransientPersistentSeparationhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonReconstruction/MuonRecExample/trunk/share/MuonRecOutputItemList_jobOptions.py

Conclusions

• The muon data flow is obviously very complex, and I can only give an overview here…

• Probably it could be simplified/cleaned up/documented better in some areas?

• Volunteers?

• But hopefully this gives an idea of where to look for problems

• … and of course, feel free to ask me, or on the muon sw mailing list, if you have further questions

�14