Edward Moyse · Edward Moyse. Introduction • Turning the raw data / detector response into Nobel...

of 14/14
Muon dataflow Edward Moyse
  • date post

    24-Oct-2020
  • Category

    Documents

  • view

    3
  • download

    0

Embed Size (px)

Transcript of Edward Moyse · Edward Moyse. Introduction • Turning the raw data / detector response into Nobel...

  • Muon dataflow

    Edward Moyse

  • Introduction

    • Turning the raw data / detector response into Nobel prizes, is a very complicated process, involving a lot of software

    • … which of course means lots of potential bugs!

    • In this talk, I will try to give an overview of the dataflow, so that you know roughly where to look in the case of problems

    • I’ll also explain a little bit about some of the details of the sub-processes…

    • Too much to cover everything!

    • One thing to note: underlined words are links - this talk is supposed to be a starting point…

    �2

  • DATAFLOW

    Overview

    �3

    Bytestream

    PRD

    Roads

    Segments MS Track SA Track

    CB Track

    ID Track

    • Bytestream - raw bits and bytes from the detector readout

    • RDO - Raw Data Object (object representation of the byte stream)

    • SDO - Simulated Data Object

    • PRD - Prepared Raw Data (after some calibration)

    Sim Hits

    RDO

    SDO

    Glossary

    Digits

    Simulation

    (Usually) real data

    Muon track Muon track extrapolated to the IP

  • DATAFLOW

    Overview - links to (some) EDM packages

    �4

    Bytestream

    PRD

    Roads

    Segments MS Track SA Track

    CB Track

    ID TrackSim Hits

    RDO

    SDO DigitsMuonDigitContainerMuonSimData

    MuonSimEvent

    MuonRDO

    MuonPrepRawData

    MuonSegment TrkTrack

    https://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonDigitContainer/trunk/MuonDigitContainerhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimData/trunk/MuonSimData/?#ada9ea6600c4c4e225be09a813a68677bhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimEvent/trunk/MuonSimEvent/?#ac8b825360c824143e6e97bf4f4d64e1dhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonRDO/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonPrepRawData/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonSegment/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/Tracking/TrkEvent/TrkTrack/trunk

  • DATAFLOW

    Overview - links to (some) EDM packages

    �5

    Bytestream

    PRD

    Roads

    Segments MS Track SA Track

    CB Track

    ID TrackSim Hits

    RDO

    SDO DigitsMuonDigitContainerMuonSimData

    MuonSimEvent

    MuonRDO

    MuonPrepRawData

    MuonSegment TrkTrack

    https://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonDigitContainer/trunk/MuonDigitContainerhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimData/trunk/MuonSimData/?#ada9ea6600c4c4e225be09a813a68677bhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimEvent/trunk/MuonSimEvent/?#ac8b825360c824143e6e97bf4f4d64e1dhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonRDO/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonPrepRawData/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecEvent/MuonSegment/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/Tracking/TrkEvent/TrkTrack/trunk

  • Overview

    �6

    `

    DATAFLOW

    Bytestream

    PRD

    Roads

    Segments MS Track SA Track

    CB Track

    ID TrackSim Hits

    RDO

    SDO Digits

    RECONSTRUCTIONDIGITIZATION

    BYTESTREAM DECODING

    DATA PREP See Niel’s talk

    Persistency

  • Configuration

    • The main configuration (as is usual for Muon SW) is handled in MuonRecExample/MuonRec_jobOptions.py

    • MuonRecFlags contains switches to turn off and on various parts of Reco/Dataflow.

    !

    • For users, parts are controlled in Reco_trf automatically by inputs and outputs

    • i.e. doing Reco_trf inputRDOFile=myRDO.pool.root outputESDFile=myESD.pool.root will run normal muon reco

    • From MuonRec_jobOptions.py, many other jobOpts files are called:

    !

    • For dataflow in particular, many live inside MuonCnvExample:

    �7

    https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonReconstruction/MuonRecExample/trunk/share/MuonRec_jobOptions.pyhttp://atlas-computing.web.cern.ch/atlas-computing/links/nightlyDocDirectory/MuonRecExample/html/namespacepython_1_1MuonRecFlags.html#a8c13e425d4973b479d425591fb1815fhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonReconstruction/MuonRecExample/trunk/share/MuonRec_jobOptions.pyhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonCnvExample/trunk/?#a54ce1fe31c8be632c9361454db65cb22

  • Digitisation

    • Most packages are found under top level SVN directory MuonDigitization

    • One (inexplicable?) exception: MuonDigitContainer, which is in the top level MuonSpectrometer directory.

    • ‘Digitizers’ Algorithms (one per technology) call a IMuonDigitizationTool.

    • Example: MDT_Digitizer calls MDT_DigitizationTool (see next slide)

    !!

    • It’s here that: wire sag, missing chambers, pileup etc etc are handled.

    • Digits are stored in “IdentifiableContainers” - basically maps of collections of digits, where the key is the IdentifierHash of the collection

    • Definition of IdentifierHash, can be found on MuonEventDataModel twiki, but for e.g. the CSC, the IdentifiableContainer has a collection (and IdentifierHash) per chamber - reading in a CSC collection, reads in an entire chamber’s worth of data.

    �8

    http://MuonDigitizationhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonDigitContainerhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonDigitization/MDT_Digitization/trunk/src/MDT_Digitizer.cxxhttps://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/MuonEventDataModel#IdentifierHash_table

  • MDT digitization

    �9

    MDT_DigitizerAlgorithm

    MdtDigitizationTool

    AlgTool, PileupTool

    RandomNumberGenerators

    MdtCalibrationDbSvc

    IMDTConditionsSvc

    • MDT_Digitizer calls an AlgTool (MdtDigitizationTool), which in turn calls a bunch of tools and services

    • CalibrationSvc - which provides the t0 for MDT tubes

    • ConditionsSvc - can provide a list of dead/masked chambers (also can be provided via jobOptions).

    • IMDT_DigitizationTool - R/T relations

    • MdtDigitizationTool reads in SimHits, and outputs Digits (MdtDigitContainer) and SDOs (MuonSimDataCollection)

    • DigitsToRDO:

    • Algorithms live in MuonByteStreamCnvTest(!) and have form, XXXDigitToMdtRDO.

    IMDT_DigitizationTool

    SDO!DigitsSimHits

    MuonDetectorManager

    DATAFLOW

    https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonByteStreamCnvTest/?#a0a7378f95c480945c4457fdbdb91fcdf

  • Bytestream conversion

    • Main package responsible is MuonByteStream

    • Has several Algorithms responsible for conversations BS->RDO and RDO->BS

    • BS->RDO: XXXRawDataProvider (XXX=Mdt/Csc/Rpc/Tgc)

    • RDO->BS: XXXContByteStreamCnv (XXX=CscRdo/MdtCsm/RpcPad/TgcRdo)

    • Both sets of algorithms call tools to do actual conversions:

    • BS->RDO: Call tools implementing the IMuonRawDataProviderTool interface

    • RDO->BS: Call tools implementing specific interfaces e.g. ICSC_RDOtoByteStreamTool

    • Actual Tools are found in packages called MuonXXX_CnvTools (XXX=Mdt/Csc/Rpc/Tgc)

    • These tools in turn call some extra tools - very technology dependent design at this point. Not much commonality

    • One other important point - it’s here that the cabling services are called. These provide conversions from the online / offline identifiers.

    • Found under MuonCablings - too detailed for this general overview talk

    �10

    https://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCnv/MuonByteStream/trunkhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCnv/MuonCnvToolInterfaces/trunk/MuonCnvToolInterfaces/IMuonRawDataProviderTool.hhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCnv/MuonCnvToolInterfaces/trunk/MuonCnvToolInterfaces/ICSC_RDOtoByteStreamTool.hhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonCablings

  • Muon RDO to Prep Data

    • Algorithms are of form XXXRdoToPrepData, and found in MuonRdoToPrepData

    • Again, most of the work is done using a convertor tool with interface IMuonRdoToPrepDataTool (in MuonCnvToolInterfaces) and concrete implementations in one of four packages, MuonXXX_CnvTools, and called XXXRdoToPrepDataTool

    • e.g. MdtRdoToMdtPrepData calls MdtRdoToPrepDataTool, and which in turn calls a bunch of further tools:

    • IMDT_RDO_Decoder, MdtIdHelper

    • One final comment: the interface for the tool is designed for use offline AND in the trigger

    • UNSEEDED MODE: Offline we decode retrieve the RDOs from Storegate and convert them all into PrepRawData

    • SEEDED MODE: In the Trigger, the tool can DIRECTLY call the byte stream conversion for a passed collection of IdentifierHashes, i.e. we can decode the data within a ‘Region Of Interest’ (ROI)

    �11

    https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonRdoToPrepData/?#ac6e3b8ec7d85e96190d1689b8ddbf005https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonCnvToolInterfaces/trunk/MuonCnvToolInterfaces/IMuonRdoToPrepDataTool.h

  • Truth

    • We already mentioned SimHits (MuonSimEvent), and SDOs (MuonSimData), but we also store some other objects which link the reconstructed objects to the truth (HepMCParticleLink)

    • Won’t go into details here, but you can see them in MuonRecOutputItemList_jobOptions

    !!!!!!!

    • Also store Muon(Entry/Exit)Layer, which store the HepMCParticleLinks at entrance/exit of spectrometer

    �12

    https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimEvent/trunk/MuonSimEvent/?#ac8b825360c824143e6e97bf4f4d64e1dhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonSimData/trunk/MuonSimData/?#ada9ea6600c4c4e225be09a813a68677bhttps://svnweb.cern.ch/trac/atlasoff/browser/MuonSpectrometer/MuonReconstruction/MuonRecExample/tags/MuonRecExample-02-05-16/share/MuonRecOutputItemList_jobOptions.py

  • EDM/AOD Persistency

    • Six (excluding truth convertors) main packages involved in Muon persistency

    • Pool convertors (think of them as the interface to Athena/Storegate)

    • MuonEventAthenaPool, TrkEventAthenaPool, TrackParticleAthenaPool

    • Main documentation: https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/WriteReadDataViaPool

    • T/P convertors, called by Pool convertors and turn transient objects into something more ROOT-compatible, and handle schema evolution

    • MuonEventTPCnv, TrkEventTPCnv, TrackParticleTPCnv

    • Main documentation: https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/TransientPersistentSeparation

    • The list of objects written to ESD/AOD is defined in MuonRecExample/MuonRecOutputItemList_jobOptions.py

    • Consists of lines like: MuonESDList+=["TrackCollection#MuonSpectrometerTracks"]

    �13

    Type Name

    https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonEventAthenaPool/?#a1f668c6977af618c00894b8891ee01c2https://svnweb.cern.ch/cern/wsvn/atlasoff/Tracking/TrkEventCnv/TrkEventAthenaPool/?#af4d963e8ae61c9fd6ffa9c792e7d7690https://svnweb.cern.ch/cern/wsvn/atlasoff/Reconstruction/TrackParticleAthenaPool/?#ab68c5245d9e5720a9dc772d780ea5837https://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonCnv/MuonEventTPCnv/?#a58711ea705062b0258676b6a5bd44841https://svnweb.cern.ch/cern/wsvn/atlasoff/Tracking/TrkEventCnv/TrkEventTPCnv/?#a2af1c40766c57795eaa08d69c55e2ff6https://svnweb.cern.ch/cern/wsvn/atlasoff/Reconstruction/TrackParticleTPCnvhttps://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/TransientPersistentSeparationhttps://svnweb.cern.ch/cern/wsvn/atlasoff/MuonSpectrometer/MuonReconstruction/MuonRecExample/trunk/share/MuonRecOutputItemList_jobOptions.py

  • Conclusions

    • The muon data flow is obviously very complex, and I can only give an overview here…

    • Probably it could be simplified/cleaned up/documented better in some areas?

    • Volunteers?

    • But hopefully this gives an idea of where to look for problems

    • … and of course, feel free to ask me, or on the muon sw mailing list, if you have further questions

    �14