ATLAS HLT in PPD John Baines, Dmitry Emeliyanov, Julie Kirk, Monika Wielers, Will Dearnaley, Fred...
-
Upload
donna-martin -
Category
Documents
-
view
356 -
download
0
Transcript of ATLAS HLT in PPD John Baines, Dmitry Emeliyanov, Julie Kirk, Monika Wielers, Will Dearnaley, Fred...
1
ATLAS HLT in PPDJohn Baines, Dmitry Emeliyanov,
Julie Kirk, Monika Wielers, Will Dearnaley,
Fred Wickens, Stephen Burke
2
Overview
• Introduction• ID trigger• e/gamma trigger• B/physics trigger• ROS & Farms • Upgrade
3
Who’s doing what
• John: HLT UK project leader (M&O and Upgrade)• Dmitry: Inner Detector Trigger & Upgrade– Lead developer for L2Star
• Monika: Electron & photon Trigger • Julie: B-Physics Trigger– B-trigger coordinator
• Will: B-Physics Trigger• Fred: ROS & HLT farms • Stephen: Upgrade
Level 1 (LVL1)• Fast Custom-built electronicsLevel 2 & Level 3 (Event Filter):• Software based running on large PC farmLevel-2: • Fast custom algorithms • reconstruction mainly in Regions of
Interest (RoI)=> limited data accessLevel 3 = Event Filter (EF)• Offline tools inside custom wrappers,• Access to full event information
The ATLAS Trigger
4
~300 Hz~300 MB/s
~3kHz
<75 kHz
Front End Pipelines
Readout Buffers
Event Builder
Full Event Buffers
Storage & offline
processing
Tracking
Level-1Fast Custom Electronics
t<2.5ms
Level-2500 PCs
<t>~40ms
Event Filter1800 PCs
<t>~4s
Calo, Muon,
Min. Bias
Requested Data in RoI
Access to full event
5
Inner Detector Trigger• UK & Genoa• UK : UCL, Sussex, Manchester, RAL• RAL: L2 Tracking – L2Star development– Parallelisation/use of coprocessors– Use of FTK tracks
6
Inner Detector Trigger : Dmitry
TrigSteering
TrigIDSCANTrigSiTrack
Data Provider
ZFinder
HitFilter
Track Fitting
Pattern recognition
TRT Track extension
Monitoring
Pre-2012 running
– common code, “service elements”
– algorithm-specific code
Two HLT algorithms: • SiTrack : Genoa
• Combinatorial method• Used for beamspot, tau & b-jet
triggers• IDSCAN : UK
• Fast histograming method• Used for muon, e/gamma & B-
physics triggersCommon:• Track Fitting• TRT extension
Written by Dmitry
7
ID Tracking Performance in
2011
L2 ID execution time v. Mean no. interactions
Electron RoI: ~10ms/RoILinear rise with luminosity(LHC design <m>=23)
Oct, 2011
ID tracking Effic. for electrons
8
L2STAR : new L2 tracking framework
TrigSteering
The unified algorithm
Data Provider
Track Fitting
Monitoring
TRT Track ext.
<<interface>> ITrigL2PattRecoStrategy
m_findTracks(RoI, data) : HLT::ErrorCodem_findTracks(data) : HLT::ErrorCode
Strategy A Strategy B Strategy F
Common interface
Configurable list of track finding strategies – plug-ins
L2STAR design Design goals:• Provides a single configurable algorithm for L2 tracking• Simplify menus• Removes duplication of code• Provides different track-finding strategies
Status:•L2STAR has replaced IDSCAN and SiTrack in the 2012 trigger menu•L2STAR is being used for data-taking in 2012• Next phase of development targets 2014 running
Dmitry: Lead developer
Fast Histogramming Combinatorial Start from FTK trk
e/gamma (Monika)
9
• Finalizing the 2011 performance – with combined e/g efficiency group – will appear in trigger e/g trigger
conference note (nearly through the approval process)
– Effic. w.r.t. offline loose/med/tight sel. – For lowest un-prescaled single triggers:
• e20_medium• e22_medium and e22vh_medium1
– Extract efficiencies in 1d as a function of η and ET
• Use tag & probe with Z->ee– Calculate systematics by variation of
the Z(ee) inv. mass window and the tightness of the tag
e/gamma (Monika)• 2011 performance
– Extraction of scale factors (SF=ε(data)/ε(MC)) using MC11a, MC11c and atlfast– These numbers are available within the e/g combined performance group tools
for use in any D3PD/AOD analysis
10
e/gamma (Monika)• 1st look at 2012 performance using data from
period A• Measure effic. using Z(ee) tag & probe • For lowest threshold unprescaled triggers:
– e22vh_medium, e22vhi_medium, – e24vh_medium, e22vhi_medium
• Prepare for higher lumi :– Selections tighter than 2011– Lower Effic.– Check for pile-up dependence
• Not conclusive yes, but hint of small of pileup dependance
11
v: variable threshold in eta bins of 0.4 at L1, h: hadronic core isolation cut, i: track isolation use at EF
12
e/gamma (Monika)• Looking in detail at where loses occur• Losses due to b-layer cut were traced
back incorrect handling of noisy modules• Other sources of losses under study
13
B-physics programme: Low pT di-muon signatures: Onia studies
(J/ψ→μ+μ−, ϒ→μ+μ−)mixing and CP violation studies
B→J/ψ(μμ)X, Rare and semi-rare B decays
B→μμ(X)
Trigger :• low pT (4,6 GeV) di-mu•2 muons at Level1•Confirmed in High Level Trigger•Require vertex fit and mass cuts
•Unprescaled ‘2mu4’ trigger for Jpsi, Upsilon and B→μ+μ− throughout 2011 data-taking Large samples of events recorded for the B-physics programme
“DiMu” prescaled in latter part of 2011 data-taking
About 50% of total 2011data sample
Trigger Mass window No. of evts (M)
2mu4_DiMu 1.4 – 14 GeV 27
2mu4_Jpsimumu 2.5 – 4.3 GeV 14
2mu4_Bmumu 4 – 8.5 GeV 3.7
2mu4_Upsimumu 8 – 12 GeV 9.1
B-physics - Julie
14
B-physics Changes for 2012“2mu4” rates are too high above 3.e33 –> baseline triggers move to “2mu6_xxxxx”:
2mu6 yield wrt 2mu4
Jpsi 25%
Upsilon 12%
Bmumu 36%
BUT lower yield for B-Physics:
To keep some of lower thresholds introduce “Barrel” triggers:
Efficiencies wrt L1_2MU4:EF_2mu4T_Bmumu 68% EF_mu4Tmu6_Bmumu 55% EF_2mu4T_Bmumu_Barrel 43% EF_mu4Tmu6_Bmumu_Barrel 34% EF_2mu4T_Bmumu_BarrelOnly 33% EF_2mu6_Bmumu 24%
Barrel Triggers: •OK for L1 rate. •EF rate still too high→ Introduce “delayed” stream
Will only be processed when space at Tier0 – possibly not until 2013
15
First look at 2012 data compared to 2011Some runs from periodA (up to 201120 - use mu18_medium)Jpsi tag-and-probe using “mu18_medium” as tag
periodL 2011periodA 2012
pT probe muon
B-trigger (Julie)
16
B-Physics Trigger : Will • Trigger efficiency
measurements• Validation of trigger in
Sample A & Sample T productions
• Muon & B-physics on-call
• B-physics systematics from trigger choice
• Currently working on providing B-Trigger info in D3PD Maker
17
ROS & FarmPeak L2 data request rates per ROS PC
ROS: UCL & RAL (Fred)UK delivered 1/3 of Read Out System:•700 ROBin boards manuf. & tested in UK•ROS PCs sustained required rates:
• Up to ~20kHz request rate per RoS PC•2011 programme of rolling replacement (motherboard, CPU, memory) prioritised ROS with highest load
Additional 4-6 kHz to Event Builder
18
Upgrade• Next phase of L2Star:– Greater modularization– Tracking for Single-node (L2 and EF combined on
same PC) – Use of FTK information in the HLT
• Parallelization of code, use of co-processors
19
Parallelisation for HLT Upgrade : Dmitry• Dmitry leading work to investigate parallelisation in the ATLAS Trigger
• Use of Graphical Processor Units (GPU)• Use of Many-core CPU
• Have successfully parallelized a number of HLT algorithms and re-implemented them for GPUs, in particular: • full data preparation chain (Bytestream decoding, pixel/strip
clusterization and spacepoint making) for Pixel and SCT – (Dmitry + Jacob Howard, Oxford)
• parallel GPU-accelerated track finder (based on SiTrack) (Dmitry)
20
GPU-based Tracking
Spectacular speed-up factor up to x26Full data preparation takes only 12 ms for the whole
Pixel and SCTFactor 12 speed-up for Pat. Rec.:
21
Parallelisation for HLT Upgrade : DmitryDmitry developed solution for integration of GPU-based code into Athena reconstruction:• “client-server” architecture with multi-process server• allows for transparent GPU “sharing” between few Athena apps on multiple CPU cores
GPU sharing test
Up to 4 parallel jobs can share GPU and get accelerated w/o significant GPU saturation
22
Trigger Operations• Trigger Monitoring Expert : John, Monika• Inner Detector & B-jet on-call : Dmitry • Trigger Release Coordination: Dmitry, Stephen• ID code maintenance/bug-fixes: Dmitry• B-trigger menu, B-trigger code development & bug fixes: Julie• Muon & B-trigger on-call: Will• Trigger Validation shift: Julie, Will• e/gamma & Tau on-call shifts: Monika• e/gamma code development: Monika• Control room shifts: Monika, Stephen, Will
23
Extras
24
Gig
abit
Ethe
rnet
Even
t dat
a re
ques
tsD
elet
e co
mm
ands
Requ
este
d ev
ent d
ata
Regi
ons
Of I
nter
est
LVL2Super-visor
Network switches
Second-leveltrigger
pROS
~ 500
stores LVL2output
LVL2 farm
Read-Out
Drivers(RODs) First-
leveltrigger
Dedicated linksVME
Data of events acceptedby first-level trigger
Read-OutSubsystems
(ROSs)
1600Read-OutLinks
RoIBuilder
~150PCs
Event data pushed @ ≤ 100 kHz, 1600 fragments of ~ 1 kByte each
Timing Trigger Control (TTC)
DataFlowManager
EventFilter(EF)
~1600
Network switches
Event data pulled:partial events @ ≤ 100 kHz, full events @ ~ 3 kHz
Event size ~1.5MB
4-code dual-socket nodesCERN computer centre Event rate
~ 200 HzData storage
6Local
StorageSubFarmOutputs
(SFOs)
~100 Event
BuilderSubFarm
Inputs
(SFIs)
Trigger / DAQ architecture
25
ARCHITECTURE
H
L
T
40 MHz
75 kHz
~3 kHz
~ 300 Hz
40 MHz
RoI data = 1-2%
~2 GB/s
FE Pipelines2.5 ms
LVL1 accept
Read-Out DriversROD ROD ROD
Event Builder
EB
~3 GB/s
ROS Read-Out Sub-systems
Read-Out BuffersROB ROB ROB
120 GB/s Read-Out Links
Calo MuTrCh Other detectors
~ 1 PB/s
Event Filter
EFPEFP
EFP
~ 4 sec
EFN
~3 GB/s
~ 300 MB/s
~ 300 MB/s
Trigger DAQ
LVL2 ~ 40 ms
L2P
L2SV
L2NL2PL2P
ROIB
LVL2 accept
RoI requests
RoI’s
LVL1 2.5 ms
CalorimeterTrigger
MuonTrigger
Min. Bias Triggers
~2.5 ms