Upgrade Letter of Intent High Level Trigger Thorsten Kollegger

Upgrade Letter of IntentHigh Level TriggerThorsten Kollegger

ALICE | Offline Week | 03.10.2012

2

RequirementsFocus of ALICE upgrade on physics probes requiring high statistics: sample 10 nb-1

Online System RequirementsSample full 50kHz Pb-Pb interaction rate (current limit at ~500Hz, factor 100 increase)

~1.1 TByte/s detector readoutHowever: • storage bandwidth limited to ~20 GByte/s• many physics probes have low S/B: classical trigger/event filter approach not efficient

ALICE | Offline Week | 03.10.2012 | Thorsten Kollegger

3

• Main physics topics, at the LHC uniquely accessible with the ALICE detector:• measurement of heavy-flavour transport parameters:

• diffusion coefficient – azimuthal anisotropy and RAA

• in-medium thermalization and hadronization – meson-baryon• mass dependence of energy loss – RAA

• study of QGP properties via transport coefficients (h/s, q)• J/y , y’, and cc states down to zero pt in wide rapidity range

• yields and transverse momentum spectra – RAA, elliptic flow• density dependence – central vs. forward production• statistical hadronization vs. dissociation/recombination

ˆ

Physics MotivationSlide from Karel Safarik


4

• measurement of low-mass and low-pt di-leptons • chiral symmetry restoration – vector-meson spectral function

• disappearance of vacuum condensate and generation of hadron masses

• QGP thermal radiation – low-mass di-lepton continuum• space-time evolution of the QGP – radial and elliptic flow of emitted

radiation• Jet quenching and fragmentation

• jet energy recuperation at very low pt

• heavy-flavour tagged jets, gluon vs. quark induced jets• heavy-flavour produced in fragmentation• particle identified fragmentation functions

• Heavy-nuclear states• high statistics mass-4 and -5 (anti-)hypernuclei• search for H-dibaryon, Ln bound state, etc.

Physics Motivation


Slide from Karel Safarik

5

Why not triggering?

Triggering on D0, Ds and Λc (pT>2 Gev/c) ~ 36 kHz@50kHz rate...

Slide from Luciano Musa


6

StrategyData reduction by (partial) online reconstruction and compression

Store only reconstruction results, discard raw data• Demonstrated with TPC clustering since Pb-Pb 2011• Optimized data structures for lossless compression• Algorithms designed to allow for offline reconstruction passes with improved calibrations

Implies much tighter coupling between online and offline reconstruction software


7

Event Size

Expected data sizes for minimum bias Pb-Pb collisions at full LHC energy

DetectorEvent Size (MByte)

After Zero Suppression

After DataCompression

TPC 20.0 1.0TRD 1.6 0.2ITS 0.8 0.2

Others 0.5 0.25Total 22.9 1.65


8

TPC Data Reduction

First steps up to clustering on FEE/FPNs (RORC FPGA)Further steps require full event reconstruction on EPNs, pattern recognition requires only coarse online calibration

Data FormatData

Reduction Factor

Event Size(MByte)

Raw Data 1 700FEE Zero Suppression 35 20

HLT

Clustering & Compression 5-7 ~3Remove clusters not

associated to relevant tracks2 1.5

Data format optimization 2-3 <1


Data Member HLT Cluster Format Optimized

Padrow Number

UShort 16 bit 6 bit

Pad Position Float 32 bit 14 bitTimebin Float 32 bit 15 bitWidth Pad Float 32 bit 8 bitWidth Time Float 32 bit 8 bitTotal Charge Short 16 bit 16 bitMax Charge Short 16 bit 10 bit

Reduction of data size/cluster: 22 Byte -> 10 Byte

Float to Fixed-Point convertion, size according to detector resolution

TPC Data Reduction

ALICE | Offline Week | 03.10.2012 | Thorsten Kollegger 9

Lossless data compression with Huffman code (entropy encoding)Data members transformed to increase performance:• e.g. Padrow Number => Drow(i) = row(i) – row(i-1)• Entropy reduced from ~6 to 1.1


TPC Data Reduction

Overall data size to tape reduced by factor 4.3Used in Pb+Pb 2011, p+p 2012... standard ALICE data format

Further reduction possible by transforming pad, time coordinates ALICE | Offline Week | 03.10.2012 | Thorsten Kollegger 11

TPC Data Reduction

12

TPC Data Reduction

First steps up to clustering on FEE/FPNs (RORC FPGA)Further steps require full event reconstruction on EPNs, pattern recognition requires only coarse online calibration

Data FormatData

Reduction Factor

Event Size(MByte)

Raw Data 1 700FEE Zero Suppression 35 20

HLT

Clustering & Compression 5-7 ~3Remove clusters not

associated to relevant tracks2 1.5

Data format optimization 2-3 <1


Discard clusters not assigned to tracks (or in the track vincinity)- Requires online calibration (at least coarse one)- Allows later offline re-productionAlternative: identify background clusters


Further data reduction

14

Processing PowerEstimate for online systems based on current HLT processing power- ~2500 cores in ~200 nodes-108 FPGAs on H-RORCs for local preprocessing

- TPC clusterfinding: 1 FPGA equivalent to ~80 CPU cores- 64 GPGPUs for tracking (NVIDIA GTX480 + GTX580)

Scaling to 50 kHz rate to estimate requirements- ~ 250.000 cores- additional processing power by FPGAs + GPGPUs1250-1500 nodes in 2018 with multicores


Name of Task

(Initialization)

Combinatorial Part(Cellular Automation)

Neighbors Finding

Evolution

Kalman Filter PartTracklet Construction

Tracklet Selection

(Tracklet Output)

Algorithm implemented as multithreaded CPU and CUDA GPU versionALICE | Offline Week | 03.10.2012 | Thorsten Kollegger 15

HLT TPC Tracking

3-fold speedup of GPU compared to optimized CPU version on 6 cores


HLT TPC Tracking

Consistency between GPU and CPU version of tracker

HLT Tracking Performance


Active GPU Threads using Dynamic Scheduling

Active GPU Threads: 67%

threadstim

e


HLT Tracking Performance

Efficiency/Clone/Fake rate calculation merged with PWG-PP/TPC code- Under review by TPC group

Old HLT efficiency macro

19

SummaryAfter the upgrade:Store only reconstruction results, discard raw data• Requires online calibration• Algorithms designed to allow for offline reconstruction passes with improved calibrations

Implies much tighter coupling between online and offline reconstruction software


20ALICE | Offline Week | 03.10.2012 | Thorsten Kollegger

21

Backup - Processing PowerEstimate of processing power based on scaling by Moore’s law

However: no increase in single core clock speed, instead multi/many-core

Reconstruction software needs to adapt to full use resources

Picture from Herb Sutte: The Free Lunch Is OverA Fundamental Turn Toward Concurrency in SoftwareDr. Dobb's Journal, 30(3), March 2005 (updated)


Upgrade Letter of Intent High Level Trigger Thorsten Kollegger

Documents

Transcript of Upgrade Letter of Intent High Level Trigger Thorsten Kollegger