CSN1 Settembre 2005 - infn.it · CSN1 Settembre 2005 ATLAS HLT/DAQ Stato e prospettive Valerio...

CSN1CSN1 Settembre 2005Settembre 2005

ATLAS HLT/DAQ ATLAS HLT/DAQ Stato e prospettiveStato e prospettive

Valerio VercesiValerio Vercesi

CSN1 Settembre 2005 V. Vercesi - INFN Pavia 2

OutlineOutline

Pre-series Status in USA15/SDX1Commissioning and exploitation

Large Scale TestActivities, experiencesLessons learnt

ActivitiesMonitoring, ROD Crate DAQAlgorithms development and deployment

FinanceAccounting2006 requests


S. Falciano (Roma1) Coordinatore Commissioning HLTA. Negri (Pavia) Coordinatore Event Filter DataflowA. Nisati (Roma1) TDAQ Institute Board chair e Coordinatore Muon Slice PESAF. Parodi (Genova) Coordinatore b-tagging PESAV. Vercesi (Pavia) Deputy HLT leader e Coordinatore PESA (Physics and Event Selection Architecture)Attività italiane

Trigger di Livello-1 muoni barrel (Napoli, Roma1, Roma2)Trigger di Livello-2 muoni (Pisa, Roma1)Trigger di Livello-2 pixel (Genova)Event Filter Dataflow (LNF, Pavia)Selection software steering (Genova)Event Filter Muoni (Lecce, Napoli, Pavia, Roma1)DAQ (LNF, Pavia, Roma1)Monitoring (Cosenza, Napoli, Pavia, Pisa)Pre-series commissioning (LNF, Pavia, Roma1)


ATLAS TDAQ system ATLAS TDAQ system

Muon

ROD ROD ROD

LVL1

LVL2

Event builder network

Storage: ~ 300 MB/s

ROBROB ROBROB ROBROB

Calo Inner

PipelineMemories

ReadoutDrivers

ReadoutBuffers~1600

High-Level Trigger

LEVEL-1 TRIGGER• Hardware-Based• Coarse granularity from calorimeter & muon systems

LEVEL-2 TRIGGER• Regions-of-Interest “seeds”• Full granularity for all subdetector systems

• Fast Rejection “steering”

EVENT FILTER • Possibly “seeded” by Level 2 • Full event access• Algorithms inherited by offline

RoI

EF farm~1000 CPUs

1 selected event

every millionTDAQ ≅Rates

40 MHz

~75 kHz

~2 kHz

~200 Hz

~2 ms

~10 ms

~ 1 s

Latency

EF

LVL2farm


TDAQ TDAQ Trigger e Data Acquisition hanno da sempre in fase di commissioning un doppio ruolo

Come “server” per il commissioning dei rivelatoriCome “client” per utilizzare le informazioni realistiche dell’esperimento per i propri studi di funzionalità e performance

La situazione si è già presentata durante il Combined Testbeam 2004

Il TDAQ di ATLAS è un progetto in piena evoluzione in cui development/commissioning/exploitation sono ancora fasi molto miscelate

Presentazione di risultati e indicazione delle prospettiveMaggiore enfasi alle componenti con forte partecipazione italianaDescrizione del piano di commissioning generale


PrePre--seriesseries designdesign“Module-0” of final system

8 racks (~10% of final dataflow)


PrePre--seriesseries realityrealityROS rack LVL2 rack

6 racks SDX1

EF rackSwitch rackOnline rack


CommissioningCommissioning and and exploitationexploitationFully functional, small scale, version of the complete HLT/DAQ

Equivalent to a detector’s ‘module 0’

Purpose and scope of the pre-series systemPre-commissioning phase

To validate the complete, integrated, HLT/DAQ functionalityTo validate the infrastructure, needed by HLT/DAQ, at point-1

Note it will be provisionally installed at point 1 (USA15 and SDX1)

Commissioning phaseTo validate a component (e.g. a ROS) or a deliverable (e.g. a Level-2 rack) priorto its installation and commissioning

TDAQ post-commissioning development systemValidate new components (e.g. their functionality when integrated into a fullyfunctional system)Validate new software elements or software releases before moving them to the experiment.


PrePre--Series CommissioningSeries Commissioning


Commissioning LVL2+ROSCommissioning LVL2+ROS

First measurements with full LVL2 rack feeded by ROS data

Using separate Control and Data networks


Commissioning EFCommissioning EF


Large Scale Large Scale TestsTestsPre-serie work will help understanding the TDAQ system in termsof functionality

Forms the basis for future deployments/exploitationsComplexity of ATLAS TDAQ system arises also from the size of bulk components involved

Topology of communications, size of LVL2/EF farms, software, …Test scalability of HLT system using presently available large installations

Understand issues like configuration, startup time, communication, control, error reporting, …

UCB/TRIUMF WestGrid Cluster (http://www.westgrid.ca)60 racks x 14 nodes = 840 Dual-CPU nodes(3 GHz CPUs / 2-4 GB RAM)

CERN LXSHARE Cluster (http://batch.web.cern.ch/batch)Up to ~700 nodes (various flavours)

Reference page for all testshttp://atlas-tdaq-large-scale-tests.web.cern.ch

http://atlas-tdaq-large-scale-tests.web.cern.ch/


Success, Success, problemsproblems, , ……High test activity at both system andcomponent levelProblem finding – solving or workaround – find new problemsGeneral feeling that

We are well advancing in theunderstanding of the system on a large scaleIn some areas we have come furtherthan hoped (HLT offline)A lot remains to be done for final AtlasRepeated questions: when are thenext large scale tests planned?

Comprehensive report is planned to be available for the Atlas Week in October


State State transitionstransitions

Luke Warm Start Luke Warm Stop

USR_RUNNING_TIME(default is 30 s)

RUNNING

CONFIGURED

INITIAL

ABSENT

Configure

Boot

Unconfigure

Shutdown

Setup Close

Cold Start Cold Stop

configure: loadconfigure

start: prepareForRunstartTrigger

stop: stopTriggerstopFrontEndstopDataCollectionstopEventFilterstopRecording

unconfigure: unconfigureunload


LVL2 transition times LVL2 transition times State transistion timing quite acceptable

No significant differences between 2 and 3 tier Run Control

2 Tier Run Control

01020304050607080

0 5 10 15 20 25

Configuration Id (# L2PU nodes 8 -> 256)

Tim

e (s

ecs)

setupbootconfstartwaitstopunconfshutdownclose


EF results @ LSTEF results @ LSTEF TrigMoore with Oracle/MySQL

0

100

200

300

400

500

600

700

800

0 100 200 300 400 500 600 700

Number of processes (1EFD+2PTs)

conf

ig. /

tota

l tim

es, s

config Oracle

config MySQL

Total test Oracle

Total test MySQL

Timings HelloWorld / TrigMoore

0

50

100

150

200

250

0 100 200 300 400 500 600 700

Number of processes (1EFD+2PTs)

time,

s

config HelloWorld

config TrigMoore MySQL

backend setup HelloWorld

backend setup TrigMoore MySQL

Effect of realistic algorithm:TrigMoore vs HelloWorld

EF standalone 1EFD+2PTs / nodeup to 200 nodesMySQL as geometry DB usedsignificant slow down due to access/reading geom. DB

MySQL vs Oracle DB in TrigMooreEF standalone, 1EFD+2PTs / nodeOracle DB – up to 160 nodesMySQL DB – up to 200 hostsMySQL works faster at “small scales”, while Oracle looks better at higher scales - to be investigated morenot able to on higher than 200 nodes with any of both partitions –to be investigate further (do we need to replicated DBs ?)


Cosmics Cosmics TileTile setupsetupMobiDAQ (Mobile DAQ):read out of 8 drawers inthe pit with temporaryRODemu but real TDAQ(tdaq-01-02-00), tests ofelectronics, cosmicmuons runs

GNAM Monitoring chainGNAM Monitoring chain

Framework per monitoring on-line a basso livelloCore: trasporto di eventi, istogrammi e comandiPlugin dinamici: decodifica e istogrammazione Possibilita’ di correlazione fra diversi rivelatori

Comandi asincroni (reset, rebin, update)

StatusIncluso in TDAQ da aprileUtilizza i servizi disponibili per il monitoringValidato al CTB04; in uso in alcuni siti di commissioning


GNAM al commissioningGNAM al commissioningAcquisizione dati nell’ambito del software TDAQCommissioning di MDT usa GNAM per monitoring online ed analisi datiSampling completo a livello di ROS [~2 KHz per noise test, ~200 Hz per pulser]Integrazione di librerie: completato per MDT, in via di sviluppo per RPCStato del monitoring MDT:

Richiesta minimale di informazioni allo shifter (nomi camere)Output: file di istogrammi e file di testo con risultati di analisi dati per ciascuna camera e per ciascun run

In sviluppo:ottimizzazione dell’analisi dati on-lineinstallazione del presenter per la visualizzazione degli istogrammion-line event display


PMP PresenterPMP Presenter

Visualizzazione asincrona on-line di istogrammiInterattivo (reset, rebin, zoom, fitting, ecc...) Operazioni grafiche su istogrammi (ROOT canvas)Grafica configurabile

StatusIncluso in TDAQ da settembreUtilizza i servizi disponibiliper il monitoringPienamente funzionaleal CTB04Riprogettato per nuovefunzionalità e maggiorescalabilità


GNAM&PMP: sviluppi futuriGNAM&PMP: sviluppi futuriGNAM

Completare l’integrazione nel framework del TDAQConfigurazione di Core e librerie dal database del TDAQMessaggistica e gestione degli errori software

Supporto per la generazione di allarmi automaticiLivelli di severità, routing

Verifica delle prestazioni e delle risorse necessarieCPU, Memoria, Banda

Studio della scalabilità

PMPCompletare la nuova versione

Minimizzazione del traffico di reteAdattamento della IGUI alla nuova struttura

Generazione allarmiPlugin di analisi degli istogrammi


ROD ROD CrateCrate DAQDAQRCD is used as interface with the RODsfor

ControlConfigurationMonitoring

StatisticsEvent sampling

Data readout (through VME)User guide for detectors developers availableValidation system in Bld. 40DAQ Commissioning – Phase 1:

The ROD Emulator system will be used in order to validate all common RCD software and infrastructureAfter adding and validating the detector sw and hw specific items, multi crate event building will be used in the absence of the full DAQ chain

VMEbus memory + CORBO

=Memory +Registers +Interrupt capability

RCC

MEMORY

CORBO

Config & Control

Data readout

REB

Event Fragments

ROD Emulator

ROS


RCD RCD exploitationexploitationExperience from Combined Testbeam extremely useful

Recall almost all detector used it in the CTBSuccessful workshop to put forward new requirements

As a consequence, several improvements during last monthsConfigurable interrupt handlingSimplified user interface to access ordered event fragmentsData driven event building for multicrate acquisition in the commissioningphaseSimplified ROD emulationHardware trigger distribution

All sub-detector commissioning (but LAr…) sites use RCDMDT and RPC on the forefront

BB5 integration, Point 1 with MROD, Lab testing with ROD emulators


Algorithms: Muon sliceAlgorithms: Muon sliceLVL1 simulation is the fundamental input for the measurement of the full muon vertical slice performanceLVL2 and EF Muon algorithms have been extensively tested on data simulated in ATLAS

Rome Physics Workshop: June 2005LVL2: µFast

Confirm the LVL1 trigger with a more precise PT estimation within a Region of Interest (RoI)Global pattern recognition, track fit, fast PT estimate via Look Up Table (LUT) with no use of time consuming fit methods

Event Filter: TrigMooreBased on offline reconstruction algorithm MooreCan run seeded (reconstruction starting from RoI of previous levels)Precise PT determination

General goal is now to achieve more realistic estimate of trigger selections and corresponding rates

Real geometry, configuration and conditions database usage, …


LVL1 Coincidence WindowsLVL1 Coincidence Windows

Athena release 10.0.4Low pT 6 GeV Threshold

Low-pT Inefficiency mapEfficiency curve


MuFastMuFast RadiusRadius approachapproachBarrel, Muon layout Q

Use of curvature radius instead of sagitta investigatedMore suitable for the endcap, recover efficiency in the barrelSame algorithm across ± 2.4 in η

Tail extension over 2 σ:

15% @ 6 GeV25% @ 20 GeV35% @ 100 GeV

misposition of the fitsegment w.r.t. the MDT wire, increasestails at higher pT.

Standard sectorsAll sectors

|η| < 1.1


MuFastMuFast endcapendcap extensionextensionEvent display #17

-2.24

-2.22

-2.2

-2.18

-2.16

-23000 -21000 -19000 -17000 -15000 -13000

Z

eta

TGC

TGC SP

MDT EM

MDT EO

ROI

Early stages of projectEndcap differs from Barrel

M and O station are outside B field Inhomogeneous B field – bending is local

AlgorithmPattern recognition and fit in TGC → position and slope in EM Extrapolate segment into MDT EM/EO → Roads in EM/EO, find hits, fit

Next pattern recognition and fit in MDT as in mFast – not done yetExtrapolation into EI and LUT

-1.40E-05-1.20E-05

-1.00E-05

-8.00E-06

-6.00E-06

-4.00E-06-2.00E-06

0.00E+000 10 20 30 40

PT

Slop

e

Forward Endcap


MuFastMuFast: MDT : MDT miscalibrationmiscalibrationCommissioning the algorithms: realistic approach to data handlingThe plot shows the muFast resolution for two different scenarios:

the correct MDT r-t function is used, red pointsa systematic shift of + 0.2 mm is added to the radius returned by the correct r-t function, blue points

+10% degradation @ 50 GeV, to be compared with a + 5% expected by a naive calculation


TrigMOORETrigMOORE updatesupdatesA check on the detectors hash ids selected by RoIs is now performed in order to avoid “duplicated” Moore tracks, that can be reconstructed twice if starting from equal or similar RoIs.The RegionSelector can be initialised with maps from MDT, RPC, TGC for different layouts

CSC recently added: checks in progress

TrigMoore can be seeded in the barrel and in the endcapsWork in progress to establish robustness and fake rate in presenceof cavern background


TrigMooreTrigMoore: MDT : MDT miscalibrationmiscalibrationSingle muons (with pT = 6, 10, 20, 40, 100 GeV/c, produced for the Rome InitialLayout) have been reconstructed in two different scenarios

Using the correct MDT r-t relation function (red squares in pictures)Applying a systematic +0.2 mm shift on the radius obtained with the correct MDT r-t function (blue circles in pictures)

pT resolution (Moore)

The relative degradation in σ(pT) is+5% for muons with a 6 GeV/c transverse momentum, increases to+13% around 50 GeV/c

This MDT miscalibration leaves almostunaffected η resolutions.


LVL2 tracking: SiTrackLVL2 tracking: SiTrackPreliminary results obtained on DC1 b-jet samples at initial luminosity


BB--taggingtagging @ LVL2@ LVL2Results obtained with the “standard” SiTrack algorithm on DC1 datab-tagging: likelihood ratio using transverse and longitudinal impact parametersUpgraded version to be tested soon: should improve both efficiency and track parameters resolution and hence significantly improve the b-tagging performance


AlgorithmsAlgorithms steeringsteering

Cluster60

eg25

eg15

eg60iSe60

iSg60

iSg25

l2g60i

l2g25i

eg60Hy

eg25Hy

eg15Hy

T2Calo

iSg15

iSe25

iSe15

EMtrackSoft

l2e25i

l2e15i

gIsol60

gIsol25

eIsol15

eIsol25

g60Hy

g25Hy

g15Hy

e60Hy

e25Hy

e15Hy

EMtrackHard

TrackSoft15EM

l2e60ieIsol60

eg20

iSg20 l2g20i

eg20Hy

iSe20 l2e20i

gIsol20

eIsol20

g20Hy

e20Hy

EM60Hy

EM25Hy

EM15Hy

EM20Hy

EM25

EM15

EM60

EM20

l2e15ieIsol15

TrackSoft02EM

EMROI

Cluster15

Cluster25

Cluster20

TrackSoft25EM

TrackHardEM

T2Calo

T2Calo

T2Calo

EMtrackSoft

EMtrackSoft


ATLAS ATLAS CommissioningCommissioning PhasesPhasesCommissioning means bringing ATLAS systems from“just installed” to “operational”. It is broken in 4 phases

Subsystem standalone commissioningIntegrate subsystems into full detectorCosmic rays, recording data, analyze/understand, distribute toremote sitesSingle beam, first collisions, increasing rates, etc…

A consistent part of commissioning activities will be doneduring the installation itselfPhases will overlap since different systems may be at different development levels

For the barrel calorimeter commissioning will start soonTile calorimeter is already taking data


HLT HLT CommissioningCommissioningCommissioning is a set of activities which spans the time interval from the installation of the HLT racks and nodes …

A rack is the elementary unit for commissioningOS, Dataflow and Online software are installed

... to the phase when the HLT is filtering physics data and recording themHLT selection algorithms are installed and running stablyThe complete trigger menu (at least for early physics) is configuredThe trigger selection efficiencies and background rejection rates are understood and can serve as input for physics measurements

Phase-1 Commissioning definition is the most urgentHeavily use the Pre-series to exercise the procedures for installation and commissioning

Important steps will cover the integration of detectors into full systemInvolve operations that have a very strong coupling with the offline commissioningactivitiesDevelopment of specific algorithms looking at simple data decoding (cabling,…)

Final commissioning phases extend far beyond the data-taking startup (interface with run coordinator team)


CosmicCosmic muons in ATLASmuons in ATLAS

Rock ~ Silicon

600m x 600m x 200m deep

(2.33 g/cm3)

AirConcrete

Surface building

PX14/16 shielding

(2.5 g/cm3)PX16

PX14

(18.0 m Inner Ø)(12.6 m Inner Ø )

ATLAS

Geant Simulation Initial detector


OutlookOutlookA lot of work during this year, system entering phase of complete deployment

Purchase plan proceeding as scheduled, with some minor delays(Wo)manpower situation not always healthyMore help and support welcome

Three big tasks awaiting us in the next monthsCommissioning the pre-series and extract a coherent and complete set of system performance measurements

Based on previous experiences and on already established partial resultsOn-line trigger selections evaluation (rates, efficiencies, physics coverage,…) evolving towards more realistic approach

Calibration, geometry “as installed”, mis-alignment, error handling, complete trigger menus, physics analysis based on trigger objects

Prepare for cosmic run next year“Cosmic” slices and trigger menu (Tile, LVL1, “digital” LVL2)


AccountingAccountingContributo INFN alla Pre-serie

Read-Out System: 51 kCHF (ROS Racks)Online Computing System: 40 kCHF (Monitoring, Operations)Online Network System: 44 kCHF (Switches, FileServer)

Inviati al CERN a Dicembre 2004VV riceve in copia tutte le fatturazioni dei singoli acquisti ed un sommario mensile dello stato finanziario

Contributo CORE 2005Online Computing System: 45 kCHF (Monitoring, Operations)

Inviati al CERN a Maggio 2005Già acquisiti due file server

Read-Out System: 275 kCHF (ROS Racks)Questo acquisto si espleta secondo una gara e non con un semplice market survey o price inquiry come fino ad ora avvenutoRichiesta alla Giunta l’autorizzazione per partecipare alla gara

Grazie a Speranza che si è prodigata per espletare le pratiche necessarieIl CERN preferisce gestire la gara su un periodo di due anni

Omogeneità dei componenti vs miglioramento delle prestazioniSi sommano i 275 kCHF del 2006


CostCost ProfileProfile ((kCHFkCHF))2004 2005 2006 2007 2008 2009 Total

Pre-series 140 0 0 0 0 0 140

INFN Percentage(%) 13.4 9.5 19.0 11.1 12.4 13.9 13.1

Detector R/O 0 275 275 0 0 0 550

LVL2 Proc 0 0 65 195 230 160 650

Event Builder 0 0 50 50 110 70 280

Event Filter 0 0 170 180 570 380 1300

Online 0 45 135 0 0 0 180

Infrastructure 0 0 80 80 20 20 200

INFN Total 140 320 775 505 930 630 3300

TDR Total 1048 3357 4087 4544 7522 4543 25101


Missioni EstereMissioni Estere

LNFCommissioning HLT/DAQ Pre-serie e pit: 6 m.u.

Ferrer, Kordas (+ Miscetti, Giovannella)

PaviaVV coordinatore PESA e duputy HLT: 1 m.u.Negri A. responsabile Event Filter: 1 m.u.Scannicchio D. commissioning HLT: 2 m.u.

Roma1Speranza responsabile commissioning HLT: 2 m.u.Leandro chair IB, coordinatore slice mu: 1 m.u.ROD crate DAQ e HLT/DAQ muoni : 4 m.u.

Pasqualucci, Di Mattia, …


MilestonesMilestones30/06/2005

TDAQ - Installazione, test e uso della "Pre-serie" (~ 10% TDAQ slice)

“ragionevolmente” raggiunta: ritardi accumulati soprattutto sugli acquisti delle componenti

24/12/2005TDAQ - Installazione e test dei ROS di Pixel, LAr, Tile, Muon (interfacciamento al ROD Crate e integrazione nel DAQ)

Parte del piano di commissioning in esecuzione: piccola dipendenza dalla data di consegna dei ROS

30/04/2006Completamento dei test sulla pre-serie e definizione delle funzionalità per il supporto al commissioning TDAQ

31/08/2006Commissioning delle slice di ROS dei rivelatori utilizzando le funzionalitàdella pre-serie (modulo-0 del sistema finale)

31/12/2006Presa dati integrata dei rivelatori nel pozzo con raggi cosmici

CSN1 Settembre 2005 - infn.it · CSN1 Settembre 2005 ATLAS HLT/DAQ Stato e prospettive Valerio...

Documents

Transcript of CSN1 Settembre 2005 - infn.it · CSN1 Settembre 2005 ATLAS HLT/DAQ Stato e prospettive Valerio...