124-10-2008 Bart Hommels for the CALICE-UK groups

Data acquisition systems for future calorimetry

at the International Linear Collider

Introduction

DAQ design at system level

DAQ for ILC Calorimetry: CALICE

224-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

Introduction

324-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

IntroductionTraditionally, DAQ in HEP is at the back of the queue for R&D effort,

often leading to a conservative system implementation.

To make optimal use of the advancements in industry, a new DAQ should be considered as a ‘generic’ system, with a ‘use case’ for a particular (sub-)detector.

Detector Readout System

D

A

QIn most cases, Detectors and associated Readout Systems are designed, tested and approved first, before substantial DAQ effort is undertaken.

424-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

Introduction: generic DAQ design

This DAQ is designed as a generic, scalable, and self-contained system, build around commercial off-the-shelf components where possible

The generic DAQ is then configured towards multiple ‘use-cases’. ILC calorimetry might not be the only customer

Using the generic, core DAQ for other interesting cases is welcomed for stress-testing the system: think of linear collider experiments or SuperLHC

524-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

DAQ design

624-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

DAQ design: global architecture

Detector modules called ‘slabs’ deliver data over ~Gbit speed optical fibre

• Layer-1 switch routes event data to PC

• Target Control re-routes data should a certain PC not be available

• After processing, data is routed through a network switch towards event builder/data store

724-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

DAQ design: candidate Level-1 switch

Candidate Level-1 switch: state-of-the art full optical switch system

Currently investigating performance: switch timing characteristics etc.

M. WarrenV. Bartsch

824-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

DAQ design: PC-based receiver card

PC-based receiver card is a key component in the generic DAQ design

• Developed & built by PLD Applications

• Large & Fast FPGA: Xilinx Virtex4-FX100

• 8x PCIexpress bus• SFP cages for multi-Gbit/s

interfaces• Many high-speed I/O

connections

To be used for the following tasks:• Data reception• Clock source• Control data transmission• Configuration data distribution

924-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

A DAQ for ILC calorimetry

1024-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: Architecture

Link Data Aggregator(LDA)

Detector Interface(DIF)

Detector Unit

Off Detector Receiver(ODR)

DAQ software

1124-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: system overview

Detector Unit: long (>1.5m) detector ‘slab’ with integrated front-end electronics

DIF: Detector InterFace, servicing the Detector UnigLDA: Link-Data Aggregator. Concentrates data, fanin/fanout for clock

and control dataCCC: Clock&Control Card: Fanout of clock and fast controlsODR: Off-Detector Receiver: PC interface for the DAQ systemPC running DOOCS for run-control and DAQ monitoring

DAQ PC

LDA

LDA

ODR

DetectorUnit

DIF

CCC

DetectorUnit

DIF

DetectorUnit

DIF

DetectorUnit

DIF

StorageControl

PC (DOOCS)

DAQ PC

ODR

1224-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: links & protocols

1. DIF-LDA: 80-160 MHz serial link. 8b/10b encoded synchronous data transfers

2. LDA-ODR: Gbit ethernet (or possibly TLK2501) over optical fibre

3. CCC-LDA: trimmed version of DIF-LDA link

LDA

LDA

DetectorUnit

DIF

CCC

DetectorUnit

DIF

DetectorUnit

DIF

DetectorUnit

DIF

Control PC

(DOOCS)

12

3

1324-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: DIF

ECAL DIF prototype:65x72mm, 8 layers1. JTAG programming header2. LDA link HMDI connector3. DIF link connector4. mini-USB connector5. Xilinx PROM6. Cypress 2MB 10ns SRAM7. Xilinx Spartan3-1000 FPGA8. FDTI FT245R USB controller9. 20p user header connector10. reset pushbutton11. 90pin SAMTEC IB connector

• 2 DIFs produced, parts available for 10 more.• DIF hardware is (at least partly) functional

The DIF concept is generic in firmware, running on detector-specific hardware.

Different DIF flavours for ECAL, HCAL

ECAL DIF status:• Test hardware in place• Firmware development started

M. Goodrick, B. Hommels, R. Shaw

1424-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: DIF -LDA linkLink shows signs of life:

pseudo-LDA sends CLK & 8B/10B data @ 100MHz over AC-coupled LVDS on HDMI cables

DIF

pseudo-LDA

HDMI cable

AC-couplingadapter

stable data loop-back in firmware

HDMI connector + cabling standard adapted for excellent price/performance

1524-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: LDA

• Aggregate data from many DIF links and send to ODR over Gbit Ethernet link

• LDA should serve 10 DIFs

• Using commercial board from Enterpoint, with semi-custom add-on boards

• However, board came back with problems acknowledged by manufacturer - re-spin now

• Major firmware development underway

• ODR-LDA protocol progressing

• LDA-DIF link being documented

M Kelly

1624-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: CCC

Tasks:• Synchronise all subsystems

upon pre-spill warning• Act as reliable global clock

source for fast clock, and phase-aligned ILC machine clock

• Distribute asynchronous fast trigger and/or busy signals

• Capable to run stand-alone in tandem with DIF

CPLD-based for implementation of complex logic.Routing to/fro many sources and destinations possible

M Warren, M Postranecki

1724-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: ODR

ODR is the use-case for the PC-based DAQ interface card:

• Commercial board with custom firmware and software

• Task-specific performance optimisation

• System-specific extensions: decode LDA event data headers, etc.

170MB/s to disk

B. Green, A. Misejuk

1824-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: Software

Software to follow system architecture:

• Modularity• Generic nature with

specific interfaces• Scalability

DAQ software requirements in ILC calorimetry:• Support for several calorimeter types, each of significant complexity: O(104) channels, O(103) ASICs, O(102) electronics boards• Preferably use existing DAQ software (framework)• Chose DOOCS being developed at DESY for XFEL

tesla.desy.de/doocs/doocs.html

V. Bartsch, T. Wu

1924-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimeter DAQ: Software application• Starting from

DAQ-ODR interface: first hardware layer & most advanced

• Have set-up LDA emulator and passed data in this system:

PC->ODR->LDA

2024-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

ILC Calorimetry DAQ: CALICE

CALICE: physics prototype of ILC calorimeter detector segment: ECAL + HCAL

First demonstrator for newly developed DAQ:

CALICE testbeam in 2009

DAQ components: Software, ODR, CCC, LDA, DIF prototyped and on track to be ready in time -before everyone else!

2124-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL

DAQ for future calorimetry at ILC: summary

Aim is to develop a DAQ system that is generic in nature, using commercial components where possible

The DAQ system should be modular and scalable

ILC Calorimeter sub-detectors demonstrators serve as the first well-described ‘use-case’ for the DAQ

All DAQ components prototyped in hardware, ready to merge into a full fledged DAQ system through firmware and software development.

First serious test opportunity: testbeam in 2009

Exciting times ahead!