Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden...
-
Upload
barry-hicks -
Category
Documents
-
view
221 -
download
0
Transcript of Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden...
Frank Lemke
Design concepts and measurements of the CBM Design concepts and measurements of the CBM DAQ networkDAQ network
DPG – Frühjahrstagung Dresden 2013
HK 10.7
University of Heidelberg
Computer Architecture Group
Frank Lemke, Sven Schatral
04.03.2013
Frank LemkeDPG – Frühjahrstagung Dresden 20132
OutlineOutline
The Compressed Baryonic Matter Experiment Motivation CBM Network Structure Front-end ASIC Communication FPGA and ASIC Based Readout Controllers Conclusion
Frank Lemke3
FAIR at GSI Darmstadt GermanyFAIR at GSI Darmstadt Germany
Facility for Antiproton and Ion Research (FAIR) extends the existingGSI accelerator and synchrotron
Starting 2018, FAIR will be used to measure atomic nuclei and the particles they are made up of
( http://www.fair-center.de , January 2013 )
DPG – Frühjahrstagung Dresden 2013
( https://www.gsi.de )
Construction work has already begun and makes visible progress
Demonstrator prototypes have been build and development for the experiment readout systems are ongoing
Frank LemkeDPG – Frühjahrstagung Dresden 20134
The Compressed BaryonicThe Compressed BaryonicMatter ExperimentMatter Experiment
One of the eight FAIR experiments is the Compressed Baryonic Matter (CBM) experiment
The CBM experiment Investigates highly compressed nuclear matter
using nucleus-nucleus collisions Is a self-triggered detector system Provides eight different kinds of detectors
( https://www.gsi.de )
Frank LemkeDPG – Frühjahrstagung Dresden 20135
MotivationMotivation
Create complete DAQ network solution usable within all stages of the network providing the required communication capabilities for CBM
Complete network approach allows design of straight forward and compact solutions fulfilling
Restricted space limits for hardware High bandwidth requirement Supporting CBM specific synchronization methodologies
No protocol conversion required Easy for user adaption and providing reusable blocks for the CBM network
Frank Lemke6
CBM Network Challenges
DAQ network system needs to deliver Flexibility for various build-up variants Efficiency for data aggregation Successive link rate increase
From 0.5 Gb/s up to 10Gb/s per lane
Clock distribution Reliable system clock from one single master Delivering up to 250 MHz over LVDS for stable FEE sampling
Precise time synchronization In the order of the link bit clock cycles -> 2 ns
Support for centralized control system solutions Dense interconnection solution Handling of up to several TB/s of raw data
DPG – Frühjahrstagung Dresden 2013
Frank Lemke
CBM Network StructureCBM Network Structure
7 DPG – Frühjahrstagung Dresden 2013
( Walter F.J. Mueller, GSI, Darmstadt, 30 March 2012 )
Frank Lemke
Front-end ASIC ChallengesFront-end ASIC Challenges
8 DPG – Frühjahrstagung Dresden 2013
Depending on type and system position FEE bandwidth varies Starting at 0.5 Gb/s using one lane
Supporting up to four lanes with 2 Gb/s
CBMnet has reusable design blocks Control and status Register File (RF) I2C support for RF access, debugging and bring-up CTRL decode for CBMnet ctrl messages ASIC SERDES implementations supporting synchronization CBMnet protocol module with Master/Slave support
Unbalanced links: 1 up-stream and up to 4 down-stream links Shift register chain and sub RF support for analog designed ASIC parts
Frank Lemke9
Front-end ASIC ModulesFront-end ASIC Modules
DPG – Frühjahrstagung Dresden 2013
Frank Lemke10
CBMnet Implementation at CBMnet Implementation at SPADIC & STSXYTERSPADIC & STSXYTER
DPG – Frühjahrstagung Dresden 2013
STSXYTER Device will be used for the Silicon Tracking System 2 Gb/s read-out bandwidth Currently in production
SPADIC - Self-triggered Pulse Amplification and Digitization asIC
( Please visit http://spadic.uni-hd.de/)
Device will be used for the transition radiation sub-detector (TRD)
Device to read-out and process small electrical detector signals on a single silicon
32-channel mixed signal Self-triggered hit detection and neighbor readout Full pulse recording 1 Gb/s read-out bandwidth Current version 1.0 using CBMnet protocol
communication and design modules
Frank Lemke
SPADIC MeasurementSPADIC Measurement
11 DPG – Frühjahrstagung Dresden 2013
Synchronization accuracy on bit clock level of 2ns was shown
Configuration registers have been written and control communication using CBMnet protocol was approved
Data was successfully streamed from the SPADIC
Frank Lemke
FPGA Readout - EvalFPGA Readout - Eval
12 DPG – Frühjahrstagung Dresden 2013
It provides
SFP to use old readout chainsFMC adapter to attach specially developed board connecting multiple FEEsHDMI cables used for first cablingIdentical FPGA type as new ROC used as emulation platform Small and low cost beam time and laboratory readout chainsLimited synchronization capability
Xilinx Spartan 6 eval-board SP605 with 45T device is used as first evaluation platform for the next generation FPGA Boards
Frank Lemke
FPGA Readout – ROC3FPGA Readout – ROC3
13 DPG – Frühjahrstagung Dresden 2013
In the collaboration a new universal readout controller is in production using a Xilinx Spartan6 150T device
FMC adapters for attaching FEE readout Almost identical first readout using SP605 HDMI solution Designed to provide the full synchronization feature set Supporting up to 8 FEEs
Substitution of former concentrator and aggregator boards Clock source or even multiplier ECS/DCS emulator Data sink and readout chain source Emulation platform for HUB testing on limited small functional blocks
HK 34.6: Modular CBM-ROC Firmware - Was bisher geschah und wie es weitergeht. SEBASTIAN MANZ und UDO KEBSCHULL (Dienstag, 5. März, 15:30, HSZ-405)
Frank Lemke14
ASIC Readout ChallengesASIC Readout Challenges
DPG – Frühjahrstagung Dresden 2013
Readout ASIC (HUB) needs to support at least 32 link FEE connections
Automatic and flexible initialization and synchronization of FEEs Aggregation of data from numerous FEE devices Rate conversion preserving reliable synchronization Creating an excellent ratio for bandwidth per area
Up to 4x, 5 Gb/s for each lane, 20 Gb/s per HUB
3 HUBs bundled to 12x links with 60 Gb/s total
Specific SERDES designs Standard cell SERDES to connect FEE devices (500Mb/s)
Full custom SERDES with CDR to connect back-end (at least 5Gb/s)
Frank Lemke15
HUB ASIC StructureHUB ASIC Structure
DPG – Frühjahrstagung Dresden 2013
Direct extraction and distribution of synchronization
Deadlock avoiding for data and control communication (VCs)
Large crossbar structures enabling flexible FEE attachment and fault tolerance
Fully remote control through RF
Flexible FEE re-initialization and control
Frank Lemke16
ConclusionConclusion
The CBM protocol and the module concept have shown their usability SPADIC front-end ASIC was successfully tested Tests for STSXYTER are prepared and the chip was submitted Basic FPGA prototyping for ROC usage and HUB ASIC emulations
are ongoing
The goal for 2013 is preparing the first HUB ASIC prototype design as miniASIC for testing ofRadiation tolerance of design partsHigh-speed SERDES functionalityData aggregation and synchronization functionalities
DPG – Frühjahrstagung Dresden 2013
Frank Lemke
Thank you for your attention !
Questions ?
DPG – Frühjahrstagung Dresden 2013