XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University...
-
Upload
colleen-hood -
Category
Documents
-
view
218 -
download
1
Transcript of XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University...
![Page 1: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/1.jpg)
XFEL Large Pixel DetectorDAQ
![Page 2: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/2.jpg)
Project TeamTechnical Team:
STFC Rutherford DAQGlasgow UniversitySurrey University
Science Team:UCLDaresburyBath Universityothers …
![Page 3: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/3.jpg)
Project Outline
1) Phase 1: Develop a digitising pipelined XFEL detector (1k by 1k pixels)
• 3Y Project given approval Dec 2007
2) Phase 2: Construct complete XFEL instruments as required before 2013
- mass produce electronics- match XFEL DAQ
![Page 4: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/4.jpg)
Phase 1 Detector 1M Pixel
4 x 4 Super Modules
![Page 5: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/5.jpg)
Module Concept
![Page 6: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/6.jpg)
XFEL ASIC
New design matched to the XFEL:
- Time Structure- Dynamic Range- Channel Count- System Interfaces etc.
![Page 7: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/7.jpg)
XFEL ASICPreamplifiers
Dynamic range stages
Pixel Resetting
Power supply Conditioning
Deep pipeline memory (786,432 samples)
IO to DAQOverload Control
Sequencing and control
ADC Stages
Power supplies
Store in Pipeline during bunch trainReadout during long gap
![Page 8: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/8.jpg)
XFEL Structure
600 s
99.4 ms
100 ms 100 ms
200 ns
FELprocess
X-ray photons100 fs
Electron bunch trains; up to 3000 bunches in 600 sec, repeated 10 times per second.Producing 100 fsec X-ray pulses (up to 30 000 bunches per second).
XFEL ~ 30 000 bunches/sbut
99.4 ms (%) emptiness
Data Sampling to Memory Serialise and Transmit to DAQ
![Page 9: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/9.jpg)
Multi-gain concept
Required dynamic range compression
– Experience with calorimetry at CERN
– Relaxes ADC requirements
– Fits with CMOS complexity
![Page 10: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/10.jpg)
On detector electronics
![Page 11: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/11.jpg)
Detector Summary
•Phase 1– Common super module design– Economic mass production– Eases test and maintenance– Scalable DAQ
•Phase 2– Mechanical design– Large scale replication– Industrial technology
![Page 12: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/12.jpg)
Tracker View showing Higgs decay to 4 muons
LHC Example
![Page 13: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/13.jpg)
The CMS Tracker• ~210 m2 of silicon, 10M channels• 75000 FE chips, 40000 optical links• 15000 modules mass produced using automatic
assembly techniques• Hybrids and assembly at CERN, FE ASIC Design at RAL
Radiation environment ~10Mrad ionising~1014 hadrons.cm-2
Inner barrel layerRod insertion
Petal assemblyCERN assembly
APV25
![Page 14: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/14.jpg)
>500 cards >20,000 BGAs
• Collaboration with Imperial College and CERN• Massively Parallel Processing ~ 30 VME crates• 10 TERA-bits / sec• 15 Exa-Bytes of raw input per year!
The CMS Tracker DAQ
![Page 15: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/15.jpg)
Project Management
•One overall project manager– Report to XFEL and provide information as
required
•Each workpackage– run as subproject in our QA system
•Approved ISO9000 system– Formal design review processes– Drawing and record control
![Page 16: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/16.jpg)
The First 3 Years:
•Included– Sensor proving tests at LCLS– XFEL ASIC development– Build and test of 1Mpixel system
•Excluded– Off detector DAQ– Sensor R&D
![Page 17: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/17.jpg)
Work Packages
WP1: SensorsWP2: Front End ElectronicsWP3: Mechanical DesignWP4: On Detector ElectronicsWP5: Data AcquisitionWP6: Software, controls and
integration
![Page 18: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/18.jpg)
WP5: Data Acquisition
Leader – John Coughlan
![Page 19: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/19.jpg)
DAQ ApproachUp to now effort has been concentrated on WPs for Mechanics / Sensors / ASICs rather than Readout.
We intend to exploit Commercial Off The Shelf (COTS) based equipment where practical (e.g. FPGA Development boards, vendor/commercial FPGA cores)
Xilinx (Virtex 5) System on ChipRocket IO serial data links , Embedded Ethernet coresEmbedded PowerPCs, Fast memory interfacesEmbedded Development Kit
Industry standards for interface protocols vs custom(GEthernet, PCIe, sFPDP …)
Develop a Scaleable system. Final detectors.
![Page 20: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/20.jpg)
DAQ XFEL IntegrationStay Flexible to adapt to FE ASIC and common XFEL DAQ Architecture specifications.
DAQ/Timing/Trigger Interface Standards & Protocols need to be agreed with XFEL DAQ group.
STFC/Rutherford has a long history of successful partnerships with DESY on Particle Physics projects (e.g. H1 DAQ)
![Page 21: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/21.jpg)
DAQ ExperienceSTFC has a large established base of DAQ hardware and firmware expertise (wide variety of projects… Particle Physics, Xray, Neutron).
STFC has a proven track record on the delivery and long term support of large scale readout systems (e.g. H1 DAQ, CMS Silicon Tracker)
![Page 22: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/22.jpg)
DAQ On Detector
3 Year Plan includes only elements local to detector.
Module Support Cards (MSC) : FPGA Gain Selection
COTS : FPGA Dev BoardsXilinx Virtex 5 + EDK PPCCOTS : FPGA-> PC cores (Qx UDP)
FPGA
Electrical LVDS Links
Switch Inputs / Farm
FEMs : FirmwareData FormattingSample selectionTraffic Shaping
![Page 23: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/23.jpg)
Data Rate Challenge
1 M Pixels x 512 x ~ 2 bytes x 10 Hz ~ 10 GBytes/secProtocol overheadsFixed length fragments?Data Selection, Sparsification?
=> 1 TeraByte recorded every 2 minutes !
![Page 24: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/24.jpg)
Modularity 1 MPixNr Super Modules 16
Nr Hybrids 256
Nr Module Support Cards 32
Nr Front End Modules? 16 x 8 links
Nr Links? (GbE) 80 MB/s 128
x N MPixel x N
Each FEM Fragment = 128 KB (64 MB / train)128 KB x 512 x 10 = 640 MB/s @ 80 MB/s link = 8 links / FEM
![Page 25: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/25.jpg)
DAQ EoI Off Detector
E.g. ATCA crate for Surface & Nuclear Science AGATA Daresbury & PadovaATCA card withFPGA->PC PCIe readout
Event BuilderAdvanced TeleComms ArchitectureCOTS Carriers + AMC MezzaninesOr MicroTCA
6 TB SFPDP Disk Storage and Servers
![Page 26: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/26.jpg)
WP6: Software, controls and integration
Leader – Tim Nicholls
![Page 27: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/27.jpg)
Software, Controls & Integration
Delivery and Operation of Detector in beam-line environments.
System Operation Timing and Controls.
Software Development and Integration with Beam-line scientists.
Real Time Data Monitoring and Analysis.
Team of Integration engineers, leader with experience on DESY H1 experiment.
![Page 28: XFEL Large Pixel Detector DAQ. Project Team Technical Team: STFC Rutherford DAQ Glasgow University Surrey University Science Team: UCL Daresbury Bath.](https://reader035.fdocuments.in/reader035/viewer/2022062423/56649e525503460f94b47aaf/html5/thumbnails/28.jpg)
Summary
1) Phase 1: Develop a digitising pipelined XFEL detector (1k by 1k pixels) 3Y starting Jan 2008
2) Phase 2: Construct complete XFEL instruments as required before 2013
- mass produce electronics- match XFEL DAQ Architecture