T OWARDS A S YSTEM D YNAMICS M ODELING M E- THOD B ASED ON DEMATEL
PCI B ASED R EAD-OUT R ECEIVER C ARD IN THE ALICE DAQ S YSTEM
19
9-13 September, 20 02 PCI-based Readout Receiver Card - LECC 2002, Colmar 1 PCI BASED READ-OUT RECEIVER CARD IN THE ALICE DAQ SYSTEM F. Carena 1 , W. Carena 1 , P. Csató 2 , E. Dénes 1 , T. Kiss 2 , J. C. Marin 1 , R. Divià 1 , K. Schossmaier 1 , C. So ós 1 , J. Sulyán 2 , A. Vascotto 1 , P. Vande Vyvre 1 1 CERN/EP-AID (Geneva) 2 KFKI-RMKI (Budapest)
description
PCI B ASED R EAD-OUT R ECEIVER C ARD IN THE ALICE DAQ S YSTEM. F. Carena 1 , W. Carena 1 , P. Csat ó 2 , E. D é nes 1 , T. Kiss 2 , J. C. Marin 1 , R. Divi à 1 , K. Schossmaier 1 , C. So ós 1 , J. Suly án 2 , A. Vascotto 1 , P. Vande Vyvre 1 1 CERN/EP-AID (Geneva) 2 KFKI-RMKI (Budapest). - PowerPoint PPT Presentation
Transcript of PCI B ASED R EAD-OUT R ECEIVER C ARD IN THE ALICE DAQ S YSTEM
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 1
PCI BASED READ-OUT RECEIVER CARD IN THE
ALICE DAQ SYSTEM
F. Carena1, W. Carena1, P. Csató2, E. Dénes1, T. Kiss2, J. C. Marin1, R. Divià1, K. Schossmaier1, C. Soós1,
J. Sulyán2, A. Vascotto1, P. Vande Vyvre1
1CERN/EP-AID (Geneva)2 KFKI-RMKI (Budapest)
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 2
ALICE DAQ SYSTEM
DAQ Network
Event builder
Local Data Collector
PRORCDDL
LDC
PRORCDDL
LDC
PRORCDDL
sub-events
full events
PRORCDDL
eventfragments
DDLDetector
DDLDetector
DDLDetector
DDLDetector
Fro
nt-E
ndE
lect
roni
cs(F
EE
)
Front-End Electronics
(FEE)
Source Interface
Unit (SIU)
Rea
d-O
utR
ecei
ver
Car
d(R
OR
C)
Destination Interface
Unit (DIU)
PCI-basedRead-out Receiver
Card (PRORC)
Multimode optical cable
Event builder
Det
ecto
r D
ata
Lin
k (D
DL
)
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 3
FEATURES
• Interface between the DIU and PCI local bus– 32 bit/33 MHz PCI version, max. throughput 132 MB/s
• PCI master capability, data push architecture– Autonomous operation with little software assistance
– Supports multi-paged memory management
• Direct data transfer to the PC memory– No local memory on the board
– Small elasticity buffers between different clock domains
• Built-in test capability– Internal pattern generator can produce formatted data
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 4
HARDWARE
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 5
PCI INTERFACE
Incoming
Mailbox
PCI to local busFIFO (8 x 32 bit)
Local bus to PCI FIFO (8 x 32 bit)
Outgoing
Mailbox
Local bus interface
PCI bus interface
(32 bit/33 MHz)
PC
I bu
s
AMCC S5935
Loc
al b
us
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 6
FIRMWARE
AM
CC
log
ic in
terf
ace
Read DMA controller
Write DMA controller
Memory manager
and command interprete
r
DDL interface
Receiver FIFO
Transmitter FIFO
Pattern Generator
PRORC firmwareD
DL
AM
CC
S59
35
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 7
FIRMWARE: DDL INTERFACE
• Handles the full-duplex bus between the PRORC and the DIU
• Provides clock domain separation using dual port FIFO memories
• Transmit data and command to the DIU
• Receive data and status from the DIU
• Inject data into the DDL or the receive FIFO using the pattern generator
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 8
FIRMWARE: INTERNAL CONTROL
• Interprets commands passed through the mailboxes
• Controls other firmware block according to the required operation
• Manages the DMA control registers in the AMCC
• Handles the Free FIFO
• Includes the read and write DMA engines
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 9
FIRMWARE: AMCC INTERFACE
• Controls the half-duplex bus between the AMCC and the FPGA
• Performs arbitration for different local bus accesses
• Manages FIFO read and write operations
• Manages mailbox read and write operations
• Handles the hardware interrupt generated by the AMCC
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 10
PRORC PC memory bank
OPERATION: THE FREE FIFO
Firmware
readout
page address
page address
page address
Free FIFO
PC CPU
Allocation of free pages
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 11
PRORCFirmware
PC memory bank
DDL
No involvement
OPERATION: DMA TRANSFER
PC CPU
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 12
PRORC PC memory bank
OPERATION: THE READY FIFO
readout
DDL
Ready FIFOFirmware
addresspage status
addresspage status
addresspage status
Delivery of filled pages
PC CPU
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 13
Front-End Emulator Interface Card (FEIC): it emulates a detector readout– It generates data blocks of random size up to 1 MB
– It drives the DDL SIU
– It is sensitive to back-pressure
Source side• Two FEIC with SIU
Destination side• Two PRORC+DIU on a PCI bus
and Gigabit Ethernet on the other
• A PC 2 x Pentium III 1 GHz with two PCI bus(32bit/33MHz and 64bit/66MHz)
• Linux operating system
• ALICE data acquisition software (DATE)
THE TEST BED
PRORC
DIU
FEIC
SIU
FEIC
SIU
PRORC
DIU
DDL
LDC
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 14
LDC
Single PRORC w/o Event Builder
DDL saturated for block size above5 kB:
– 101 MB/s
Event rate saturated for block size below 5 kB:
– 35 000 events/s
– PRORC handling overhead in LDC: 28 µs
• The firmware/software system withstands the DDL rate• The nominal system specification has been met
One PRORC
0
20
40
60
80
100
120
0 10 20 30 40 50 60 70
Block size (kB)
Da
ta r
ate
(M
B/s
)
0
5000
10000
15000
20000
25000
30000
35000
40000
Ev
en
t ra
te (
1/s
)
Data rate MByte/sec Event rate per sec
FEIC SIU PRORCDIU
TEST I: SINGLE PRORC
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 15
LDC
Two PRORCs w/o Event Builder
Same saturation pattern, but the PCI bus is the limiting factor:
– 127 MB/s
Software overhead scales:
– 18 500 events/s
– PRORC handling overhead in LDC: 54 µs
The firmware/software system fully exploits the PC architecture
FEIC SIU PRORCDIU
FEIC SIU PRORCDIU
Two PRORCs
0
20
40
60
80
100
120
140
0 10 20 30 40 50 60 70
Block size (kB)
Dat
a ra
te (
MB
/s)
0
5000
10000
15000
20000
Ev
en
t ra
te (
1/s)
Data rate MByte/sec Event rate per sec
TEST II: TWO PRORC CARDS
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 16
LDC
Single PRORC with Event Builder
The overall performance is determined by the speed of the Gigabit Ethernet card:
– 70 MB/s
Software overhead does not change:
– 37 000 events/s
– PRORC handling overhead in LDC: 27 µs
The system behaves as expected and the performance fulfills the needs
FEIC SIU PRORCDIUEvent
Builder
PRORC + LDC + Event Builder
0
10
20
30
40
50
60
70
80
0 10 20 30 40 50 60 70
Block size (kB)
Dat
a ra
te (
MB
/s)
0
5000
10000
15000
20000
25000
30000
35000
40000
Ev
en
t ra
te (
1/s)
Data rate MByte/sec Event rate per sec
TESTS III: PRORC + EVENT BUILDER
GbE
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 17
APPLICATIONS
• Inner Tracking System (ITS), Silicon Drift (SD) detector– Dedicated ASIC performing DSP functions (CARLOS)
– Interface to the DDL (CARLOS-rx)
– Test patterns generated by a pattern generator have been collected using DATE
• Time Projection Chamber (TPC)– Prototype Readout Control Unit (RCU) is tested with the PRORC
– TPC sector is going to be tested at CERN
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 18
SUMMARY
• Hardware, firmware and software have been developed– The card is ready for production
• PRORC provides efficient readout of the DDL– 100 MB/s nominal speed has been achieved
• PRORC and DDL have been successfully integrated in DATE– Long-term tests show stable performance
– Tested with the DDL using Front-end Emulator Cards
– Tested in standalone mode using the Embedded Data Generator
9-13 September, 2002 PCI-based Readout Receiver Card - LECC 2002, Colmar 19
