Atlas SCT ROD FDR Implementation Model
description
Transcript of Atlas SCT ROD FDR Implementation Model
Atlas SCT ROD FDRImplementation Model
August 20, 2002
Atlas Wisconsin GroupKhang Dao, Damon Fasching, Douglas Ferguson, Owen Hayes,
Richard Jared, John Joseph, Krista Marks, Mark Nagel
Sriram Sivasubramaniyan (Oklahoma), Alden Stradling and
Lukas Tomasek (Institute of Physics AV CR, Prague)
2
ROD Overview Notes• FPGA real time data path with DSPs for back-end processing and
operation control.– Different firmware and part loaded for SCT or Pixel RODs
• 9U VME card, custom back plane.• Back-of-Crate Card for optical interface to modules.• Clock and Trigger.
– Clock and trigger signals are available on the back plane from the Trigger Interface Module in the crate.
• Supports 100 kHz trigger rate without an upgrade.• SCT: 96 input links at 40 MHz• Diagnostic FIFO Memories on board for testing and data monitoring• Register based interface with DSP list processor.• Fiber Optic S-Link to Readout Buffers.• 12 FPGAs, 4.7 Million gates of programmable logic.• 5 DSPs, One for control and 4 for calculations.• Cypress Semiconductor VME Slave Interface chipset.
– Block Transfers - 2.2 Mbytes/second Slave DSP read access.
– A32/D32 data transfers
– Simple interface and supports standard VME transactions w/ interrupts.
8/20/2002 3
ROD Implementation Model: FPGA Front End / DSP Back End
96Data RCVR
ROD Controller: Operation,
Command, Trigger
43
Formatter and EFB
FPGA96
Bac
k of
Cra
te C
ard
/ Fro
nt E
nd E
lect
roni
cs
Input Memory & XCVR Control
VME Slave Interface
Sli
nk o
n B
OC
VM
E B
us
32
Cont / Addr
Data Bus
Con
trol
, Sta
tus,
Mas
k
Trig Data (TTC Bus)
Tim
ing
Inte
rfac
e
Mod
ule
BOCSetup Bus
Serial Output Link
Serial Input Link
Xon/Xoff
ROD Busy
Dec
. Tri
g C
ount
Control & Status for Block Xfer/ DMA / S-Link
DSP Event Trapping,
Histograms (Four DSP Chips)
DM
A 1
Router FPGAS-Link & DSP DMA
Host Port Interface
Data Valid
S-link Data
DM
A 4
Router Halt Output
DMA Cont.
Eve
nt D
ata/
Tir
g T
ype
R/W Bus
R/W Bus
R/W Bus
...
Bus
y
Program ResetManager
Tok
en a
nd D
ynam
ic M
ask
S-link Data
S-link WR
S-link Clk
S-link Test
48
R/W Bus
Cnf
g, R
eset
Cnfg, Reset
8/20/2002 4
48
Link or Diagnostic
Data to Formatter
48Bac
k pl
ane
Con
nect
or
Input FIFO
4k words
48
48
Data
DataInput FIFO
4k words
Link Input from BOC
R/W Bus Data
Input Memory & XCVR Control
Control FIFO Operation
ADV OE
48 bits are read as 32 bit word bits 0:3116 bit word bits 32:47
Advance FIFO and latch new input
Enable FIFO output data
Read or write data
Select 16 or 32 bit part of word
Retransmit FIFO
Data Receiver Implementation
FIFO O
ut R
egister
FIFO In
Register
FIFO D
ata R
egister
FIFO D
ata R
egister
ADV OE RT
8/20/2002 5
• Input Diagnostic Memories.– Play or record data to perform diagnostics and tests.
– Start of record operation is triggered by the ROD Controller.
– Trapping (record) of Link Data can be synchronized to triggers or configuration commands.
– The controller can retransmit the list in a loop. The retransmit can be issued using the serial inputs (SP0 or SP1) from the DSP, or by a preset counter used in one of the Test Bench modes. Trigger information (from the ROD TIM FIFO) can also be retransmitted so that the ROD can have trigger rates up to 100kHz.
– The SCT ROD implementation of the Input Diagnostic Memories is two groups of 4k x 48 bit FIFOs. The Pixel ROD will use 64k deep FIFOs.
• Drivers and Registers– The input drivers allow the input link data to be routed to the Formatters and FIFOS
simultaneously. The FIFO can not be read by the DSP while the Input Links are connected to the Formatters.
– The Register ICs are used to latch the data from the ROD bus or the FIFO. The FIFOs receive data from the ROD Bus from the DSP through the Controller. The ROD Bus is not setup to handle a burst from a FIFO.
Data Receiver Notes
8/20/2002 6
32
96
Control, Status, MaskDec.
Trig Count
BusyMode Bits
Formatter 0Serial to Parallel ConversionFlag Header and Trailer ErrorsData Modes: Raw, Condensed, ExpandedHeader Trailer Limit Busy LimitDerandomizing FIFO4k Deep Debug Memories
R/W Bus
TokenCmd
Input Links
12Input Links
Formatter 1
Formatter 7
Formatter 6
Formatter 5
Formatter 4
Formatter 2
Formatter 3
12Input Links
12Input Links
12Input Links
12Input Links
12Input Links
12Input Links
12Input Links
*Proposed for Pixel ROD: Formatters 0,1,4,5 only
Event ID Checking,
Dual EFB Design
Event Formatting
Error Counters
Event Fragment Builder FPGA
Event ID Data /
Trig Type Dynamic
Mask
46
46
16Kx46 FIFO
32
Control
EV Fragment Header/Trailer
16Kx46 FIFO
Control
Data Valid
Gatherer Halt Output
Almost Full
Almost Full
Data
32
46
Halt Output
Halt Output
Data
Data Valid
Data Valid
Format #
Format #
R/W Bus
To/
Fro
m R
oute
r F
PG
A
To/From Controller FPGATo/From Controller FPGA
Token01
Cont header,trailer or data
Debug FIFO
4k x 39B
Debug FIFO
4k x 39B
FPGA Formatter and EFB
Token45
TokenCmd
Token30
Token74
8/20/2002 Page 7
Formatter 0: Top Level
Link FormatterAnd
Link OutputFIFOs
Link[11..0]DATA 12b
FifoDATA 32b
Link Number:12:4 Encoder
Link# hasTOKEN
HOLDoutput 1b
Formatter Writes Trailer Encoder:
latch Trailers and store new internal stateon ShowTrailerFLAGS
FormatterBUS 26bfrom RodContoller
ShowTrailerFLAGS 1b
L1TriggerPulse 1b
TOKENin 1bfrom previous chip
FormatterWritesTrailer 12b
DFF
HeaderTrailerLimit
EFB
RodController
FormatterBUSStrobe 1b
HeaderTrailerLimit[11..0] 12b
FormatterWritesTrailer[11..0] 12b
TimeOutError[11..0] 12b
Link[11..0]hasTOKEN 12b
TOKENout 1b
to next chip
ModeFifoEmpty[11..0] 12b ModeFifoEmpty 1b
FifoDATA 32b
HOLDoutput 1b
ModeBitsIN[11..0] 12b
RegTransferACK 1b OperationsController:
Register Block
MasterNSlave 1b
DataValid[11..0] 12bCondensedMode[11..0] 12b
RODBusyLimit[11..0] 12b
HeaderTrailerLimit[11..0] 12b
RODBusyLimit[11..0] 12b
Link[11..0]hasTOKEN 12b
ThisChipHasToken
RODBusyLimit
DataValid 1bDataValid[11..0] 12b
TimeOutError 1bTimeOutError[11..0] 12b
CondensedMode 1bCondensedMode[11..0] 12b
StoreTrailerHit
1ea2 state state machine
ClearCurrentFlag
TokenToReadOutTimeout 7bDataOverflowLimit 8b
HeaderTrailerLimit 8bRodBusyLimit 8b
OverflowError[11..0] 12b
OverflowError 1bOverflowError[11..0] 12b
RodBusyLimit 8b
Configuration (12 x 4b)
Status (12 x 16b)
TokenToReadOutTimeout
DataOverflowLimit
HeaderTrailerLimit
RodBusyLimit
Configuration(11:0) FIFO Word Counts(12 x 8b)
8/20/2002 8
Static Link Mask 1 bit per
link
Data Valid to EFB
EFB Halt Output
Condensed Mode and Raw Mode
Selects R/W Bus
Serial to Parallel Conversion and correction of 1 bit errors in header and trailerRecord error in data words
...
12 ea. converters
Link 0
Link 1
Link 11
Link 10
Config, Raw ABCD format per link bases Condensed 2 hits per 16 bits & not Config, RawNot condensed and not Config, Raw Expanded 32 bits for 1 or 2 hits
Serial to Parallel Conversion and correction of 1 bit errors in header and trailerRecord error in data words
FIFO 256 W by 32 bits
FIFO 256 W by 32 bits
...
12 ea. FIFOsandControl
Readout Control
Token from Controller
ContTrailer & Word in
Data to EFB
Token to Next Link
Readout Control
ContTrailer & Word in
Token to Next Formatter
Token from Previous Link
ValidTrailer
Readout controller provides: Readout of data on token Link time out error detection, header generation Data over flow detection Header trailer limit Busy function and limit Dynamic mode bits Value Meaning 00 Normal 01 Skip Readout 10 Mask 11 Dump 1st read 2nd Trailers to Controller
ValidTrailer
Busy
Busy
Busy toController
Trailers to ControllerDecrementTriggerCounters
Mux
Dynamic Mode Bits
2 bits per link
Link on/off
32 b
32 b32 b
7 b
7 b
39 b
39 b
2 b
2 b
HeaderTrailer Limit to Controller
FatalError to Controller
Edge ModeTimeoutdata_to_Large
R/W Bus
Formatter 0: 12 Links Shown
8/20/2002 9
.Formatter FPGAs have separate versions of logic for SCT and Pixel Modules.
.Link output FIFOs are implemented with FPGA internal memory. The XCV400E has 40 4Kbit RAM Blocks that can be implemented as 256x16 Dual Port RAM. The SCT Formatter must accept 12 input links, and each Link FIFO must be 32 bits wide. The maximum size of the Link FIFO with 12 links of input data is 256 words deep.
.The next version of the Formatter will provide dynamic memory allocation to provide a 512 words deep FIFO if one link from a module is masked off. This modification will combine 256x32 FIFOs on adjacent links in the case that the Static Mask Bit is set.
.Formatter can keep up with any input data rate (within memory availability).
.Only headers and trailers shall be written to the Link FIFO when the occupancy reaches almost full (Header/Trailer Limit). The Header Trailer Limit value is set by writing to the HT Limit Register in each Formatter. The value a global setting for each group of 12 links.
.Individual links can be put in raw, expanded or condensed hit data mode (SCT) by writing the appropriate value to the Formatter Configuration Register.
Formatter
8/20/2002 10
. Sorts data by sequentially transferring one event of data from each link to the Event Fragment Builder when the Controller issues a FE Command Pulse (Token Command).
. When FIFO Dual Port Memory is almost full:– The Decoder forces the links that are almost full (Header/Trailer Limit)into write
header/trailer only mode to reduce bandwidth.– If Memory continues to fill, and the Link FIFO reaches it set value of ROD Busy
Limit, full flag, the ROD Busy signal is issued to to stop triggers. At this point, an error flag is issued to interrupt the ROD controller - fatal error condition(ROD busy)
.This should not be possible unless flow control has been asserted by the S-Link for an extended period of time, Header/Trailer Limit is set to the same value as ROD Busy Limit, or the front end electronics have malfunctioned.
. Support flow control - EFB can assert pause signal. The Link FIFO will continue to fill and can become full if back pressure is asserted for an extended period of time. In this condition, the ROD will lose any event data that is on the FE links, but will retain all events that are in the Link FIFOs.
Formatter
8/20/2002 11
. If the token has been sent to a specific link, and there is no data in the FIFO, the link will timeout after an internal counter reaches the value set in the Token to Readout Timeout Register. In this case the Link Formatter will output a header and a trailer with the timeout error bit flagged, and pass the token to the next link.
. Events that exceed the length limit defined for a specific subset of links will have data deleted and a header trailer generated with an error flag.
. Individual links can be masked off by writing a 1 to the Static Mask bit in the Formatter Configuration Register or dynamically using the Mode Bits. This allows the user to mask off known malfunctioning links from a list or table or on the fly.
. Correct for loss of synchronization (module data is ahead or behind other modules) in a module (not required but is in VHDL). The dynamic mode bits correct for a link by advancing or withholding the link data with respect to the other links. The values are shown below: Dynamic mode bits: Value Meaning
00 Normal Readout Mode 01 Skip Readout of FIFO for this Trigger 10 Mask Link for this Trigger (full contents of the FIFO will be flushed) 11 Dump current event, read second for this trigger
The L1 ID number is corrected in the EFB. BCID numbers are ignored.
Formatter
8/20/2002 12
.Serial-to-Parallel conversion with error checking. Errors Detected: Header 1 bit error Trailer 1 bit error Condensed mode data error detection Link time out error in readout Header trailer limit Data too large limit
Header Errors Accepted: 111010 Head Det. Head Error 111010 Yes No1111010 Y Yes 011010 Y y 101010 Y y 110010 Y y 111110 Y y 111000 Y y 111011 Y y1110100 Y y
Trailer Errors : Trailer 1000000000000000A changed bit in any position is accepted but trailer error is set.If in state “new_cluster” and trailer_det=1 set trailer error. Example below:<11101><0><nnn><bbbb,bbbb><1><01><aaaa><ccc,cccc><1><ddd> chip channel hit ^ Trailer Detect error
Condensed mode error detection:The ROD variable “edge_mode” is used to determine if the input hit data is consistent with the state of this variable. If not a condensed mode error bit is set.
Link time out error detection:If a link has not received data in the time set in the time timeout register, a header and trailer will be written with error code.
Data too large error detection:This is a historic state that is not longer needed. If the event is to large (based on a register) a status bit is set.
Header trailer limit error detection:If the Formatter header trailer limit is passed. The formatter will not place new data in the FIFO. When a trailer is detected the trailer is written with the header trailer limit error code.
Synchronization error detection:If one of the field separator bits is incorrect. The decoding is confused. The link will be changed to raw data till a trailer is detected.
Formatter
8/20/2002 13
.Formatter Data Format SCT all data is in 16 bit packets packed in 32 bit words bit 31:0 Data bit 35;32 Link Channel address (for all 32 bit words) bit 36 Time Out Error (for all 32 bit words) bit 37 Condensed mode bit (for all 32 bit words)
Formatter Output (bits 31:0 bits 37:32 for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBtrailer 010zhvxxxxxxxxxx1 hit cond 1FFFFCCCCCCCxfx02 hits cond 1FFFFCCCCCCCsfx11st hit clus exp 1FFFFCCCCCCC0DDD1 hit clus exp 1xxxxxxx0xxx1DDD2 hit clus exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWcond=condensed, clus=clusterexp=expanded
Key :x= do not care (The ROD fills these with 0’s)B=BCIDW=raw data C=cluster base addressD=3 bit hit dataE=ABC error codeF=FE numberh=header trailer limit errorL=L1IDn=count of raw dataf=error in condensed mode data, 1st hits=error in condensed mode data, 2nd hitp=preamble errorz=trailer bit errorv=data overflow error
Condensed example W1 001pLLLLBBBBBBBB 1FFFFCCCCCCCxfx0 Header+1 hitW2 010zhvxxxxxxxxxx 0000000000000000 trailerW1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hitW2 1FFFFCCCCCCCxUx0 010zhvxxxxxxxxxx 1 hit+trailer
Raw data example W1 001pLLLLBBBBBBBB 011NNNxxBBBBBBBB Header+8bitW2 011NNNxxBBBBBBBB 010zhvxxxxxxxxxx 8 bit+trailer
Expanded example W1 001pLLLLBBBBBBBB 1FFFFCCCCCCC0DDD Header+1 hitW2 1xxxxxxx1DDD1DDD 1xxxxxxx0xxx1DDD 2 hit +1 hit W3 010zhvxxxxxxxxxx 0000000000000000 trailer
Condensed to raw on errorCorrectW1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hitW2 1FFFFCCCCCCCxfx0 010zhvxxxxxxxxxx 1 hit+trailerError on 1 before 3rd hit (could have done it before 2nd hitW1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hitW2 011nnnxxWWWWWWWW 010zhvxxxxxxxxxx raw+trailer
Format will switch to raw on the following errors for the rest of the event.0 after beam crossing numberfirst bit of hit data leader is 1bit before hit pattern is 0
Formatter
8/20/2002 14
39
Link or Diagnostic
Data to EFB
39
Input FIFO
4k words
39
39
Data
DataInput FIFO
4k words
Data Input from Formatter
R/W Bus Data
Input Memory & XCVR Control
Control FIFO Operation
ADV OE
48 bits are read as 32 bit word bits 0:3116 bit word bits 32:47
Advance FIFO and latch new input
Enable FIFO output data
Read or write data
Select 16 or 32 bit part of word
Retransmit FIFO
Debug FIFO Implementation
FIFO O
ut R
egister
FIFO In
Register
FIFO D
ata R
egister
FIFO D
ata R
egisterADV OE RT
OE
OE
8/20/2002 15
• Debug Diagnostic Memories.– Play or record data to perform diagnostics and tests.
– Start of record operation is triggered by the ROD Controller.
– Trapping (record) of Formatter Output Data can be synchronized to triggers.
– The SCT and Pixel ROD implementation of the Debug Diagnostic Memories is two groups of 4k x 39 bit FIFOs.
Debug FIFO Notes
8/20/2002 16
39
Control, Status,
Event Data /
Trig Type
43
43
16Kx46 OutputFIFO
32
Output Control
EV Fragment Header/Trailer
Event Data Valid
Router Halt Output
Almost Full
Almost Full
Data
43
Halt Output
Data Valid
Format #
R/W Bus
To/
Fro
m R
oute
r F
PG
A
To/
Fro
m C
ontr
olle
r F
PG
A
EFB Data Engine 1Event ID checkingError CountersFormat event
39Data
Halt Output
Data Valid
Format #
DerandomizingFIFO 4k x 16b
Header FIFO 64w x 32b
Engine FIFO 32w x 107b2 ea.
EFB Data Engine 2Event ID checkingError CountersFormat event
Event Data
Event Data
Output Control
ErrorSummary
Word counts
Header/trailer EventFragment Generation
Data Output Control
16Kx46 OutputFIFO
Read Event
Dyn Mask &L1-BCO
Data Type
Event Fragment Builder (EFB)
8/20/2002 17
. Provides two parallel streams with data processing engine from half of the input links to mask the Formatters link-to-link latency.
– Each engine has a 16k word output FIFO for derandomizing of data and to hold large events.
– The two data processing engines helps achieve 40 MWords/sec bandwidth to the S-Link. . If “Router Halt Output” is issued, data can still flow through the EFB and fill the output memory.
. If the output memory is full back pressure will be applied to the Formatter.
. EFB checks Event ID (BCID and L1ID) and generates errors. The BCID error checking can be disabled by setting a bit in an EFB Internal Register.
. Performs summary error counts with mask Counting of errors can be masked on a link by link basis by writing the appropriate value to the Error Mask Registers in the EFB.
. The EFB formats input data event for the S-Link.
. Generates S-Link compatible headers and trailers. The information for this step is sent to the EFB from the Controller over the Dynamic Mask Data bus. The Event ID and Dynamic Mask for a specific Event must be written to the EFB before a Token can be issued to the Formatter to read out the link FIFOs.
EFB: Overview
8/20/2002 18
• The Event ID,Trigger Type, ECR, and Dynamic Mask are sent to the EFB from the Controller. This data arrives after TIM has transmitted an L1 trigger, Serial ID and Trigger Type data to the Controller and before a FE Command Pulse can be sent to the Formatters.
• The Event ID and Dynamic Mask is data set of 16 half words. The data format is shown on the next slide.
• The EFB stores the 16 16bit words per event in a 4kword x 16 FIFO, transfers this group of data to 3 FIFOs used for Event Header generation, L1ID/BCID checking, and synchronization correction.
• The Event Header FIFO is 16 words by 64 bits, stores the L1ID, BCID, ECR, and Trigger Type, and is used to build the Event Headers when the EFB is ready to send an event to the Router
• Each EFB engine require a Dynamic Mask FIFO. This FIFO is 16 words by 64 bits, stores the L1ID, BCID, and the Dynamic Mask bits required by each link for correction.
• The Event Header and Dynamic Mask FIFOs are ready for data processing 20 clocks after the EFB receives the full data set from the Controller.
EFB: Event ID and Dynamic Mask
8/20/2002 19
Word Name Engine FIFO Header FIFO Word 0 L1 ID [15:0] [3:0] yesWord 1 ECR ID [7:0] & L1 ID [23:16} no yesWord 2 BC ID [11:0] & 0000 [7:0] yesWord 3 TIM EvtType [1:0] & TrigType [7:0} no yesWord 4 ROD Event Type [7:0]/(spare bits) [15:8] no yesWord 5 Dyn Mask [7:0] yes noWord 6 Dyn Mask [15:8] yes noWord 7 Dyn Mask [23:16] yes noWord 8 Dyn Mask [31:24] yes noWord 9 Dyn Mask [39:32] yes noWord 10 Dyn Mask [47:40] yes noWord 11 Dyn Mask [55:48] yes noWord 12 Dyn Mask [63:56] yes noWord 13 Dyn Mask [71:64] yes noWord 14 Dyn Mask [79:72] yes noWord 15 Dyn Mask [87:80] yes noWord 16 Dyn Mask [95:88] yes no
Dynamic mask 95:0 is 2 bits per link, used to correct module synchronization. DM Bits [1:0] “00” no change to L1 ID for this event DM Bits [1:0] “01” Increment L1 ID for this event DM Bits [1:0] “10” Decrement L1 ID for this event
ECR = Event Counter Reset, TrigType = ATLAS Trigger Type
EFB: Event ID and Dynamic Mask Data Format
8/20/2002 20
3943
Data Link 1-48
BCID / L1ID
Check
Error Detect, Count
Format Data & Count
Halt OutputOutput FIFO Almost Full
Output FIFO Data
Error Summary Datalnput bits to error summary Header bit error (0) Trailer bit error (1) Flagged error (2) Sync error (3) Hit pattern error (4) L1ID error (5) BCID error (6) Timeout error (7) Almost full (8) Data overflow (9)If summary AND Mask =TrueCount = Count +1Summary Error = Summary Error | current Error
Data Valid
Format #
Add or subtract from L1 number
Dynamic Mask &L1-BCO
Mode bits add or sub 1
from L1ID
L1IDand
BCO
R/W Bus
Mask (96 ea.) Summary and Count to Error FIFO
Word count to WC FIFO
EFB: Data Engine - 1 Engine Shown
WR Output Data
Mask BCID Check
8/20/2002 21
• Correct L1 ID Number
Based on the Dynamic Mask bits, the L1 ID increments or decrements. This is used when a link is resynchronized on the ROD without stopping ATLAS triggers.
• BC ID / L1 ID Check:For Link Header, compare expected BCID and L1ID with the decoded data and flag any errors. All other data types are ignored.Separate Event FIFOs provide L1/BCID data so each EFB data path can work independently on different events.BCID check can be globally disabled if required.
• Error Detect:See EFB Data Engine above for error bit definition. Looks at data bit fields to identify errors.Generates the error summary block for the event.One maskable error counter per link.One global error counter BCID errors can be ignored for a link that has been resynchronized
EFB: Data Engine
8/20/2002 22
• Error Bits Summary in the Link Error Mask Register:
Bit(0) => Header bit errorBit(1) => Trailer bit errorBit(2) => Flagged errorBit(3) => Sync errorBit(4) => Hit pattern errorBit(5) => L1ID errorBit(6) => BCID errorBit(7) => Timeout errorBit(8) => Almost fullBit(9) => Data overflow
• Format Data and Count:1. Removes front end trailer words unless the trailer error flag or the almost
full flag was set.2. Removes header from links with no hit data and no errors.3. After unnecessary half words are removed, it packs all the data into 32 bit
words so there are no gaps in the data.4. Counts number of words sent to the output FIFOs over an event and
writes the count into a FIFO.
EFB: Data Engine
8/20/2002 23
Header/Trailer Event Fragment Generation
Gather Error Summary, Word Counts and Event ID data and raise Event Data Valid
Then Control Event Readout
Halt output suspends readout
32
Event Header Data
FIFO:Error Summary Data
FIFO:Word Count
FIFO: Header Event ID Data
Router Halt Output
FIFO:Word Count
FIFO:Error Summary Data
Engine 1Engine 1Engine 1 Engine 2
Event Trailer Data
32
Event Data Valid
Data Type
To\from
Rou
ter
Data FIFOs Output Control
3
EFB: Header/Trailer Generation
8/20/2002 24
• Header/Trailer Block Generation, Data Output Control:– After the data counts are available from both EFB engines, an event is
delivered to the Router then to the S-LINK (data is pushed).• A complete event must processed and written to the output FIFOs
before Event Data Valid is true.• An Event Header block is first transmitted to the S-Link.• Data is then transmitted from the output FIFOs. • Finally, the summary and event fragment trailers are transmitted.
• Pause:– If The DSP asserts Stop (calibration mode only) or the S-Link asserts Xoff,
data will not be transmitted, but the EFB can still process data and write to the output memories.
– If any of the internal EFB FPGA FIFOs are almost full or the output FIFOs are almost full, the FIFO controller will be told to pause and back pressure will be applied to the Formatters.
– Output FIFOs are big enough to hold largest SCT/Pixel Event.
EFB: Header Generation
8/20/2002 25
Event HeaderWord Contents Comment
0 0xB0F00000 (beginning of fragment)1 0xEE1234EE (start of header)2 0x9 (header size)3 TBD (format version number)4 TBD (source identifier)5 L1 ID (level 1 ID)6 BC ID (bunch crossing ID)7 L1 TT (ATLAS Level 1 trigger type)8 DET (detector event type ROD or TIM)
Event TrailerWord Contents Comment
0 Error count (Status 1: count of words with error)1 Error flags (Status 2: Bit error type occurrence)2 0x2 (Number of status words)3 nData (word count of data words)4 0x1 (status block position: 0/1 = before/after data)5 0xE0F00000 (end of fragment)
EFB: Output Format
8/20/2002 26
EFB Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBtrailer 010zhvxxxxxxxxxx1 hit condensed 1FFFFCCCCCCCxfx02 hits condensed 1FFFFCCCCCCCsfx11st hit clust expan 1FFFFCCCCCCC0DDD1 hit clust exp 1xxxxxxx0xxx1DDD2 hit clust exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWclust=cluster, exp=expanded
Key :x= do not care (The ROD fills these with 0’s)B=BCIDW=raw data C=cluster base addressD=3 bit hit dataE=ABC error codeF=FE numberh=header trailer limit errorL=L1IDn=count of raw dataf=error in condensed mode data, 1st hits=error in condensed mode data, 2nd hitp=preamble errorz=trailer bit errorv=data overflow errort=time out errorK=condensed model=L1 errorb=BCID errorM=link number
.Link Data Format SCT all data is in 16 bit packets packed in 32 bit words bit 31:0 Data bit 42-32 are valid for header word only bits 42:32 blktMMMMMMM
EFB Output examples for two eventsBits 42:32 Bits 31:16 Bits 15:0 Event header 9 words (Normal event) blKtMMMMMMM link header CondensedblKtMMMMMMM Condensed link header Note:link trailers Condensed condensed are removed on noblKtMMMMMMM link header condensed trailer error, link condensed condensed by link event trailer 6 words
Event header 9 words (a trailer error) blKtMMMMMMM link header CondensedblKtMMMMMMM Condensed link header Condensed condensed blKtMMMMMMM link trailer link header Note:error in trailer condensed condensed condensed link trailer Note:special trailer
fill event trailer 6 words
If there are no errors in trailer and header and no data the link header is not written.
EFB: Output Link Format
8/20/2002 27
Router Implementation: Top Levelevent_info_in(45:0)
tristate_n_in(2:0)
data_valid_in(1:0)
slink_xoff_in
router_bus_data_inout(15:0)
slink_bad_in
slink_data_out(31:0)
slink_wen_out
slink_rst_out
dsp0_event_data_out(31:0)
dsp0_dataframeready_out
dsp3_event_data_out(31:0)dsp3_ dataframeready _out
slink_test_out
stop_output_out
dsp0_sbsram_clk_indsp0_send_data_in
dsp0_mask_dbr_in
dsp0_sbsram_me_in
router_bus_addr_in(7:0)
router_bus_rnw_in
router_bus_strb_n_inrouter_bus_ack_out
dsp1_cntrl_in(3:0)
dsp2_cntrl_in(3:0)
dsp3_ cntrl_in(3:0)
clk40_in
rst_n_in
Register Block
Write link and error info into Link Headers, Format data for S-Link and Error Checking
S-Link Write Control
46b
slink_cmd_reg(15:0)
Stop Output
slink_uctrl_out
DSP FIFO Buffer
Trap Event Datatrap_event_type_reg(31:0)
DSP 1
DSP 2
DSP 3
dsp2_event_data_out(31:0)dsp2_ dataframeready _out
dsp1_event_data_out(31:0)dsp1_ dataframeready _out
DSP 0
router_bus_registers
header detect
Error Format Data
slink_data_out(31:0)
dsp0_trap_enable_in
stop_out_override
stop_out_override
8/20/2002 28
43
Sli
nk o
n B
OC
Control & Status for Block Xfer/ DMA / S-Link
DM
A 1
DSP Error Data
DM
A 4 DMA
Cont....
S-link DataEvent Header Data
Event Trailer Data 32
Event Data Valid
Data Type
Output FIFO Data
32Read event data and format for S-Link and DSP error data.Detect Errors in the Link Headers, and create error code words for the Error Format.Detect Event Headers to enable trapping algorithms
R/W Bus
4 DMA transfer engines. Each engine has a 1k 32b word derandomizing buffer. Each buffer is broken into 256 word blocks.DMA moves full blocks of data. The blocks are filled as shown below:small event word count & continue bit in last word large event word count & continue bit in last word exactly one block word count & continue bit in block 2 last word
4 event trapping filtersFilters can trap on event types ATLAS, TIM , ROD or error dataEach trap has a divided by factor that will reduce data to DSPs
Header Detect
Router: Data Format and Trap
8/20/2002 29
. Write link and error information into the Link Headers for the S-Link Data Format.. When a link header is detected in the data, the Router moves the link number and error bits information from bits 45 down to 32 and overwrites the L1ID and BCID fields. (Shown on a later slide)
. Write to S-Link if Data Valid is true, and the User has not set the S-Link Masking bit or setup to mask specific Events.
. Detect errors in the Link Headers, and generate an error code for each link in Error Data Format. (Shown on a later slide). The error codes are stored in a block of registers for the duration of the Event. When the data
from EFB Engine 2 is done in the Router, the Error codes are written into the Event Trailer
Two bits error codes for each link00 = no data01 = good data10 = BCID or L1ID error11 = time out error
. Router FPGA routes data to the S-Link and the Slave DSPs.
- Internal FPGA dual port memory are used to asynchronously connect to DSPs.- During runtime monitoring, data is DMA’d into the internal SRAM of each DSP at
80 Mwords/sec. This channel must have the highest DMA priority or errors will be seen in the transfers.
• DSP captures the data it needs, throws away the rest.
Router: Overview
8/20/2002 30
Router: Overview• The Register Block allows VME access to the Router over the ROD Bus. The registers
in the Router include control and status for the DSP trapping algorithms and the S-Link.
• Because the Router needs to monitor header words 8 and 9 to sense an Event to trap, the Router uses a 9 clock pipeline for the Event Data path.
• If the S-Link asserts Xon/Xoff or S-Link Bad, the Router will mask the S-Link Write Enable to stop data transmission to the S-Link.
• In Calibration Mode only, if the DSP FIFO is almost full, and the StopOutputOverride bit is set, the Router can assert the StopOutput signal and temporarily apply back pressure to the EFB and Formatters. The Router can not apply back pressure from the DSPs to the ROD data path in Normal Data Taking Mode.
8/20/2002 31
• The Router supports four sets of trapping filters, one set per DSP. Each filter set is comprised of two filters per DSP.
• The first filter per DSP takes priority over second filter. Each filter has a mask bit to select a specific type of event (ATLAS, TIM, ROD).
• The first filter can be set to trap all events which disables the second filter.• Each filter has a Trap Match (8 bits per filter) value used to match the event type.• Each filter has a divide by factor (8 bits per filter) which allows pre-scalable trapping.• The trap variables must be loaded by setting the Load Trap Bit in the Reset/Load Trap Register.
This is done to protect from changing the trap type while the Router is in the middle of trapping an event.
• Each DSP DMA port has a bit to select between S-Link data format or error data format.
• Each DSP must set the Trap Enable bit to enable the Event trapping. Any the time that the Trap Enable bit is cycled from 0 to 1, the trap is reset and the FIFO counters are reset.
• The Mask Data Frame Ready input to the Router masks the Data Frame Ready signal until the DSP ISR is complete. If this bit stays high for an extended period, the FIFO will become full, and DSP data will be lost. This does not affect the Real Time Data Path signals
• When the an event has been trapped, a special event completion word is written to the last location of the FIFO block (256 words per block) that the event occupies. This word is made up of a pattern that does not appear in the data, and the total word count of the event. If an event end exactly on a block boundary, the word is written to the end of the next boundary.
Router: DSP Trapping Algorithm
8/20/2002 32
The format for the module link header is changed. The L1ID and BCID is replaced with error codes and the link number as shown below:
Key :x= do not care (The ROD fills these with 0’s)B=BCIDW=raw data C=cluster base addressD=3 bit hit dataE=ABC error codeF=FE numberh=header trailer limit errorL=L1IDn=count of raw dataf=error in condensed mode data, 1st hits=error in condensed mode data, 2nd hitp=preamble errorz=trailer bit errorv=data overflow errort=time out errorK=condensed model=L1 errorb=BCID errorM=link number
EFB Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBtrailer 010zhvxxxxxxxxxx1 hit condensed 1FFFFCCCCCCCxfx02 hits condensed 1FFFFCCCCCCCsfx11st hit clust expan 1FFFFCCCCCCC0DDD1 hit clust exp 1xxxxxxx0xxx1DDD2 hit clust exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWclust=cluster, exp=expanded
Bits 42:32 blKtMMMMMMM
Router Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001ptlbKxMMMMMMMtrailer 010zhvxxxxxxxxxx1 hit condensed 1FFFFCCCCCCCxfx02 hits condensed 1FFFFCCCCCCCsfx11st hit clust expan 1FFFFCCCCCCC0DDD1 hit clust exp 1xxxxxxx0xxx1DDD2 hit clust exp 1xxxxxxx1DDD1DDDflagged error 000xxxxxxFFFFEEEraw data 011nnnxxWWWWWWWWclust=cluster, exp=expanded
ReplacedL1ID andBCID in Event Header
Router: S-link Module Output Data Format
8/20/2002 33
The format for the header is not changed except for the addition of UCTL from the EFB input. The format is shown below:
Event HeaderWord Contents Comment
0 0xB0F00000 + UCTL (beginning of fragment)1 0xEE1234EE (start of header)2 0x9 (header size)3 TBD (format version number)4 TBD (source identifier)5 L1 ID (level 1 ID)6 BC ID (bunch crossing ID)7 L1 TT (ATLAS Level 1 trigger type)8 DET (detector event type ROD or TIM)
Event TrailerWord Contents Comment0 Error count (Status 1: count of words with error)1 Error flags (Status 2: Bit error type occurrence)2 0x2 (Number of status words)3 nData (word count of data words)4 0x1 (status block position: 0/1 = before/after data)5 0xE0F00000 + UCTL (end of fragment)
Router: S-link Event Header Output Format
8/20/2002 34
One of the slave DSPs is concerned with error diagnostics. It will need all error flags, module numbers and L1ID and BCID fields. The event header format is changes as shown below:
Router error data output same asEFB Output (bits 31:0 bits for all words)Name Bits 15:0 or 31:16header 001pLLLLBBBBBBBBAll other bits are also the same
EFB Event Header DSP Error HeaderWord Contents Word Contents Comment
0 0xB0F00000 0 same (beginning of fragment)1 0xEE1234EE 1 same (start of header)2 0x9 deleted (header size)3 TBD deleted (format version number)4 TBD deleted (source identifier)5 L1 ID 2 same (level I ID)6 BC ID 3 same (bunch crossing ID)7 L1 TT 4 same (ATLAS Level trigger type)8 DET 5 same (detector event type ROD or TIM)
EFB Event Trailer DSP Error Trailer CommentWord Contents Word Contents0 Error count 0 same (Status 1: count of words with error)1 Error flags 1 same (Status 2: Bitter error type occurrence)2 0x2 2 value (error status for links 1-16)3 nData 3 value (error status for links 17-32)4 0x1 4 value (error status for links 33-48) 5 value (error status for links 49-64) 6 value (error status for links 65-80) 7 value (error status for links 81-96) 5 0xE0F00000 8 same (end of fragment)
Two bits error codes for each link00 no data01 good data10 BCID or L1ID error11 time out error
Router: Error Data Output Format for the DSP
8/20/2002 35
Back End DSPs
• The Back End DSPs are TI TMS320C6701, floating point processors. The CPU clock is 160MHz, and each DSP has 32MBytes of SDRAM.
• The DSPs are removed from the data path allowing them to work independently without affecting the ROD data taking
• The VME host can access the Slave DSPs through a 16 bit Host Port Interface.
• SDSPs are hardwired for HPI Boot and Little Endian byte order.
• The interface to the Slave DSP is through the Master DSP to the ROD bus.
• The Slave DSPs respond to D32 block transfers from the VME master. (2.2Mbytes/S)
• The back end DSP performs histogramming, spying, and event trapping tasks.
– Calibrations: raw data, histograms, summaries to reduce VME traffic and load on crate processor, or comparison to reference histograms.
– Can trap specific events, i.e. embedded calibrations.
• Uses Trigger Type Info.
8/20/2002 36
TIM Data
Token and Mode Bits
Dec. Trig Count
R/W Bus to
FPGAs, BOC,
FIFOs, SDSPs
FE Command Streams48
Busy to TIM
Formatter
EFB
Input Memory & XCVRControl
ROD Controller: Implementation
ROD Controller
FPGAMaster DSP
TMS320C6201HPI Bus (VME)
EMIF Bus
ASYNC
DSP Serial Ports(1:0)
Interrupts
Event ID/ Dynamic Mask
FIFO Occupancy Status
Event ID Error
BOC
TIM
GPI/O PRM FPGA
8/20/2002 37
ROD Controller: Implementation
Test Bench Interface
Rod Controller FPGA:
RodController DSPmodule: SPI TCUTMS320C6201
HPI
EMIF
VMEbusIInterface:
FPGA Program Manager:Reset Manager:
RodResourcesInterfaceProgramBits 1b
RodResourcesInterfaceProgramBits, rst 5b
AFormattersProgramBits, rst 5b
BFormattersProgramBits, rst 5b
EventFragmentBuilderProgramBits, rst 5b
RouterProgramBits, rst 5b
EvntFrgBldrBUS 16b
FormatterABUS 16b
RouterBUS 8b
BOCsetupBUS 8b
busBuffer:
busBuffer:
busBuffer:
busBuffer:
busBuffer:
INMEM R/W & Control 32b
FormatterTrailerInformation 12b
Formatter Readout Mode Bits 16b
FE Command Streams 48b
EFB Dynamic Mask 12b
ControlREG,DebugMEMInterface:
busBuffer:DEBUGMEM R/W & Control 32b
FrontEndCommandEncoder:
FormatterReadoutModeBitsEncoder:
EvntFrgHeaderAndControlEncoder:
TIM 8b
busBuffer:
busBuffer:
busBuffer:
busBuffer:
BackEndDSP0 16b
BackEndDSP1 16b
BackEndDSP2 16b
BackEndDSP3 16b
BOOT Strap:PROM
Interrupt 1b
16b
32b
2b
DSP MEMORY:16Meg Bytes SDRAM
busBuffer:EVNTMEM R/W & Control 32b
En, AckR/NW
4b
Front Panel Status/Indicator LEDs 8b
32b
8b
32b
Interrupt 1b32b
DSP CE1 Mtype=000b rom
DSP CE2 Mtype=011b sdramDSP CE3 Mtype=011b sdram
BootMode=01101xBootProcess=8bitROM defualt timings MemoryMap=Map1
FrontEndOccupancyCounters:
AddressDecoder:
FormatterBBUS 16bbusBuffer:
dspReset[4..0] 5b
32b
32b
DSP CE0 Mtype=010b async
8/20/2002 38
• The Master DSP is a TI TMS320C6201B, fixed point processor. The CPU clock is 160MHz, and the DSP has 16MBytes of SDRAM (upgradeable to 32MBytes).
• The VME host can access the Master DSP through a 16 bit Host Port Interface.
• The Master DSP is hardwired for 8bit ROM Boot and Little Endian byte order.• The Boot ROM can be accessed on the VME Bus so that new versions of code can be
downloaded and started when required by the DAQ system.
• The Master DSP responds to D32 block transfers from the VME master. (2.2Mbytes/S)
• The Master DSP provides the main path of communication to the ROD from the VME through the Host Port Interface.
• The ROD Bus, controlled by the ROD Controller, is connected to the Master DSP EMIF asynchronous port, and utilizes CEn memory space 0. This is the path for access to all of the data path FPGAs, the Slave DSPs, the diagnostic memories, and the BOC.
• The Host Port Interrupt out from the DSP is wired to a VME interrupt input.
• The Master DSP has 2 serial communication ports (SP0 and SP1) that are used in the ROD to send serial commands to the FE Modules
ROD Controller: Master DSP
8/20/2002 39
• The Master DSP allows high level control operations to be performed on the ROD.– Because the control path of the ROD is not required to operate in real-time,
performance is less critical. – Using Primitive Lists, the MSDP performs control of ROD functional setup (the
specific mode of the ROD), histogramming, spying, and event trapping.
ROD Controller: Master DSP
• When the periodic reset is issued, the ROD raises busy and interrupts the master DSP. The master DSP then waits for all events to be played out of the front end chips by monitoring that the counts of all events pending is zero. Configuration as needed is then performed within approximately 1 thousandths of a second. Near the end of the cycle, the reset is issued to the modules and busy is removed.
• It is also possible to continue triggers while the ROD is correcting the state of the front end modules. In this mode busy would not be raised while the modules are configuring. Triggers would just return empty event with an error flag.
• The exact details of the periodic reset if needed will have to be worked out with ATLAS Trigger/DAQ.
Periodic Reset
8/20/2002 40
TIM Data
Event ID Data
Trig TypeDynamic
Mask FIFO
Bac
k-of
-Cra
te C
ard
Tok
en a
nd M
ode
Bit
s
Dec
. Tri
g C
ount
Back End DSPs
Host Port Interface (HPI) Bus
Master DSP r/w
R/W bus to
FPGAs and BOC
R/W Control
Registers
r/w
Control links 48 48
2 inputor
Circuits
Busy to TIM
48 bitMask
& Inputbit by bit
48 bitMask
& Inputbit by bit
Serial ExpansionEvent Data
TIM FIFO
4k W 8b
48
48
ToFormatter
Trigger OperationControl
ROD BCID, L1ID and Trigger Type
in ROD based triggers
To EFB
DSP SerialPort 0
DSP SerialPort 1
r/w
AllZero
Generate Dynamic
Mask
To data RCVR
Input Memory & XCVRControl
964 bit
Counters
Decr
Incr
FromFormatter
r/w
r/w
r/w
FromFormatter
ROD Controller FPGA
8/20/2002 41
• The main function of the ROD Controller FPGA is to control the operation of the ROD and coordinate the flow of data on the board in real time.
• Listing of the real time functions:1. Process commands from the TIM: Trigger, fast and slow commands and calibration signals2. Process serial data from the TIM: Event ID and Trigger Type.3. Count and store ECR commands, and include the count in the Event Header in the L1ID field.4. Send FE Commands to the Modules.5. Supply the Formatter Mode Bits and the EFB Event ID and Dynamic Mask Bits to the Data Path when an
L1 trigger has been received and the appropriate data has arrived from the TIM.6. Count and store triggers, and issue them to the Data Path when ROD is ready to process an Event.
• Listing of quasi real time functions:1. Issue and count corrective triggers sent to the data path.2. Issue Corrective Mode Bits for an Event.3. Issue a corrective Dynamic Mask and transfer it to the Default Mask.4. Change the masking of FE Command Streams.5. Interrupt the Master DSP to indicate the type of trigger sent to the data path.6. Trap Configuration or Event data in the Input Diagnostic FIFOs.
• Listing of non real time functions:1. Read/Write to the internal memory mapped register set.2. Read/Write to the BOC, diagnostic FIFOs, and the Slave DSPs.
ROD Controller: FPGA
8/20/2002 42
• The controller has an internal test bench block and a set of diagnostic registers to allow the ROD to be tested in a stand alone mode. This is required for initial debugging and some system testing.
• Listing of the Test Bench functions:1. Internal FIFO to mimic the TIM inputs2. Separate states to run stand alone test on the ROD.
A. Trap all Link inputs to Inmem FIFOsB. Trap all Link inputs to Inmem, send to Formatters, trap Formatter Output to Debug FIFOs, play data through the ROD
to the SDSPs (Event ID data must be provided by a TIM or internal registers) C. Load Inmem and TIM FIFOs with simulation data files and play through the RODD. Load Inmem and TIM FIFOs with simulation data files and play through the ROD with a set number of retransmits (1 to
65536).E. Load Inmem FIFOs with simulation data files and trap in the Debug FIFOsF. Load Debug and TIM FIFOs with simulation data and trap in the Event FIFOsG. Load Event FIFOs with 1 Event of simulation data and trap in the Slave DSPsH. Load Inmem FIFOs with 1 Event of simulation data, set the internal Event ID, Mode Bits and Dynamic Mask registers
to the required values, send L1 Triggers over SP0 or SP1, trap in the SDSPs
• Event ID FIFO stores a list of expected trigger IDs that the EFB can check against.
• Process the DSP Serial ports to provide for module configuration and ROD based trigger information.
ROD Controller: FPGA
8/20/2002 43
• The controller has a memory mapped set of internal registers required for all ROD operations.
• Listing of the internal registers by functional group:1. Main control and status2. FE Command Mask settings3. Calibration Commands4. Formatter Mode Bits/EFB Event ID and Dynamic Mask control and status5. Formatter Mode Bits /EFB Event ID and Dynamic Mask Look-up Tables6. Diagnostic control, status and counters7. DSP interrupts8. FE Occupancy Counter Control and Status9. Diagnostic Event ID, L1ID and BCID values10.Corrective Events FIFO access
• Can execute a long command sequence without tying up the ROD Crate Controller.
• Keeps count of the number of pending triggers to sense all events processed and for diagnostics. All events processed is used in a periodic reset to sense that the data is not in the front end chips.
• The ECR Counter can only be reset by writing a 1 to the ECR reset bit in the RCF Spare Control Register
ROD Controller: FPGA
8/20/2002 44
• Normal Data Taking
• The ROD receives L1A, Event ID, and Trigger Type signals from the TIM.
• With a latency of 2 clocks after receiving the L1A, the ROD sends the L1 Trigger command to the FE modules on the output links that are active.
• The FE Trigger Counter and the FE Occupancy Counters are incremented by 1.
• The Formatter Mode Bits Encoder sends the default Mode Bits to the Formatters.
• The EFB Event ID Dynamic Mask Encoder receives the deserialized ID and Trigger Type data, processes and sends the Event ID and default Dynamic Mask data to the EFB.
• The Token is issued to the Formatters, the FE Trigger Counter is decremented by 1, and an interrupt is sent to the Master DSP to indicate that a default trigger has been processed by the ROD Controller.
• The FE Occupancy Counters are decremented when a Link Formatter writes a trailer into the Link FIFO.
ROD Controller: FPGA
8/20/2002 45
• Resynchronization of modules during data taking
• Slave DSP determines what link requires L1 ID adjustment. The information is transferred to the Master DSP.
• The Master DSP writes the correction information to the Corrective Mode Bits and Corrective Dynamic Mask Registers in the ROD Controller FPGA then sets the “New DM Mask Ready” bit in the RCF Main Control Register.
• When the next L1A is detected from the TIM, the ROD performs the steps listed in the Normal Data Taking description with the following exceptions:
• The Corrective Trigger is trapped in the Corrective Events FIFO.
• The Mode Bits and Dynamic Mask Encoders issue the corrective Mode Bits and Dynamic Mask, and send an interrupt to the Master DSP to indicate that a Corrective Mask has been sent. This mask is valid for 1 L1 Trigger only for the Formatter. It is permanent for the EFB. (This feature is in the VHDL, but not tested)
ROD Controller: FPGA
8/20/2002 46
• FE triggers from Master DSP Serial Streams
• The ROD Controller FPGA contains a BC counter, L1 counter, and Event Type Register per serial stream. When the Calibration Trigger Decoder block detects a L1 Trigger from either serial stream, the L1 counter is incremented, the value of the BC counter is latched into a register, and the Event Type Register value is written to the Event ID FIFO.
• The Default Mode Bits and Dynamic Mask Look-up tables are used when a trigger is detected on SP0
• The Corrective Mode Bits and Dynamic Mask Look-up tables are used when a trigger is detected on SP1
• Fast Reset Commands will reset the counters as specified. A BCR command will reset the ROD BC counter and the FE module BC counter simultaneously, an ECR will not produce this result because the BC reset happens after other resets are performed by the chip controller.
• The BC ID register can be set to a static value.
• This mode is used for triggers during histogramming and diagnostics.
ROD Controller: FPGA
8/20/2002 47
ROD Program Reset Manager: Implementation
Flash Interface
FPGA Configuration
Data
FLASH 12MBytes
90ns
FPGA Config Control
ID(7:0)
FPGA Cnfg/RST Controller
Internal
R/W Registers
VME Address Decoder
and Data
Bridge
IA(18:0)
Control
ID(7:0)
ID(7:0)
FPGA Config Status
IA(20:0)
FPGA Reset Control
DSP Reset Control
Control
Status
D(31:0)
A(31:0)
Master DSP
Control
Control
AckN
D(31:0)
A(31:0)
Control
AckN
To VME
D(31:0)
Control
HR
DY
N
HHWIL
HD(15:0)
HCNTRL
HDS1
To MDSP
HPI
HRDYN
To FPGAsGP I/O
To ROD Controller
8/20/2002 48
ROD Program Reset Manager: Overview• Controls power on sequence of the ROD by configuring the data path
FPGAs in a specific order, and controlling resets to the DSPs.
• Allows VME host to configure and reset Data Path FPGAs.
• Allows VME Host to reset all DSPs.
• Allows VME Host to monitor status of the configuration status of Data Path FPGAs
• Acts as a bus-bridge between the ROD and the VME Bus.
• Enables D32 transfers between the VME and the Master DSP (16 bit HPI)
• Controls the read and write access to the 12 Mbytes of Flash memory that stores the FPGA configuration data.
• Stores the specific Serial Number of each board
8/20/2002 49
ROD Program Reset Manager
• Power on sequence of the ROD: • Configure ROD Controller FPGA
• Activate Master DSP
• Configure Formatters, EFB and Router
• Active Slave DSPs
• VME to Master DSP D32 access:
• Write Access: The PRM receives a D32 data word from the VME bus, splits it into 2 half words and sends the first half word to the HPI data port. After HRDY_N indicates that the HPI is ready for the second half word, the PRM toggles the half word ordering bit, moves the upper word to the lower half of the bus and writes it to the HPI data port. At this point, the PRM asserts LACKN to the VME Chipset, and is ready for the next transfer.
• Read Access: Same as above functionally.
8/20/2002 50
ROD Program Reset Manager• Controls the configuration and reset of Data Path FPGAs, resets of all DSPs, and status of
configuration and reset signals
• Writes to the Configuration and Reset registers on the VME bus allow the configuration and reset of all of the Data Path FPGAs and DSPs
• Reads to the Status registers show the state of configuration and reset signals
• Controls the read and write access to the 12 Mbytes of Flash memory that stores the FPGA configuration data.
• Internal registers and algorithms allow for simple access to the Flash memory.
• The Address and Write Data register is used to set the memory address for writes and reads and the data to write if required
• The Read Data register stores the data from the last read to the Flash
• The control register allows reads, host algorithm writes and PRM algorithm writes
• The host algorithm write gives complete control to the host to program the part. This is useful during chip or sector erase functions.
• The PRM algorithm write keeps control of the programming operation in the PRM. It is most useful for programming the part because it takes care of the repetitive commands required to program a Flash, and only requires address and data from the host.
8/20/2002 51
ROD Clock Distribution: Implementation
Clock Distribution
IC
1 In / 36 Out
PLL
BOC CLK+From BOC
PECL =>
LVTTL
40MHz XTAL OSC
BOC CLK-
SEL CLKTo
ROD36
To S-Link
S-LINK CLK
8/20/2002 52
ROD Clock Distribution: Overview
• Accepts the BOC clock PECL input, and converts to LVTTL to match the input of the Phase Locked Loop used on the ROD.
• Internal crystal oscillator can be used in stand alone mode
• Distributes separate clock lines for all parts that require clocking