Switching for BTeV Level 1 Trigger Jinyuan Wu (For the BTeV Collaboration)

Switching for BTeV Level 1 Trigger

Jinyuan Wu(For the BTeV Collaboration)

Jinyuan WuBTeV trigger

Level 1 vertex trigger architecture

FPGA segment finders

Merge

Trigger decision to Global Level 1

Switch: sort by crossing number

track/vertex farm(~2500 processors)

30 station pixel detector


DAQ From M. Bowden’s Talk


GlobalLevel-1

ITCH

Information Transfer Control Hardware

GL1

Level-1 Buffers

12 x 24-port Fast Ethernet Switches

Level 2/3Processor Farm

Pixel Processors

FPGA Segment Finder

Track/Vertex Farm

Gigabit Ethernet Switch

Data Combiners +Optical Transmitters

OpticalReceivers

BTeV Detector

Front End Boards

8 Data Highways

Data Logger

Cross Connect Switch

BTeV Trigger (From M. Wang’s Talk)


Data From Pixel, Organized in Highways

This is DCB.

The PDCB and DCB look similar.

There are 10 sub-racks for pixel system.

There are 8 cables per sub-rack.

Each cable has 12 fibers.

• 80 cables, 960 fibers• 10 cables/hwy

Each card sees ¼ station, 12

cards/sub-rack.

Each cable goes to one

highway.


Pre-Processor

Pre-Processor

Pre-Processor

Pre-Processor

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

Optical Receivers to Segment Finder

Optical Receivers, 10 cards/hwy Pre-processors, 32 cards/hwy

SegmentFinder

Segment Finders, 32 cards/hwy

TO Farmlet

Pre-Processor

SegmentFinder

TO L1 Buffer:

From PDCB


Optical Receiver

0,10,20…

1,11,21…

2,12,22…

F,1F,2F…

IN: 12 fibers 1 cable/card

OUT: 16 cables/cardLVDS 4 pairs/cable<470 Mb/s per pair

1

2

F

0

Rate/cable:2.5 Gb/s x 12/16= 2.5 Gb/s x 3/4LVDS 4 pairs/cable<470 Mb/s per pair

Rate/fiber:2.5 Gb/s

Optical Receivers, 10 cards/hwy, 80 cards total


Pre-processor

Rate/cable:2.5 Gb/s x 12/16 x 5/4= 2.5 Gb/s x 15/16LVDS 4 pairs/cable<<585 Mb/s per pair

Pre-processors, 32 cards/hwy, 256 cards total

Pre-Processor

IN from Optical Receivers5 cables/card

OUT to Segment Finders: 4 cables/card

OUT to L1 Buffers: 4 cables/card

Rate/cable:2.5 Gb/s x 12/16= 2.5 Gb/s x 3/4LVDS 4 pairs/cable<470 Mb/s per pair



Segment Finder


Segment Finders, 32 cards/hwy, 256 cards total

IN from Segment Finders: 4 cables/card

OUT to DSP Farmlets: 4 cables/card


SegmentFinder


Pre-Processor

Pre-Processor

Pre-Processor

Pre-Processor

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0


IN: 120 fibers 10 cables/hwy

OR_PP: 160 cables

Optical Receivers, 10 cards/hwy Pre-processors, 32 cards/hwy

Ch: (1/120) BCO: (1/8)

Ch: (1/10) BCO: (1/8 x 1/16)

SegmentFinder


Ch: (1/2) BCO: (1/8 x 1/16 x 5/4)

Ch: (all) BCO: (1/8 x 1/16 x 5/4 x 2/4)

TO Farmlet 128 cables

PP_SF:128 cables

Pre-Processor

SegmentFinder

TO L1 Buffer: 128 cables


Short Compares

TDR New

L1 Switching Function

To Farmlet: Yes

To L1 Buffer: No

To Farmlet: Yes

To L1 Buffer: Yes

L1 Trigger Switch Hardware

Yes ($800K) No ($0)

Bandwidth from Preprocessor to Segment Finder

2.8 x

Input bandwidth

1 x

Input bandwidth

Number of cards/hwy

OR: 10

PP: 120 (60, 30?)

SF: 56 (28?)

OR: 10

PP: 32

SF: 32 (64?)

Optical Receiver Simple, FPGA not even needed

Need FPGA

Latency Minimum Longer, Need Study


To Do and Questions

• Optical receiver.• Latency budget.• Last stations.• Reduce switching in DAQ stages?• Further absorb segment finder into preprocessor?


End

End


Pixel data readout

Counting RoomCollision Hall

Pixelprocessor

Pixelprocessor

Pixelprocessor

FPGAsegment finder

to neighboring FPGAsegment finder

to neighboring FPGAsegment finder

Pixel stations

Optical links

Pixel processor

time-stamp expansion

time ordering

clustering algorithm

xy table lookup

FPIX2 Read-out chip

DCB

DCB

DCB

Data combiners

Row (7bits) Column (5bits) BCO (8bits) ADC (3bits)

sync (1bit)

Chip ID (13bits)


Optical Receiver

Optical Receivers, 80 cards full system

12 fibers in 1 cable/card

8 highways/card16 cables/card

0,8,16,24,32,40,48,56,64,72,80,88,96,104,112,120

1,9,17,25,33,41,49,57,65,73,81,89,97,105,113,121

2,10,18,26,34,42,50,58,…

3,11,19,27,35,43,51,59,…

4,12,20,28,36,44,52,60,…

5,13,21,29,37,45,53,61,…

6,14,22,30,38,46,54,62,…

7,15,23,31,39,47,55,63,…

0,64,…8,72,…16,80,…24,88,…

32,96,…40,104,…48,112,…56,120,…

LVDS 4 pairs/cable LVDS 8 pairs/card/hwy

<470 Mb/s per pair


The 4x4 Cable Bundle


Switching in the Processing Stages

Optical Receivers, 80 cards full system


8 highways/card16 cables/card

LVDS 4 pairs/cable<470 Mb/s per pair

Pre-processors, 32 cards/hwy

160 cables/hwy640 pairs/hwy

80 cables960 fibers

5 cables20 pairs/card

To L1Buffer

To SegmentFinder

Cable bundle4 x 4

Ch: (1/80) BCO: (all)

Ch: (1/80) BCO: (1/8 x 1/2)

Ch: (1/4) BCO: (1/8 x ½ x ¼ )

Segment Finders,40 cards/hwy 4 cables/card

Ch: (all) BCO: (1/8 x ½ x ¼ x 1/5)

To DSPFarmlet


L2/L3 PC

To L1 Buffer, Etc.


5 cables20 pairs/card

To L1Buffer

To SegmentFinder

Cable bundle4 x 4

Ch: (1/4) BCO: (1/8 x ½ x ¼ )


Ch: (all) BCO: (1/8 x ½ x ¼ x 1/5)

To DSPFarmlet

4 cables/card

Ch: (all) BCO: (1/8 x ½ x ¼ x 1/5)

L1 Buffer

SF SF SF SF


In One Highway:

Optical Receivers, 10 cards/hwy


16 cables/cardLVDS 4 pairs/cable<470 Mb/s per pair


160 cables/hwy

10 cables/hwy120 fibers/hwy

5 cablesTo L1Buffer

To SegmentFinder

Ch: (1/10) BCO: (1/8)

Ch: (1/10) BCO: (1/8 x 1/16)

Ch: (1/2) BCO: (1/8 x 1/16 )


Ch: (all) BCO: (1/8 x 1/16 x 2/5)

To DSPnodes


In One Highway:


IN: 12 fibers 1 cable/card

OUT: 16 cables/cardLVDS 4 pairs/cable<470 Mb/s per pair


160 cables/hwy

10 cables/hwy120 fibers/hwy

IN: 5 cablesOUT: 8 cables

To L1Buffer

To SegmentFinder

Ch: (1/10) BCO: (1/8)

Ch: (1/10) BCO: (1/8 x 1/16)

Ch: (1/2) BCO: (1/8 x 1/16 )

Segment Finders,64 cards/hwy IN: 4 cables

Ch: (all) BCO: (1/8 x 1/16 x 4/8)

To DSPnodes


1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0

1

2

F

0


IN: 120 fibers 10 cables/hwy

OR_PP: 160 cables


Pre-Processor


Ch: (1/120) BCO: (1/8)

Ch: (1/10) BCO: (1/8 x 1/16)

Pre-Processor

Pre-Processor

Pre-Processor

Pre-Processor

SegmentFinderSegment

FinderSegment

Finder


Ch: (1/2) BCO: (1/8 x 1/16 x 1/8)

Ch: (all) BCO: (1/8 x 1/16 x 1/8 x 2/4)

TO Farmlet 256 cables

PP_SF:256 cables


BTeV trigger overview

BTeV detector

L1 muon

L1 vertex

GlobalLevel-1

Level-1

Level 2/3 Crossing Switch

Data Logging

Front-end electronics

Level-1 Buffers

Level-2/3 Buffers

Information Transfer Control Hardware

ITCH

Level-2/3 Processor Farm#1

#2#m-1

#m

RDY

Crossing #N

Req. data for crossing #N

Level-3 accept

GL1 accept

PIX

> 2 x 10 channels7

800 GB/s7.6 MHz

L1 rate reduction: ~100x

L2/3 rate reduction: ~20x

4 KHz

Switching for BTeV Level 1 Trigger Jinyuan Wu (For the BTeV Collaboration)

Documents

Transcript of Switching for BTeV Level 1 Trigger Jinyuan Wu (For the BTeV Collaboration)