The Gigabit Link Interface Board (GLIB) Paschalis VICHOUDIS CERN PH-ESE-BE
description
Transcript of The Gigabit Link Interface Board (GLIB) Paschalis VICHOUDIS CERN PH-ESE-BE
1xTCA IG, 07-March-2011
The Gigabit Link Interface Board (GLIB)
Paschalis VICHOUDIS
CERN PH-ESE-BE
xTCA Interest Group meeting – 07 March 2011
2xTCA IG, 07-March-2011
Sophie Baron
Manoel Barros Marin
Vincent Bobillier
Stefan Haas
Magnus Hansen
Markus Joos
Francois Vasey
Paschalis Vichoudis
The Team
PH-ESE-BE
GLIB project homepage: https://espace.cern.ch/project-GBLIB/public
3xTCA IG, 07-March-2011
Outline
1. INTRODUCTION
2. IMPLEMENTATION
3. STATUS & DELIVERABLES
4xTCA IG, 07-March-2011
Introduction
THE GLIB IS: an evaluation platform and an easy entry point for users of high speed
optical links
THE GLIB IS TARGETED FOR: optical link evaluation in the laboratory control, triggering and data acquisition from remote modules in beam or irradiation
tests
CONCEPT
GBT
Versatile Link
TIA
LD
PD
Laser
GBT13
TRx
FPGA
Payload
GBT GLIB
E-Link
5xTCA IG, 07-March-2011
Introduction
OVERVIEW
Mid-size double-width Advanced Mezzanine Card (AMC)
Serves as an evaluation platform and an easy entry point for users of high speed
optical links.
4 SFP+ transceiver modules
Virtex-6 FPGA with twenty 6.5Gbps transceivers.
I/O capability can be further enhanced with two FPGA Mezzanine Cards (FMC).
Gigabit Ethernet link to PC for bench-top operation.
6xTCA IG, 07-March-2011
Introduction
TYPICAL USE CASES (1/6)
BENCH-TOP: beam test setup
= SFP+= TTC FMC
7xTCA IG, 07-March-2011
Introduction
TYPICAL USE CASES (2/6)
BENCH-TOP: front-end module test setup
= SFP+= TTC FMC= E-LINK FMC
8xTCA IG, 07-March-2011
Introduction
TYPICAL USE CASES (3/6)
BENCH-TOP: system test setup
= SFP+= TTC FMC
9xTCA IG, 07-March-2011
GLIB
Timing/Trig to FE
FRONT-END
FPGA
GLIB configGbE:
Slow CTRLDAQ
Timing/TrigGBT:
Power Supply
FRONT-ENDFRONT-END
GBT payload10GbE:3.2Gbps/link
GBT(s)
Introduction
TYPICAL USE CASES (4/6)
BENCH-TOP: system test setup [remote control/readout]
= SFP+= TTC FMC= 10GbE FMC
10xTCA IG, 07-March-2011
GLIB
HARDDISK
GBT payloadGLIB config
Timing/Trig to FE
FRONT-END
GbE:
FPGA
Backplane
CrateGLIB
FPGA
GbESlow CTRL
DAQTiming/Trig
GBT:
FRONT-ENDFRONT-END
SRAM SRAM
GbEswitch
ClockDistr.
PCIeswitch
MCHμP
Commercial MCH
Crate Management
GBT(s)
Introduction
TYPICAL USE CASES (5/6)
CRATE: beam test setup
= SFP+= TTC FMC
11xTCA IG, 07-March-2011
GbEswitch
ClockDistr.
PCIeswitch
MCHμP
Commercial MCH
GLIB
CPU StorageMedium
GLIB config
Storage
Timing/Trig to FE
FRONT-END
CPU
FPGA
Backplane
CrateGLIB
FPGA
Slow CTRLDAQ
Timing/TrigGBT:
GBT payload
PCIe:
FRONT-ENDCrate Management
GBT(s)
Introduction
TYPICAL USE CASES (6/6)
CRATE: system test setup
= SFP+= TTC FMC
12xTCA IG, 07-March-2011
AMC edgeconnector
Port [0:1]
Port [4:7] MGT quad
CLK1/TCLKA
JTAG circuitry(CPLD-based)Module Management
Controller (MMC)IPMI
JTAG
FPGA
GbE
Port [12:15]
4x SFP+
4
SRAM
I/O
CLK
JTAG
FMC#2
FMC#1
I2C
ClockDistribution
Circuitry
GbEPHY
20
I/O
CLK
JTAGI2C
TRx
4
160
160
SRAM
4
4
CLK2/TCLKB
CLK3/FCLKA
Port [2:3]
Port [8:11] MGT quad
M-LVDS TRx
Port [17:20]MGT quad
MGT quad
Diff. I/O pairs
ARCHITECTURE
Implementation
13xTCA IG, 07-March-2011
First Prototype
CMS xTCA workshop, 01-Feb-2010
14xTCA IG, 07-March-2011
First Prototype
CMS xTCA workshop, 01-Feb-2010
15xTCA IG, 07-March-2011
INTERFACES (1/2)
Optical
Four SFP+ cages
AMC
Port [0-1]: GbE.
Port [4-7] (Fat Pipe): PCIe x4 GEN2. Possibility to implement other protocols.
Port [8-11] (Extended Fat Pipe): PCIe x4 GEN2. Possibility to implement other
protocols.
Port [2:3]: LVDS I/O pairs. Possibility to implement other differential I/O standards.
Port [12-15]: LVDS I/O pairs. Possibility to implement other differential I/O standards.
Port [17-20]: M-LVDS.
CLK1/TCLKA: M-LVDS clock input.
CLK2/TCLKB: M-LVDS clock input/output.
CLK3/FCLKA: HCSL/M-LVDS clock input.
Implementation
16xTCA IG, 07-March-2011
INTERFACES (2/2)
FMC
2 High-pin count (HPC) sockets
160 user-specific I/Os (single-ended or differential pairs)
2 differential clock inputs and 2 differential clock outputs.
The primary FMC is accessible from the front panel
The primary FMC also provides four optional 6.5Gbps transceiver lines.
PC (only in bench-top operation)
GbE RJ45 socket (1000BASE-T).
PCIe 4x GEN2 adapter board.
Possibility to implement additional PC interfaces on the FMC mezzanines.
Implementation
17xTCA IG, 07-March-2011
FPGA
XILINX Virtex-6 LXT FPGA ( VLX130T, FF1156 package)
600 I/O that can be configured to various differential or single-ended standards.
4 Ethernet MAC and 2 PCIe Hard-IP blocks.
20 6.5Gbps transceivers (MGTs) organized in 5 quads.
~10Mb of block RAM.
Pin Compatible with higher capacity FPGAs
– VLX195, VLX240, VLX365, VSX315
Implementation
18xTCA IG, 07-March-2011
RAM & MMC
On-board memory
Two 72Mb (2M x 36bit) SRAM devices (CY7C1470 by Cypress)
Operating frequency at up to 250MHz.
Upgradeable up to 1.125Gb (once available)
Module Management Controller (MMC)
Implements the Intelligent Platform Management Interface (IPMI) for the AMC initialization and monitoring in μTCA environment.
Mezzanine card based on an ATMEL microcontroller.
Developed by CPPM (microcontroller firmware in collaboration with DESY).
Dimensions: 39mm x 20mm x 10mm.
Implementation
19xTCA IG, 07-March-2011
Implementation
CLOCK DISTRIBUTION
20xTCA IG, 07-March-2011
Implementation
MODULE MANAGEMENT CONTROLLER
MP
GA0GA1GA2
ENABLE#SCLSDA
PS0#PS1#
MMCmezzanine Board
Temperature #1
FPGAtemperature
FPGA
CPLD Control of regulators & other devices
IPMI
I2C
JTAG
GND
VCCGND
AMC front panel LEDS
I/O
AMC edgeconnector
I/OI/OI/O I/O
FrontPanel
Modulehandle
I2C
BoardTemperature #2
Diode for module detecton
I/O
I/O
I/O
I/O
Note: Not required for bench-top applications
21xTCA IG, 07-March-2011
CPLDFPGA
FPGA configuration
EEPROM
MMC
DedicatedJTAG lines
I/O
I/O
DedicatedJTAG lines
I/O I/O
I/O
AMC edgeconnector
DedicatedJTAG lines
I/O
JTAGheader #2
JTAGheader #1
Port0Port1
GbE
I2C I2CIPMI
FPGAconfig
I/O
I/O
Other devicesJTAGchain
JTAG lines
I/O
ADDRDATA
FPGA/EEPROM parallel bus
FPGAconfig
Implementation
JTAG CIRCUITRY
Important for Testing
22xTCA IG, 07-March-2011
Implementation
POWERING
LTM4601
LTM4606
LTM4606
LTM4606
12VMP
1V
1V5
3V3
2V5
MP
MAX8556
Ext. PowerConnector
MAX85561V2A
1VA
3V3
JumperSelector
3V3MP
LT3021
LT3021
LTC6902
CLKSYNC1
CLKSYNC2
CLKSYNC3
CLKSYNC4
2V5MP
1V8MP
FPGA MGT VTT
FPGA MGT VCC
MMC, CPLD I/O
CPLD I/O
CPLD core
FPGA core
FPGA I/O, various ICs, FMCs, master tracking voltage for LTMs
FPGA I/O, EEPROM I/O, various ICs, FMCs
AMC edgeconnector
1V8EEPROM coreLT3021
Power budget: Up to 0.5W for Management and 80W for Payload Power
23xTCA IG, 07-March-2011
Implementation
POSSIBLE FMC IMPLEMENTATIONS
24xTCA IG, 07-March-2011
Implementation
FIRMWARE ARCHITECTURE
SRAMinterface
FPGA EEPROMinterface
PatternGenerator
FPGA EEPROM
SRAM
SRAM
ClockDistribution controller
ClockDistribution
chipset
FPGA LOCAL BUS
GBTIP
Fat Pipe interface
(typically PCIe IP)
GbEinterface
Port[8:11]
Port[4:7]
USER LOGIC
SFP+
Port[0]
Port[1]
GbE PHY
FMC#1interface
FMC#2interface
25xTCA IG, 07-March-2011
Implementation
TESTING
- Possibility for boundary scan testing
- Use of commercial hardware for creating loopbacks and providing clock I/O for connectivity and clock distribution testing, respectively.
- Testing of optical parts (loopbacks w/ optical fibres, BER measurements
etc)
- Various testpoints on board
- Special testing software and firmware
AMC edgeconnector
Port [0:1]
Port [4:7] MGT quad
CLK1/TCLKA
JTAG circuitry(CPLD-based)Module Management
Controller (MMC)IPMI
JTAG
FPGA
GbE
Port [12:15]
4x SFP+
4
SRAM
I/O
CLK
JTAG
FMC#2
FMC#1
I2C
ClockDistribution
Circuitry
GbEPHY
20
I/O
CLK
JTAGI2C
TRx
4
160
160
SRAM
4
4
CLK2/TCLKB
CLK3/FCLKA
Port [2:3]
Port [8:11] MGT quad
M-LVDS TRx
Port [17:20]MGT quad
MGT quad
Diff. I/O pairs
26xTCA IG, 07-March-2011
** Presented at TWEPP 2010 **
Summary
Specifications: v1.9 available.
Hardware: First prototype available, on-going tests.
Testing tools: Commercial hardware, boundary scan software.
Infrastructure: Crate, MCH available, commercial AMCs available. Getting
familiar with.
Software/firmware: Development on-going.
Latest news: GLIB has entered the open hardware repository.
https://edms.cern.ch/nav/EDA-02180-V1-0
specs from GLIB website
GLIB at OHR
27xTCA IG, 07-March-2011
Deliverables
GbEswitch
ClockDistr.
PCIeswitch
MCHμP
Commercial MCH
GLIB
CPU StorageMedium
GLIB config
Storage
Timing/Trig to FE
FRONT-END
CPU
FPGA
Backplane
CrateGLIB
FPGA
Slow CTRLDAQ
Timing/TrigGBT:
GBT payload
PCIe:
FRONT-ENDCrate Management
GBT(s)
The GLIB team envisages to deliver and support software, firmware and
hardware for the following 3 setups:
- Bench-top beam test setup
- Bench-top front-end module test setup
- Crate system test setup
The required FMCs (TTC & E-Link) will also be delivered and supported.
= SFP+= TTC FMC= E-LINK FMC
Bench-top front-end module test setupBench-top beam test setup Crate system test setup