Post on 12-Jan-2016
Technology and Functionality of the EVLA Correlator
(Next Corr Workshop, June 27, 2006)
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Outline
• EVLA project overview.
• Correlator capabilities.
• Correlator architecture.
• Technology.
• Schedule.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
EVLA Project Overview
• NRAO:
– Upgrade VLA from the current ~100 MHz BW to 16 GHz (8 GHz/pol’n).
– New receivers: 1-50 GHz contiguous frequency coverage.
– Replace waveguide system with digital fiber system.
– 3-bit (8x2 GHz) and 8-bit (4x1 GHz) sampling.
• New correlator (NRC Canada).
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Correlator Capabilities
• 16 GHz/antenna (8 GHz/poln). 32-antenna system…expandable.
• Can tradeoff BW for #ants and #beams; can proc multiple narrow (<1 GHz) bands; VLBI ready.
• 3/8-bit ADC (at antenna), 4/7-bit re-quantization and correlation…more bits/less BW possible.
• All digital +/-1/32 sample delay tracking; 0.26 sec delay range.
• 3-level phase rotation in correlator chip: sub-sample delay, LO offset removal, earth-rotation phase, bias/birdie/aliasing de-correlation.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Correlator Capabilities
• 16k channels/baseline wideband; up to 4 Mchannels/baseline narrowband with “recirculation”.
• Up to 144 tunable multi-stage digital filters per antenna…’slot’ restrictions in 1st stage of filter.
• Use filter logic for sub-band multi-beaming and/or narrowband delay.
• ~60 dB spectral dynamic range.
• “WIDAR” technique (hybrid).
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Correlator Capabilities• High-speed pulsar phase binning; 2000 bins/product, min ~15 usec bin width.
• High-speed dumping. 1 Gbps Ethernet/board standard. Up to 10 Gbps/board. 25 Mbytes/sec to archive.
• All digital phased output…1 GHz…expandable to full BW.
• Unlimited sub-arrays.
• Multiple VSI I/O.
• Configurable for virtually any radio telescope configuration…including auto-correlation.
• Uses VLBI standard frequencies (256 MHz…). Can incorporate VLBA into “excess” correlator capacity.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Correlator Architecture• Station Board:
– 6U Fiber receiver mezzanine card (NRAO).– Cross-bar switch FPGA.– Delay module mezzanine card (+/-0.5 samples delay).– Wideband autocorrelator.– Multi-stage digital filters (1 FPGA per sub-band).
• Cross-bar switch and sub-and multi-beaming delay memory.
• 4 stages…128 MHz…31.25 kHz BW out.
• Power measurements for sub-band stitching.
• Real-time RFI blanking per sub-band.
• CPU-settable scaling factors/re-quantization for max sensitivity/dynamic range.
• State counts, lag-0 power, phase-cal.
• Narrowband sub-sample delay with 16-step FIR interpolation.
– Cross-bar switch and mux to 1 Gbps for data transport to Baseline Board.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
FIR Interpolator 1
FIR Interpolator 2
FIR Interpolator 3
FIR Interpolator 4
FIR Interpolator 5
FIR Interpolator 16
Fractionaldelay model
Output
Input
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Correlator Architecture• Fanout Board:
– Dual 1-8 fanout of signals (4-wafer ‘stack’, 1 sub-band, all basebands, one antenna) for baseline rack signal distribution.
• Baseline Board:– 8x8 matrix of corr chips fed by 8 ‘X’ and 8 ’Y’ FPGAs
(recirculation, phase generation, 7-bit handling, cross-bar).– Corr chip: 2048 c-lags, in 16 128 c-lag “cells”. Full 4-bit
multiply, no trunc, 10-500 sec int. Cells can be concatenated.– Dedicated LTA FPGA +256 Mbit RAM for each corr chip; high-
speed phase binning and recirculation.– Output 1 Gbps Ethernet via FPGA.– Can set corr chip and board for autocorr mode: 64 independent
2048 channel, 128 MHz autocorrelations.– Can process up to 32 stations (1024 correlations), 128 MHz, 1
poln product, at 64 channels/baseline.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
RCRC RC
CCRC
Baseline Board“i”
in the “k”th sub-band correlator
wafer 0
Filter Bank1
(18 sub-bandfilters)
Filter Bank0
(18 sub-bandfilters)
DelayModule
DelayModule
Fib
er R
ecei
ver
Mo
dule
Station Board
0
17
k
0
17
k
0
17
0
17
Filter Bank1
(18 sub-bandfilters)
Filter Bank0
(18 sub-bandfilters)
DelayModule
DelayModule
Fib
er R
ece
iver
Mo
dule
Station Board
0
17
k
0
17
k
0
17
0
17
Filter Bank1
(18 sub-bandfilters)
Filter Bank0
(18 sub-bandfilters)
DelayModule
DelayModule
Fib
er R
ece
iver
Mo
dule
Station Board
0
17
k
0
17
k
0
17
0
17
Filter Bank1
(18 sub-bandfilters)
Filter Bank0
(18 sub-bandfilters)
DelayModule
DelayModule
Fib
er R
ece
iver
Mod
ule
Station Board 0of quad “n”
0
17
k
0
17
k
0
17
0
17
4 wafer cablebundle fromstation “n”,
physical output“k”, sub-band “s”
s
s
4 wafer cablebundle fromstation “m”,
physical output“k”, sub-band “s”
wafer 1
wafer 2
wafer 3
BB-0,1
BB-2,3
BB-4,5
BB-6,7
LTA
COTS GigESwitch “i"
GigE FPGA
BaselineBrd
“i",”k”output
k+1
BE CPU BE CPU
Ant
enn
a “
n”,
BB
-0,1
X
Y
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Technology
• Use Xilinx Virtex-IV, and Altera Stratix II/GX/Cyclone FPGAs.
• Standard cell correlator chip, 4 million gate, 0.13 m. PD~3.7 W each @ 256 MHz, 1 V. Use ‘Accel’ point-of-load reg to minimize power, track chip speed with time.
• BGA packages throughout, some gull-wing memories/drivers.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Technology• 8 different boards in total:
• Station Board, Baseline Board: 12U x 400 mm, 28 layer, 0.0035” trace-’n-space, 0.125” thick. Phasing Board: ~6U x 160 mm(?).
• Fanout Board: 6U x 100 mm, with re-sync Stratix II FPGA.
• Common Backplane/midplane (3U x 25 mm); signal I/O, power, ID…used everywhere…no monolithic backplanes. HM 2.0 mm straight-thru. All press-fit.
• Delay module (~5”x3.5”)mezzanine card: FPGA+DDR SDRAMs
• PC/104+ COTS CPU + PCMC card.
• RPMIB (very simple; diodes, optos) for rack/fan power monitor and control
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Technology
• 1 Gbps station-baseline data transport on high-density Meritec 2 mm hardmetric cable assemblies. Make correlator highly configurable for various correlator configurations (e.g. e-MERLIN).
• Use Altera’s 1 Gbps mux/demux with dynamic phase alignment…built into Stratix chips.
• About 500 rack-to-rack cables, 2500 intra-rack cables (<1 m) in EVLA correlator. All cables plug into 3U x 25mm Common Backplane.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Baseline Board (12Ux400 mm) PCB-back
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Baseline Board PCB-front
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Thermal vias “pocket” for corr chip heatsink
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Technology
• Use central -48 VDC COTS power supply, DC-DC supplies on each board. Provides battery backup (5 min req).
• 24, 24” racks (2.5’ x 3’ x 7’), each one holding up to 16 large boards in two crates, and Fanout Boards in 6U crate.
• All hot-swappable, including fans.
• On-line detection of communications errors, temps, voltages…remote shutdown/power cycle of any board through central PXI chassis/CPU.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Thermal mock-up test rack
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Technology
• Software:– Embedded PC/104+ COTS CPU on each large board. Cheap,
fast, easily available, easily replaceable, many vendors.– RT Linux…”Unix-style” device drivers.– System…board…chip-level GUIs for initial testing. Useful
during operations for continued debugging, low-level access.– MCCC—central host computer maps high-level requests from
EVLA M&C to embedded processors, via XML messages.– CPCC—central power control computer.– Backend—array of COTS PCs connected to Baseline Boards
via commercial GigE switches.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Schedule• Slipped about 1.5 yrs from original 2001 schedule…various reasons.
• 10 proto corr chips arrived June 20/06.
• Baseline Board…probably August/06…some PCB fab problems have delayed delivery.
• Station Board…signed-off design for fab quotation…probably get assembled board late September/06.
• Depending on how well testing goes:
– 1st prototype corrs at VLA/Jodrell Bank…mid ’07.– Full production mid 2008. Full installation early 2009.– Turn off old correlator end of 2009.
Next Gen Corr Workshop: EVLA Correlator—Carlson
June 27, 2006
Summary Response• Type of correlations: all• Output req: spectral + continuum• Special processing req: tunable sub-bands, high spec res, pulsar, 4/7-bit corr, RFI
blanking/robust; RFI excision in backend CPUs eventually; possible RT RFI cancel.• Input BW + digitization: 8x2GHz 3-bit; 4x1GHz 8-bit; dedicated fiber I/F on mezzanine
card; Other BW/digitization supported; VSI I/O.• #spec channels at max BW: 16k, increases by 2X with each decrease in BW by ½.• #baselines correlated: 528 baselines, 163,840 cross-corrs• Integration times: min ~15 usec with binning. 11 msec standard…scalable with backend
and O/P link; unlimited max.• Dynamic range: ~60 dB spectral.• Scale: large, near limits of technology but not “bleading edge”.• Technology: FPGAs, custom hardware, 1 ASIC, COTS CPUs/network, standard form
factor.• Scalability: unlimited but high flexibility not necessarily best approach for ‘large N’. 4X
baselines possible with new ASIC, same architecture.• Flexibility: high: BW/Nant/Nchan/Nbeam/Nbits; RT, nRT VLBI, auto, connected element,
integration/dumping independent of system timing.• Architecture: “WIDAR” (hybrid XF).• RFI mitigation: Nbits, RT blanking, high dynamic range.