Technology and Functionality of the EVLA Correlator (Next Corr Workshop, June 27, 2006)

Technology and Functionality of the EVLA Correlator

(Next Corr Workshop, June 27, 2006)

Next Gen Corr Workshop: EVLA Correlator—Carlson

June 27, 2006

Outline

• EVLA project overview.

• Correlator capabilities.

• Correlator architecture.

• Technology.

• Schedule.


June 27, 2006

EVLA Project Overview

• NRAO:

– Upgrade VLA from the current ~100 MHz BW to 16 GHz (8 GHz/pol’n).

– New receivers: 1-50 GHz contiguous frequency coverage.

– Replace waveguide system with digital fiber system.

– 3-bit (8x2 GHz) and 8-bit (4x1 GHz) sampling.

• New correlator (NRC Canada).


June 27, 2006

Correlator Capabilities

• 16 GHz/antenna (8 GHz/poln). 32-antenna system…expandable.

• Can tradeoff BW for #ants and #beams; can proc multiple narrow (<1 GHz) bands; VLBI ready.

• 3/8-bit ADC (at antenna), 4/7-bit re-quantization and correlation…more bits/less BW possible.

• All digital +/-1/32 sample delay tracking; 0.26 sec delay range.

• 3-level phase rotation in correlator chip: sub-sample delay, LO offset removal, earth-rotation phase, bias/birdie/aliasing de-correlation.


June 27, 2006

Correlator Capabilities

• 16k channels/baseline wideband; up to 4 Mchannels/baseline narrowband with “recirculation”.

• Up to 144 tunable multi-stage digital filters per antenna…’slot’ restrictions in 1st stage of filter.

• Use filter logic for sub-band multi-beaming and/or narrowband delay.

• ~60 dB spectral dynamic range.

• “WIDAR” technique (hybrid).


June 27, 2006

Correlator Capabilities• High-speed pulsar phase binning; 2000 bins/product, min ~15 usec bin width.

• High-speed dumping. 1 Gbps Ethernet/board standard. Up to 10 Gbps/board. 25 Mbytes/sec to archive.

• All digital phased output…1 GHz…expandable to full BW.

• Unlimited sub-arrays.

• Multiple VSI I/O.

• Configurable for virtually any radio telescope configuration…including auto-correlation.

• Uses VLBI standard frequencies (256 MHz…). Can incorporate VLBA into “excess” correlator capacity.


June 27, 2006

Correlator Architecture• Station Board:

– 6U Fiber receiver mezzanine card (NRAO).– Cross-bar switch FPGA.– Delay module mezzanine card (+/-0.5 samples delay).– Wideband autocorrelator.– Multi-stage digital filters (1 FPGA per sub-band).

• Cross-bar switch and sub-and multi-beaming delay memory.

• 4 stages…128 MHz…31.25 kHz BW out.

• Power measurements for sub-band stitching.

• Real-time RFI blanking per sub-band.

• CPU-settable scaling factors/re-quantization for max sensitivity/dynamic range.

• State counts, lag-0 power, phase-cal.

• Narrowband sub-sample delay with 16-step FIR interpolation.

– Cross-bar switch and mux to 1 Gbps for data transport to Baseline Board.


June 27, 2006


June 27, 2006

FIR Interpolator 1

FIR Interpolator 2

FIR Interpolator 3

FIR Interpolator 4

FIR Interpolator 5

FIR Interpolator 16

Fractionaldelay model

Output

Input


June 27, 2006

Correlator Architecture• Fanout Board:

– Dual 1-8 fanout of signals (4-wafer ‘stack’, 1 sub-band, all basebands, one antenna) for baseline rack signal distribution.

• Baseline Board:– 8x8 matrix of corr chips fed by 8 ‘X’ and 8 ’Y’ FPGAs

(recirculation, phase generation, 7-bit handling, cross-bar).– Corr chip: 2048 c-lags, in 16 128 c-lag “cells”. Full 4-bit

multiply, no trunc, 10-500 sec int. Cells can be concatenated.– Dedicated LTA FPGA +256 Mbit RAM for each corr chip; high-

speed phase binning and recirculation.– Output 1 Gbps Ethernet via FPGA.– Can set corr chip and board for autocorr mode: 64 independent

2048 channel, 128 MHz autocorrelations.– Can process up to 32 stations (1024 correlations), 128 MHz, 1

poln product, at 64 channels/baseline.


June 27, 2006


June 27, 2006

RCRC RC

CCRC

Baseline Board“i”

in the “k”th sub-band correlator

wafer 0

Filter Bank1

(18 sub-bandfilters)

Filter Bank0


DelayModule

DelayModule

Fib

er R

ecei

ver

Mo

dule

Station Board

0

17

k

0

17

k

0

17

0

17

Filter Bank1


Filter Bank0


DelayModule

DelayModule

Fib

er R

ece

iver

Mo

dule

Station Board

0

17

k

0

17

k

0

17

0

17

Filter Bank1


Filter Bank0


DelayModule

DelayModule

Fib

er R

ece

iver

Mo

dule

Station Board

0

17

k

0

17

k

0

17

0

17

Filter Bank1


Filter Bank0


DelayModule

DelayModule

Fib

er R

ece

iver

Mod

ule

Station Board 0of quad “n”

0

17

k

0

17

k

0

17

0

17

4 wafer cablebundle fromstation “n”,

physical output“k”, sub-band “s”

s

s

4 wafer cablebundle fromstation “m”,

physical output“k”, sub-band “s”

wafer 1

wafer 2

wafer 3

BB-0,1

BB-2,3

BB-4,5

BB-6,7

LTA

COTS GigESwitch “i"

GigE FPGA

BaselineBrd

“i",”k”output

k+1

BE CPU BE CPU

Ant

enn

a “

n”,

BB

-0,1

X

Y


June 27, 2006

Technology

• Use Xilinx Virtex-IV, and Altera Stratix II/GX/Cyclone FPGAs.

• Standard cell correlator chip, 4 million gate, 0.13 m. PD~3.7 W each @ 256 MHz, 1 V. Use ‘Accel’ point-of-load reg to minimize power, track chip speed with time.

• BGA packages throughout, some gull-wing memories/drivers.


June 27, 2006

Technology• 8 different boards in total:

• Station Board, Baseline Board: 12U x 400 mm, 28 layer, 0.0035” trace-’n-space, 0.125” thick. Phasing Board: ~6U x 160 mm(?).

• Fanout Board: 6U x 100 mm, with re-sync Stratix II FPGA.

• Common Backplane/midplane (3U x 25 mm); signal I/O, power, ID…used everywhere…no monolithic backplanes. HM 2.0 mm straight-thru. All press-fit.

• Delay module (~5”x3.5”)mezzanine card: FPGA+DDR SDRAMs

• PC/104+ COTS CPU + PCMC card.

• RPMIB (very simple; diodes, optos) for rack/fan power monitor and control


June 27, 2006

Technology

• 1 Gbps station-baseline data transport on high-density Meritec 2 mm hardmetric cable assemblies. Make correlator highly configurable for various correlator configurations (e.g. e-MERLIN).

• Use Altera’s 1 Gbps mux/demux with dynamic phase alignment…built into Stratix chips.

• About 500 rack-to-rack cables, 2500 intra-rack cables (<1 m) in EVLA correlator. All cables plug into 3U x 25mm Common Backplane.


June 27, 2006


June 27, 2006

Baseline Board (12Ux400 mm) PCB-back


June 27, 2006

Baseline Board PCB-front


June 27, 2006

Thermal vias “pocket” for corr chip heatsink


June 27, 2006


June 27, 2006

Technology

• Use central -48 VDC COTS power supply, DC-DC supplies on each board. Provides battery backup (5 min req).

• 24, 24” racks (2.5’ x 3’ x 7’), each one holding up to 16 large boards in two crates, and Fanout Boards in 6U crate.

• All hot-swappable, including fans.

• On-line detection of communications errors, temps, voltages…remote shutdown/power cycle of any board through central PXI chassis/CPU.


June 27, 2006

Thermal mock-up test rack


June 27, 2006

Technology

• Software:– Embedded PC/104+ COTS CPU on each large board. Cheap,

fast, easily available, easily replaceable, many vendors.– RT Linux…”Unix-style” device drivers.– System…board…chip-level GUIs for initial testing. Useful

during operations for continued debugging, low-level access.– MCCC—central host computer maps high-level requests from

EVLA M&C to embedded processors, via XML messages.– CPCC—central power control computer.– Backend—array of COTS PCs connected to Baseline Boards

via commercial GigE switches.


June 27, 2006

Schedule• Slipped about 1.5 yrs from original 2001 schedule…various reasons.

• 10 proto corr chips arrived June 20/06.

• Baseline Board…probably August/06…some PCB fab problems have delayed delivery.

• Station Board…signed-off design for fab quotation…probably get assembled board late September/06.

• Depending on how well testing goes:

– 1st prototype corrs at VLA/Jodrell Bank…mid ’07.– Full production mid 2008. Full installation early 2009.– Turn off old correlator end of 2009.


June 27, 2006

Summary Response• Type of correlations: all• Output req: spectral + continuum• Special processing req: tunable sub-bands, high spec res, pulsar, 4/7-bit corr, RFI

blanking/robust; RFI excision in backend CPUs eventually; possible RT RFI cancel.• Input BW + digitization: 8x2GHz 3-bit; 4x1GHz 8-bit; dedicated fiber I/F on mezzanine

card; Other BW/digitization supported; VSI I/O.• #spec channels at max BW: 16k, increases by 2X with each decrease in BW by ½.• #baselines correlated: 528 baselines, 163,840 cross-corrs• Integration times: min ~15 usec with binning. 11 msec standard…scalable with backend

and O/P link; unlimited max.• Dynamic range: ~60 dB spectral.• Scale: large, near limits of technology but not “bleading edge”.• Technology: FPGAs, custom hardware, 1 ASIC, COTS CPUs/network, standard form

factor.• Scalability: unlimited but high flexibility not necessarily best approach for ‘large N’. 4X

baselines possible with new ASIC, same architecture.• Flexibility: high: BW/Nant/Nchan/Nbeam/Nbits; RT, nRT VLBI, auto, connected element,

integration/dumping independent of system timing.• Architecture: “WIDAR” (hybrid XF).• RFI mitigation: Nbits, RT blanking, high dynamic range.

Technology and Functionality of the EVLA Correlator (Next Corr Workshop, June 27, 2006)

Documents

Transcript of Technology and Functionality of the EVLA Correlator (Next Corr Workshop, June 27, 2006)