The Mark 5B VLBI Data System

Alan WhitneyMIT Haystack Observatory

10 Jan 2006IVS General Meeting

Concepion, Chile

Mark 5A Data Acquisition System(introduced 2002)

Main features of Mark 5A Data System

• Direct plug-compatible replacement for 64-track Mark4 or VLBA tape drives

• Record/Playback at rates up to 1024 Mbps• Two independent ‘8-pack’ disk modules can be used in

‘ping-pong’ fashion for nearly continuous recording• Will record 8, 16, 32 or 64 tracks from Mark4 formatter (1024

Mbps max) or VLBA formatter (512 Mbps max)

Current Mark 5A Status

• ~120 Mark 5 systems deployed; >90% of tape systems have been replaced

• >1000 Mark 5A ‘8-pack’ disk modules deployed (>2 PB of storage!); growing rapidly!

• Many stations are now entirely Mark 5A, except in some cases for data which goes to VLBA correlator

• 1 Gbps experiments are now routine for both geodesy and astronomy

• VLBA is converting to Mark 5; all VLBA stations have Mark 5A, but correlator not yet fully outfitted

Mark 5B Data System Features• Full VSI-H compatibility• Same chassis as Mark 5A; uses same disk modules; requires Mark 5B I/O card• 1024 Mbps record/playback• Eliminates need for external formatters, but requires sampler adapter for Mark

4/VLBA DASs to provide VSI-compatible output• Station Unit capabilities for connection to Mark 4 correlators is designed into

Mark 5B• Extensive built-in phase-cal extraction and state counting on both data record

and data playback• Front-panel status display – 8 tri-color LEDs• DIM and DOM capabilities are separate FPGA downloads• FPGA is programmable via software

Development support from Mark 5 development consortium – BKG, EVN, KVN, JPL, MPI, NASA, NRAO, USNO

DPSCLOCK

DPS1PPS

ROT Clock(Requested Observe

Time)

DOT Clock(Data Observe Time)

QVALID

QDATA

RBS 0

RBS 31

Control

DOM(Data Output Module)

DTS(Data Transmission System)

Figure 1: VSI-H Functional Block DiagramVSI6.DRWARW 21 Jun 2000

ALT1PPS

DPS(Data Processing System)

TransmissionMedium

R1PPS

RCLOCK

80

-pin

MD

R co

nn

ecto

r; all LV

DS

LVDSRS-232

PDATA

BS 0

BS 31

1PPS

CLOCK

DIM(Data Input Module)

DOTMON

TVR TVG

TV0

TV31

TVRTVG

TV0

TV31

32x32C

rossbar

Notes: 1. Shaded items are for illustrative purposes only. 2. PVALID is optionally transmitted from DIM to DOM. 3. PDATA is optionally transmitted from DIM to DOM. 4. Data delay in DOM is required only for storage-based systems. 5. See text for discussion of use of optional use of P/QSPARE1/2 signals. 6. If DIM/DOM in single box, ALT1PPS/DPSCLOCK/DPS1PPS share single MDR-14 connector. 7. This diagram does not show all functions and options -- see VSI-H specification for details.

PVALID

LVDS EthernetTTL

80

-pin

MD

R co

nn

ecto

r; all LV

DS

DAS(Data Acq System)

Compareand

Synchronize

High-orderTime

Delay

ROT1PPS

ROTMON

QCTRLPCTRL

PSPARE1

TTL

(TVGCTRL)

PSPARE2(DOT1PPS)

QSPARE1(DPSCLOCKX)

QSPARE2(DPS1PPSX)

VSI-H VSI-H

VSI-S

VSI-E

Mark 5A/B System Comparisons

Mk5A Mk5B

Data Interface Emulates Mk4/VLBA

tape transport

VSI-H(32MHz max clock rate)

Max data rate 1024 Mbps 1024 Mbps

Record modes 8, 16, 32, 64“tracks”

1,2,4,8,16,32bitstreams

Disks Mk5 “8-pack” Same

Chassis Mk5 Same

I/O card Mk5A Mk5B

SS card XF2 XF2

I/O-SS intf Modified FPDP FPDP

Mark 5B Functional Block Diagram

CLOCK

1PPS

PVALID

BS

RCLOCK

QVALID

QDATA

RBS

DPSCLOCK

DPS1PPS

Fan-out

DOTClock

Add NDRheader

Stripheader

Da

ta

FPDPBus

PDATA

32Fan-in

Databuffer(min

256MB)

WriteRead

Xbar

32

Clo

ck

TO

T1P

PS

PCAL/Statecount

PCI

Va

lidity

Readcontrol

VSIIn

VSIOut

LVDS/RS232Conversion

To MOBOSerial Port

Internal Clock

INTCLK

Int

DIM

1PP

SD

OT

1P

PS

CF

INT

TOT1PPS

TO

T1P

PS

DOT1PPS

DPS1PPSXDPSCLOCKX

ALT1PPS

DOMCLK

DOM1PPS

DIM1PPS

ALT2PPS

CFInt

Gen

DOM1PPS

DOMCLOCK

CFINT

DO

M1P

PS

BSCLOCK

DOT1PPS

PVALID

From DIM Select

CLOCK

1PPS

PVALID

BS

RCLOCK

R1PPS

QVALID

QDATA

RBS

DPSCLOCK

DPS1PPS

DIM DOM

Fan-out

DOTClock

Add NDRheader

Stripheader

Da

ta

FPDPBus

PDATA

StreamStor

3232Fan-in

Databuffer(min

256MB)

WriteRead

Delayparams &

CF hdr databuffer

XbarAdd

CF hdr(for SUmode)

32

Clo

ck

TO

T1P

PS

PCAL/Statecount

PCI

Va

lidity

Readcontrol

TVR

TVG

VSIIn

VSIOut

LVDS/RS232Conversion

To MOBOSerial Port

Internal Clock

INTCLK

Int

DIM

1PP

SD

OT

1P

PS

CF

INT

/RO

TIN

T

TOT1PPS

TO

T1P

PS

DOT1PPS

DPS1PPSXDPSCLOCKX

ALT1PPS

DOMCLK

DOM1PPS

DIM1PPS

INTCLK

ALT2PPS

CFInt

Gen

DOM1PPS

DOMCLOCK

CFINT

DO

M1P

PS

BSCLOCK

DOT1PPS

PVALID

From DIM Select

ROT1PPS (BOCF)

TVG

TVR

INTCLK

SPARE1SPARE2

Mark 4 Station Unit emulation

 

Station delay model

Applydelaymodel

Phase-calextraction

(16 tones/chn)

Statecounting

Playback

Phase, delay model(per channel)

To correlator(16 channels)

Insert

Station unit function

Disks

Mark 4 Station Unit

Mark 5B I/O Board

FPDP VSI Output VSI Cntl/Mntr

VS

I In

put

256MB memoryPCI Connector

Mark 5B Status

• Checkout of Mark 5B hardware is complete• Successful correlation and fringes from Mark 5B data (software-converted

to Mark 5A format and correlated on Mark 4 correlator)• Direct connection to Mark 4 correlator is coming along well • Mark 5B support software is functional but needs some additional

capability and testing before release• 15 Mark 5B I/O boards have been built and tested – will be distributed to

Mark 5 development consortia members in near future• Design is being transferred to Conduant Corp for replication and sale –

expect Mark 5B to be orderable ~early Spring 2006

 

Reasons to adopt Mark 5B

• Eliminate need for expensive external formatters; particularly important for new stations or stations without existing Mark 4 or VLBA formatters

• With a 14-BBC Mark4 or VLBA4 system, up to 1792 Mbps can be recorded with two parallel Mark 5B systems; current 14-BBC systems can only generate a maximum of 1024 Mbps of formatted data

• Extensive phase-cal extraction and state counting capabilities for better diagnostics and better system calibration

• Replace unreliable Station Units on Mark 4 correlators; SU capability is built into Mark 5B

Mark 5A/B Compatibility

• Mark 5B can play only Mark 5B recordings (VSI format in/out)• Upgraded Mark 5A (“Mark 5A+”) will be able to play:

- All Mark 5A recordings- Mark 5B recordings made in selected modes;

playback will be VLBA-track-format• Mark 5A+ design is now just starting and will take several

months to complete

 

Bottom line: Existing Mark 4 correlators with only Mark 5A units will be able to process data from both Mark 5A and Mark 5B units during transition period to Mark 5B.

Mark 5B+ (2048Mbps)

• Conduant has introduced an upgraded StreamStor (dubbed “Amazon”) that supports up to ~3 Gbps an FPDP2 interface which allows clocking at 64 MHz

• Mark 5B I/O card has been designed to support input VSI-H clock rate of 64MHz, as well as FPDP2 compatibility, to support max recording rate of 2048 Mbps

• Testing with prototype SATA Amazon board at Haystack was successful; PATA version compatible with Mark 5 is under development by Conduant

• May be desirable to record across 2 disk modules (16 disks) simultaneously• Playback is limited to 1024 Mbps• Much software work and testing remains to be done• No promises yet on date of general availability

 

Mark 5A/B/B+ System Comparisons

Mk5A Mk5B Mk5B+

Data Interface Emulates Mk4/VLBA

tape transport

VSI-H(32MHz max clock rate)

VSI-H(64MHz max clock rate)

Max data rate 1024 Mbps 1024 Mbps 2048 Mbps

Record modes 8, 16, 32, 64“tracks”

1,2,4,8,16,32bitstreams

Same as Mk5B

Disks Mk5 “8-pack” Same Same

Chassis Mk5 Same Same

I/O card Mk5A Mk5B Mk5B

SS card XF2 XF2 Amazon

I/O-SS intf Modified FPDP FPDP FPDP2(clocks on both

edges

Mark 5 Upgrade Costs

Target

Existing

Mk5A Mk5B Mk5B+

0 $16.3K ~$18-20K ~$20-22K

Mk5A - ~$2-2.5K(Mk5B I/O)

~$11.5-12K

(Mk5B I/O plus Amazon)

Mk5B - - ~$9555

(Amazon)

Note: Does not include external cabling costs, typically a few hundred dollars

Disk-Media Status

• Hard disk price vs capacity/performance continues to drop- Now at or below ~$0.50/GB and continues to drop

(Mark 4/VLBA tape is ~$2.00/GB)

• 250 GB disks now common – 8-pack of 250GB disks comparable to ~7 VLBA/Mark 4 tapes

• 400 GB disks are available – 8-pack of 400GB disks comparable to ~11 VLBA/Mark 4 tapes

• 700 GB disks expected by end 2005 – 8-pack of 700GB disks comparable to ~19 VLBA/Mark 4 tapes;

~24 hours @ 1 Gbps unattended!

-2

-1

0

1

2

3

4

5

6

7

1980 1985 1990 1995 2000 2005 2010

Year

log($/GB)

Disc Drive Street Prices

Computer Tape Media

1998 IBMDisc Projection

Mark IV/VLBA/K4 Media

S2, S3 Media

Disc industryProjections

1998 NSICDisc Projection

LTO Media Projection

c

Tape vs. Disc Price Comparison

Disk-Media Reliability

• Based on statistics collected at Haystack, average disk drive failure rate is ~0.5% per year

• Failure rate of Hitachi 250GB has been higher than average

• Conduant has qualified drives from Maxtor, WD and Seagate

• 400GB – Seagate

• 250GB – Western Digital RAID Edition (high-reliability)

• 250GB – Maxtor Maxline III (high-reliability)

• Conduant is shipping these disks in assembled Mark 5 disk modules

• Disk reliability at high altitudes is under investigation

Summary

• Mark 5B is nearly ready for prime time

• Upgrading to Mark 5B may be particularly advantageous in certain situations

• Mark 5B+ (2 Gbps) is on the horizon

• Expect continuing upgrades to Mark 5 over the next few years