FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava.

FLARE Workshop November 4-6, 2004 1

Data AcquisitionData Acquisition

Mark Bowden, Margaret VotavaMark Bowden, Margaret Votava

2FLARE Workshop November 4-6, 2004

ParametersParameters

250,000 channels

4 Bytes/chan

2 MHz sample rate

Front-end data rate = 2 TBytes/sec


Front-end (ICARUS)Front-end (ICARUS)

Caen V789 (Digital Board) + V791 (32 channel A/D Board) Rack 18 X 32 channels


FLARE Front-end AssumptionsFLARE Front-end Assumptions

• Newer components allow greater packaging density

128 or 256 channels (if new amplifier/shaper ASIC)

Digital board can be reduced to single FPGA andcombined with Analog board

FE Board count drops from 16,000 to 2,000

• Eliminate VME

VME is expensive and provides no added value

Direct Ethernet connections to FE boards

6U Eurocard or 1U (pizza box) packaging


FLARE Front-endFLARE Front-end

40 X 128 channels (6U packaging) 20 X 256 channels (1U packaging)

48V Power


Data RatesData Rates

1 MByte/sec

2 GByte/sec

1 KByte/sec

2 MByte/sec

1 MByte/sec

2 GByte/sec

1 GByte/sec

2 TByte/sec

Without FE Data Compression

With FE Data Compression

Spill Only*

Continuous

* 2 msec per 2 sec cycle ** assuming 128 channels/FEB

- per FEB**

- system total

Note: BTeV is 0.5 TByte/sec


Data NetworkData Network

…

FE switches (100) $15KGBE switches (50) $25KCables $10KTotal $50K

… … … … …

……

……

……

……

……

……

5 GByte/sec


Switch-based DAQSwitch-based DAQ

• Any size/bandwidth system can be built from low cost switch components.

• Cost scales as NlogN.

• Up to 100m (copper) between switches.


Processing & Data StorageProcessing & Data Storage

1U dual processor $2K

DVD writer $7K

(Delay purchase of computers to maximize performance/cost.)


““Event” ProcessingEvent” Processing

• An “Event” is an arbitrary time slice of data from all channels

2 msec 4 MBytes

20 msec 40 MBytes

200 msec 400 MBytes

2 sec 4 GBytes

• Choose largest time slice that fits in processor memory

• For this application, ~100 msec



Front-end (2000 data sources)

Ethernet Network

Processors

1 2 2000

1 2 200

Source 1

Source 2

Source 3

1 2 3 200

1 2 3 200

1 2 3 200

1 2 3

1 2 3

1 2 3

Destination



Front-end

Ethernet Network

Processors

1 2 2000

1 2 200

Data transfer time = ~40 MBytes/sec * time_slice [4 sec]

Processing time = Nprocessors * time_slice [20 sec]


Data StorageData Storage

Tape$300K/PByte

Hard Disk$800K/PByte

Blu Ray$900K/PByte

<$100K/PByte

DVD-R$100K/PByte

DVD-RW$300K/PByte


Data StorageData Storage

• continuous readout, without online tracking

- 2 GBytes/sec X 107 sec = 20 PBytes/yr

- $2-6M/yr media cost

• continuous readout, with online tracking

- 2 MBytes/sec X 107 sec = 20 TBytes/yr

- $10K/yr media cost

Note: BTeV includes 2 PBytes of hard disk (@ $600K/PByte) in baseDAQ construction cost. Offline storage is part of operating cost.


Ethernet Network(~$50K)

Processing Farm(~$500K)

Data Storage(~$50K @ 2MB/sec,~$1M @ 2GB/sec+ media cost)

DAQ CostsDAQ Costs

Note: DAQ based on all commercial hardware


Additional hardware costs

Slow Controls ~$200KTiming System ~$50KDevelopment System ~$100KInfrastructure ~$50K

DAQ CostsDAQ Costs


• with online tracking

DAQ hardware ~$1.0MDAQ software ~$2.0MDAQ management ~$0.5M

DAQ operating cost ~$100K/yr

• without online tracking

(fewer online processors, but more data recorders)

DAQ hardware ~$1.7MDAQ software ~$1.5MDAQ management ~$0.5M

DAQ operating cost ~$4M/yr

DAQ Cost SummaryDAQ Cost Summary


• data compression at front-end is preferred(“necessary” for continuous readout)

• online tracking will significantly reduce operating cost by reducing storage media expense

• DAQ base cost estimate is ~$3.5M

• DAQ contingency and overhead estimate is ~$1M

ConclusionsConclusions

FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava.

Documents

Transcript of FLARE Workshop November 4-6, 2004 1 Data Acquisition Mark Bowden, Margaret Votava.