National Radio Astronomy Observatory June 13/14, 2005 EVLA Phase II Proposal Review EVLA Phase II...

National Radio Astronomy ObservatoryJune 13/14, 2005 EVLA Phase II Proposal Review

EVLA Phase IIComputing Development

Bryan Butler (EVLA System Engineer for Software)

National Radio Astronomy Observatory

13/14 June, 2005 EVLA Phase II Proposal Review

EVLA Software

The primary goals of EVLA software are:

• maximize the scientific return of the EVLA;

• be easy to use, for all astronomers;

• provide a look-and-feel and functionality which is consistent with all NRAO telescopes.



EVLA Phase II Computing

The EVLA Phase II computing effort:

• Develops software, by the end of the construction project, which can be used for accessing or operating those parts of the EVLA telescope included in Phase II (E configuration; NMA; use of WIDAR for VLBA correlation);

• turns this software over to Operations - there is therefore a close relationship between software development (by the EVLA Phase II project) and operation, maintenance, and upgrade (by Operations);

• Mostly a simple extension of EVLA Phase I software effort.



EVLA Software - Methodology

1. Scientists deliver use cases and requirements;

2. Overall design developed based on these;

3. Each subsystem with the overall design then developed in

greater detail;

4. For all three of the above, iterate with short cycles (as short

as a week, depending on the particular item).



Overall NRAO Design (e2e)

Telescope Data Model

Export Data Format

Science Data Model

Feedback to telescope

Proposal SubmissionAnd Handling

Observation Preparation

EVLA VLBA ALMA

GBT

EVLASched

EVLAControl

ALMASched

ALMAControl

GBTSched

GBTControl

DataCapture

Archive

Telcal

Offline VO

ObserverDomain

Mostly Telescope-Independent

Common Software

VLBASched

VLBAControl

Quick Look

Pipeline

GBT Postproc

TelescopeDomain

ScienceDomain

Mostly Telescope-Specific

Project Software

Mostly Telescope-Independent

Common Software



Overall EVLA Design (1)

Observation Preparer

Ast

rono

mer

Default Program Block(with ‘suggestions’ filled in)

One Program

EVLA ObservingHeuristics

Program Block(Set of Scheduling Blocks for one Program)

Refinements

Proposal Preparer

To Observation Scheduler

ObserverDomain




TelescopeDomain

ObservationScheduler

ObservationExecutor

Hardware M&C

Next SBExecutionState

Equipment State

Metadata to DCAF

Operator

Environment

From Observation Preparer

Results from TelCal

Sequence of ConfigurationsAntenna Delays

Archive

Archive

Operator

Raw Visibility Data

Properties

Data Addressing InfoEquipment State

Archive

Operator

Heuristics

Metadata to DCAF

To DCAF To Archive




ScienceDomain

Quick Look Pipeline

Astronomer

Observation Monitor

Image Cubes

SDM

ArchivePost-Processing

Image Cubes

Astronomer

Default Image Pipeline

Image Cubes

Data CaptureAnd Format

From M&C

TelCal

To DCAF & Executor

SDM

Archive

Archive



Phase II Specific Developments

The major developments for EVLA Phase II are:• Additional hardware to support much higher

archive data rates;• Software to schedule the WIDAR correlator to

support VLBA recorded media, and simultaneously correlate EVLA, NMA, and/or VLBA;

• Post-processing system development in support of the much higher resolution full EVLA and NMA.



Archive Data RatesExpected data rates are determined by computing the dump time necessary to avoid time-bandwidth smearing. We know however that we cannot support the highest rates immediately (in 2014), so we have a staged plan, to allow storage (and processing) speeds to catch up:

We do not expect these rates to be difficult to support (they are quite conservative), given current storage capabilities and expected increases, and in fact we may increase at a faster rate.

DateMax Data Rate

(MB/s)

Total Volume

(TB/yr)

2008 (Phase I) 25 75

2014 500 1500

2017 1600 4500



SchedulingAt the completion of Phase II we will have 37 antennas which must be flexibly scheduled in various combinations. The software must support this, which will require changes to most of the subsystems. All are minor, except the Observation Scheduler.

In addition, we plan to use the WIDAR correlator to correlate VLBA disk-recorded observations, and to concurrently correlate combinations of all of the possibilities. This will require development of new software, including a Correlator Scheduler subsystem.



Post-ProcessingEffort in several areas of post-processing is needed, but much of this development is common with either ALMA or EVLA Phase I or both. The primary issue is the sheer size of the databases and the computational load. We estimate the following CPU power necessary:

This is not trivial, but we do not see it as a major problem.

DateAverage CPU rate

TFlopnormalized*

compute power

2008 (Phase I) 0.5 1

2014 10.0 0.6

2017 32 0.5 * - normalized by assuming doubling every 18 months (Moore’s Law), and

scaling to the 2008 compute power. This implies 125 GFlop right now.



Post-Processing

Recent changes:• major reorganization;• rewrite of code base (CASA) and interface

(Python);• development driven by ALMA and EVLA

deliverables;• ALMA already incorporating outside users

in testing, EVLA planning to do so soon.



Post-Processing - DetailsUse of a common post-processing package is the baseline plan for all three of

ALMA, EVLA Phase I, and EVLA Phase II. We understand that there is some skepticism that NRAO can successfully develop such a post-processing package, given recent past performance. We point out, however, the following salient features of the new post-processing package:

• the project has undergone a major reorganization, including the disbanding of the AIPS++ consortium and replacement of top level management;

• the code base is undergoing a massive rewrite (the CASA libraries are the result), with a revised interface based on Python instead of glish;

• development is now based upon project needs for ALMA and EVLA, including a coherent testing plan (for both ALMA and EVLA) which incorporates NRAO staff scientists and external users, and responds to deliverables based on requirements provided by the projects.

Because of these changes, we are confident that the post-processing package can deliver what is necessary for our processing needs. This is not to say that there are no remaining problems - just that they are not insurmountable.



Personnel (1)61.5 FTE years budgeted:• Observer Domain - Total = 7.5

• Proposal - 0.5• Observation Preparation - 2.5• Telescope and Correlator Scheduling - 4.5

• Telescope Domain - Total = 21.5• E configuration - 0.5• Other (MIBs, Executor, VLBA antenna rewrite, etc…) - 11.0• Correlator Backend - 1.0• Using WIDAR for VLBA - 9.0

• Science Domain - Total = 29.5• Post-processing - 24.0• Data Archiving - 2.0• Pipelines - 3.5

• Computing Infrastructure - Total = 3.0



Personnel (2)

We can do this with only 61.5 FTE years (which would seem too small for a project of the size of EVLA Phase II) because of:

• heritage from EVLA Phase I - with ~100 FTE years of effort;

• overlap with ALMA - with ~250 FTE years of effort.

In this way, just as we are leveraging EVLA Phase I hardware (which leveraged VLA hardware) we are also leveraging the software.



Hardware BudgetIn addition to the 61.5 FTE years of effort, there is ~$2M budgeted for computing hardware. This includes:• networking upgrades;• pipeline hardware;• M&C (CBE and CALC) hardware;• Post-processing hardware:

• Development cluster;• Final cluster;• Observers workstations;• Data storage and distribution media.



Summary

• EVLA software is designed to:• maximize the science derived from the instrument;• be easy to use, for all astronomers;• provide a look-and-feel and functionality which is consistent with

all NRAO telescopes.

• It builds extensively on the foundation of Phase I;• There are three main areas specific to Phase II:

• Scheduling - telescope and correlator;• Archive (mostly hardware);• Post-processing.

• 61.5 FTE years and ~$2M hardware budgeted, with ~100 FTE years and $2M hardware leveraged from Phase I; some part of ~250 FTE years leveraged from ALMA.

National Radio Astronomy Observatory June 13/14, 2005 EVLA Phase II Proposal Review EVLA Phase II...

Documents

Transcript of National Radio Astronomy Observatory June 13/14, 2005 EVLA Phase II Proposal Review EVLA Phase II...