Solar-B Global DataGrid 21 st September 2006 UK e-Science All Hands Meeting 2006 RALMSSL Tim Folkes...

Solar-B Global DataGrid21st September 2006UK e-Science All Hands Meeting 2006

RAL MSSL

Tim Folkes Elizabeth Auden

Jens Jensen Paul Lamb

Matthew Wild Matthew Whillock

Len Culhane

Elizabeth Auden 21 September 2006 e-Science AHM 2006

Overview

The Solar-B mission will be launched tomorrow, Friday 22 September 2006, from the Uchinoura Space Center in Japan.

Mission duration: 3 years nominal, +10 years lifetime expected

Scientists and software engineers at MSSL and RAL have designed the Solar-B global data grid to provide mission data to solar research communities in the UK and abroad.

• Solar-B science objectives• Data transfer and processing• Data storage• User access to data• Data security


Solar-B Mission Overview


EIS - MSSL/NRL EUV Imaging SpectrometerSOT - ISAS/NAOJ Solar Optical TelescopeXRT - SAO/ISAS X-ray TelescopeFPP - Lockheed/NAOJ Focal Plane Package

Solar-B Science


Key Science Questions

1. How is the solar corona heated? Are active regions heated in a different way to the quiet Sun?

2. What causes solar flares and coronal mass ejections?

3. How is energy transferred in the quiet Sun? How are solar magnetic fields affected?

Solar events like flares and coronal mass ejections can have a profound affect on the Earth’s magnetosphere. A geomagnetic storm in 1989 triggered a blackout of the Hydro Quebec power system.

Transfer of Solar-B Data to ADS

Solar-B data transferred via GridFTP from ISAS and MSSL to an NFS mounted disk at ADS.


Data Migration FacilityManages NFS area

• Produced by SGI• User transparent file

migration• Licence for 500Tbytes

storage

• Backend: STK SL8500– 10,000 tape slots

– STK 9940B drives (200GB 30MB/sec)

– STK T10000 drives (500GB 120MB/sec)

ADS RAL

ISAS Japan

MSSL processing

Satellite data LL data

LL data

HL data

HL

LL

ADS Data Storage


DMF STK SL8500 robot

The DMF monitors the Solar-B NFS area and removes disk copies of data to keep free space at required level.

Small files have primary copy on disk. (Speeds up read/writes.)

ADS

InterfaceNFS

Tape

Firesafe

Larger files have primary copy on tape.All files have secondary copy

on tape. These are periodically moved to the firesafe.

Hardware• Based on Altix 350• Numaflex architecture• 40 Tbytes raid disk Network• Local server: 1Gbit network• Part of 10Gbit back-bone• Coming soon: SJ5 10Gbit link

User Access to Solar-B Data


Instrument Data

SOT: Solar Optical TelescopeMagnetic field imagesOptical imagesMagnetic field velocity

measurements

XRT: X-Ray TelescopeX-ray coronal images

EIS: EUV Imaging SpectrometerCoronal line strengths and profiles Plasma diagnostics

DMF

Users access data as URLs through the AstroGrid workbench, STAP web services, browsers, or wget.

Files via URLs and DMF system at RAL’s UKSSDC

Accessing Solar-B data through AstroGrid


MySQL

AstroGrid DSA

FITS keywords,Data URLs

DSA: DataSet AccessADQL to SQL

AstroGrid STAP service

INPUT: ADQL queries

STAP: Simple TimeAccess Protocol

OUTPUT: FITS keywords, URLs

INPUT: start / stop dates

OUTPUT: URLs

AstroGrid workbench

AstroGrid HelioScope,

STAP enabled tools

ADS Web-visiblefile system

AstroGRID PLASTIC tools, Solar-B website, wget, browser

INPUT: URLs

OUTPUT: Solar-B files

AstroGrid User Components


HelioScope: solar data results from STAP services.

AstroGrid workbench: STAP service access through HelioScope, DSA access through the Task Launcher.

Task Launcher: locate DSAs, submit ADQL queries

Workbench:

http://www2.astrogrid.org/desktop/index

STAP: Simple Time Access Protocolhttp://personalpages.manchester.ac.uk/staff/s.dalla/astrogrid/strap.html

DSA: DataSet Access

http://www.astrogrid.org/maven/docs/HEAD/pal/index.html

ADQL: Astronomical Data Query Language

http://www.ivoa.net/Documents/latest/ADQL.html

Mission Data Flow


Solar-B Studies / Observation Planning


• Study Proposal– All study proposals are tested on the Flight Spare model

before acceptance into the master Study Database– The Study Database contains information about the science

(target, pointing, rationale) as well as meta-data (author, joint ops id, etc)

• Database Distribution– This is done using the automatic Solarsoft update mechanism

through Goddard Space Flight Centre• Daily Planning

– Performed at ISAS using the Study Database– Studies are entered into the Timeline (As-planned) Database

FITS File Reformatting


• Fits Reformatting (1)– Level-0 performed at ISAS– Accesses As-Planned Database– Populates As-Run Database– Transfers files to ADS and ISAS web system– Accesses other instruments’ Fits files and transfers them to the

ADS for storage• Fits Reformatting (2)

– EIS Level-0 Fits files are used to generate Level-2 Fits files at MSSL

– Updates the data catalogue– Transfers Level-2 Fits files back to the ADS

Data Security

GridFTP used for file transfers


ADS(RAL)

ADS(RAL)

ADS(RAL)

GridFTP

GridFTP

GridFTP

HTTP USERS

GRIDFTP

• Secure transfers

• 3rd party transfers

• Good performance

• Certificates from UK e-Science CA and AIST (Japan)

STORAGE

• 10 TB per year

• 30 TB over nominal 3 year duration of mission

GridFTP: Clients / hosts authenticate with X509 certificates

Conclusions

• Solar-B launch tomorrow. 22 September 2006

• Data transferred via GridFTP to ADS at RAL

• ADS facility uses DMF to manage files on disk and tape

• Data searchable through AstroGrid DSA and STAP web service

• Solar community accesses data through AstroGrid workbench, HelioScope or a web browser

LINKS:

AstroGrid: http://www.astrogrid.org

Solar-B countdown: http://www.mssl.ucl.ac.uk/www_solar/


Questions?

http://www.astrogrid.org/

Solar-B Global DataGrid 21 st September 2006 UK e-Science All Hands Meeting 2006 RALMSSL Tim Folkes...

Documents

Transcript of Solar-B Global DataGrid 21 st September 2006 UK e-Science All Hands Meeting 2006 RALMSSL Tim Folkes...