GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories Grant...

GENESI-DR:Ground European Network for

Earth Science Interoperations - Digital Repositories

http://www.genesi-dr.eu

Grant agreement no: 212073

EGEE 4th User Forum – Catania, 4 Mar 2009EGEE 4th User Forum – Catania, 4 Mar 2009

EGEE and GENESI-DR: an interface for accessing and processing Earth Science data

Roberto Cossu – ESA

EGEE and GENESI-DR: an interface for accessing and processing Earth Science data

Roberto Cossu – ESA

EC Grant Agreement no. 212073

24/02/2009 2EGEE 4th UF - Catania - March 4th, 2009

Outline

• Introduction to GENESI-DR– The Earth Science data deluge– The needs for data discovery and access

•The deployed GENESI-DR infrastructure

•EGEE – GENESI-DR collaboration

•The GOME processing and validation scenario



The EO data deluge…

Data (Petabytes)

1980

1985

1990

1995

2000

2005

2010

2015

2020

2 4 6 8 10 12 14

Petabytes of already acquired data are presently underexploited, because to get the results in a reasonable time not enough computing resources,

tools and algorithms are available.



The ES data deluge…

Similar situation with other ES sensors airborne, on ground and in the sea – e.g. sensor webs



Several Digital Repositories

Difficulties in discovering and accessing data and productsDifferences in data policies, formats and quality…



What is unique in GENESI-DR (1)

GENESI-DR

A centralized discovery service

to seamlessly access heterogeneous sources of data…

…in situ, satellite, airborne…



How it works…

User can discover via the GENESI-DR Web portal (or web services) data and services on the base of free text string and

metadata…

such as geographical

area, temporal range..

…and other specific parameters as applicable



about=rdf:about="http://earth.esa.int/object/index.cfm?fobjectid=1454"

title=MERIS Full Resolution Geophysical Product (MER_FR__2P)

abstract=MERIS Full Resolution Geophysical Product (MER_FR__2P) MERIS MER_FR__2P is a Full-Resolution Geophysical Product for Ocean, Land and Atmosphere. The product is also specific to Water Leaving Radiance (Reflectance). Each MERIS Level 2 geophysical product is derived from a MERIS Level 1b product and auxiliary parameter files specific to the MERIS Level 2 processing and distributed following the floating scene concept. NRT version is available 1 day after acquisition, OFL (fully consolidated version) 2 weeks after acquisition. The file size is 187 Mbytes per scene and auxilary data include: Surface identification flags included in the level 1b product, Orbit state vector, Time correlation parameters, Latitude, Longitude, altitude and topographic corrections, Sun azimuth, Sun elevation, view azimuth, view elevation, Mean Sea Level pressure, Total column ozone, Total column water vapour, Wind speed.

datasetname=MER_FR__2PprocessingLevel=Level 2satellite=ENVISATsensor=MERISsubject1=Atmosphere (Clouds)subject2=Ocean and Coast (Sea Surface

Temperature,Ocean Colour/Biology)subject3=Atmosphere (Air Quality (Ozone),Atmospheric

Winds,Clouds,Precipitation)subject4=Land (Soil)rights=Free Usage with Rigthsprojection=pseudo-satellite projection along trackresolution=265spatial=POLYGON((-180.0 -90.0, -180 90, 180 90, 180

-90, -180 -90))fileFormat=N1 (ENVISAT)contact=rdf:resource="http://eohelp.esa.int/"

How it works…

GENESI-DR central site owns a

description of the series of data archived at the federated DRs



How it works…

DRs reply with all data and services matching the query

CNR

ENEADR

DR

DRDR

DRDR



How it works…

User is provided with

the direct access to the DR

catalogue, data

and services. (Data are not copied to the central site).

ENEA

CNR

ENEA

ESA

Individual DR

data policies are considered and respected



How it works…

User can access and manipulate data …

…buoys archived at ENEA

…SST maps previously archived at CNR

…SST generated on the fly from satellite data at ESA

using Grid resourcesCNR

…to perform new science (and collaborate)

…to generate new products

… to populate new repositories

GENESI-DR



The deployed operational infrastructure

– Currently:• 8 DRs have been GENESI-fied• 42 different dataset series

– Data discovery, data access, processing– “GENESI-DR Operations Manual” and “GENESI-

DR User Manual”

satellite

Oceanographic ships

Buoys

Airplanes Lidar

Model results



Which services/features we demonstrate

GOME Processing GOME Processing & Validation& Validation

In DRsIn DRs

Data access/ downloadData access/ download ProcessingProcessingData discoveryData discovery

Web portalWeb portal

GlobModel GlobModel Visualization and Visualization and Validation ServiceValidation Service

SeaDataNetSeaDataNet

SST and VOSSST and VOS

AlgalBloomAlgalBloom

SST generationSST generation

Hetero-Hetero-genous genous

datadata

APIAPI

In the In the central citecentral cite

Multiple Multiple data data

downloaddownload

Results Results registration registration in GENESI-in GENESI-

DRDR

Multiple Multiple access access pointspoints



EGEE – GENESI-DR collaboration

•A.1 : Enable EGEE ES community access to GENESI-DR data.

•A.2 : To enable the EGEE ES community and GENESI-DR users to share applications across the two infrastructures



Ozone processing and validation

• Satellite-borne GOME sensor is able to measure the total column of Ozone in the atmosphere.

• Algorithms allow the scientists to estimate the vertical profiles of Ozone starting from the total column

• Lidar stations measure the vertical profile of Ozone • While satellites provide global coverage, Lidar stations

can provide measures only in a narrow area of around them

• Scientists are interested in comparing/validating the Ozone profiles obtained processing GOME acquisitions with Lidar measures




• Aim: to discover, access, massive processing and validation of Ozone data and products

• Data:– ERS-2 GOME data @ESA– LIDAR profiles @ IPSL/CETP

• Processing:– The algorithms for the computing the vertical

profiles of Ozone and for validating these results with Lidar profiles are integrated in GENESI-DR




Webportal

Central SiteESA

GOME data

IPSL

Lidar in-situ data

Ozone processing

Profiles comparision

GENESI-DR

“External” world




ESA IPSL

User Portal Central Site Digital repositoriesComputing Elements

Log in, service selection, parameter

definition

Data discovery

Data discovery

Available data

Service triggering Send processing

Results

Selected data

Task status

Query

Dataset Series available



Conclusions

• GENESI-DR is an ambitious project with large implications on present operations in ES data providers

• Getting the right set of demonstration and validation scenarios are of fundamental importance to convince both Data Providers and Data Users to join the GENESI-DR federation of Earth Science Digital Repositories

• EGEE & GENESI-DR collaboration has already allowed the deployment of a “validation application”

• The application is accessed via web portal. More in general GENESI-DR exposes a discovery web service

• Future work within the collaboration will enable the two communities to share data and resources

GENESI-DR:Ground European Network for

Earth Science Interoperations - Digital Repositories

http://www.genesi-dr.eu

Grant agreement no: 212073

EGEE 4th User Forum – Catania, 4 Mar 2009EGEE 4th User Forum – Catania, 4 Mar 2009

EGEE and GENESI-DR: an interface for accessing and processing Earth

Science dataRoberto Cossu – ESA

EGEE and GENESI-DR: an interface for accessing and processing Earth

Science dataRoberto Cossu – ESA

GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories Grant...

Documents

Transcript of GENESI-DR: Ground European Network for Earth Science Interoperations - Digital Repositories Grant...