INFSO-RI-508833 Enabling Grids for E-sciencE A service oriented framework to create, manage and...

INFSO-RI-508833

Enabling Grids for E-sciencE

www.eu-egee.org

A service oriented framework to create, manage

and update metadata for earth system science

K. Ronneberger, DKRZ, GermanyS. Kindermann, DKRZ, Germany

EGEE Userforum 2008 – Clermont-Ferrand 11.-14.02.08 2


INFSO-RI-031688

Structure

• Background: – What is Earthsystem science about

– Motivation of this framework

• Design of the metadata framework– Requirements– Concept – Implementation

• Potential impact and vision



INFSO-RI-031688

Distributed Climate Data

Process

Result Dataset

Earthsystem Sciences

• Goal: learn about the past, the present, and possible futures of the earth system

• Method: Modelling, analysing, comparing and processing data

• Input: data from observations and/or other modelling studies

• Community: internationally and interdisciplinary distributed but strongly interconnected

Model DataObservation Data

Scenario data

Typical workflow

Analysis Dataset

Find & Retireve

Share

Result Dataset

Result Dataset

Compare/reuse

Compare/reuse



INFSO-RI-031688

Data grid for Earthsystem Science

• A grid to

– Share data

– Find data

– Process data in modular, independent steps

– Compare/reuse data

…needs Metadata to describe

Content (unique variable description, temporal & spatial bound)

Discovery (where to find, how to access)

Use (format, size, etc.)

Provenance (origin of data, performed processing steps etc.)



INFSO-RI-031688

ISO-Metadata for earthsystem science

• Complex enough to reference geographic data

• Offers a hierarchical description• XML implementation ISO 19139 available• Used for ESS data by academic and

business:– ANZLIC– ESRI– con terra GmbH

• C3Grid -> adapted ISO format for grid world

ISO 19115

Content Discovery Use Provenance

Tools are needed to create, manage and update metadata



INFSO-RI-031688

Share DataF

ind

& retrieve

Process Data

Metadata in the grid

EGEEEGEE

ISO Metadata

C3Grid dataprovider

C3Grid dataprovider

ISO Metadata

C3Grid dataprovider

C3Grid dataprovider

ISO Metadata

C3Grid dataprovider

C3Grid dataprovider

ISO Metadata

C3Grid dataprovider

C3Grid dataprovider

ISO Metadata

EGEEEGEE

ISO Metadata

Central Metadata catalogue

Web Portal C3

Co

mp

are/

reu

se

Data

ISO MD

Data‘

ISO MD’

Create ISO MD for existing data

resource

Automatically update ISO MD

during processing



INFSO-RI-031688

Metadata workflows

Create ISO MD for existing data

resource

Automatically update ISO MD

during processing

ISO 19115

Content Discovery Use Provenance

(1) Understand ISO-MD implementation

?

tempBB

geogrBB

vertBB

extent

timeBegin

timeEnd

timeInt

(2) Interactively create default file

<iso:abstract> this data ..

</iso:abstract>

…

<iso:contactInfo>

<iso:name> DKRZ

</iso:name>...

(3) Produce property file

Default A

Ref_xml B,C

update timeEnd with t1

add vertBB from A

update cfvar = a with b

delete keyword = c



INFSO-RI-031688

System designStep (1) (2) (3)

Function view structure, content and organization of the

metadata

Parsed creation/change of metadata via GUI

Automatic, property file driven update of MD

Requirements • graphical depiction• Schema template & specification file• Example MD files

• GUI• Secure login• Parsing functionality• Schema template & specification file• Example MD files

• API• property file language• A&A• Update & parsing functionality• Schema template, specification & default file• Original MD files

System layout • Interface

•Specification layer• XML Databases

• Interface

•Security layer• Manipulation layer• Specification layer• XML Databases

• Interface• Abstraction layer• Security layer• Manipulation layer• Specification layer• XML Databases



INFSO-RI-031688

Implementation

Metadata basisMetadata basis ISO XMLs

Specification layerSpecification layer ISO XML template, Specification file, default XMLs

Manipulation layerManipulation layer XSLT & XQuery

Secure login

HTML + css

Orbeon

Exist

BrowserBrowser

OGSA-DAI

Property file interpreter

API

Program interface



INFSO-RI-031688

Summary

• Modular Differentiated access

• Common basisEasy maintainable

• Layered systemEasy extendable/reusable

• Standardized technologyEasy adaptable

Metadata basis ISO XMLs

Specification layer ISO XML template, Specification file, default XMLs

Manipulation layer XSLT & XQuery

Secure login

HTML + css

Orbeon

Exist

BrowserBrowser

OGSA-DAI

Property file interpreter

API

Program interface



INFSO-RI-031688

Impact & vision

• Tools are used in C3-EGEE frameworkEase integration of further data

providers/processors

• ISO is increasingly used as metadata format for ES dataFoster the collaboration on ES data

• The schema related XMLs are limited and easy to exchangeAdapt tools for further schemas/communities

INFSO-RI-508833 Enabling Grids for E-sciencE A service oriented framework to create, manage and...

Documents

Transcript of INFSO-RI-508833 Enabling Grids for E-sciencE A service oriented framework to create, manage and...