Data access to distributed databases A case study from oceanography Taco de Bruin

1NIOZ is part of the Netherlands Organisation for Scientific Research (NWO)

Koninklijk Nederlands Instituut voor ZeeonderzoekRoyal Netherlands Institute for Sea Research

Data access to distributed databases

A case study from oceanography

Taco de Bruin

SCADM/SCAGI meetings

Amsterdam, NL, Sep. 7-9, 2009



Goal of all our data management activities is to provide the user access to the data holdings

“Collect once, use many times”



Situation of data exchange within NL-NODC a couple of years ago:

Online accessibility and availability of data

• Waterbase (RIKZ)• CODIS (NIOZ)• KODAC (KNMI)• Watermarkt (RIKZ, many other institutes)• HYDRA (KNMI)• COREBASE (NIOZ)• DINO (TNO- B&O)• Getij.nl (RIKZ)• Waterstat.nl (RIKZ)





• Waterbase (RIKZ)• CODIS (NIOZ)• KODAC (KNMI)• Watermarkt (RIKZ, many other institutes)• HYDRA (KNMI)• COREBASE (NIOZ)• DINO (TNO- B&O)• Getij.nl (RIKZ)• Waterstat.nl (RIKZ)

MEDIN Meeting 20090630





• Waterbase (RIKZ)• CODIS (NIOZ)• KODAC (KNMI)• Watermarkt (RIKZ + many partner institutes)• many more …..

- user had no overview of data availability

- user had to do all format conversions and syntheses

- user had to spend at least 50% of time on finding data and data conversion, instead of doing science



SeaDataNet:

A Pan-European distributed data access infrastructure

http://www.seadatanet.org/data_access

NODC-i:

A national distributed data access infrastructure, based on SeaDataNet technology

http://www.nodc.nl – Data Access Services



Total 49 partners, links with JCOMM, WDC, DMAC, MMI, EDIOS, EU-MarBEF (NoE), EU-Seased, ESEAS, Esonet, EuroGeoSurveys, POGO, etc.

SeaDataNet, consortium of national oceanographic data centres:

35 countries

IOC/IODE

ICES

JRC



SeaDataNet System

SeaDataNetPortal

Data centreData centreData centreData centre

Search&

shop

Metadata + transaction data



• CDI is a fine-grained index to individual data measurements (such as a CTD cast or moored instrument record).

• CDI: ‘common’ metadata of individual measurements (location, time, parameter, availability, owner, etc.)

Common Data Index (CDI)

CDI records from >29 countries



SeaDataNet system

CDI

User interface

DownloadManager

datadata

data

Shopping basket

Request Status Manager

DownloadManager

DownloadManager

Central User

Register



CDI Search – home page



Search – records selected for shopping basket



Contents of shopping basket



Submit Shopping list => User Authentication



Authentication, Authorization and Administration

- Single Sign On system set up for access to distributed system

- User’s authentication information based on personal login / password

- User must register to get one login, thereby agreeing with the “SeaDataNet User Licence”

- Central User Register, but management is decentralized

- Authorisation based on “User Roles”



Data access restrictions in CDI V1

CC Collection cost charge

FE Commercial charge

LI Licence

MO Moratorium

NA No access

NC Distribution cost

OG Organisation

RS Restricted

SR Academic

UK Unknown

UN Unrestricted

See Vocabulary L081 for definitions Multiple restrictions may apply



User Roles in AAA Register, of relevance for data access

SDRN02 Public

SDRN03 Academic

SDRN04 Commercial

SDRN05 National and local government

SDRN06 Pan-National government

See Vocabulary C866 for definitions

Multiple roles may apply



Business Matrix

W

V

U

DCBA Restriction

Role

OK

OK

OK

TBDOK

OKTBD

OK

OK

NO

TBDNO



Request confirmed and link to RSM (also by e-mail)



User RSM – Overview of request per Data Centre and Status



Downloading data from SeaDataNet infrastructure:

Standard formats (depending on data type):

- ODV ASCII

- NetCDF

- MedAtlas

Agreed QC flags



SeaDataNet is a working, distributed data delivery

infrastructure, featuring all elements required by SCADM,

SCAR or the Southern Ocean Observing System (SOOS).

The big challenge will be to couple the European SeaDataNet

infrastructure to the emerging Australian and American

systems and prevent these developments from diverging.

The International Oceanographic Data and Information

Exchange Committee (IODE) is working on achieving this

with its Ocean Data Portal project.

Data access to distributed databases A case study from oceanography Taco de Bruin

Documents

Transcript of Data access to distributed databases A case study from oceanography Taco de Bruin