Third Session

of the Joint WMO-IOC Technical Commission for Oceanography and Marine Meteorology (JCOMM)

Marrakech, Morocco, 4-11 November 2009

IODE Ocean Data Portal

Nikolay Mikhaylov,JCOMM/IODE ETDMP Chair

Objective The objective of the Ocean Data Portal is to facilitate and promote the exchange and dissemination marine data and services.

Participants: ODP-WIGOS for JCOMM Steering Group, JCOMM/IODE ETDMP, IODE Project Office, IODE NODC/DNA’s

Project implementation period:• 2008 – 2009 (IODE-XX/JCOMM-3) – ODP (V.1)• 2009 – 2011 ODP (V.2)

Work packages:• Project coordination• ODP standards development• Portal implementation

IODE Ocean Data Portal ProjectIODE Ocean Data Portal Project

IODE Ocean Data Portal Concept IODE Ocean Data Portal Concept

• ODP is a standards based infrastructure for integration of data and services from IODE and other IOC/JCOMM data centres

• ODP provides on-line access and dissemination:– operational and delayed-mode data– data and services from the oceanographic and marine meteorological

domains– from multiple data formats and local data systems (DBMS, data files, map

services, electronic documents)– from multiple data and service providers in different geographic regions

• Data/service providers need to accept and implement a set of interoperability arrangements

• It will not required to change their local data structures and formats for distributed data integration

• ODP will provide the interactions with other systems (WIS, SDN,..)

• ODP is build on the existing components: ISO and OGC standards, DiGIR, OpenDAP, Russian system – ESIMO, Australian IMOS and other solutions

IODE Ocean Data Portal StagesIODE Ocean Data Portal Stages

The ODP development is a complex task and therefore it was defined that two versions of the portal would be developed one after another:

version 1 - with initial capabilities operating on the basis of the E2EDM technology

version 2 - with full capabilities which should be fully based on ISO/OGC interoperability standards and tools

During the reported period the ODP (version 1) has been developed (http://www.oceandataportal.org).

Data/Service registration, Metadata management, Data delivery,

Monitoring, …

Administrating, Discovery, Viewing, Analysis, Download

Layer of Distributed Data Net

IODE ODP Architecture

Interfaces

Data Catalogue

Data System Interaction

Data Provider

Metadata and Data

Request

External System

InterfacesInterfacesInterfaces

Response

Service Catalogue

Processing System Interaction

Service Provider

InterfaceInterface

Interfaces

Interfaces

InterfacesInterfaces

Interfaces Interface

End-User

Internet-network

User service layer

Integration layer

ODP V.1 Operation FrameworkODP V.1 Operation Framework

1. Integration of Distributed Data Sets

Data centers install the Data Provider software or use the light DP to serve as a data source for the distributed system according to the commitments of the centers.

The IODE Project Office uses the Integration Server for the metadata management and other operations for the distributed data system support

Each ODP data provider is responsible for provided data quality and data storage. There is an Integration Server function for “cashing” and storage of the data sets and this function can be implemented according to the agreements (driven by requirements).

User access

Integration Server Integration Server

ODP Services

ODP Services

ODP V.1 Operation FrameworkODP V.1 Operation Framework

2. Access to distributed data sets and end-user services

Discovery service disseminates a data source catalogue with descriptions of resources. The data source catalogue can be accessed from external systems directly or after reformatting into other metadata structures. 

Viewing service is based on Web-application and provides: (i) data search using a map, time period, resource and etc.; (ii) access to remote data sources and tabular-graphic/map visualization of data.

Analysis service has been developed recently – the GIS applications for presentation of multidisciplinary data and products on the map and WMS/WFS services.

Download service - the loading the data on user computer via search or a time schedule (in this case it is required to register a point for downloading, list of required data sets and sampling criteria). Transport data file formats: NetCDF, ASCII and XML.

ODP V.1 ODP V.1 Discovery Service

Resource catalogue

Parameters

Additional search options

ODP V.1 Search Service

Search status

NODC, Russia

• ODP Web site www.oceandataportal.org

• ODP applications

- IODE Project Office (VLIZ) www.oceandataportal.net

- RIHMI-WDC (mirror) http://data.meteo.ru/e2edm/

ODP V.1 Viewing Service

Point data grid dataprofile data

ODP V.1 GIS Analysis ServiceODP V.1 GIS Analysis Service

IODE ODP V.1 Operation Framework IODE ODP V.1 Operation Framework

3. Administrative management

The administrative section of the ODP provides authorization, authentication and audit against user roles and rights on particular data and tools.

Service is provided according to the IOC Data Exchange Policy. The following access rules were defined:(i) metadata are freely accessible without any condition.(iI) data and products require:

- registration- acceptance of user license conditions:

referencing of data sources

Statistics on data and metadata usage, requests and ranking of data products are based on the actions of authorized users only.

Integration Server Integration Server

ODP Services ODP Services

SDN Portal

WIS GISC OBIS

OAI Client(GISC)

OAI Server

DCPCObninsk

WMOSchema

«Register»

WMO Core Metadata records (v.0,2, v.1.0)

«Request»

«Response»

E2EDM records are reformatted into WMO standard records are migrated from (A) to (B)

«A»

«B»

GISC catalogue / registry

Resource catalogue

Integration Server

Data Providers

VGISC protocol communicationInterexchange rules

1. Intersystem’s connection is provided

2. Data exchange list is agreed

3. Discovery metadata catalogue is used

4. Metadata reformatting is provided

IODE ODP V.1 Operation Framework IODE ODP V.1 Operation Framework

4. Interaction with other systems

JCOMM DCPC Prototype (Obninsk) – 2007, WIS prototype project

ODP Documentation

• Overview of the IODE/JCOMM Ocean Data Portal (April 2009);

• E2EDM Technology Overview (V.1.0, July 2008);

• The Checklist of technical requirements for data providers for E2E technology implementation (V.1.0, August 2008)

• E2EDM technical specification (V.1.0, January 2007)

• E2EDM Concept XML Schema (V.1.1, April 2009);

• E2EDM Global XML Schema (V.1.0, May 2007);

• E2EDM Integration Server User Guide (V.1.0, March 2007);

• E2EDM Data Provider User Guide - Installation and setup (V.1.1, July 2008);

• E2EDM Data Provider User Guide

• Ocean Data Portal. User Guide (V.1.1, April 2009)

IODE ODP V.1 Operation Framework IODE ODP V.1 Operation Framework

ODP for ODP for IOC/IODE Ocean IOC/IODE Ocean Data Data Infrastructures Infrastructures (ODIN’s)(ODIN’s)

IODE ODP V.1 Implementation IODE ODP V.1 Implementation for ODIN’sfor ODIN’s

ODINCARSA ODINCINDIA ODINECET

ODINAFRICA ODIN-BlackSeaODIN-WESTPAC

Training courses on ODP:

• the IODE Project Office (Oostende, Belgium, October 2007),

• RIHMI-WDC/Russian NODC (Obninsk, Russia, March 2009)

• NODC of Korea (Seoul, Republic of Korea, August, 2009)

IODE ODP V.1 Implementation for JCOMMIODE ODP V.1 Implementation for JCOMM

ODP has been designated as a WIS Data Collection and Processing Centre (DCPC) and will be a resource that can be discovered through the WIS Global Information System Centre (GISCs).

JCOMM Pilot Project for WIGOS:(i) define and agree on common standards to manage the

instrument/platform metadata in cooperation with JCOMM ET (META‐T, VOS/VOSC and etc.)

(ii) build a distributed data system and provide its interoperability with WIS.

(iii) prepare and submit the ocean and marine meteo standards to be adopted via IODE/JCOMM Standards Process.

WIS Compliance Specifications

• Existing centres within WMO Member States may apply for designation as one of the functional centres forming the core infrastructure of WIS: – Global Information System Centres (GISCs)– Data Collection or Production Centres (DCPCs)– National Centres (NCs)

• Designation requires a statement of compliance

with WIS requirements, compiled by the Inter-Commission Coordination Group on WIS (ICG-WIS)

• The WIS Compliance Specifications document is

the authoritative source for specifications applicable to WIS GISCs, DCPCs, and NCs

WIS-TechSpec-1, Uploading of Metadata for Data and Products. WIS-TechSpec-2, Uploading of Data and Products. WIS-TechSpec-3, Centralization of Globally Distributed Data.

Status – not required . WIS-TechSpec-4, Maintenance of User Identification and Role

Information. WIS-TechSpec-5, Consolidated View of Distributed Identification

and Role Information. WIS-TechSpec-6, User Authentication WIS-TechSpec-7, User Role Authorization WIS-TechSpec-8, DAR Catalogue Search and RetrievalWIS-TechSpec-9, Consolidated View of Distributed DAR Metadata

CataloguesWIS-TechSpec-10, Downloading Files via Dedicated NetworksWIS-TechSpec-11, Downloading Files via Non-dedicated Networks WIS-TechSpec-12, Downloading Files via Other MethodsWIS-TechSpec-13, Maintenance of Dissemination MetadataWIS-TechSpec-14, Consolidated View of Distributed

Dissemination Metadata CataloguesWIS-TechSpec-15, Reporting of Quality of Service

WIS and ODP Interoperability

Functional view

IODE ODP V.1 Implementation for JCOMMIODE ODP V.1 Implementation for JCOMM

Technical view

ODP will make appropriate ocean and marine meteo datasets to WMO and IOC applications. A number of organizations and programmes have been identified as potential partners and data providers for ODP.

Пользователь

Сервер Интеграции

Транспортный файл данных

Транспортный файл данных

ЗапросОтвет Интерфейсное взаимодействие

Сообщения -ответы Сообщения -запрос

Транспортный

файл данных

ПоставщикДанных

Поставщик

Данных

Поставщик

Данных

IODE ODP V.1 Implementation for JCOMM IODE ODP V.1 Implementation for JCOMM

Core JCOMM for WIGOS partners 1) US NODC: World Ocean Atlas; World Ocean Database; US NODC GTSPP; Surface currents from HF radar; 2) Russian Federation NODC: GTS observation data, analysis and forecast data gridded fields 3) Canada, ISDM: Upper-ocean T & S gridded fields; and Ocean currents derived from surface drifters; 4) BODC GLOSS Sea-level data 5) UK Met Office or DWD via Virtual GISC or direct impact to ODP Marine Climatological Summaries and Global Collecting Centres (GCCs)

Potential partners 1) In situ data sets from the JCOMM Observations Programme Area (Argo, OceanSITES, TAO, DBCP, SOT, GOSUD 2) Model output fields (e.g., GODAE); 3) Metadata about the platforms / instruments (e.g., META-T, ODAS); 4) Fast delivery sea level data (University of Hawaii Sea Level Center);

IODE ODP for JCOMM – current main issue IODE ODP for JCOMM – current main issue

ODP implementation processes are strongly connected with the expansion of data providers

How to become the ODP data provider?

Data Catalogue

Data System Interaction

Data Provider

User access

Integration Server Integration Server

ODP Services

ODP Services

ODP profile for Climate

ODP profile for Marine Hazards

ODP profile for

Maritime Safety

ODP profile….

IODE/JCOMM distributed data system – E2E technology realization

IODE ODP for JCOMM – looking forward IODE ODP for JCOMM – looking forward

Thank you for attention

Marrakech, Morocco, 4-11 November 2009