Status report on the implementation of GISC at CMA

Li XiangNMIC, CMA

2010.2

To support WIS/GISC’s functions and provide the services of DAR, Data request, Data subscription and GTS data ingest to WIS centers and authorized users.

• Metadata generation

• Metadata Synchronization

• Metadata search

• Data subscription

• Data dissemination

• Web interfaces

Development of GISC service

Metadata Information Generating

VolC1VolC1

......Table B1-B6 from OP for GTS code

list.Vol A

Static Referance Information.

XML File

VolC1 +VolC1 +

Template

GTS Bulletin

CMA Satellite DataCMA TIGGE Data

CMA NWP Data

Metadata Basic Info.

Metadata Generation

Metadata File Generating

• Metadata files: 114,495, including the metadata files for

– GTS data (Vol C1) ： 59,175

– CMA NWP + FY satellite data : 1,222

– TIGGE products: 54,098 (10 centers’ products)

Metadata Generation

Auto-generation - DAR metadata for GTS Bulletin

• WMO Volume C1 (Catalogue of Meteorological Bulletins)

- CMA NWP, FY Satellite Data, TIGGE data

Repositories

M e t a d a t a

D a t a B a s e

Metadata Files

Harvest Service

Metadata CheckProviders

Metadata

FTP collect

Provider Service

HarvestersMetadata Files FTP out

Metadata synchronization system

JMA,DWD …

JMA,DWD …

Provider System

Harvest System

Metadata Synchronization (OAI, FTP)

Development environment

• Java jdk1.5

• Oracle 10g

• OAI framework:

- OAICat

- OAIHarvester2

• Web container

- Tomcat

Metadata Synchronization – provider configuration, monitoring

Metadata Synchronization – harvester configuration, monitoring

• Directory navigation

Metadata Search

• Simple Search (keywords, title, time, …)

• Complex Search (geographical location+keywords+…)

• SRU service, being implemented based on OCLC’s SRW/SRU open source

Data subscription

• Web Interface (subscribe data)

• Subscription parser (data order, dissemination metadata)

• Service management (queue, monitoring)

• Data dissemination (ftp, http, email)

BeginOrders

PreparationOrders

Submission Orders Database

Order Metadata

Metadata Database

Delivering

Metadata

OrdersAnalysis

Queue Management

Data Collection

Local Database

Packing, compression and other operational

configuration

Regulation for the file naming

Delivering SystemEnd

Send

Data dissemination

• GTS– protocols: ftp, tcp sockets

– other GTS centers

• Internet– protocols: ftp, http, email

– other centers, end-users

• Implemented the services– Metadata synchronization

• OAI-PMH, FTP

– Remote data request/reply• Access JMA data from CMA WIS portal• Access CMA data from JMA WIS portal

– Web Data Ingest• CMA:bulletin-based interface, TAC->BUFR• JMA: report-based interface• software shared between CMA and JMA

Collaboration with JMA

• Data Synchronization using blog-based technology

• Metadata– DWD Metadata generator– Dissemination metadata

• Contents, XML template• Service backup

– CMA Metadata synchronization components: harvester and provider

Collaboration with DWD

• Data synchronization test (2009.9)– AFD/FTP/SFTP, BLOG/HTTP/HTTPS

– test results: AFD has a better performance on making routine data exchange

Collaboration with DWD

• Metadata synchronization test (2010.1)

• Test beds– CMA: in-house software– DWD: GeoNetwork

• Test results– CMA harvesting metadata from DWD provider ： 12387 metada

ta in 14940s （ 1.206s/rec ）– DWD harvesting metadata from CMA provider ： 47326 metada

ta in 64441s （ 1.361s/rec ）

-> more tests planned in March-> analyze performance and make improvement

• Participants: • CMA, DWD, JMA• WISPO

• Discussion on• DAR Metadata • Data Synchronization• Metadata Synchronization • Service Monitoring and Service Backup

• Outcomes• test plans for metadata synchronization

• the document related to DAR metadata, including metadata file identifier, usage of keywords, usage of ISO element <CI_OnlineResource> for accessing online data and products, and submited it to CBS expert team.

Collaboration meeting on WIS development

Collaboration with HKO

• HKO will operate a DCPC in connection with the World Weather Information Service (WWIS) and Severe Weather Information Centre (SWIC) under the WMO Public Weather Services Programme.

• Beijing will be one of its the associated GISC

• NMIC, CMA and HKO made discussion on WIS development in Beijing in November, 2009– DAR metadata

– metadata provision

• Metadata provision tests– planned in March

Progress

Metadata generation √√Metadata Synchronization

√√

Metadata search 80%

(SRU service will be available in March)

Data subscription 80%

Data dissemination √√Web interface 70%

GISC service backup Planned in 2010