GCMD/IDN STATUS AND PLANSStephen WhartonCWIC MeetingFebruary19, 2015

Outline

2

1. GCMD/IDN Operational Services2. DIF Support3. Continuity of Services

1. GCMD Operational Services

3

– IDN– Web Search Interfaces– Metadata QA Process– docBuilder– Operations Client– Web Services (KMS, MWS, CSW,

OpenSearch)

IDN• What it does:

o NASA’s contribution to CEOS Working Group on Information Systems and Services (WGISS) community.

o Offers researchers a search interface using GCMD’s taxonomy to locate publicly available datasets and services.

o Offers partner agencies customized look & feel of metadata.

• How it works:o Leverages GCMD’s portal

technology (MD server, free text, and relational database) to create IDN subset.

o Uses the GCMD web interface to produce a customized view (the IDN portal).

o Uses a Content Management System (CMS) for static content such as images and articles.

Web Search Interfaces

5

• What it does:o “Search By” menu allows users to

click the tab and search by combinations of keywords and free text search.

o “Search portals” allow users to search a subset of records associated with a particular science discipline or organization.

• How it works:o A dynamic query is built and then

translated to a combined data base and free text call.

o Templates (JSPs) are used at the display level for portal customization.

Metadata QA Process

Metadata QA Steps:1. Automated validation 2. Triage (prioritization and routing)3. Manual QA check4. Notify metadata provider of issues and recommend changes

QA Frequency:• New and/or updated metadata records are reviewed as they are submitted. • “Link Checker” tool used monthly to scan for broken links and notify the science coordinator/providers when links are broken. • Annual check and review for all records that have not been reviewed or updated over the past 2 years.

6

• Tools used:o OPS cliento docBuildero MWS & KMS

docBUILDER• What it does:

o Online authoring tool to add or modify DIFs and SERFs and other formats.

o Provides visual check list of fields to populate.

o Offers choices from controlled vocabulary (intended to make results more accurate and consistent).

o Provides validation of controlled fields and syntax.

o Provider Customizable

• How it works:o MWS to access recordso KMS to access keywordso Portal framework supports

provider customizability o Extensible to other schemas

Operations Client (OPS)

• What it does:o Tool to manage and QA metadata. o Display queue of incoming

metadatao Load, extract, or delete metadata by

ID or by queryo Launch docBuilder

• How it works:o Uses MDServero Uses automated checks to prioritize

queueo Uses RMI calls to provide an

interactive load process

8

Keyword Management Service API (KMS)

• What it does:o RESTful web service provides access to GCMD keywordso Supports multiple keyword sets (RDF, OWL, CSV)o Multiple REST pathso Used by DAACs to retrieve keywords and sync keywords

between provider and cliento Provides a normalized and persistent keyword ontology

• How it works:o Applies a query interface to RDBMS.o Uses a caching layer for improved performance on large

keyword sets. o Utilizes UUIDs to provide persistent keyword identifiers

9

Metadata Web Service API (MWS)• What it does:

o Provides dynamic access to all GCMD metadata record types (DIF, SERF, CD, Supplemental)

o Retrieves a list of: ALL metadata documents, filtering by a query expression (e.g.,

by instrument). A specific metadata document, using its unique identifier. Unique identifiers of metadata documents, filtering by a query

expression Keywords for a specific metadata field.

How it works:o Uses RESTful service to translate web requests to database

statementso Uses caches to improve performanceo Uses a custom query language to support ‘portalization’

10

Catalog Service for the Web API

• The API uses OGC standards to provide access to metadata.

• The IDN provides distributed CSW queries against the IDN CSW server and is used to support of the following projects.o GEOSS/CI - Global Earth Observation System of Systems Common

Infrastructureo CEOSS/CWIC - Committee on Earth Observation Satellites WGISS

Integrated Catalogo EURO-GEOSS - European approach to GEOSS

11

OpenSearch API

• What it does:o Supports distributed searches using free text and

GCMD keywords.o Results can be viewed in either HTML, Atom,

RSS, or CSV. o Results can be bounded by GCMD portal subsets

(portals; predefined queries forces on one discipline and/or provider contribution).

• How it works:o Leverages existing GCMD search interfaces to

translate OpenSearch querieso Employs standards based APIs

12

2. DIF Support

• Develop DIF-10 extension of DIF-9 to support UMM-C Compliance.• DIF-10 Changes:

o 15 required fields for DIF-10 (versus 6 required fields for DIF-9)

o New Fields include: Platform/Instrument/Sensor, Granule Link, Version ID

o GCMD is prototyping adapters to transform DIF-9 to DIF-10.

o DIF-10 will also support the ECHO-10 fields.o DIF-9 will continue to be supported.

13

3. Continuity of Services

- GCMD Services- IDN Support

14

GCMD Services

• Near Term: GCMD services will continue unchanged.o DIF-9 support will continueo DIF-10 support will be addedo Science keywords will be supportedo GCMD Services will be supported

• Longer Term:o DIF metadata will migrate to CMRo GCMD will continue to support Science QAo GCMD services will be provided via CMR

15

IDN Support

16

Maintain continuity of CEOS/GEOSS Services (Near-Term)• Support IDN portals• Support CWIC (records, members, QA, metadata mappings)• Support discovery of CWIC records (OpenSearch or CSW servers.

o Support OpenSearch servero Support CSW server for GEOSS services

• Deploy GEODataCore tags on request from agencies

Maintain continuity of DIF format/content • DIF format evolution (In parallel with UMM-C)• Work with IDN partners to ensure high quality DIF content

IDN records would eventually move to the CMR • Transition IDN (CEOS) metadata records into the CMR.• Support QA of CEOS and other non-NASA metadata in the CMR