ESDIS and International Standards

ESDIS and International StandardsTed Habermann, NASA ISO Metadata WorkshopNovember, 2012

ISO Process

Preliminary Proposal Preparatory Committee Enquiry Approval Publication

New Work Item

Proposal

Draft New Work Item Proposal

Working Draft

or PAS

Technical Consensus

(DIS)

Final Draft International

Standard(FDIS)

International Standard

(IS)

5 months 2 months

0-6 months 6 months 12 months 9 months 3 months? months

ISO Project Options

Stage Proposal Preparatory Committee Enquiry Approval Publication

Develop and accept

committee draft

Develop and accept

enquiry draft

Publication of International

Standard

NormalProcedure

Acceptance of Proposal

Preparation of working

draft

Approval of FDIS

Draft With Proposal

FastTrack

Technical Specification

Technical Report

Publically Available Specification




Standard


Standard

Publication of Technical

Specification



Approval of FDIS

Approval of FDIS

Study by Working

Group

Prepare Draft Accept Draft

Develop and accept

committee draft

Develop and accept

enquiry draft

Accept enquiry draft

Approve draft PAS

Publication of Technical

Report

Publication of PAS

Accept Draft

ISO Timeline

P PAEC P PAEC

3 years 3-5 years 3 years

6-8 yearsImplementation

19115-1 (Metadata Conceptual Model: IS 2013/06)*19115-3 (Metadata Schema, Transform, Schematron: IS 2015/05)*

19115-2 (Metadata Acquisition / Images Conceptual Model)*19139-2 (Metadata Acquisition / Images Schema: TS 2012/10)*

19157 (Data Quality Conceptual Model: IS 2013/04) *19157 (Data Quality Schema)*

TS 19130 Parts 1-2 (Imagery Sensor Models for Geopositioning Conceptual Model: TS 2012/12)*

Conceptual Models and Implementations

191152003-05

191392006-083.3 Years

19115-22009-02

19139-22012-103.6 Years

19115-12013-06

19115-32015-051.9 Years

191572013-04

19157-2??

~50% Decrease

Now creating XML schemas directly from UML models using software developed in OGC Testbed. This capability is also being added directly into the tool by the vendor.

Test schemas for 19115-1 and 19157 with proposed NASA extensions are available now.

191302012-12

???

Model SchemaDelay

5.8 Years

Conventions and Extensions

Conventions are agreements that ensure consistent usage of the standard and facilitate interoperability across communities. They are expressed as guidance and best practices for the community.

Requirements not considered during the development of the original standard or that emerge during the implementation period can be addressed using extensions to the standard.

These extensions should be considered as candidate changes when the standard is revised every five years.

Developing NASA conventions and extensions:1) makes it possible to use the standard now and 2) increases the chances of addressing NASA needs in the next version of the

standard.

P PAEC P PAEC

Conventions and ExtensionsImplementation

NASA Documentation Requirements

ISO Standards

Users

GCMD

ECHO

DAACs/ECS

Missions

Science Teams


(Some) Existing Documentation Requirements

More than 2600 Collections have been described using the ECHO metadata dialect.

Almost 6000 DIFs include the word NASA.

ECHO Dialect

~150 Elements with content. Keywords, identifiers, and contact information are the most commonly occurring fields.

324 AdditionalAttributes occur over 3000 times

Most Common: PROCESSVERSION QAPERCENTGOODQUALITY QAPERCENTOTHERQUALITY QAPERCENTNOTPRODUCEDCLOUD QAPERCENTNOTPRODUCEDOTHER

DIF Dialect

~130 elements with content. Keywords, identifiers, and contact information are the most commonly occurring fields.

99% translation to ISO, ~79% translation to ECHO

More Overlap Than Difference

DIF

ISO

ECHO

The metadata dialects currently used by ESDIS have much more overlap than difference.

The mappings are generally well understood: we are in a tweaking stage.

The translations can be implemented using well-known, standard tools that are designed for XML processing.

These are different from the programming languages generally used for scientific data processing.

Additional AttributesThe ECHO AdditionalAttributes are an undifferentiated pile of information.

They can be translated to ISO, but we need to know where to put them. In order to do that, we need to identify their types.

AdditionalAttributes and ISO

citation.datecitation.identifiercitation.pointOfContactcontentInformationdescriptiveKeywordsdistributionInformationextentgeographicIdentifierinstrumentInformationlineageprocessingInformationqualityInformation


Many of the AdditionalAttributes fit into standard ISO elements.

Others are implementation specific elements of existing ISO types.

Rules:Percent/Pct > contentInformationQuality > qualityInformationGain/Bias > instrumentInformation

75% of the attributes, 81% of content


<eos:otherPropertyType> <gco:RecordType xlink:href="http://www.echo.nasa.gov/ingest/schemas/operations/Collection.xsd# xpointer(//element[@name='AdditionalAttribute'])">Echo Additional Attribute</gco:RecordType></eos:otherPropertyType><eos:otherProperty> <gco:Record xmlns:echo="http://www.echo.nasa.gov/ingest/schemas/operations"> <echo:AdditionalAttribute> <echo:Name>ScanAngle</echo:Name> <echo:DataType>float</echo:DataType> <echo:Description>The angle between the sensor view vector and the nadir axis</echo:Description> <echo:MeasurementResolution/> <echo:ParameterRangeBegin/> <echo:ParameterRangeEnd/> <echo:ParameterUnitsOfMeasure>degrees</echo:ParameterUnitsOfMeasure> <echo:ParameterValueAccuracy/> <echo:ValueAccuracyExplanation/> <echo:Value>47.5</echo:Value> </echo:AdditionalAttribute> </gco:Record></eos:otherProperty>

RecordType/Record - ISO with XML Reference

RecordType

Record

gmi:MI_AcquisitionInformation

MD_Metadata

MI_Instrument

+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ type : CharacterString+ description[0..1] : CharacterString

MI_EnvironmentalRecord

+ averageAirTemperature : Real+ maxRelativeHumidity : Real+ maxAltitude : Real+ meteorologicalConditions : CharacterString

+ environmentalConditions 0..*

+ operation

0..*

MI_Platform

+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ description[0..1] : CharacterString+ sponsor[0..*]: CI_ResponsibleParty

MI_Plan

+ type[0..1] : MI_GeometryTypeCode+ status : MD_ProgressCode+ citation[0..*] : CI_Citation

MI_Operation

+ description[0..1] : CharacterString+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ status : MD_ProgressCode+ type[0..1] : MI_OperationTypeCode0

MI_AcquisitionInformation

+ platform 0..*

+ acquisitionPlan 0..*

+ instrument 0..*

+ objective 0..* + requirement

0..*

MI_Objective

+ identifier[1..*] : MD_Identifier+ priority[0..1] : CharacterString+ type[0..*] : MI_ObjectiveTypeCode+ function[0..*] : CharacterString+ extent[0..*] : EX_Extent

MI_Requirement

+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ requestor[1..*] : CI_ResponsibleParty+ recipient[1..*] : CI_ResponsibleParty+ priority : MI_PriorityCode+ requestedDate : MI_RequestedDate+ expiryDate : DateTime

+ mountedOn 0..1

eos:MI_AcquisitionInformation

MD_Metadata

MI_Instrument

+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ type : CharacterString+ description[0..1] : CharacterString+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record

MI_EnvironmentalRecord

+ averageAirTemperature : Real+ maxRelativeHumidity : Real+ maxAltitude : Real+ meteorologicalConditions : CharacterString

+ environmentalConditions 0..*

+ operation

0..*

MI_Platform

+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ description[0..1] : CharacterString+ sponsor[0..*]: CI_ResponsibleParty+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record

MI_Plan

+ type[0..1] : MI_GeometryTypeCode+ status : MD_ProgressCode+ citation[0..*] : CI_Citation

MI_Operation

+ description[0..1] : CharacterString+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ status : MD_ProgressCode+ type[0..1] : MI_OperationTypeCode0

MI_AcquisitionInformation

+ platform 0..*

+ acquisitionPlan 0..*

+ instrument 0..*

+ objective 0..* + requirement

0..*

MI_Objective

+ identifier[1..*] : MD_Identifier+ priority[0..1] : CharacterString+ type[0..*] : MI_ObjectiveTypeCode+ function[0..*] : CharacterString+ extent[0..*] : EX_Extent

MI_Sensor

+ citation[0..*] : CI_Citation+ identifier : MD_Identifier+ type : CharacterString+ description[0..1] : CharacterString+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record

MI_Requirement

+ citation[0..1] : CI_Citation+ identifier : MD_Identifier+ requestor[1..*] : CI_ResponsibleParty+ recipient[1..*] : CI_ResponsibleParty+ priority : MI_PriorityCode+ requestedDate : MI_RequestedDate+ expiryDate : DateTime

+ sensor 0..*

+ mountedOn 0..1

+ mountedOn 0..1

gmi:DQ_Lineage

MD_Metadata

DQ_DataQuality

+ scope : DQ_Scope+ lineage 0..1

LI_Lineage

+ statement [0..1] : CharacterString

LE_Source

+ description [0..1] : CharacterString+ scaleDenominator [0..1] : MD_RepresentativeFraction+ sourceReferenceSystem [0..1] : MD_ReferenceSystem+ sourceCitation [0..1] : CI_Citation+ sourceExtent [0..*] : EX_Extent+ processedLevel[0..1] : MD_Identifier+ resolution[0..1] : LE_NominalResolution

LE_ProcessStep

+ description : CharacterString+ rationale [0..1] : CharacterString+ dateTime [0..1] : DateTime+ processor [0..*] : CI_ResponsibleParty

+ source 0..*+ processStep 0..*

+ output, source0..*

If(count(source) + count(processStep) =0) and(DQ_DataQuality.cope.level = 'dataset' or 'series')then statement is mandatory

LE_Processing

+ identifier : MD_Identifier+ softwareReference[0..*] : CI_Citation+ procedureDescription[0..1] : CharacterString+ documentation[0..*] : CI_Citation+ runTimeParameters[0..1] : CharacterString

LE_Algorithm

+ citation: CI_Citation+ description : CharacterString

+ processingInformation0..*

+ algorithm 0..*LE_ProcessStepReport

+ name : CharacterString+ description[0..1] : CharacterString+ fileType[0..1] : CharacterString

+ report 0..*

+ sourceStep0..*

eos:DQ_Lineage

MD_Metadata

DQ_DataQuality

+ scope : DQ_Scope+ lineage 0..1

LI_Lineage

+ statement [0..1] : CharacterString

LE_Source

+ description [0..1] : CharacterString+ scaleDenominator [0..1] : MD_RepresentativeFraction+ sourceReferenceSystem [0..1] : MD_ReferenceSystem+ sourceCitation [0..1] : CI_Citation+ sourceExtent [0..*] : EX_Extent+ processedLevel[0..1] : MD_Identifier+ resolution[0..1] : LE_NominalResolution

LE_ProcessStep

+ description : CharacterString+ rationale [0..1] : CharacterString+ dateTime [0..1] : DateTime+ processor [0..*] : CI_ResponsibleParty

+ source 0..*+ processStep 0..*

+ output, source0..*

If(count(source) + count(processStep) =0) and(DQ_DataQuality.cope.level = 'dataset' or 'series')then statement is mandatory

LE_Processing

+ identifier : MD_Identifier+ softwareReference[0..*] : CI_Citation+ procedureDescription[0..1] : CharacterString+ documentation[0..*] : CI_Citation+ runTimeParameters[0..1] : CharacterString+ otherPropertyType [0..1]: RecordType+ otherProperty [0..1]: Record

LE_Algorithm

+ citation: CI_Citation+ description : CharacterString+ additionalCitation: CI_Citation+ version: CharacterString

+ processingInformation0..*

+ algorithm 0..*LE_ProcessStepReport

+ name : CharacterString+ description[0..1] : CharacterString+ fileType[0..1] : CharacterString

+ report 0..*

+ sourceStep0..*


ISO Standards

Users

GCMD

ECHO

DAACs/ECS

Missions

Science Teams

Granules/HDF

Identifiers

Semantic Mashups

User Input

Lineage / PROVData Quality

Preservation Objects

GIS

ServicesNASA Documentation

Requirements

Questions?

[email protected]

ESDIS and International Standards

Technology

Transcript of ESDIS and International Standards