From Earth Science Observations to GI Coverages:
Towards an harmonization framework for coverages
Stefano NativiItalian National Research Council
(Institute of Methodologies for Environmental Analysis) and
University of Florence
Scientific Data Type and Feature modeling-----
Rutherford Appleton Lab 7-9 Mar 07
Outline• Rationale• From events to datasets
– An harmonization framework for the coverage domain
• netCDF Coverage implementation– Abstract, content and encoding models– ncML-Gml
• O&M Coverage implementation• Scientific Feature Coverage implementation
• Future work• Conclusions
Rationale Provide Information Society with an effective, NRT and easy-to-use
fruition of multidimensional Earth Sciences datasets (e.g. 4/5-D)
Geospatial datasetsAcquisition and
Encoding
Knowledge Extraction
and Harmonization
StandardModels andInterfaces
Explicit Semantic level /Interoperability level
SOCIETY
INFRASTRUCTURES,
PLATFORMS
and SYSTEMS
Over-simplified Worldviews• To the Geographic Information community, the world
is:– A collection of featuresfeatures (e.g., roads, lakes, plots of land)
with geographic footprints on the Earth (surface).– The featuresfeatures are discrete objectsdiscrete objects described by a set of
characteristics such as a shape/geometryshape/geometry
• To the Earth Science community, the world is:– A set of event observationsobservations described by parametersparameters
(e.g., pressure, temperature, wind speed) which vary as continuous functionscontinuous functions in 3-dimensional space and time.
– The behavior of the parametersparameters in space and time is governed by a set of equations.equations.
[from Ben Domenico]
ES and GI Data Models• Diverse semantic and portability levels• Diverse attention to domain multi-dimensionality• Diverse aggregation structures and Data types
[from J. Caron]
Observ.s Vs. Features: Value-added Chaining
• (Event) Observation – estimate of value of a property for a single
specimen/station/location – data-capture, with metadata concerning procedure, operator, etc
• Feature – object having geometry & values of several different
properties 1. classified object
– snapshot for transport geological map elements 2. object created by human activity
– artefact of investigation borehole, mine, specimen
[from S.Cox Information Standards for EON]
• Coverage – compilation of values of a single property across the domain of
interest – data prepared for analysis/pattern detection
An Harmonization Framework• A unified framework with: O&M model, GF Model and IGCD Model • Harmonization
– O&M classes implementations for the Coverage domain – Scientific feature implementations for the Coverage domain
• GI:coverage is a GI:feature type
<<O&M Model>>OGC model
<<General FeatureModel>>
ISO 19107,…
<<IGCD Model>>
ISO 19123
O&M Coverages
<<IGCD Model>>
CF- NetCDF Grid Model
CF-NetCDF Coverage
<<O&M Model>>
CF- NetCDF Observation Model … .
This allows to develop crosswalks towards well-accepted ES data models
- Example: CF-netCDF community
ES
GI
<<IGCD Model>>
ISO 19123
<<IGCD Model>>
CF- NetCDF Grid Model
multidimensional Observation
dataset(e.g. 4/5D hypercube)
N-Dimension Coordinate Systems
ES Dataset content
<dimension>,<coordinateSystem><coordinateAxis>
<netcdf type>
explicit/semi-implicit/implicit Geometry
<dimension>,<variable>
0110110011111010101010010101…01101100111
11010101010010101…
Scalar measured quantities
<variable>
IGCD
CF-netCDF
2D+elev+timedataset
2D Spatial Coordinate System + elev + time
Range set
GI coverage content
<_CoordinateSystem>,<coordinateSystem Axis>
<rangeSet> <_Coverage>
explicit/implicit Geometry
Spatial Reference System (SRS)
<gridDomain>,<rectifiedGrid Domain>,<multipointDomain>
<GeographicCRS>
IGCD
CF-netCDF
2 Dimension Coordinate System
Implicit/explicit Geometry
Range set
Spatial Reference System (SRS)
2 Dimension Coordinate System
Implicit/explicit Geometry
Range set
Spatial Reference System (SRS)
2 Dimension Coordinate System
Implicit/explicit Geometry
Range set
Spatial Reference System (SRS)
2 Dimension Coordinate System
Implicit/explicit Geometry
Range set
Spatial Reference System (SRS)
The Mediation Process
ES hyperspacedataset(3/4/5D)
2D + elev + time Coverages
2D+elev+time dataset
2D SCS + elev + time
Implicit/explicit Geometry
Range set
Spatial Reference System (SRS)
a Coverage
0110110011111010101010010101…01101100111
11010101010010101…
N-Dimension Coordinate Systems
explicit/semi-implicit/implicit Geometry
Scalar measured quantities s S
SSS
S
IGCD
CF-netCDF
The Implementation
• ES data model– netCDF– Extra metadata: CF conventions
• GI Coverage model– ISO 19123: DiscreteGridPointCoverage
• Harmonization implementation-style– Declarative style
• Mediation Markup Language• Rule-based procedure
IGCD
CF-netCDF
CF-netCDF Model
• NetCDF data model was extended adding a set of conventions– One of the most
popular convention is the Climate and Forecasting metadata convention (CF)
– Introduce more specific semantic elements (i.e. metadata) required by different communities to fully describe their datasets
Coordinate
{ shape->size = 1}
units as defined inUdunits package
TimeCoordinate
-calendar:String-month_lengths :String-leap_year: int-leap_month :int
Identified either via:- units (length, pressure, etc.)- positive attribute
Vertical Coordinate
-units:String-positive:up|down-formula_terms:String
LatLonCoordinate
CF-CoordinateVariable
-axis:X|Y|Z|T
DimensionalQuantity
-units:String
- same name as related dimension - numeric values- monotonic
Dimension
CoordinateVariable
local attributes override global ones
CF-Variable
-institution :String-source:String-references:String-comment:String-long_name :String-standard_name:String
Variable
-shape:Dimension[]
Dataset
-Conventions:String-history:String-title:String
CF-Dataset
-Conventions:String=CF-1.0-institution :String-source:String-reference:String-comment:String
CF-netCDFProfileModel
*
*
*
*
*
coordinates
netCDF Model
DiscreteGridPointCoverage
<< DataType >>CV_GridCoordinates
+coordValues:Sequence<Integer>
<< DataType >>CV_GridEnvelope
+low:CV_GridCoordinates+high:CV_GridCoordinates
<< Type >>CV_RectifiedGrid
+dimension:int+axisNames:Sequence<CharacterString>+extent:CV_GridEnvelope+origin:DirectPosition+offestVectors:Sequence<Vector>
<< Type >>CV_GridValueMatrix
+values:Sequencee<Record>+sequencingRule:CV_SequenceRule+startSequence:CV_GridCoordinates
<< Abstract >>SC_CRS
<< metaclass >>GF_FeatureType
<< Type >>CV_DiscreteGridPointCoverage
+domainExtent:EX_Extent[1..*]+rangeType:RecordType+commonPointRule :CommonPointRule
DiscreteGridPointCoverageProfile model
<< instantiates >>
*
CRS+
Coordinate Reference System
evaluator+ 0..1
valueAssignment+
PointFunction
valuation
Record
+attributes:Dctionary<AttributeName, Any>
<< metaclass >>RecordType
+typeName:TypeName+attributeTypes :Dictionary<AttributeName, TypeName>
TypeName
+aName:String
AttributeName
+aName:String+attributeType:TypeName
record+*
recordType+
RecordType
Coordinate
{ shape->size = 1}
units as defined inUdunits package
TimeCoordinate
-calendar:String-month_lengths :String-leap_year: int-leap_month :int
Identified either via:- units (length, pressure, etc.)- positive attribute
Vertical Coordinate
-units:String-positive:up|down-formula_terms:String
LatLonCoordinate
CF-CoordinateVariable
-axis:X|Y|Z|T
DimensionalQuantity
-units:String
- same name as related dimension - numeric values- monotonic
Dimension
CoordinateVariable
local attributes override global ones
CF-Variable
-institution :String-source:String-references:String-comment:String-long_name :String-standard_name:String
Variable
-shape:Dimension[]
Dataset
-Conventions:String-history:String-title:String
CF-Dataset
-Conventions:String=CF-1.0-institution :String-source:String-reference:String-comment:String
CF complete Class Diagram
*
*
*
*
*
coordinates
<< DataType >>CV_GridCoordinates
+coordValues:Sequence<Integer>
<< DataType >>CV_GridEnvelope
+low:CV_GridCoordinates+high:CV_GridCoordinates
<< Type >>CV_RectifiedGrid
+dimension:int+axisNames:Sequence<CharacterString>+extent:CV_GridEnvelope+origin:DirectPosition+offestVectors:Sequence<Vector>
<< Type >>CV_GridValueMatrix
+values:Sequencee<Record>+sequencingRule:CV_SequenceRule+startSequence:CV_GridCoordinates
<< Abstract >>SC_CRS
<< metaclass >>GF_FeatureType
<< Type >>CV_DiscreteGridPointCoverage
+domainExtent:EX_Extent[1..*]+rangeType:RecordType+commonPointRule :CommonPointRule
DiscreteGridPointCoverageProfile model
<< instantiates >>
*
CRS+
Coordinate Reference System
evaluator+ 0..1
valueAssignment+
PointFunction
valuation
Record
+attributes:Dctionary<AttributeName, Any>
<< metaclass >>RecordType
+typeName:TypeName+attributeTypes :Dictionary<AttributeName, TypeName>
TypeName
+aName:String
AttributeName
+aName:String+attributeType:TypeName
record+*
recordType+
RecordType
0…1
0…1
1…n
Mapping Rules
Concept type Definition NotesAn observation is a function from a given multidimensional real domain (d) to a multidimensional real co-domain (c).Note: a netCDF variable is a special case of Observation (with domain in d and c=1).
d = {b1, b2, …, bn}A dataset is a set of observation data.Note: a netCDF file is a special case of Dataset.
S: {3, SCS} A Spatial Domain is 3 with a law from 3 to a location in the physical universe (Spatial Coordinate System). A 2D Spatial (Planar) Domain is the restriction of S to 2.
c: {S, T} n
n
C = {c}
A coverage is a function defined from a Spatio-Temporal Domain (e.g. Lat, Lon, Height, Time) to a multidimensional real co-domain (n).Note: if a set of CF-netCDF coordinate variables is a Spatio-Temporal Domain, then CF-netCDF variables defined over the corresponding dimensions can be mapped to Coverages
Domain and Functional Definitions
Observation Data/Observation
b: d c
d, c
B= {b}Dataset
Spatial Domain
Coverage
Concept type Definition Notes
g(b) =c
g: B C
Given an observation data, the Observation to Coverage operator generates a coverage.
s = {g1, g2, …, gn}
A Dataset to Coverages operator consists of a set of Observation to Coverage operators.
Hence, Given an dataset element, the Dataset to Coverages operator generates a set of coverage elements.
(Another task is the metadata elements mapping from dataset to the whole set of coverages).
p(c) = m p: C M
A Coverage Portrayal operator transforms a coverage to a map, by means of a combination of the following operations:
– Domain restriction (to a certain Z0 and T0);
– Co-domain restriction (to a scalar quantity).
Domain and Functional Definitions
Observation to Coverage Operator
Dataset to Coverage Operator
Coverage Portrayal Operator
Data model harmonization: Implementation style
Abstract model level
HyperspatialDataset Coverage/Feature
netCDF + CFContent model level
ISO 19123 Coverage Model
GIS InformationCommunity
Earth SciencesInformationCommunity
Encoding level ncML GMLGGMLMLGGMLML
Mapping rules
Mapping rules
IGCD
CF-netCDF
Data model harmonization
ISO 19123Data Model
GIInformationCommunity
Earth SciencesInformationCommunity
Information Society(e.g. Spatial Data Infrastructure)
ncML-GMLncML
EncodingModel
GML 3.xEncoding
Model
WCS 1.xContent Model
WFSContent Model
Data Models Mediation
netCDFData Model
CFMetadata
IGCD
CF-netCDF
ncML-GML• Mediation Markup Language• An extension of ncML (netCDF Markup Language) based on GML
(Geography Markup Language) grammar
IGCD
CF-netCDF
Content Feature of interest
Content Models
Encoding Models
Available Language specification and Tools
• The ncML-GML markup language implements the presented reconciliation model
• It is a Mediation Markup Language between ncML (netCDF Markup Language) and GML– An extension of ncML core schema, based on GML grammar
• NcML-GML version 0.7.3– based on GML 3.1.1
• N2G version 0.8– Java API for ncML-GML ver. 0.7.3
• WCS-G– WCS 1.0 server which supports ncML-GML/netCDF documents
• Subsetting (domain and range-set)– netCDF– ncML-GML 0.7.3
• Client for WCS-G– GI-go thick client
GGMLMLGGMLML
IGCD
CF-netCDF
An Harmonization Framework• A unified framework with: O&M model, GF Model and IGCD Model • Harmonization
– O&M Coverage implementation for the Coverage domain– Scientific feature implementations as Coverage
• GI:coverage is a GI:feature type
<<General FeatureModel>>
ISO 19107,…
O&M Coverages
<<IGCD Model>>
CF- NetCDF Grid Model
CF-NetCDF Coverage
<<O&M Model>>
CF- NetCDF Observation Model … .
This allows to implement and harmonize other well-accepted data models
ES
GI<<O&M Model>>
OGC model
<<IGCD Model>>
ISO 19123
From Event to Datasets (O&M)• Instruments and sensors observe and measure properties of Feature-of-
interests• Observations and measurements generate datasets • Datasets can be modeled/stored as either boundary or coverage data
– It depends on the observed property variability over the Feature_of_interest domain
Observation
PropertyFeatureOfInterest+domain
monitors
0..*
detects
characterizes 0..*
Dataset
+result0..*
Georelational Dataset CompositeDataset
Event
O&M Model
IGCD Model
CoverageObservation implementation
Event
«FeatureT ype»observ ation::Observ ation
«FeatureT ype»DiscreteCov erageObserv ation
+ result: CV_DiscreteCoverage
«FeatureT ype»PointCov erageObserv ation
+ result: CV_DiscretePointCoverage
«FeatureT ype»TimeSeriesObserv ation
+ result: CV_DiscreteT imeInstantCoverage
«FeatureT ype»Cov erageObserv ation
+ result: CV_Coverage+ resultDefinition: RecordT ype
«FeatureT ype»ElementCov erageObserv ation
+ result: CV_DiscreteElementCoverage
proximateFeatureOfInterest
1
0..*
«FeatureType»Observ ation
+ quality: DQ_Element [0..1]+ responsible: CI_ResponsibleParty [0..1]+ result: Any
«FeatureType»Event
+ eventParameter: TypedValue [0..*]+ time: TM_Object
«DataType»TypedValue
+ property: ScopedName+ value: Any
«Union»Procedure
+ procedureType: ProcedureSystem+ procedureUse: ProcedureEvent
AnyIdenti fiableObject
«FeatureType»AnyIdentifiableFeature
AnyDefinition
«ObjectType»Phenomenon
+followingEvent 0..*+precedingEvent 0..*
+generatedObservation
0..*
+procedure 1
+observedProperty1{Definition must be of aphenomenon that is a propertyof the featureOfInterest}
+propertyValueProvider
0..*
+featureOfInterest
1
<<CoverageRangeSet>>Un-
UniformFeatureProperty
un-uniformObservedProperty
<<CoverageDomain>>
FeatureDomainExtentSet<GM_Object>
CoverageObservation. featureOfInterest is the Feature Domain Extent
CoverageDomain is a proximate FeatureOfInterest
CoverageObservation. observedProperty is un-uniform over the Feature Domain Extent
Observed Phenomenon is the Range-set of generated coverages.
O&M Model
IGCD Model
SamplingFeature implementation
Event
«FeatureType»Observ ation
ObservationProcedure
«ObjectType»Surv eyProcedure
«FeatureType»SamplingFeature
+ property: TypedValue [0..*]+ responsible: CI_ResponsibleParty [0..1]
«DataType»SamplingFeatureRelation
+ role: GenericName
AnyIdentifiableObject
«FeatureType»AnyIdentifiableFeature
«DataType»TypedValue
+ property: ScopedName+ value: Any
+propertyValueProvider 0..*
+featureOfInterest 1
+relatedObservation 0..*
+relatedSamplingFeature 0..* +source
0..* +target
+surveyDetails 0..1
+sampledFeature
0..1
CV_featureOfInterest
1
Event
«FeatureType»observ ation::Observ ation
«FeatureType»DiscreteCov erageObserv ation
+ resul t: CV_DiscreteCoverage
«FeatureT ype»PointCov erageObserv ation
+ result: CV_DiscretePointCoverage
«FeatureT ype»TimeSeriesObserv ation
+ result: CV_DiscreteT imeInstantCoverage
«FeatureType»Cov erageObserv ation
+ resul t: CV_Coverage+ resul tDefini tion: RecordT ype
«FeatureT ype»ElementCov erageObserv ation
+ resul t: CV_DiscreteElementCoverage
CoverageSamplingFeature
<<CoverageDomain>>
FeatureDomainExtent
ObservationProcedure
tileShape: GM_Object [0..*]tileDistributionType: TypedDistr [0..*]tileResolution: TypedRes [0..*]
CoverageSurveyProcedure
0..1CV_surveyDeta
ils
observableShape [0..1]observableAxes: CS_Axis [1..*]observableFrequency: TM_int [1..*]Set<GM_Object> [0..1
0..*
CV_relatedObservation
In a coverage observation framework, to define a CoverageSamplingFeature as a SamplingFeature subtype.
CoverageSamplingFeature is a proximate Feature of interest
FeatureDomainExtent class is the simplest way to implement the CoverageSamplingFeature concept
O&M Model
IGCD Model
• The CoverageSamplingFeature is characterized by the following fields: – the possible shapes of the observation coverage domain– the possible observation domain axes– the possible observation frequencies
• The CoverageSurveyProcedure characterizes the sampling observation procedure
– described by the following fields: shape(s), domain distribution(s) and resolution(s) of coverage tiles worked out by observations.
CV_featureOfInterest
1
Event
«FeatureT ype»observ ation::Observ ation
«FeatureT ype»DiscreteCov erageObserv ation
+ resul t: CV_DiscreteCoverage
«FeatureT ype»PointCov erageObserv ation
+ resul t: CV_DiscretePointCoverage
«FeatureT ype»TimeSeriesObserv ation
+ resul t: CV_DiscreteT imeInstantCoverage
«FeatureT ype»Cov erageObserv ation
+ resul t: CV_Coverage+ resul tDefini tion: RecordType
«FeatureT ype»ElementCov erageObserv ation
+ resul t: CV_DiscreteElementCoverage
CoverageSamplingFeature
<<CoverageDomain>>FeatureDomainExtent
ObservationProcedure
tileShape: GM_Object [0..*]tileDistributionType: TypedDistr [0..*]tileResolution: TypedRes [0..*]
CoverageSurveyProcedure
0..1CV_surveyDetails
observableShape [0..1]observableAxes: CS_Axis [1..*]observableFrequency: TM_int [1..*]Set<GM_Object> [0..1
0..*
CV_relatedObservation
SamplingFeature implementationO&M Model
IGCD Model
ExtensiveSamplingFeatures implementations
• Most of extensive sampling features may be implemented as CoverageSamplingFeature sub-types.
– Specific values of CoverageSurveyProcedure and CoverageSamplingFeature fields characterize the site types for observation sampling.
Fields / ExtensiveSamplingFeature
FeatureDomainExtent.Observable (domain) Axes
CoverageSurveyProcedure.Tile Shape
CoverageSurveyProcedure.Tile Distribution
Scene Space (2-3 D) quadrilateral Regular/Irregular
Swath Space (2D) ellipse Regular
Profile Space (1D) andVertical (i.e. pressure or density)
quadrilateral Irregular
Scanning Radar Space (2D) circle sector Regular
Time Series (Profile Series)
Space (1D) or Vertical (i.e. pressure or density) and Time (Actual or Forecast time)
quadrilateral Regular/Irregular
O&M Model
IGCD Model
An Harmonization Framework• A unified framework with: O&M model, GF Model and IGCD Model • Harmonization
– O&M Coverage implementation – Scientific feature implementations for the Coverage domain
• GI:coverage is a GI:feature type
O&M Coverages
<<IGCD Model>>
CF- NetCDF Grid Model
CF-NetCDF Coverage
<<O&M Model>>
CF- NetCDF Observation Model … .
This allows to implement and harmonize other well-accepted data models
ES
GI<<O&M Model>>
OGC model
<<General FeatureModel>>
ISO 19107,…
<<IGCD Model>>
ISO 19123
Coverage is a Feature type• Discrete coverage modeled as a logical complex feature
– implicit geometry (derived from the coverage function concept)– a collections of tiles which cover/sample the coverage spatial&temporal domain
• Tiles may be modeled as simple features, characterized by explicit geometry.Location Properties
Property 1 Property 2 … Property m
(x1, y1) Value11 Value1
2 … Value1m
(x2, y2) Value21 Value2
2 … Value2m
(x3, y3) Value31 Value3
2 … Value3m
(xn, yn) Valuen1 Valuen
2 … Valuenm
GF Model IGCD Model
Scientific Feature types are ExtensiveSamplingFeatures
(Coverage sub-types implem.)• Scientific Feature types (Extensive Sampling Features)
– Certain feature types are only associated with sampling, and have no significant function outside of their role in the observation process …. Feature types having this behaviour are similar across all application domains [O&M]
Fields / Scientific Feature types
FeatureDomainExtent.Observable (domain) Axes
CoverageSurveyProcedure.Tile Shape
CoverageSurveyProcedure.Tile Distribution
Grid Space (2-3 D) quadrilateral Regular/Irregular
Swath Space (2D) ellipse Regular
Ragged section Space (1D) andVertical (i.e. pressure or density)
quadrilateral Irregular
Scanning Radar Space (2D) circle sector Regular
Time Series (Profile Series)
Space (1D) or Vertical (i.e. pressure or density) and Time (Actual or Forecast time)
quadrilateral Regular/Irregular
GF Model IGCD Model
Scientific Feature types are ExtensiveSamplingFeatures (Coverage sub-types implem)
• ISO 19123 recognizes some coverage sub-types which can be easily mapped to Scientific feature types
Coverage sub-types
GF Model IGCD Model
Content Vs Container Features• ScientificFeatures ContainerFeatures
(ExtentSamplingFeatures) (Coverage sub-type implementations)• In order to facilitate interoperability, coverage can be modeled as a general container feature (i.e. proximate Feature-of-interest)
Observation values
Observation Sampling/functions
Encoding ModelAbstract Model
Application Elements
Application Elements Behavior
Applicationspecific
Applicationagnostic Arrays
Proximate FOI(Coverage Subtypes)
Application FOI
Inference Rules
Content Model
Binary File
XML elements
Semantics level
Dat
aset
sMet
adat
a XML elements
XML elements
Container
Content Impl
icite
ness
GF Model IGCD Model
Future Work
• To extent the harmonization model between netCDF and GI Coverage realms– O&M concepts– Scientific Features (Sampling Features)
• To encode it in XML– Encode and access netCDF datasets
• ncML-Gml • CSML, O&M ML, other GML dialects?
Main Conclusions• ES and GI data model interoperability is more
and more important for Society’s applications• The GI coverage concept seems to be a good
solution to bridge GI and ES data models• We need a unified framework to harmonize
concepts coming from O&M, GF and IGCD models in the coverage domain
• To facilitate the mediation between ES and GI communities– A valuable case in point: the FES realm (CF-netCDF)
Back-up slides
Top Related