Content-Infused OGC Web Services Enabling Dynamic Quality Assessment in Observing Systems
-
Upload
cybera-inc -
Category
Technology
-
view
418 -
download
0
description
Transcript of Content-Infused OGC Web Services Enabling Dynamic Quality Assessment in Observing Systems
Content-Infused OGC Web Services
Enabling Dynamic Quality Assessment in Observing Systems
Janet J. Fredericks Applied Ocean Physics & Engineering Woods Hole Oceanographic Ins=tu=on
Carlos Rueda
Monterey Bay Aquarium Research Ins=tute
Workshop on Sensor Web Enablement 2011 (SWE 2011) As part of The 2011 Cybera Summit on Data For All -‐ Opening up the Cloud October 6-‐7, 2011, Banff, AB, Canada
1
Research and survey data served with
associated metadata in a few speci5ic formats with
associated software installations
Sensor Manufacturers <html/> and manuals
NOAA/NDBC provides 24/7 QC;
Feeds National IOOS backbone;
NOAA/NODC provides national archival for valued
data sets (they can determine the value)
NSF/OOI; NSF/R2R; NSF/BCODMO provides community-‐based integration with tools and QC, along with discovery
and mapping opportunities
Real-‐time Rapid Response integration can be accomplished quickly and reliably by communicating metadata
in standards-‐based systems
Modeling using translation tools from the cloud, modelers have access to a broader
source of information
ANYONE By fully describing data, sensors and
processing with associated provenance, data can be discovered and explored for
any program
Data Provider Nightmare!
User-‐based Output 2
Research and survey data served with
associated metadata in a few speci5ic formats with
associated software installations
Sensor Manufacturers <html/> and manuals
NOAA/NDBC provides 24/7 QC;
Feeds National IOOS backbone;
Data Provider (and Consumer)
Nightmare!
User-‐based Output
Research and survey data served with
associated metadata in a few speci5ic formats with
associated software installations
Research and survey data served with
associated metadata in a few speci5ic formats with
associated software installations
IOOS
GEOSS
3
Research and survey data served with
associated metadata in a few speci5ic formats with
associated software installations
Sensor Manufacturers <html/> and manuals
NOAA/NDBC provides 24/7 QC;
Feeds National IOOS backbone;
NOAA/NODC provides national archival for valued
data sets (they can determine the value)
NSF/OOI; NSF/R2R; NSF/BCODMO provides community-‐based integration with tools and QC, along with discovery
and mapping opportunities
Real-‐time Rapid Response integration can be accomplished quickly and reliably by communicating metadata
in standards-‐based systems
Modeling using translation tools from the cloud, modelers have access to a broader
source of information
ANYONE By fully describing data, sensors and
processing with associated provenance, data can be discovered and explored for
any program
Data Provider (and Consumer)
Nightmare!
User-‐based Output 4
Standards-‐based (machine-‐to-‐machine harves=ng)
Research and survey data served with
associated metadata in a community-‐
adopted, standards-‐based framework
Sensor Manufacturers and domain experts develop sensor and
processing descriptions in standards-‐based
encodings
NOAA/NDBC provides 24/7 QC;
Feeds National IOOS backbone;
NOAA/NODC provides national archival for valued
data sets (they can determine the value)
NSF/OOI; NSF/R2R; NSF/BCODMO provides community-‐based integration with tools and QC, along with discovery
and mapping opportunities
Real-‐time Rapid Response integration can be accomplished quickly and reliably by communicating metadata
in standards-‐based systems
Modeling using translation tools from the cloud, modelers have access to a broader
source of information
ANYONE By fully describing data, sensors and
processing with associated provenance, data can be discovered and explored for
any program
Converters; QC algorithms; vocabularies & ontologies; analysis and
visualization tools
GOAL: two paths Described well enough for assessment of
data for specified use and for a repurposed applica<on
User-‐based Frameworks 5
Data Provider needs to communicate how the sensible proper=es were turned into observa=ons!
Sensor Logging/Processing
Web Service
6
Quality Assurance (QARTOD)
QC Tests
MetaData Requirements
Seman=c Tools (MMI)
Vocabulary Registry & Term Mapping
Ontology Development & Registry
Standards (OGC)
Syntactac=c Interoperabilty (SensorML/O&M)
Standards-‐based web services (SOS)
Guides/Implementa=on (OOStethys/OGC-‐OIE/OpenIOOS)
So_ware Packages/ cookbooks
Observa=ons Based SOS
Quality – to – OGC (Q2O) -‐ IntegraKon of sensor & processing descripKons aimed towards the ability to assess quality of observaKons
Project to Address Data Quality in Sensor Web Enablement Frameworks
BACKGROUND
7
Domain Experts
Sensor Mfgrs
Operators
IT Specialists
Content Specifica=ons/
SWE Implementa=on
Model
What informaKon is needed to assess quality of data and how do we implement it into an Sensor ObservaKon Services (SOS)?
Community-‐based Development
8
OWL/RDF Vocabularies/ontologies
SensorML
Data
HARVESTS
REFERENCES RESOLVES
DescribeSensor (SensorML)
GetObserva@on (O&M)
CL I ENT
SOS
9
OWL/RDF Vocabularies/ontologies
SensorML
Data
SOS
HARVESTS
REFERENCES RESOLVES
DescribeSystem (SensorML)
GetObserva@on (O&M)
CL I ENT
<sml:output name="swell"> <swe:Quan=ty defini=on="hkp://mmisw.org/ont/mvco/proper=es/swell"> <swe:uom code="cm"/> </swe:Quan=ty> </sml:output>
10
OWL/RDF Vocabularies/ontologies
SensorML
Data
SOS
HARVESTS
REFERENCES RESOLVES
DescribeSystem (SensorML)
GetObserva@on (O&M)
CL I ENT
11
Building ontologies
12
Five Role-‐based Categories of SensorML
Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on)
Process Files (SensorML -‐> DescribeSensor)
QC Tests -‐ are classified as QC tests (QcCategory) and may have associated QC flags as output
Processing Descrip=ons -‐ to describe how an observa=on is derived from sensor output
Sensor/Deployment Files (SensorML -‐> DescribeSensor)
Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Model
Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and
Event History Details
Observable Proper=es
SML system
13
Five Role-‐based Categories of SensorML
Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on)
Process Files (SensorML -‐> DescribeSensor)
QC Tests -‐ are classified as QC tests (QcCategory) and may have associated QC flags as output
Processing Descrip=ons -‐ to describe how an observa=on is derived from sensor output
Sensor/Deployment Files (SensorML -‐> DescribeSensor)
Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Model
Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and
Event History Details
Observable Proper=es
SML system OEM Model DescripKon Created by
manufacturer and available for anyone using the par<cular model – accuracy; error analysis etc
specific to the model
14
Five Role-‐based Categories of SensorML
Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on)
Process Files (SensorML -‐> DescribeSensor)
QC Tests -‐ are classified as QC tests (QcCategory) and may have associated QC flags as output
Processing Descrip=ons -‐ to describe how an observa=on is derived from sensor output
Sensor/Deployment Files (SensorML -‐> DescribeSensor)
Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Model
Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and
Event History Details
Observable Proper=es
SML system
ConfiguraKon and Deployment File Working with OEM
file/Sensor Manufacturers and Marine Operator
describe this instance: contacts (operator), parameters (set-‐up specifica=on that can affect accuracy or
relevance to repurposed
applica=on), posi=ons, events rela=ng to sensor health
15
Five Role-‐based Categories of SensorML
Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on)
Process Files (SensorML -‐> DescribeSensor)
QC Tests -‐ are classified as QC tests (QcCategory) and may have associated QC flags as output
Processing Descrip=ons -‐ to describe how an observa=on is derived from sensor output
Sensor/Deployment Files (SensorML -‐> DescribeSensor)
Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Model
Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and
Event History Details
Observable Proper=es
SML system
QC Tests Data manager
describes QC tests and associated flags; inputs
to tests and parameters are specified – the
parameters can be =me-‐stamped
16
Five Role-‐based Categories of SensorML
Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on)
Process Files (SensorML -‐> DescribeSensor)
QC Tests -‐ are classified as QC tests (QcCategory) and may have associated QC flags as output
Processing Descrip=ons -‐ to describe how an observa=on is derived from sensor output
Sensor/Deployment Files (SensorML -‐> DescribeSensor)
Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Model
Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and
Event History Details
Observable Proper=es
SML system
Processing DescripKons
Data managers and domain experts provide authorita<ve reference and descrip<ons of processing used for derived proper=es
17
Five Role-‐based Categories of SensorML
Observed and Derived Proper=es and QC Flags (O&M -‐> GetObserva=on)
Process Files (SensorML -‐> DescribeSensor)
QC Tests -‐ are classified as QC tests (QcCategory) and may have associated QC flags as output
Processing Descrip=ons -‐ to describe how an observa=on is derived from sensor output
Sensor/Deployment Files (SensorML -‐> DescribeSensor)
Original Equipment Manufacturer (OEM) File Descrip=on of Sensor Model
Configura=on/Ownership/Deployment (CONDEP)File Descrip=on of Sensor Configura=on, Deployment and
Event History Details
Observable Proper=es
SML system
18
How does this model enable dynamic quality assessment? 1) ROLES -‐ Provides a template for instrument manufacturers/data managers/
marine operators to describe details that describe quality related informa=on in a standards-‐based encoding
2) CONNECTIONS -‐ Through the connec=ons list in SensorML, the QC flags can be associated with the QC tests with associated parameters
3) ENABLING SEMANTIC MAPPINGS -‐ Through inclusion of associated URLs encoded with each term, ontologies and mappings can be built to define rela=onships across poli=cal and research domains promo=ng interoperability and interdisciplinary research for all geospa=al, sensor-‐based observa=ons.
4) Encoding thorough descrip=ons of processing and process lineage promotes beker understanding of the observa=ons, which enhances the value and reliability of the data.
The original provider has no knowledge of how the data may be used! We need to communicate enough informa=on to enable assessment for a par=cular use beyond the project design! Did they sample fast enough for the new applica=on? Or long enough? Is the repor=ng frequency adequate?
19
Next Steps • Build beker SensorML editors and registries -‐-‐ making things easier and promo=ng fully-‐described sensor and processing lineage. This will promote adop<on of the use of standards and more fully-‐described systems!
• Encourage manufactures, data managers and domain experts to create meaningful vocabularies including authorita=ve references to processing algorithms, with figures, equa=ons, etc. and to register the vocabularies, providing resolvable links in a standards-‐based encoding (OWL)
• Provide tools and opportuni=es for domain experts to create and register ontologies, associa=ng terms in RDF (Is ThisQCtest the same as ThatQCtest? Does this QC flag have th same meaning as thatQCflag)
20
Conclusions • Structured Q2O (hkp://q2o.whoi.edu) SensorML serves as a
model for any sensor-‐based, in situ observa=ons; each component can be implemented by the responsible party and adop=on of the model can happen in stages.
• By associa=ng QC flags with qc tests, processing methods with observa=ons, and fully-‐describing how observable proper=es become observa=ons knowledge about quality will be shared.
• By referencing (encoding) resolvable terms, ontologies can be built and registered to foster interoperability across-‐domains and poli=cal boundaries.
The ability to dynamically assess data quality will provide a
trusted founda<on for observing systems
21