INSPIRE WCS: From Mystery to Mastery · INSPIRE WCS :: © 2017 Peter Baumann Coverage Coordinates...

77
INSPIRE WCS :: © 2017 Peter Baumann Peter Baumann Jacobs University | rasdaman GmbH Jordi Escriu Institut Cartogràfic i Geològic de Catalunya, INSPIRE Thematic Cluster #3 Facilitator INSPIRE WCS: From Mystery to Mastery [gamingfeeds.com]

Transcript of INSPIRE WCS: From Mystery to Mastery · INSPIRE WCS :: © 2017 Peter Baumann Coverage Coordinates...

INSPIRE WCS :: © 2017 Peter Baumann

Peter Baumann

Jacobs University | rasdaman GmbH

Jordi Escriu

Institut Cartogràfic i Geològic de Catalunya,

INSPIRE Thematic Cluster #3 Facilitator

INSPIRE WCS:

From Mystery to Mastery

[gamingfeeds.com]

INSPIRE WCS :: © 2017 Peter Baumann

Datacube Research @ Jacobs U

Large-Scale Scientific Information Systems

research group

• focus: large-scale n-D raster services & beyond

• www.jacobs-university.de/lsis

Spin-off: rasdaman GmbH

Main visible results:

• pioneer Array DBMS, rasdaman

• Big Data standards

INSPIRE WCS :: © 2017 Peter Baumann

INSPIRE WCS :: © 2017 Peter Baumann

Learning Objectives

This seminar will explain:

- The coverage data model shared by OGC and ISO

- OGC’s coverage service model: WCS, WCPS

- OGC, ISO, and INSPIRE

By the end of the day, you should be able to

- Understand principles of coverage data, service, and clients

- Understand differences between service functionality of WMS, WFS, WCS

- Understand how coverage services provide analysis-ready data

- Understand how common Annex II and III data can be modelled as coverage

- Understand how coverages can be orchestrated in overall service ecosystem

- Express core coverage retrieval tasks as WCS requests

Prerequisites

- Basics of Web services & GIS

INSPIRE WCS :: © 2017 Peter Baumann

Overview

Introduction

The data: Coverages

The service: WCS

Conformance Testing

Selected Applications

Conclusion

6

INSPIRE WCS :: © 2017 Peter Baumann

Overview

Introduction

The data: Coverages

The service: WCS

Conformance Testing

Selected Applications

Conclusion

7

INSPIRE WCS :: © 2017 Peter Baumann

Why Standards?

[www.jhchoppers.com]

INSPIRE WCS :: © 2017 Peter Baumann

Facing the Coverage Tsunami

coverage

server

9

sensor feeds[OGC SWE]

INSPIRE WCS :: © 2017 Peter Baumann

[OGC SWE]sensor feeds

Taming the Coverage Tsunami

10

coverage

server

INSPIRE WCS :: © 2017 Peter Baumann

Serving Coverages

SWE SOS: upstream

data capturing

CIS & WCS: downstream

access & analytics

SOS WCS

coverage

server

INSPIRE WCS :: © 2017 Peter Baumann

EarthServer: Datacubes At Your Fingertips

Agile Analytics on x/y/t + x/y/z/t Earth & Planetary datacubes

- Rigorously standards: OGC WMS + WCS + WCPS

- EU rasdaman + US NASA WorldWind

- 700+ TB 1+ PB

Intercontinental initiative,

3+3 years: EU + US + AUS

www.earthserver.eu,

www.planetserver.eu

INSPIRE WCS :: © 2017 Peter Baumann

INSPIRE WCS :: © 2017 Peter Baumann

MEA: Daily Sentinel 2A Availability

INSPIRE WCS :: © 2017 Peter Baumann

MEA: Sample Datacubes Coverages

INSPIRE WCS :: © 2017 Peter Baumann

MEA: NDVI Timeseries Extraction

INSPIRE WCS :: © 2017 Peter Baumann

MEA: Daily Hydro Estimator

INSPIRE WCS :: © 2017 Peter Baumann

ECMWF: River Discharge

INSPIRE WCS :: © 2017 Peter Baumann

MEA: Land Surface Temperature, Cloudfree

INSPIRE WCS :: © 2017 Peter Baumann

Coverages in INSPIRE

Jordi Escriu,

INSPIRE Thematic Cluster #3 Facilitator

INSPIRE WCS :: © 2017 Peter Baumann

Overview

Introduction

The data: Coverages

The service: WCS

Conformance Testing

Selected Applications

Conclusion

22

INSPIRE WCS :: © 2017 Peter Baumann

Features & Coverages

The basis of all: geographic feature

Special kind of feature: coverage

- aka space-time varying phenomenon

- regular & irregular grids, point clouds, meshes

• Usually, coverages are Big Geo Data

INSPIRE WCS :: © 2017 Peter Baumann

MultiSolid

Coverage

Coverages- abstract: OGC Abstract Topic 6 = ISO 19123

- concrete, interoperable: Coverage Implementation Schema ISO 19123-2• (CIS, aka GMLCOV)

MultiPoint

Coverage

MultiCurve

Coverage

MultiSurface

CoverageReferenceable

GridCoverage

Rectified

GridCoverage

Grid

CoverageCIS 1.1

General Grid Coverage

«FeatureType»Abstract

Coverage

INSPIRE WCS :: © 2017 Peter Baumann

MultiSolid

Coverage

Coverages- abstract: OGC Abstract Topic 6 = ISO 19123

- concrete, interoperable: Coverage Implementation Schema ISO 19123-2• (CIS, aka GMLCOV)

AbstractCoverage

MultiPoint

Coverage

MultiCurve

Coverage

MultiSurface

CoverageGeneral Grid

Coverage

general meshes point clouds regular & irregular grids

INSPIRE WCS :: © 2017 Peter Baumann

Coverage Implementation Schema

Coverage Definition

«Feature Type»

Coverage

«Feature Type»

GML::Feature

«Data Type»

DomainSet

«Data Type»

RangeSet

«Data Type»

SWE Common::DataRecord

rangeSetdomainSet rangeType

[OGC 09-146r6]

0..1 metadata

from SWE Common

INSPIRE WCS :: © 2017 Peter Baumann

INSPIRE WCS :: © 2017 Peter Baumann

INSPIRE WCS :: © 2017 Peter Baumann

The Integrated Geo Warehouse

Comprehensive

geophysics data mgmt

Integrating

data + meta data

1D

2D

3D

nD

[GFZ Potsdam]

3D

+time

INSPIRE WCS :: © 2017 Peter Baumann

Coverage Coordinates and CRSs

Uniform handling of horizontal space, height, time, etc.

Native CRS = CRS of coverage- srsName attribute in DomainSet, auxiliary attributes accordingly

Ex: <gmlcov:GridCoverage … gml:id="C0001">

<gml:boundedBy>

<gml:Envelope srsName=" http://www.opengis.net/def/crs-compound?

1=http://www.opengis.net/def/crs/EPSG/0/4326&

2=http://www.opengis.net/def/crs/OGC/0/ansiDate“

axisLabels="Lat Long Time" uomLabels="deg deg day" srsDimension=“3">

<gml:lowerCorner>1 1 2012-03-10</gml:lowerCorner>

<gml:upperCorner>3 10 2012-03-10 </gml:upperCorner>

</gml:Envelope>

</gml:boundedBy>

</gmlcov:GridCoverage >

INSPIRE WCS :: © 2017 Peter Baumann

Grid Types Axis Types

CIS 1.0 (from GML): coverage classification by grid type

- GridCoverage, RectifiedGridCoverage, ReferenceableGridCoverage

- non-intuitive, hard to describe, missing cases

CIS 1.1 approach: axis types

- Index axis: not georeferenced, Index CRS

- Regular axis: georeferenced, constant spacing

- Irregular axis: georeferenced, variable spacing

- Displacement axis nest: georeferenced, arbitrary grid point locations

- Algorithmic coordinates: such as sensor model

- All combinations possible

Further integrating GML 3.3 and SensorML grids

INSPIRE WCS :: © 2017 Peter Baumann

Interpolation

Often requested:

coverage should inform about desirable/allowed interpolations

Approach: optional list of interpolation methods

Identifiers defined elsewhere, not hardwired into coverage model

<interpolationRestriction>

<allowedInterpolation>

http://www.opengis.net/def/interpolation/OGC/0/nearest-neighbor

</allowedInterpolation>

<allowedInterpolation>

http://www.opengis.net/def/interpolation/OGC/0/linear

</allowedInterpolation>

<interpolationRestriction>

INSPIRE WCS :: © 2017 Peter Baumann

Adding Metadata To Coverages

Slot „metadata“ for <any> kind of metadata

- WCS will deliver without knowing contents

Inverse possible, too

- metadata record

contains (or links)

coverage

INSPIRE WCS :: © 2017 Peter Baumann

A Simple Coverage, in GML

INSPIRE WCS :: © 2017 Peter Baumann

{ "type": "CoverageByDomainAndRangeType",

"domainSet":{

"type": "DomainSetType",

"generalGrid":{

"type": "GeneralGridCoverageType",

"srsName": "http://www.opengis.net/def/crs/OGC/0/Index2D",

"axisLabels": ["i", "j"],

"axis": [{ "type": "IndexAxisType“, "axisLabel": "i", "lowerBound": 0, "upperBound": 2

},{ "type": "IndexAxisType“, "axisLabel": "j", "lowerBound": 0, "upperBound": 2 }]

}

},

"rangeSet": { "type": "RangeSetType",

"dataBlock": { "type": "VDataBlockType", "values": [1,2,3,4,5,6,7,8,9] } },

"rangeType": { "type": "DataRecordType",

"field":[{ "type": "QuantityType",

"definition": "ogcType:unsignedInt",

"uom": { "type": "UnitReference", "code": "10^0“ } }]

}

}

A Simple Coverage, in JSON

INSPIRE WCS :: © 2017 Peter Baumann

<http://www.opengis.net/cis/1.1/examples/CIS_05_2D>

<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>

<http://www.opengis.net/cis/1.1/CoverageByDomainAndRangeType> .

<http://www.opengis.net/cis/1.1/examples/CIS_05_2D>

<http://www.opengis.net/cis/1.1/domainSet>

<http://www.opengis.net/cis/1.1/examples/CIS_DS_05_2D> .

<http://www.opengis.net/cis/1.1/examples/CIS_DS_05_2D>

<http://www.opengis.net/cis/1.1/generalGrid>

<http://www.opengis.net/cis/1.1/examples/CIS_DS_GG_05_2D> .

<http://www.opengis.net/cis/1.1/examples/CIS_DS_05_2D>

<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>

<http://www.opengis.net/cis/1.1/DomainSetType> .

<http://www.opengis.net/cis/1.1/examples/CIS_DS_GG_05_2D>

<http://www.opengis.net/cis/1.1/axis>

<http://www.opengis.net/cis/1.1/examples/CIS_DS_GG_I_05_2D> .

<http://www.opengis.net/cis/1.1/examples/CIS_DS_GG_05_2D>

<http://www.opengis.net/cis/1.1/axis>

<http://www.opengis.net/cis/1.1/examples/CIS_DS_GG_J_05_2D> .

<http://www.opengis.net/cis/1.1/examples/CIS_DS_GG_05_2D>

<http://www.opengis.net/cis/1.1/axisLabels>

<http://www.opengis.net/cis/1.1/axisLabels0> .

<http://www.opengis.net/cis/1.1/axisLabels0> <http://www.w3.org/1999/02/22-rdf-syntax-ns#first> "i" .

<http://www.opengis.net/cis/1.1/axisLabels0> <http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> <http://www.opengis.net/cis/1.1/axisLabels1> .

<http://www.opengis.net/cis/1.1/axisLabels1> <http://www.w3.org/1999/02/22-rdf-syntax-ns#first> "j" .

<http://www.opengis.net/cis/1.1/axisLabels1> <http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> <http://www.w3.org/1999/02/22-rdf-syntax

<http://www.opengis.net/cis/1.1/examples/CIS_DS_GG_05_2D> <http://www.opengis.net/cis/1.1/srsName> <http://www.opengis.net/def/c

A Simple Coverage, in RDF

INSPIRE WCS :: © 2017 Peter Baumann

Encoding Coverages

Single file encoding:

- Informationally complete: GML, JSON, RDF, …

- Further formats: GeoTIFF, NetCDF, JPEG2000, GRIB, …

Coverage

Domain set

Range type

Range set

App Metadata

Coverage

Domain set

Range type

xlink

App Metadata

NetCDF

Multipart: container( “header” + file1 + file2 + … )

- Multipart/MIME, zip, GMLJP2, SAFE, GeoPackage, ...

- Collections, tiling, streaming

Coverage

Domain set

Range type

xlink

App Metadata

NetCDF

INSPIRE WCS :: © 2017 Peter Baumann

Classical Approach vs Datacubes

WaterML 2.0: time slices

- Good performance in x/y

- Disastrous performance on

timeseries analysis

- same with TimeseriesML

OGC Coverages:

Implementation can

choose efficient layout

- Same performance

in all directions,

space & time

t

INSPIRE WCS :: © 2017 Peter Baumann

Managing Complexity

Separate conformance classes for core,

gridded and discrete data, partitioning, encoding

coverage

grid-regular

grid-irregular

grid-transformation

discrete-pointcloud

discrete-mesh

gml-coverage multipart-coverage

other-format-coverage

json-coverage

coverage-partitioning

json-ld-coverage

«depends-on»

«depends-on»

«depends-on»«depends-on»

«depends-on»

«depends-on»

«depends-on»

«depends-on» «depends-on»

INSPIRE WCS :: © 2017 Peter Baumann

Summary

Coverage Implementation Schema 1.1

= backwards-compatible evolution of GMLCOV 1.0

- All CIS 1.0 coverages remain valid

Grids: Regular + irregular (generalizing GML 3.3), SensorML

Interpolation

Representations: domain/range, „position/value pair“, partitioned, JSON,

RDF

Practice-driven packaging of functionality

OGC CIS 1.0, 1.1 ISO 19123-2

INSPIRE WCS :: © 2017 Peter Baumann

Aligning the Implementation of INSPIRE

Coverages to OGC standards

Jordi Escriu,

INSPIRE Thematic Cluster #3 Facilitator

INSPIRE WCS :: © 2017 Peter Baumann

Overview

Introduction

The data: Coverages

The service: WCS

Conformance Testing

Selected Applications

Conclusion

47

INSPIRE WCS :: © 2017 Peter Baumann

OGC Web Coverage Service (WCS)

WCS Core: access to spatio-temporal coverages & subsets

- Encoding on the fly

- subset = trim | slice

WCS Extensions: optional functionality facets

- Extraction, transformation, …up to flexible analytics (WCPS)

WCS Application Profiles: domain-oriented bundling

Large, growing

implementation basis:

rasdaman, GDAL,

QGIS, OpenLayers,

OPeNDAP, MapServer,

GeoServer, GMU,

NASA WorldWind, EOx-

Server; Pyxis, ERDAS,

ArcGIS, ...

INSPIRE WCS :: © 2017 Peter Baumann

WCS Service Model: Operations

GetCapabilities – OWS Common

- Service metadata

- Coverage list

DescribeCoverage

- coverage metadata

GetCoverage

- coverage, or subset thereof

INSPIRE WCS :: © 2017 Peter Baumann

<?xml version="1.0" encoding="UTF-8"?>

<wcs:Capabilities ...>

<ows:Title>MyLittleWCS</ows:Title>

<ows:Profile>http://www.opengis.net/spec/GMLCOV/1.0/conf/gml-coverage</ows:Profile>

<ows:Profile>http://www.opengis.net/spec/.../1.0/conf/get-kvp</ows:Profile>

<wcs:formatSupported>image/tiff</wcs:formatSupported>

<Contents xmlns="http://www.opengis.net/wcs/2.0">

<CoverageSummary>

<CoverageId>sampleCoverage</CoverageId>

<BoundingBox crs="http://www.opengis.net/def/crs/OGC/0/4326" dimensions=„2">

<LowerCorner>0 0 </LowerCorner>

<UpperCorner>49 11</UpperCorner>

</BoundingBox>

</CoverageSummary>

</Contents>

</wcs:Capabilities>

WCS GetCapabilities

information about Service capabilities + Coverage offering

INSPIRE WCS :: © 2017 Peter Baumann

http://www.acme.com/ows?

SERVICE=WQS & VERSION=1.0 & REQUEST=Query &

QUERY=/Capabilities/Contents/CoverageSummary/CoverageId &

FORMAT=application/gml+xml

Better Than DescribeCoverage: OGC WQS

Web Query Service [OGC 14-121]

- Discussion Paper, adopted May 2016

Can query any hierarchically organized server offering

- XML, JSON, ...

Query = XPath expression

Ex:

QUERY=/Capabilities/formatSupported

QUERY=/OfferedCoverage/RectifiedGridCoverage/domainSet/Grid/[@dimension=“3”]

INSPIRE WCS :: © 2017 Peter Baumann

WCS Core GetCoverage

Download a coverage (or a subset thereof), values guaranteed unchanged

Ex: „download coverage c001“

Ex: „coverage c001, lat/long cutout, time slice t=2009-11-06T23:20:52”

Ex: “coverage c001, in GeoTIFF”

http://www.acme.com/wcs ? SERVICE=WCS & VERSION=2.0

& REQUEST=GetCoverage & COVERAGEID=c001

& SUBSET=Long(100,120) & SUBSET=Lat(50,60)

& SUBSET=time("2009-11-06T23:20:52")

http://www.acme.com/wcs ? SERVICE=WCS & VERSION=2.0

& REQUEST=GetCoverage & COVERAGEID=c001

http://www.acme.com/wcs ? SERVICE=WCS & VERSION=2.0

& REQUEST=GetCoverage & COVERAGEID=c001 & FORMAT=“image/tiff“

INSPIRE WCS :: © 2017 Peter Baumann

Server-Side Processing: Performance

xxx

INSPIRE WCS :: © 2017 Peter Baumann

WCS Suite Big Picture

INSPIRE WCS :: © 2017 Peter Baumann

WCS Range Subsetting Extension [12-040]

Select components from compound cells

- „channel“, „band“ subsetting

Ex: ...& RANGESUBSET=red &…

…& RANGESUBSET=nir,red,green &…

…& RANGESUBSET=green,red,blue &…

…& RANGESUBSET=nir:green &…

…& RANGESUBSET=band01,band03:band05,band19:band21 &…

INSPIRE WCS :: © 2017 Peter Baumann

WCS CRS Extension [11-053]

Preamble

- WCS Core does not define Coordinate Reference System (CRS) handling• accessible only in Native CRS = CRS in which coverage is stored on server• srsName attribute

- CRS Name Type Specification [OGC 11-135] defines CRS URL framework• EPSG & more: Horizontal, vertical, time, index, ...• Not part of WCS

CRS Extension: GetCoverage with additional CRS parameters

- subsettingCrs: CRS in which bounding box is expressed• Default: coverage‘s Native CRS

- outputCrs: CRS in which result coverage is expressed• Default: subsettingCrs

Ex: http://www.acme.com/ows?SERVICE=WCS & VERSION=2.0

& REQUEST=GetCoverage & COVERAGEID=c001

& OUTPUTCRS=http://www.opengis.net/def/crs/EPSG/0/4326

INSPIRE WCS :: © 2017 Peter Baumann

Which CRSs Supported? See Capabilities!

<wcs:crsSupported>

http://www.opengis.net/def/crs/EPSG/0/4326

</wcs:crsSupported>

<wcs:crsSupported>

http://www.opengis.net/def/crs/EPSG

</wcs:crsSupported>

<wcs:crsSupported>

http://www.opengis.net/def/crs?authority=OGC&version=1.3&code=

AUTO42003&UoM=m&CenterLongitude=-100&CenterLatitude=45

</wcs:crsSupported>

<wcs:crsSupported>

http://www.acme.com/def/this-is-EPSG-4326

</wcs:crsSupported>

<wcs:crsSupported>

http://www.opengis.net/def/crs/OGC/0/AnsiDate

</wcs:crsSupported>

INSPIRE WCS :: © 2017 Peter Baumann

WCS Transaction Extension [OGC 13-057]

= WCS-T: Modify coverage offerings on a server via Web

New requests:

InsertCoverage + DeleteCoverage + UpdateCoverage (partial replacement)

Ex:

Compatibility: GetCoverage InsertCoverage

In rasdaman, enhanced with recipes & ingredients layer

http://www.acme.com/wcs

? SERVICE=WCS & VERSION = 2.0

& REQUEST=InsertCoverage

& COVERAGEREF=http://bcme.com/archive/hurricane.nc

& USEID=new

INSPIRE WCS :: © 2017 Peter Baumann

WCS-T With Recipes & Ingredients

Problems:

- WCS-T supports single requests, but often many files + regular updates

- Broad range of situations (metadata!)

rasdaman Approach (http://rasdaman.org/wiki/WCSTImportGuide):

- Single import command for all situations, based on WCS-T

- Import controllled by recipe = JSON import config + parameters = ingredients

Definitions:

- Data source for import

- Defaults: CRS, null values, ...

- Mosaicking

- Tiling structure in database

- ...and many more

INSPIRE WCS :: © 2017 Peter Baumann

WCS-T With Recipes & Ingredients

INSPIRE WCS :: © 2017 Peter Baumann

WCPS: Datacube Analytics

Web Coverage Processing Service

= spatio-temporal datacube analytics language

for $c in ( M1, M2, M3 )where some( $c.nir > 127 )return encode( $c.red - $c.nir, “image/tiff“ )

• "From MODIS scenes M1, M2, M3: difference red & nir, as TIFF"• “…but only those where nir exceeds 127 somewhere”

• s/t regular & irregular grids

INSPIRE WCS :: © 2017 Peter Baumann

...But That‘s Not What You Want to See

Let users remain in comfort zone of well-known tools

- Map navigation: OpenLayers, Leaflet, ...

- Virtual globe: NASA WorldWind, Cesium, ...

- Web GIS: MapServer, GeoServer,

QGIS, ArcGIS, ...

- Analysis: GDAL, R,

python (OWSLIB, Jupyter notebooks), ...

...via WCS / WCPS / WMS

as standard client/server APIs

[screenshots: diverse clients accessing rasdaman]

INSPIRE WCS :: © 2017 Peter Baumann

Synopsis of Coverage Service Suite

WCS -- spatio-temporal coverage access

WCPS -- spatio-temporal geo analytics language

WPS -- predefined server functionality

WCS-T -- ingest & update

coverage

offerings

(any format)

coverages1...

coverages2

WCS

coverages3

WCPS

sensor data

WCS-T...

coverages4

WPS...

image [timeseries]

simulation data

statistics data

INSPIRE WCS :: © 2017 Peter Baumann

OGC Sensor Observation Service (SOS)

W*S: downstream

download, processing, visualization

SOS

WMS

WCS

WCPS

WPS...coverage

server

SOS: upstream

sensor data capturing

SOS

sensor, image [timeseries], simulation, statistics data

INSPIRE WCS :: © 2017 Peter Baumann

OGC Web Processing Service (WPS)

WCPS: semantics in language, ready semantic interoperability (m2m!)

WPS: semantics in human-readable text syntactic interoperability

for $c in ( M1, M2, M3 )

return encode abs( $c.red - $c.nir ), "netcdf" )

INSPIRE WCS :: © 2017 Peter Baumann

W...whatever...S ?

OGC standards cover full range

from data-intensive to processing-intensive „Big Data“ coverage

services

WCS WCPS WPS

data access ad-hoc analytics predefined process

INSPIRE WCS :: © 2017 Peter Baumann

Overview

Introduction

The data: Coverages

The service: WCS

Conformance Testing

Selected Applications

Conclusion

84

INSPIRE WCS :: © 2017 Peter Baumann

WCS Testing: Down to Single Pixel Level

Credits: Jinsongdi Yu,

Stephan Meissl

INSPIRE WCS :: © 2017 Peter Baumann

Overview

Introduction

The data: Coverages

The service: WCS

Conformance Testing

Selected Applications

Conclusion

86

INSPIRE WCS :: © 2017 Peter Baumann

Ortho Image Timeseries

[Diedrich et al 2002, using rasdaman]

INSPIRE WCS :: © 2017 Peter Baumann

Elevation

for $s in (SatImage), $d in (DEM)

where $s/metadata/@region = “Glasgow"

return

encode(

struct {

red: (char) $s.b7[x0:x1,x0:x1],

green: (char) $s.b5[x0:x1,x0:x1],

blue: (char) $s.b0[x0:x1,x0:x1],

alpha: (char) scale( $d, 20 )

},

“image/png"

)

[JacobsU, Fraunhofer; data courtesy BGS, ESA]

INSPIRE WCS :: © 2017 Peter Baumann

WMS via WCPS

for $p in (OrthoPhoto),

$wl in (WaterLines), $wa in (WaterAreas),

$d in (DEM)

return

encode( (unsigned char) (

$p * { 1, 1, 1 }

overlay

$wl * { 0, 128, 255 }

overlay

$wa * { 191, 255, 255 }

overlay

switch $d

case $d > 260 return { red:255, green:0, blue:0 }

case $d > 262 return { red:0, green:255, blue:0 }

case $d > 264 return { red:0, green:0, blue:255 }

default return { red:0, green:0, blue:0 }

end

),

"image/png" )

INSPIRE WCS :: © 2017 Peter Baumann

rasdaman: Agile Array Analytics

= „raster data manager“: SQL + n-D arrays

- Scalable parallel “tile streaming” architecture

- Spatio-temporal regular & irregular grids

Mature, in operational use

- blueprint for Big Datacube standards: ISO, OGC, INSPIRE

INSPIRE WCS :: © 2017 Peter Baumann

external

archives

rasserver

databasefile system

rasdaman

geo services

Web clients (m2m, browser)

Architecture

distributed query

processingNo single point of failure

Internet

alternative

storage

[SSTD 2013]

optional compression[SSTD 2013]

INSPIRE WCS :: © 2017 Peter Baumann

Parallel, Distributed Processing

1 query 1,000+ cloud nodes

[ACM SIGMOD DanaC 2014]

[VLDB BOSS 2016]

Dataset B

Dataset A

Dataset D

Dataset C

select

max((A.nir - A.red) / (A.nir + A.red))

- max((B.nir - B.red) / (B.nir + B.red))

- max((C.nir - C.red) / (C.nir + C.red))

- max((D.nir - D.red) / (D.nir + D.red))

from A, B, C, D

INSPIRE WCS :: © 2017 Peter Baumann

Server-Side Processing: Federation

INSPIRE WCS :: © 2017 Peter Baumann

Overview

Introduction

The data: Coverages

The service: WCS

Conformance Testing

Selected Applications

Conclusion

96

INSPIRE WCS :: © 2017 Peter Baumann

Big Datacube Standards

Open Geospatial Consortium (OGC) :

- WCS „Big Geo Data“ standards suite

- rasdaman WCS Core Reference Implementation

ISO:

- TC211: coverages & WCS• ISO 19123 19123-1• OGC CIS 19123-2• OGC WCS ISO WCS

- SC32: SQL/MDA („Multi-Dimensional Arrays“)

INSPIRE:

- coverages & WCS

- rasdaman INSPIRE Reference Implementation

Research Data Alliance:

- Big Data Interest Group, Geospatial Interest Group, Array Database Assessment WG

INSPIRE WCS :: © 2017 Peter Baumann

Coverages in ISO

Historically, ISO 19123 defines abstract coverage model

- Identical to OGC Abstract Topic 6

- Abstract = not interoperable

– many diverging implementations can (and are known to) exist

Currently (2016):

- 19123 19123-1

- OGC CIS 1.1 19123-2

- Later: OGC WCS 2.1 ISO WCS

INSPIRE WCS :: © 2017 Peter Baumann

ESA Tech Harm Strat Meetg, 2017-feb-21

INSPIRE WCS :: © 2017 Peter Baumann

Standards: ISO Array SQL [SSDBM 2014]

select id, encode((scene.band1-scene.band2)/(scene.band1+scene.band2)), „image/tiff“ )

from LandsatScenes

where acquired between „1990-06-01“ and „1990-06-30“ and

avg( scene.band3-scene.band4)/(scene.band3+scene.band4)) > 0

create table LandsatScenes(

id: integer not null, acquired: date,

scene: row( band1: integer, ..., band7: integer ) mdarray [ 0:4999,0:4999] )

INSPIRE WCS :: © 2017 Peter Baumann

Resources OGC authoritative standards page

- http://www.opengeospatial.org/standards/wcs

- http://www.opengeospatial.org/standards/wcps

OGC Coverages.DWG wiki (background info, stds pre-releases)

- http://external.opengeospatial.org/twiki_public/CoveragesDWG

INSPIRE: http://inspire.ec.europa.eu/id/document/tg/download-wcs

Training material

- Webinars: www.earthserver.eu/webinars

- Online interactive demos & sandbox: http://standards.rasdaman.com/

- Tutorials: http://tutorial.rasdaman.org/rasdaman-and-ogc-ws-tutorial/

- Wikipedia: coverages, WCS, WCPS

- Jupyter notebooks: http://nbviewer.jupyter.org/github/earthserver-eu/

rasdaman: www.rasdaman.org

INSPIRE WCS :: © 2017 Peter Baumann

What‘s Next?

INSPIRE:

- Re-harmonize INSPIRE coverages with OGC/ISO coverages

- INSPIRE conformance tests (based on OGC testing, only specifics in addition)

OGC:

- Protocol bindings: REST (spec ready...)

- Raster/polygon clipping

- MetOcean Application Profile

ISO:

- OGC CIS ISO 19123-2

- Modernize ISO 19123-1

- OGC WCS ISO „19123-3“

INSPIRE WCS :: © 2017 Peter Baumann

Cloud Demo

INSPIRE WCS :: © 2017 Peter Baumann

Conclusion Coverage Suite: data & service model for spatio-temporal data

Coverage Implementation Schema (CIS)- raster data (timeseries) and beyond

Web Coverage Service (WCS)- access (WCS Core) up to analytics (WCPS)

- conformance tests down to pixel level, interoperable

Convergence: stds bodies, open-source & proprietary implementers

- OGC, ISO, INSPIRE

- Growing number of implementations

- rasdaman OGC & INSPIRE Reference Implementation

Federations enabling for INSPIRE

107

[Dali]