20100512 Workflow Ramage

Workshop on WorkflowsWorkshop on Workflowson Earth Observationon Earth Observation

Steven Ramage, [email protected] June 2010, Sir Clive Granger Building, University of Nottingham

mailto:[email protected]

Making location countCopyright © 2010, Open Geospatial ConsortiumCopyright © 2010, Open Geospatial Consortium

The importance of workflowThe importance of workflow

• Workflows are important because real jobs rarely involve few, simple tasks. Creating value in almost any area of endeavor requires the sequencing and organisation of human efforts, or of digital or mechanical processes, e.g.– Supply chain manufacturing– Insurance claims processing– Building design

All can be studied as workflows. • The geospatial realm is no different. Image processing and

spatial modeling can also be characterized as workflows.


Description of workflowDescription of workflow

• In information systems workflows are a series of coordinated analytical and information processing steps

• Typically to transform data into information or knowledge • Workflows could be described as Web-based scripts or

macros that automate serial information tasks.


OGC Interoperability ProgramOGC Interoperability Program

• Collaborative, standards-based interoperability initiatives• Sponsors define operational issues where standards can

help and initiatives are created• Technology providers come together to prototype solutions

to tackle sponsors’ interoperability problems. • OGC Web Services (OWS) initiatives underway for years• OWS-5 had geospatial workflows as a primary focus, a

detailed video report is available at:

http://www.opengeospatial.org/projects/initiatives/ows-5

http://www.opengeospatial.org/projects/initiatives/ows-5


OGC Web Services standardsOGC Web Services standards

OGC has four main OGC Web services standards for geospatial data sharing and processing:

1.OGC Web Mapping Service Interface Standard (WMS) defines a Web API for requesting a picture of data, such as a PNG, JPEG, or GIF.

2.OGC Web Feature Service Interface Standard (WFS) defines a Web API for accessing raw vector data in formats such as GML, KML or GeoRSS and adding, updating and deleting data.


OGC Web Services standardsOGC Web Services standards

OGC has four main OGC Web services standards for geospatial data sharing and processing:

3.OGC Web Coverage Service Interface Standard (WCS) defines a web API for accessing raster data in formats such as GeoTIFF, JPEG2000, or HDF WCS also supports adding, updating and deleting data.

4.OGC Web Processing Service Interface Standard (WPS) defines a web API for running an algorithm or model, specifying data inputs and outputs


““Chaining” Web Services for decision supportChaining” Web Services for decision support

…

WCS WPS – ClassificationWPS - WCTS WFS

Internet

Web servers

OGC Interfaces

Service chaining creates value-added products

Decision Support Client

Geoprocessing worklow developed in OGC testbeds since 2004

Assess Wildfire Activity


BPEL workflow in OWS-5BPEL workflow in OWS-5

Participants chained combinations of services together to address the requirements of geospatial data conflation and image processing.

It also addressed an important gap, which was the lack of SOAP (Simple Object Access Protocol) and WSDL (Web Service Description Language) descriptions for those APIs.


BPEL workflow in OWS-5BPEL workflow in OWS-5

Both W3C standards are critical for creating and managing workflows using industry standards such as OASIS’s BPEL (Business Process Execution Language).

BPEL is the workflow description language used in many end-user service chaining tools such as Oracle BPEL Designer and George Mason University’s BPEL Power.


Conflation workflow architectureConflation workflow architecture


Conflation workflow architectureConflation workflow architecture

The OWS-5 geo-processing workflow thread bound together WFS and WPS services in BPEL scripts to model data conflation in a consistent, repeatable fashion.

This successful prototype signaled the utility of this approach for a host of geospatial tasks that require the combination of many different geospatial services and operations.

Fusion activities developed in OWS-7 and will have more focus in this area for OWS-8.


OGC Web Processing Service (WPS)OGC Web Processing Service (WPS)

WPSGetCapabilities ExecuteDescribeProcess

Algorithms Repository

…

…

Algorithm 1

Data Handler Repository…

…

Data Handler A

Communication over the web using HTTP

WPS-client

Web Processing Service


REST-oriented workflow in OWS-5REST-oriented workflow in OWS-5

The SOAP/WSDL/BPEL approach to chaining Web services is one of the most widely used ways to implement service oriented architectures (SOA).

However, other approaches, notably REST (Representational State Transfer) services, are also used to implement services in an enterprise.

Some feel that Web services do not need the extra layer of description provided by SOAP and WSDL to build reliable workflows.


REST uses HTTP and other parts of the basic Web platform.

In the SWE (Sensor Web Enablement) thread of OWS-5, participants used a REST-based architecture and WfXML to create workflows that allowed raw earth observation data to be acquired from assets, such as the EO-1 satellite, and then utilized by different WPS.

Relevant data were then published and distributed to end-users in formats such as GeoAtom (Atom with GeoRSS extensions) and KML.

REST-oriented workflow in OWS-5REST-oriented workflow in OWS-5


Integrating smoke WPS into WfXML-R workflowIntegrating smoke WPS into WfXML-R workflow

GMU GRASS SOAP/WSDL

GRASS Image Algebra

EO-1 SOS Google Earth

ExecuteInputs: •URL Grid 1•URL Grid 2•OutputType

EO Sensor Web Workflow

GMU WCS-T

GetCapabilities DescribeProcess

NGC EO-1 Smoke WPSCalculated smoke geotiff

Classified smoke geotiff

Call calculation

Call classifier

Processes:•FineSmoke•CoarseSmoke

Inputs: •URL Grid 1•URL Grid 2•OutputType

Output: •URL Grid 3

OWS-6 WPS Grid ProcessingOWS-6 WPS Grid Processing

• Bastian Baranski and Bastian Schäffer, Univ. Muenster Institute for Geoinformatics (IfGI) and 52° North, Germany

• Andrew Woolf, Science and Technology Facilities Council (STFC), UK

• Lan-Kun Chung, GIS Center, Feng Chia University, Taiwan


OWS-6 WPS grid processingOWS-6 WPS grid processing

Grid processing profiles of WPS - specification– WPS grid processing profile integrated with grid

computing infrastructure:• Job Submission Description Language (JSDL)• High Performance Computing (HPC) Basic Profile

(HPC-BP)• Simple API for Grid Applications (SAGA)• Data Access and Integration set of specifications

(WS-DAI-*)• Web Services Resource Framework (WSRF)


OWS-6 WPS grid processingOWS-6 WPS grid processing

OGF enabled WPS - implementation– WPS to benefit from and integrate with distributed

computing resources and technologies– Two potential ways to make use of OGF specifications,

concepts and their implementations where identified• encapsulating other resources• integration alongside other services


Grid-enabled WPS implementationsGrid-enabled WPS implementations

Airport Scenario– Trajectory Service (WPS JSDL Profile)

Science and Technology Facilities Council (STFC), UK– Plume Rendering Service (WPS JSDL Profile)

Institute for Geoinformatics (IfGI), University of Münster, Germany


Grid-enabled WPS implementationsGrid-enabled WPS implementations

Debris Flow Scenario– Rainfall Data Interpolation (WPS HPC Basic Profile)

Institute for Geoinformatics (IfGI), University of Münster,

Geographic Information Systems (GIS) Research Center, Feng Chia University (FCU), Taiwan

– Geophone Data Analysis (WPS HPC Basic Profile)• Institute for Geoinformatics (IfGI), University of

Münster, Germany• Geographic Information Systems (GIS) Research

Center, Feng Chia University (FCU), Taiwan


OGC online resourcesOGC online resources

OWS-6 Geoprocessing Workflow Architecture

Engineering Report

0.3.0 09-053r5

Bastian Schäffer 2009-10-09

This document covers geo-processing workflow

best practices and methods in a SOA environment.


Debris Flow Monitoring System


Debris flow – scenarioDebris flow – scenario

A debris flow is a fast moving mass of unconsolidated,

saturated debris that looks like flowing concrete.

Based on existing debris flow monitoring system

(proprietary and monolithic software infrastructure)

in Taiwan (running since 2003).

Develop and implement open (OGC) standards-based

service oriented architecture (SOA) for debris flow

monitoring.


Debris flow - scenarioDebris flow - scenario


Debris flow – sensors and geo-processingDebris flow – sensors and geo-processing

Landslides and flooding are a threat on the mountainous

island of Taiwan, due to typhoons and earthquakes.

The Geographic Information Systems Research Center,

Feng Chia University (GIS.FCU) in Taiwan has implemented

OGC services for use in workflow, detecting and analyzing

sensor data for emergency response.

There is a demonstration showing the working network of

debris flow sensors and examples of distributed services

performing analysis and processing the sensor data.


Debris flow monitoring systemDebris flow monitoring system


Debris flow operational scenarioDebris flow operational scenario


From portal select desired theme and area of interest

Wizard picks appropriate workflow for desired result

Wizard

Mozambique

Disaster Management Information System (DMIS)

Workflows

Estimated rainfall accumulation and flood prediction model

Flood Model

Selected workflow automatically activates needed assets and models

Baseline water level, flood waters and predicted flooding

GEOSS AIP-2 flood prediction and responseGEOSS AIP-2 flood prediction and responseLed by NASA, Spot Image, Northrop Grumman, ERDASLed by NASA, Spot Image, Northrop Grumman, ERDAS


IP3 Client & Workflow engine

IP3 Distributed Community Catalog/MediatorIP3 Distributed Community Catalog/Mediator

WCS - TOther Non-OGC Services GBIF

Non-OGC Services

OGC WPS Access to Model

CSW

GEOSS Portal

WCS WFS

reqreq req

respresp resp

req

resp

reqreq

respresp

req

resp

req

resp

req

resp

Broker

GEOSS AIP-2 biodiversity & climate changeGEOSS AIP-2 biodiversity & climate changeLed by CNR, Univ of Colorado, GBIFLed by CNR, Univ of Colorado, GBIF

Research Scientist


SWE and geo-processing workflowSWE and geo-processing workflow

36

SOS

!

SAS

Sensor Net

SAS

Mission Control Center

Access & Processing Node

WPS

CSW

Register Measurement

Types

SPS CSWSOS


The power of workflowThe power of workflow

Workflows can be formally described as programmes that invoke Web services that invoke other Web services that invoke other Web services.

Analytical models, decision support tools and other complex resources can quickly deliver results in this way and optionally, provide documentation of the steps.

There is an OGC Workflow Domain Working Group: this is a forum for describing, discussing and providing best practice guidance related to geospatial workflows using existing IT workflow standards.


Contact meContact me

Thanks for your attentionThanks for your attention

20100512 Workflow Ramage

Documents

Transcript of 20100512 Workflow Ramage