Workflow Enactment of Grid-Enabled Geospatial Web Services

20
Workflow Enactment of Grid-Enabled Geospatial Web Services Gobe Hobona, David Fairbairn, Philip James UK e-Science All Hands Meeting - 12 th September 2007 - Nottingham

description

Workflow Enactment of Grid-Enabled Geospatial Web Services. Gobe Hobona , David Fairbairn, Philip James. UK e-Science All Hands Meeting - 12 th September 2007 - Nottingham. Overview. Background OGC Web Services Workflow Enactment SAW-GEO Architecture User interface – Geoportal - PowerPoint PPT Presentation

Transcript of Workflow Enactment of Grid-Enabled Geospatial Web Services

Page 1: Workflow Enactment of Grid-Enabled Geospatial Web Services

Workflow Enactment of Grid-Enabled Geospatial Web Services

Gobe Hobona, David Fairbairn, Philip James

UK e-Science All Hands Meeting - 12th September 2007 - Nottingham

Page 2: Workflow Enactment of Grid-Enabled Geospatial Web Services

Overview

• Background

• OGC Web Services

• Workflow Enactment

• SAW-GEO Architecture

• User interface – Geoportal

• Conclusions

Page 3: Workflow Enactment of Grid-Enabled Geospatial Web Services

Background

• Open Geospatial Consortium (OGC)

• ISO19119 and OGSA

• JISC Grid/OGC Collision Programme• Security : SEE-GEO

• Workflow : SAW-GEO

• OGC Interoperability Experiments (OWS-4)

Page 4: Workflow Enactment of Grid-Enabled Geospatial Web Services

OGC Web Services

• Web Feature Services (WFS)• Disseminates vector geospatial data

• Web Map Services (WMS)• Generates geovisualisations/maps from any geo-data source

• Web Coverage Services (WCS)• Disseminates raster geospatial data

• Web Processing Services (WPS)• Runs geocomputational models or geospatial operations on

user-supplied datasets

Page 5: Workflow Enactment of Grid-Enabled Geospatial Web Services

Sample OWS Request Messages

<wfs:GetFeature service="WFS" version="1.0.0" outputFormat="GML2" xmlns:topp="http://www.openplans.org/topp" xmlns:wfs="http://www.opengis.net/wfs" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.0.0/WFS-basic.xsd"> <wfs:Query typeName="topp:states"> <wfs:PropertyName>topp:STATE_NAME</wfs:PropertyName> <wfs:PropertyName>topp:PERSONS</wfs:PropertyName> <ogc:Filter> <ogc:BBOX> <ogc:PropertyName>the_geom</ogc:PropertyName> <gml:Box srsName="http://www.opengis.net/gml/srs/epsg.xml#4326"> <gml:coordinates>-75.102613,40.212597 -72.361859,41.512517</gml:coordinates> </gml:Box> </ogc:BBOX> </ogc:Filter> </wfs:Query></wfs:GetFeature>

<GetCoverage service="WCS" version="1.0.0" xmlns="http://www.opengis.net/wcs" xmlns:nurc="http://www.nurc.nato.int" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wcs http://schemas.opengis.net/wcs/1.0.0/getCoverage.xsd"><sourceCoverage>nurc:Arc_Sample</sourceCoverage><domainSubset><spatialSubset><gml:Envelope srsName="EPSG:4326"><gml:pos>9.420000076293944 42.20000076293945</gml:pos><gml:pos>11.819999694824217 43.90000049273173</gml:pos></gml:Envelope>…..</domainSubset><rangeSubset><axisSubset name="Band"><singleValue>1</singleValue></axisSubset></rangeSubset><output><crs>EPSG:4326</crs><responseCrs>EPSG:32626</responseCrs><format>TIFF</format></output></GetCoverage>

Page 6: Workflow Enactment of Grid-Enabled Geospatial Web Services

Sample OWS Response Messages…</gml:featureMember><gml:featureMember><topp:states fid="states.40"><topp:the_geom><gml:MultiPolygon srsName="http://www.opengis.net/gml/srs/epsg.xml#4326"><gml:polygonMember><gml:Polygon><gml:outerBoundaryIs><gml:LinearRing><gml:coordinates decimal="." cs="," ts=" ">-77.476082,39.719528 -78.096222,39.725368 ……74.984978,41.099285 -74.989265,41.081665 -74.966675,41.082581 -77.476082,39.719528</gml:coordinates></gml:LinearRing></gml:outerBoundaryIs></gml:Polygon></gml:polygonMember></gml:MultiPolygon></topp:the_geom><topp:STATE_NAME>Pennsylvania</topp:STATE_NAME><topp:PERSONS>1.1881643E7</topp:PERSONS></topp:states></gml:featureMember><gml:featureMember>….

NCOLS 522

NROWS 300

XLLCENTER -1.6878262291489

YLLCENTER 54.93369175561999

CELLSIZE 3.013533976228086E-4

NODATA_VALUE -9999.0 62.20000 61.70000 61.20000 61.20006 60.7000 60.40000 60.40000 60.09999 59.561.79999 61.5 61.09999 61.09999 60.7000 60.40000 60.40000 60.20000

60.0999984761.79999 61.5 61.099998 61.09999 60.7999 60.5 60.5 60.5 60.561.90000 61.59999 61.20000 61.20000 60.9000 60.70000 60.70000 60.79999 60.562.5 62.20000 61.799999 61.79999 61.79999 61.40000 61.40000 61.09999

60.9000015362.70000 62.40000 62.40000 62.40000 62 61.70000 61.70000 61.40000

61.0999984763.09999 63 62.59999 62.59999 62.4000 62 62 61.79999 61.563.70000 63.5 63.20000 63.20000 62.9000 62.59999 62.59999 62.40000

62.0999984763.90000 63.70000 63.40000 63.400001 63.0999 62.90000 62.90000 62.59999

62.4000015364.19999 64 63.70000 63.70000 63.5 63.29999 63.299999 63.09999

62.7999992464.40000 64.19999 64 64 63.7000 63.59999 63.599997 63.29999

63.0999984764.90000 64.69999 64.5 64.5 64.4000 64.19999 64.19999 64

63.7999992464 64.90000 64.80000 64.80000 64.5999 64.5 64.5 64.40000

64.1999969564 64 64 64 64 64.80000 64.80000 64.69999 64.564 64 64 64 64 64 64 64 6464 64 64 64 64 64 64 64 6464 64 64 64 64 64 64 64 6464 64 64 64 64 64 64 65

65.3000030564 64 64 64 64 64 65.19999 65.5

Page 7: Workflow Enactment of Grid-Enabled Geospatial Web Services

Grid-enabling OWS

• SOAP-based proxy service• Created from OGC XML schemas (XSD)• Requests create temporary datasets

• Resource management at two levels• Temporary datasets managed by proxy service• Persistent data managed through OWS e.g. WFS-T

• Security limitations• OWS currently do not offer message encryption• Approach to wrap OWS in firewall and allow external access

only through proxy service

Page 8: Workflow Enactment of Grid-Enabled Geospatial Web Services

Geospatial Data is Very Large

• Vector data• Thousands of coordinates

• Multiple dimensions

• Raster data• Thousands of pixels (very high resolution)

• Multispectral imagery (200+ band imagery)

• Solution• Pass data by URL references and get client to stream data in

Page 9: Workflow Enactment of Grid-Enabled Geospatial Web Services

Load Experiment

0

100

200

300

400

500

600

number of features

res

po

ns

e t

ime

(s

ec

on

ds

)

Start of serialisation ofresponse messageGML-based SOAP responsereceived

URL-reference to GMLdocument received

Page 10: Workflow Enactment of Grid-Enabled Geospatial Web Services

Workflow Enactment

• Recognised by both ISO19119 and OGSA

• Several options for workflow enactment• SCUFL, BPEL, Keppler etc

• Selected BPEL because• OASIS Standard, i.e. WS-BPEL 2.0

• Multi-vendor support including IBM, Sun Microsystems, ActiveEndpoints, Oracle etc

• Availability of open source enactors

Page 11: Workflow Enactment of Grid-Enabled Geospatial Web Services

WS-BPEL Overview

• Based on Activities• Basic or Structured

• Variables• Declaration• Assignment

• Conditionals

• Loops

• Fault handlers

Page 12: Workflow Enactment of Grid-Enabled Geospatial Web Services

An Example Geospatial Workflow

Based on OGC OWS-4 GeoProcessing Workflow Scenario

client

WorkflowEnactor

WPS 1Generalise

WPS 2Clip

WFS

Page 13: Workflow Enactment of Grid-Enabled Geospatial Web Services

Possible Applications for Geo-Workflows

• Emergency Management• Where each activity depends on the result of a previous activity

• Geographic modelling• Where several steps are needed before a final model is produced e.g.

ESRI Model Builder

• Climate Change scenarios• Where a number of possible routes for workflows are possible

depending on the state of certain variables

Page 14: Workflow Enactment of Grid-Enabled Geospatial Web Services

Components

• Geoserver

• 52North WPS

• ActiveBPEL Open Source Engine

• Globus Toolkit (GT4) …or Apache Axis 2

Page 15: Workflow Enactment of Grid-Enabled Geospatial Web Services

SAW-GEO Architecture

Page 16: Workflow Enactment of Grid-Enabled Geospatial Web Services

Geoportal

• User• Browser-based BPEL Upload

• Hot deployment

• AJAX-based SOAP client

• Workflow Monitoring• ActiveBPEL Admin Console

• Offers views of deployed processes

• Exports a list of available services to the SOAP client as XML

• Red font: Developed for SAW-GEO

Page 17: Workflow Enactment of Grid-Enabled Geospatial Web Services

ActiveBPEL Workflow Engine

Page 18: Workflow Enactment of Grid-Enabled Geospatial Web Services

Discussion

• Importing OGC XSD into SOAP services offered high interoperability

• However, variations in how tiers interpreted the XSD• varying namespaces• cyclic imports

• Necessary to throw faults at SOAP tier from faults at OGC tiers, because BPEL can handle SOAP faults

• Variations in GML between workflow nodes, potentially requiring ‘shim services’ when chaining OWS

Page 19: Workflow Enactment of Grid-Enabled Geospatial Web Services

Conclusions

• SOAP wrappers needed for Grid-enabling OGC web services

• OGC XSD appropriate for creating SOAP wrappers

• Geospatial data best passed through URL references to avoid overwhelming the workflow enactor

• BPEL can support Grid-enabled OGC web services

Page 20: Workflow Enactment of Grid-Enabled Geospatial Web Services

Future Work

• Experimental deployment on the Belfast e-Science Centre (BeSC) DEBUT system

• Investigate deployment on upcoming NGS GT4 infrastructure

• User Evaluation (November-December 2007)• Volunteers needed – Please register on the website

• http://www.edina.ac.uk/projects/seesaw

• Integration with SEE-GEO