ESIP Environment Oriented Satellite Image Processing Platform ( Middleware Extension)

www.see-grid-sci.eu

SEE-GRID-SCI

The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no. 211338

ESIPEnvironment Oriented Satellite

Image Processing Platform (Middleware Extension)

Dorian Gorgan, Victor BacuTeodor Stefanut, Denisa RodilaTechnical University of Cluj-Napoca

Romania

JRA1 Application-Specific Service Description10 September 2009

JRA1 Application-Specific Services and Middleware Extensions - Introductory Presentation 2/28

Primary contactDorian [email protected]

Technical University of Cluj-Napoca

UTCN, third party SEE-GRID-SCI partner acronym

Romania

Current status Beta

Expected availability 1st April 2010

Licensing status GPL 3

ESIP Platform


ESIP Platform Description

ESIP - Environment Oriented Satellite Image Processing Platform

Area(s) of usage Environment oriented applications using data provided through Grid based

satellite image processing

Description

ESIP provides a Grid based software platform for satellite image processing and experiments this approach on pilot environmental applications. It is a collection of services available in Web and Grid applications. ESIP will be extended to a larger production Grid infrastructure by processing real data for SEE regions

The Environment oriented Satellite Data Processing Platform (ESIP) is developed through the SEE-GRID-SCI (SEE-GRID eInfrastructure for regional eScience) project (2008-2010), funded by the European Commission.


ESIP Related References

Wiki ESIP

http://wiki.egee-see.org/index.php/JRA1_Commonalities

Wiki GreenView

http://wiki.egee-see.org/index.php/GreenView


Partners and User Communities

Identified SEE-GRID-SCI partners and communities ESIP Platform development, and Methodological Approach for ESIP based

application developmentPartners: 1. Technical University of Cluj-Napoca (UTCN), ESIP coordinator

2. National Center for Information Technology (NCIT) Bucharest3. West University of Timisoara (UVT)4. ICI Bucharest, national coordinator and SEE-GRID-SCI partner

ESIP is used to develop the GreenView applicationPartners: 1. UTCN, NCIT, UVT, and ICI (Romania)

2. Eötvös Loránd University (ELU) from Budapest (Hungary)3. Research and Educational Networking Association of Moldova (RENAM)

Other related usages

The main beneficiaries of this VO are Government Organizations, Environment Agencies, Hydrological Institutes, and Research Groups involved in environment supervision and behavior prediction of natural phenomena, especially in vegetation related studies


Grid Infrastructure: middleware (gLite)

ESIP Platform: service oriented architecture; collection of services providing creation, execution, and scheduling of the jobs; job resource management; access to image database; file transfer; data replication; workflow based execution; image processing.

Applications: environmental applications based on process description graphs.

Environmental Data

Image and Spatial Data

Process Description

Grid Infrastructure

ESIP Platform

Application Oriented Level

GreenView Appl. Other Env Appl’s. . .

Functional levels


Architecture

Server component – represented by WebServicesClient component – API used to develop client applications (web or desktop based applications)User oriented level application – a set of user interaction supported components (EditorIC, ManagerIC, ViewerIC)Application level – combines the editor, manager, and viewer related functionality into complex functionalities

Client Java API

User oriented application level

ViewerICEditorIC ManagerIC

Applications

Clie

nt

GRID Infrastructure

ViewerWSExecutorWSEditorWS ManagerWS

Ser

ver


ESIP functional levels

WebService level (first implementation) EditorWS, ManagerWS, ExecutorWS, ViewerWS

Application oriented level (Interaction Supported Components) (work in progress) EditorIC, ManagerIC, ViewerIC

Application level (first implementation of a subset of the GreenView application components) Editor Application, Manager Application, Viewer Application Complex application (e.g. GreenView)


Service supported functionality

EditorWS Provides information about the available resources (e.g. lists of operations,

sub-graphs, services, satellite image types, data types etc) WSDL URL: gPEditorWS

http://gisheo01.mediogrid.utcluj.ro:8185/gPEditor/gPEditorWSService?wsdl

ManagerWS Provides information about workflow (i.e. PDG, IPDG) Fetch and upload resources (e.g. workflow, operators, services, sub-graphs,

data) WSDL URL: gPManagerWS

http://gisheo01.mediogrid.utcluj.ro:8185/gPManager/gPManagerWSService?wsdl

Reference: Wiki ESIP (http://wiki.egee-see.org/index.php/JRA1_Commonalities)


Service supported functionality

ExecutorWS Executes the instantiated workflows (IPDGs) over the Grid Monitor the execution of the workflows WSDL URL: gPExecutorV02gLite

http://gisheo01.mediogrid.utcluj.ro:8185/gPExecutorV0.2gLite/gPExecutorWSgLiteV02Service?wsdl

ViewerWS Get and format the input and output data (e.g. initial and processed satellite

images)

Reference: Wiki ESIP (http://wiki.egee-see.org/index.php/JRA1_Commonalities)


Interaction supported functionality

Components used to build up the application’s complex functionality

EditorIC Support the user’s editing operations for the workflow development

ManagerIC Instantiate workflow for particular satellite data Manage the model resources (e.g. operators, services, sub-graphs, satellite

data)

ViewerIC Display the input and output data (e.g. initial and processed satellite images)

on the client site Get and display the monitoring data


ESIP grid workflow description

ESIP aims to provide a flexible diagrammatical description solution for image processing workflow by defining PDG patterns

Workflows or PDG (Process Description Graph) represented as DAGs describes the execution workflow and dependencies between tasks

Input data – satellite spectral bandsNodes – resources, operators, sub-graphs, services

Abstract PDG – used to define the workflow Instantiated PDG can be scheduled and executed must be mapped onto a physical satellite image


Water detection algorithm (Gond)

Bands: NIR, RED

Bands: NIR, SWIR


Process Description Graphs

• Graph nodes: • Resources• Operators• Services• Sub-graphs

Input

OP1

S1

SG

Input

Input

Input

OP2

OP3 S2

Output


Image Processing Operators

Experiments on TIFF format satellite image oriented operators:

Add, Subtract, Multiply, Divide, AddConst, SubtractConst, MultiplyConst, LogicAnd, LogicOr, LogicXor, LogicNot, Blur, Sharpen, EdgeDetection, ThresholdFilter, MeanValue, HistogramEq, HistogramScale, Blend, PseudoColoring, Dilation, Erosion, Opening, and Closing

Reference:

http://wiki.egee-see.org/index.php/JRA1_Commonalities


Web Service Composition

Design and implement a business workflow architectural model which allows the extension of the workflow description and execution tools in the Grid environment with Web service based computation nodes

Automatic discovery of Web services Automatic binding of Web services Semantic annotation Composition of Web services based on templates Automatic invocation of composed Web services from a Grid environment

The extension of workflow description and execution process is needed for defining a hyper-graph structure in a Grid environment having as computation nodes

Operators Sub-graphs Web services


Workflow Example

Band 1

AddBand 5

Band 7

Band 3

Band 4

MultiplyNot

Sub

Business Workflow

Add


Solution

Composition of Web services in two major phases:

Automatically searching the Web services UDDI private repository

Semantic annotation

Automatically binding the information of selected Web services to predefined templates used for BPEL business process creation and instantiation


System Architecture


Dissemination

Participation to the Regional Environmental Science Workshop, Sofia, May 6-8, 2009. Presentations:

1. Victor Bacu, Dorian Gorgan, Teodor Stefanut, "GreenView”. Reference: SEEGRIDSCI-NA4-RO-001-GreenView_EnvVO-a-2009-05-07.ppt

2. Victor Bacu, Dorian Gorgan, Teodor Stefanut, "ESIP Platform”. Reference: SEEGRIDSCI-JRA1-RO-001-ESIP-a-2009-05-07.ppt

Participation to SCICOM09 - 7th International Conference on "Large-Scale Scientific Computations", 4-8 June, 2009, Sozopol, Bulgaria.Presentation:

1. Dorian Gorgan, Teodor Stefanut, Victor Bacu, Danut Mihon, Denisa Rodila, “Grid based Environment Application Development Methodology”

Participation to HiPerGRID - 3rd International Workshop on High Performance Grid Middleware, 28 May, 2009, Bucharest. Presentation:

1. Victor Bacu, Teodor Stefanut, Denisa Rodila, Dorian Gorgan, “Process Description Graph Composition by gProcess Platform"

Participation to the Student Scientific Communication Session, 22 May 2009, Technical University of Cluj-Napoca. Presentations:

1. Denisa Rodila, Victor Bacu, Dorian Gorgan, “Integration of Satellite Image Operators as Workflows in the gProcess Application”

2. Danut Mihon, Dorian Gorgan, Victor Bacu, Teodor Stefanut, "Grid Based Environment Application Development- Green View"


Papers to Conferences

1. Gorgan, D., Bacu, V., Stefanut, T., Rodila, D., "Grid based Satellite Image Processing Platform for Earth Observation Applications Development". IDAACS'2009 - IEEE Fifth International Workshop on "Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications", 21-23 September 2009, Rende, Cosenza, Italy (2009)

2. Gorgan, D., Stefanut, T., Bacu, V., Mihon, D., “Grid based Environment Application Development Methodology”, SCICOM09 - 7th International Conference on "Large-Scale Scientific Computations", 4-8 June, Sozopol, Bulgaria, (to be published in LNCS, 2009)

3. Bacu V., Stefanut T., Rodila D., Gorgan D., “Process Description Graph Composition by gProcess Platform”. HiPerGRID - 3rd International Workshop on High Performance Grid Middleware, 28 May, Bucharest. Proceedings of CSCS-17 Conference, Vol.2. ISSN 2066-4451, pp. 423-430, (2009)

4. Rodila D., Bacu V., Gorgan D., “Integration of Satellite Image Operators as Workflows in the gProcess Application”. ICCP2009, 27 Aug, 2009 Cluj-Napoca (accepted for publication)


5. Mihon D., Bacu V., Stefanut T., Gorgan D., “Grid Based Environment Application Development – GreenView Application”. ICCP2009 - IEEE 5th International Conference on Intelligent Computer Communication and Processing, 27 Aug, 2009 Cluj-Napoca (accepted for publication).

6. Denisa D. Rodila, Victor Bacu, Dorian Gorgan, "Semantic Annotation based Service Composition for Grid Workflow Description and Execution". SYNASC Symposium, 26-29, September 2009, Timisoara, (Accepted for publication in IEEE Computer Press)

7. Gorgan D., "Earth Science Application Development Methodology by ESIP and gProcess Platforms". Abstract sent for the workshop "Federation and interoperability of Earth Science Digital Repositories: data discovery, access and visualization" within the Conference EGEE'09 - Uniting our strengths to realise a sustainable European grid, 21-25 Sept, Barcelona (2009)


Users, Results, Plans for Future

Possible long-term developments The ESIP Platform and the set of tools and components from the Application

Oriented Level could be used in: Development of other environmental and geographical data based applications in

SEE and CE regions

Development of tools and application required by other related European projects such as EnviroGRIDS, http://www.envirogrids.net/

Basic software platform for related national programmes from the SEE and CE regions

Center for Ecological-Noosphere Studies of National Academy of Sciences of the Republic of Armenia - CENS NAS RA

Development of environment applications. For instance, soil pollution by heavy metals

Data access services to data repositories provided by the GENESI-DR Project


Developing plan


Thanks for attending!

ESIP Environment Oriented Satellite Image Processing Platform ( Middleware Extension)

Documents

Transcript of ESIP Environment Oriented Satellite Image Processing Platform ( Middleware Extension)