NEC, Tokyo, Japan, May 2003 Use of Workflow Techniques for Grid Management Junwei Cao ( 曹军威 )...

NEC, Tokyo, Japan, May 2003

Use of Workflow Techniques for Grid Management

Junwei Cao (曹军威 )C&C Research Laboratories

NEC Europe Ltd.Germany


CCRLE

Bonn, Germany ~20 research staffs 3 working teams:

Numerical simulation MPI development Grid Computing

GEMSS


My Previous Experience PACE: performance prediction of

parallel and distributed systems Titan: prediction based job

scheduling on clusters and multiprocessors

ARMS: agent-based resource management for grid computing

GridFlow: workflow management for grid computing (CCGrid 2003)


Outline

Background GEMSS objectives Medical simulation applications Service performance prediction Workflow simulation and

scheduling Dynamic service federation Summary


Grid Workflow Management

Workflow DefinitionWPDL, BPEL4WS, GSFL, ASCI Grid Workflow SystemsWebFlow, Symphony, GridAnt, BPWS4J,

TENT Component-based SystemsCCA/XCAT, SCIRun, CXML Other SystemsCondor DAGMan, UNICORE, USC Grid

failure handling


GEMSS Objectives Demonstrate that the grid can improve pre-operative planning &

near- real-time surgical support by providing access to advanced simulation and image-processing services.

Build middleware on existing or developing grid technology standards to provide support for authorization, workflow, security and Quality of Service aspects.

Develop, evaluate and validate a test-bed for the GEMSS system, including its deployment in the end-user’s working environment.

Anticipate privacy, security and other legal concerns by examining and incorporating into its grid services the latest laws and EU regulations related to providing medical services over the Internet.


Medical Applications

Pre- surgery Maxillo-facial surgery Post- surgery

Medical simulation supports the optimization of operation procedures and the planning of therapeutic strategies.


Image Pre-processing

An intensity based algorithm (adaptive fuzzy C-means algorithm) is used to provide good quality segmentations for structures of the human head.

CT Images Identification of substructures


Numerical Modeling

Next to the image processing step follows the geometric modeling of the structures suitable for Finite Element simulations.

Mesh manipulation using the halo positioning tool

Mesh generation withOr without smoothing


HPC Simulation & Visualization

The geometric face change Calculated deformation of the skull


The Problem …

The application includes the use of a complete chain of tools necessary for the entire process from geometric model generation from scan data (segmentation, mesh generation and mesh manipulation) to computer simulation and visualisation.


Application Workflow

Image Segmentation

Mesh Generation

Mesh Manipulation

Finite Element Simulation

Visualization

Computation intensive / Semi interactive

Computation intensive / parallel

Full interactive

Computation intensive / parallel

Data intensive / Full interactive


Grid Enabling …

Interface definition of each module using the WSDL

Implementing each module as a web service using Apache Axis or a grid service using the GT3

Definition of the whole process using the BPEL4WS

Using a BPEL4WS engine for service invocation

Applied only to Computation intensive parts


Challenges

Could you finish the process in 1 hours? – QoS and adaptation support is becoming the most active research topic in grid computing community.

Service performance prediction Workflow simulation and scheduling Dynamic service federation


Grid Performance Services

Building performance services as high-level grid services based on OGSA core and base services:

Managing performance-related data Defining performance metrics Developing performance analysis

algorithms Developing new APIs for grid service

performance prediction


Performance-related Data

Application parameters that have an impact on application performance

System status e.g. CPU load, job queue and network bandwidth

Managing performance-related data using OGSA service data support


Performance Metrics & Analysis Algorithms

Performance metrics Execution time Memory usage Price, and more …Performance analysis – historic

information based Statistical analysis algorithms Self organizing mapping More …


Workflow Simulation, Scheduling & Rescheduling

S2

startT = 0exeT = 3endT = 3

S1


S3


S4

startT = 5exeT = 7

endT = 12

S5


S6

startT = 12exeT = 0

endT = 12

/ 7/ 12

S2

S1

S3

S4

S5

S6

/ 5/ 5


Dynamic Service Federation

Using the BPEL4WS service references to select and assign actual partner service dynamically.

Extending BPEL4WS <partner> with some kind of <candidate> element to indicate candidates of a grid service partner.


The Solution … Grid Service (WSDL)

Performance Services

Service data

Application portTypes

WSDL

WorkflowSimulation Engine

WorkflowExecution Engine

Grid Service (WSDL)

Performance Services

Service data

Application portTypes

UDDI

User


Other Aspects Accuracy of performance prediction Workflow execution monitoring Security and legal issues Grid workflow GUIs Grid data management Enabling interactive applications Using workflow techniques for business

process management in GEMSS


Summary Programming models for the grid –

workflow specification – a candidate? QoS support and application adaptability –

performance prediction + workflow simulation –> dynamic service federation – a solution?

Medical simulation applications – a right target application of the grid?

AgileGrid: agile computing on business grids – a next generation computing paradigm?

NEC, Tokyo, Japan, May 2003 Use of Workflow Techniques for Grid Management Junwei Cao ( 曹军威 )...

Documents

Transcript of NEC, Tokyo, Japan, May 2003 Use of Workflow Techniques for Grid Management Junwei Cao ( 曹军威 )...