Porting applications to Globus Toolkit 3.0 and designing an OGSI-based architecture Pawel Plaszczak...

Porting applications to Globus Toolkit 3.0 and

designing an OGSI-based architecture

Pawel Plaszczak and Rich WellnerPerfringo Grid Solutions

http://perfringo.com

Tutorial materials: http://perfringo.com/events

Porting to GT3 & designing OGSI - Perfringo Grid Solutions http://perfringo.com

About the authors

• From Foster’s Globus team in Argonne• Co-founded Perfringo Grid Solutions• Focus: industry level Grid applications• Currently: NeesGRID - distributed

earthquake simulations• [email protected],

[email protected]


Prerequisities

• Necessary: distributed systems & software design, purpose of Grid

• Useful: Java, Web services, XML, Grid services (will be briefly explained)

• Not necessary: coding experience• We will speak on decision making

and design. No hands-on programming


Plan

• Part I: Grid services - why and when?• Part II: OGSI and OGSA technology

review• Part III: Use case: NTCP • Part IV: Porting to Grid services and

GT3


Plan of Part I: Grid services - why and

when?

• Overview: Globus Alliance and Toolkit• Overview: GT2 vs GT3, OGSA• Why go Grid?• Why comply with OGSA?• Why port to GT3?


The Globus Alliance

• Globus Project ™, since 1996 Ian Foster (Argonne National Lab), Carl Kesselman (University of Southern

California’s Information Science Institute)

• Develop protocols, middleware and tools for Grid computing

• Globus Alliance, since Sept 2003• International scope

University of Edinburgh’s EPCC Swedish Center for Parallel Computers (PDC) Advisory council of Academic Affiliates from

Asia-Pacific, Europe, US


Globus Toolkit

• GT2 (2.4 released in 2002): reference implementation of Grid fabric protocols GRAM for job submissions MDS for resource discovery GridFTP for data transfer GSI security

• GT3 (3.0 released July 2003): redesign OGSI based Grid services, built on SOAP and XML


OGSI and OGSA

• OGSI = Open Grid Service Infrastructure Specs from GGF OGSI working group Defines what makes a Grid service Based on Web service Naming, life cycle, state, notification portTypes definitions, WSDL 1.2 draft

• OGSA = Open Grid Service Architecture Specs from GGF OGSA working group Defines a list of fundamental Grid services, and

how they cooperate Work in progress


What’s new in GT3.0

• Emphasis on OGSI compliant Grid services

• OGSA implemented• Uniform SOAP/XML communication

layer• Focus on Java implementation• Old GT2 services redesigned• Backward compatibility where

possible


Meaning of GT3 in the community

• Most commonly referred project• Traditionally “de facto standard”• Acknowledged leadership by academia

and industry (IBM,…)• BSD style license allows for commercial

usage• However, it is only a reference

implementation. Now standards = GGF• GT undergoes constant changes• With business entering grid, commercial

implementations may soon catch up


Alternatives to GT3

• Protocol level interoperability: “Grid compliant” >= “implements OGSI”

• Other OGSI implementations OGSI.NET (U.Virginia) pyGlobus (LBNL) .NET (U.Edinburgh) PERL (U. Manchester) UNICORE (Fujitsu)

• Commercial OGSI compliant products by: Avaki, Platform, Data Synapse, …

• Web service alternative: Grid App Framework http://www.neresc.ac.uk/ws-gaf


Part I contd.

Overview: Globus Alliance and ToolkitOverview: GT2 vs GT3, OGSA

Why?• Why go Grid?• Why comply with OGSA?• Why port to GT3?• upgrading from GT2


Why go Grid?

• Hot subject• Try it, experience it to learn the potential• Will enable true ubiquitous computing in

future• Today, proven in some areas: intraGrids• But still long way to World Wide Grid• State of art techniques, tools are difficult• Short term goals? Use another technology• Does your system have Grid characteristics?

Distributed users, large scale and heterogeneous resources, across domains


OGSI architecture: why and when?

• Commonly accepted Grid standard But there are alternatives: WS-GAF, GT2,…

• Extends Service-Oriented Architecture Modular, scalable, maintainable, interoperable

• Extensions: use only when needed• Uniform communication layer• Be compatible and inter-operable with

other grid systems (that’s what grids are about!)


Porting to GT3: why and when?

• Implementing OGSI is plenty of work!• Piggyback on GT3 for OGSI/A compliance

(watch commercial products)• Most serious middleware to consider• Language bindings: traditionally Unix/C,

now Java leads• But: unstable, new versions bring changes• Traditionally, for hard core hackers• Traditionally, documentation was lacking

recent signs of improvement, also IBM redbook


GT3 stability

• GT3 took only 1.5 year to develop. It will change many times!

• Anticipated or considered changes: Refactor GridFTP control channel into XML, DAI Security: add CASS GRAM: ongoing improvements OGSA-C hosting environment to come

• OGSA specs is draft• OGSI specs are completed (v. 1.0)


GT3.2

• Expected January 2004• Features:

CAS (Community Authorization Service) Index Service reimplemented GridFTP server reimplemented GridFTP java client: connection reuse, MLST GRAM performance improvements CRL support (certificate revocation list) several bug fixes in core


Java CoG Kit status

• Before: Java client for GT2• Now integrated with GT3• Provides low level API for GT3

services: security, GridFTP• Experimental features to come soon

(not part of GT3): Workflow management, compliant with

GT2 and GT3 Comes with example Jetspeed portlets


roadmap

Part I: Grid services - why and when?• Part II: OGSI and OGSA technology


GT3


Plan of Part II: OGSI and OGSA technology review

• Web service• XML and SOAP• Service Oriented Architecture• Grid service (OGSI)• OGSA• How does GT3 implement OGSA• Examples


What is a Web service

• Not much to do with HTML

• Interface to a well-explained, self-contained function (module, component)

• Usu. small grained• Defined in WSDL• Communication: XML

and SOAP

OnlineStoreWeb service

queryItems()orderItem()

Web serviceclient

SOAP/XML


More on Web services

• Specs W3 Consortium www.w3.org, IBM, Microsoft OASIS (WS security)

• For developers: “Building Web services with Java” by Steve Graham et al

• For managers: “Web services and Service-Oriented Architectures” by Doug Barry (the savvy manager’s guide)

• Stay tuned: IBM developerWorks articles http://www-106.ibm.com/developerworks/

webservices/


Service-Oriented Architecture

• System composed of several components exhibiting WS interfaces

• Discovery through a well-known directory, such as UDDI

• External services use internal ones• Modular, scalable, maintainable,

interoperable


Service-Oriented Architecture in practice

OnlineStore WS

queryItems()orderItem()

Web serviceclient

StockMgr WS

query()ship()

OrderMgr WS

order()bill()

query order

ship

warehouse


What is a Grid service

• Defined by OGSI (GGF working group)• Is a Web service with additions, which are:

Name (handle GSH, reference GSR) Lifetime management (factories, persistent

and transient services) State (Service Data) Notification as well as querying

• WSDL 1.2 draft (gwsdl: namespace)• Definitions of portTypes


Grid services in practice: notifications and service data

OnlineStore GS

query()subscribe()

Grid serviceclient

Subscribe:“tell me when youhave it on stock”

Notify:“it just arrived.”

Notification source

Notification sink

StockItem SDE

Service data element


Grid services in practice: naming and life cycle

OnlineStore Persistent GS

queryItems()

StockMgr Persistent GS

getItem()

StockItemAF45X Transient GS

Grid serviceclient

Item SDEs: dimension, price

3. create

Factory

2. Get reference

1. query

4. query


What is OGSA

• Open Grid Service Architecture, being defined by GGF OGSA working group

• In ubiquitous Grid platform, there is common need for some basic services

• OGSA defines this set of base services

• OGSA builds on OGSI


How does GT3 implement OGSA

• Java implementation, C to follow• Container approach: host small-grained

services • Hosting environment takes care of basic

GS functions• GT3 provides core and base services,

users to implement their own• Transient and persistent GS API• Tools to generate Java <-> GWSDL• Axis engine used for SOAP handling• Run with Tomcat or standalone


roadmap

Part I: Grid services - why and when?Part II: OGSI and OGSA technology


GT3


Plan of part III: NTCP

• NeesGRID project overview• NTCP requirements & high level design• Choosing Grid services for NTCP

subsystem• Choosing Grid services for Matlab

subsystem• Not choosing Grid services for

ShoreWestern subsystem• Deployment• On-going work


NeesGRID

• Make collective use of geographically distributed experiment facilities, heterogeneous hardware

• Users dislocated from facilities• Applying Grid to earthquake science• 3 year project involving several US

institutes• Distributed simulation of a building in an

earthquake• Scale: models costs millions of $ and

several months to build. Some experiments are not repeatable (model used once)


The MOST experiment

gx&&

NCSA Computational Model

m1

f1

UIUC

Experimental Model

gx&&

f1

m1

f2f2

U. Colorado

Experimental Model

gx&&

Slide courtesy of Bill Spencer, UIUC


MOST requirements

• Distributed simulation of a building in an earthquake

• 3 experimental sites• Hybrid experiment: 2 physical and 1

computational simulations• Results at one time-step used to calculate inputs

at the next time-step• Pseudodynamics: order of 10 ms per time-step• Successfully run on 30th July 2003


NTCP: NEESGrid Teleoperation Control

Protocol

• Vital component of NeesGRID and MOST

• Telecontrol system, used to coordinate the experiment with commands

• Output data collected with other NEESGrid systems like NSDS


NTCP system architecture

Coordinator(NTCP client)

NTCPserver

NTCPserver

NTCPserver

Backend ShoreWestern

Backend Matlab sim.

Backend Matlab + XPC

Localprotocol

Localprotocol

Localprotocol

NTCP NTCP NTCP

NTCP subsystem

local subsystems

NCSA U ColoradoUIUC


NTCP protocol

• Coordinator follows the script with recorded earthquake data

• Coordinator orders servers to simulate one time-step at a time (pseudodynamics)

• Servers can refuse if parameters out of range

• Core commands Propose() negotiates an action Execute() starts an action Cancel() issued when negotiation failed Other: get/setParameter, open/closeSession…


Sequence diagram (simplified)

coordinatorNTCPServer A

openSession

NTCPServer B

propose

execute

results

(accepted)

(started)

openSession

proposeexecution

1 time step


NTCP subsystem: choosing Grid services and

GT3

• Long term goals• System characteristics: distributed,

heterogeneous resources across administrative domains

• Interoperability, manageability, scalability

• Risk: GT3 released early July, MOST scheduled end of July


NTCP server simplified design


NTCP server as a Grid service

• Front end: Permanent Grid service Operates on real hardware; only admin can

restart

• methods Open/closeSession, propose, execute, cancel,

get/setParameter, getControlPoint

• Service data: Transaction state• Notification: Execution results• Back end: LocalControlPlugin interface for

implementing adapters to the local non-OGSI technologies


Sequence diagram revisited in Grid service terms

Coordinator(GS client)

NTCPGrid service

Subscribe for transaction status

Propose transaction

Execute transaction

results

(accepted)

(started)

execution

1 time step

Transactionresults notification

TransactionService Data

create

update


Matlab subsystem: plugin and backend


NTCPserver

NTCPserver

NTCPserver


Backend Matlab sim.


Localprotocol

Localprotocol

NTCP NTCP NTCP

Localprotocol


Matlab subsystem: characteristics

• Command flow:1. Coordinator2. NTCP server frontend3. NTCP server Matlab control plugin (Mplugin)4. Backend machine running Matlab simulation

• Matlab single threaded, not event driven• Information pull model enforced• Matlab has Java interface, fully compliant with Java 1.3. Can

use Java class library, third party jar files, multiple threads


Matlab subsystem: choosing Grid services

• Choose network communication technology between Mplugin and backend TCP/IP, RMI, Corba, Web services…

• Although no Grid characteristics… why not Grid services? Pulling scenario -> “server” on the same

machine as the NTCP server OGSA container is already there Ockham razor: one technology instead of many


Typical Service-Oriented Arch compared to NTCP + Matlab


Client, server and backendin Grid service model

• Methods: hasCommand, pullCommand, pushResult

• No service data, notification• Could as well be a Web service • Reversed information flow, as

compared to typical Service-Oriented• Both NTCP and commandBuffer

services are persistent; must match at start-up


Testing the NTCP and Matlab subsystem

• Junit and TestServer from GT3• 3 test suites, 4 test units• Some units implement same interfaces• 3 simulators, some deployed inside gar


ShoreWestern subsystem


NTCPserver

NTCPserver

NTCPserver


Backend Matlab sim.


Localprotocol

Localprotocol

NTCP NTCP NTCP

Localprotocol


ShoreWestern subsystem:plugin and backend

• ShoreWestern hardware driver to operate physical experiment facility

• Push communication OK• ShoreWestern already speaks a

proprietary protocol based on TCP/IP, equivalent in functionality to NTCP


TCP/IP based prototype of NTCP reused for ShoreWestern

Coordinator

Backend sim.ShoreWestern

Backend sim. Matlab

Backend sim. Matlab

TCP/IP

NTCP server

ShoreWesternbackend

TCP/IP

TCP/IP

TCP/IP

Grid service Front end

ShoreWestern ctrl plugin

Same protocol


ShoreWestern subsystem:*not* choosing Grid services

• Backend has server role• Reuse existing TCP/IP protocol• Ockham razor (again!)• ShoreWestern plugin = adapter to the old

protocol


Complete NTCP system for the MOST experiment


Backend e.g. ShoreWestern Backend e.g.

Matlab sim.

Backend e.g. Matlab + XPC

TCP/IP

OGSIOGSI

OGSA container

ShoreWestern ctrl plugin

OGSA container

NTCPGrid service

OGSA container

CmdBufferGrid service

CmdBufferGrid service

NTCPGrid service

NTCPGrid service

OGSI

OGSIOGSI


MOST experiment deployment

• First GT3-based application deployed in production

• 3 remote sites: NCSA, UIUC, University of Colorado

• 5.5 hours and almost 1500 steps• Monitored by several engineers and

scientists• Remote observers through advanced Grid

technology: multiple life video, life data, chat, all via Web interface

• Output data to be analyzed by earthquake scientists


NTCP: ongoing work

• Very high requirements Reliability Safety Error checking Security

• Involved: Information Science Institute, Argonne National Lab, Perfringo Grid Solutions


NTCP Acknowledgements

• Bill Spencer, UIUC (NeesGRID PI)• Laura Pearlman, ISI (NTCP lead)• Erik Johnson, USC (Matlab

engineering)• Ravi Madduri, Mike D’Arcy (Globus

team developers)


roadmap

Part I: Grid services - why and when?Part II: OGSI and OGSA technology

reviewPart III: Use case: NTCP • Part IV: Porting to Grid services and

GT3


Plan of Part IV: Porting to Grid services and GT3

• All depends on your starting point• Choose the best approach• High or low abstraction level?• Study early experience with working

systems• Stay tuned


All depends on your starting point

• Already have a GT2 based system? Porting GT2 -> GT3

• Already have Web services based system? Porting Web services -> Grid services

• Have a system in other technology?• Starting from scratch?


Porting GT2 -> GT3: GridFTP

• GT2 GridFTP is still there.

• Consider using Java client API from CoG (org.globus.ftp aka jftp)

• For 3rd party, consider 3-tier architecture with reliable file transfer (RFT)

• Uncertain long term protocol stability

GridFTPserver

GridFTPserver

jftpclient API

RFT

data

Grid serviceClient API


Porting GT2 -> GT3: GRAM

• GT2 GRAM now refactored into XML• Server side: new XML front end, old

backend submission scripts• New XML clients: Java or C• Client side API-level backward

compatible: old apps only need recompiling


Porting GT2 -> GT3: MDS

• GT3 Information Services, aka MDS3 fundamental change in technology

• Old GRIS: now Service Data• Old GIIS: now Index Service• Old LDAP/LDIF: now SOAP/XML• RFC2252 MDS schema: now SDD in

GWSDL• Porting old data providers

Substantial amount of work Generally, need an adapter Grid service


Porting GT2 -> GT3: the rest

• GT2 GSI security still there• New: GT3 allows creating services

not based on GridFTP, GRAM and MDS. Think about redesigning your architecture


Porting Web services to Grid services: theory

• Grid service *is* a Web service• Naming: need to introduce handle and

reference• Life cycle

Web services are like persistent Grid services

• Service data - not necessary Get/set methods can now be implemented as

SDE

• Notification - not necessary• Discovery:

GT3 uses Index Service as directory Web services sometimes use UDDI


Porting Web services to Grid services: your options

• Implement from scratch Need to implement (at least) handle and

reference management Can retain your old Web service code

• Implement with GT3 Get complete OGSI feature set for granted Hidden porting work

Is the WS already in Axis? Handler based approach WSDL -> GWSDL Different procedure to deploy in container GSI security (builds on top of WS-security) Dependencies, versioning…

• Use non-GT3 OGSI middleware


Experience of NEReSC CGM team: workflow service

Courtesy of Saras Parastatidis and Paul Watson,http://www.neresc.ac.uk/projects/CGM/web

Workflow Enactment Service

Workflow ID

Consumer

Submit workflow

Workflow Enactment Service

Completed

Consumer

Workflow status (+ workflow ID)

Factory

Locator

Consumer

Create new instance

Workflow Enactment Grid Service Instance

ok

Consumer

Submit workflow

Workflow Enactment Grid Service Instance

Completed

Consumer

Workflow Status

Web Service Grid Service


Courtesy of Saras Parastatidis and Paul Watson,http://www.neresc.ac.uk/projects/CGM/web

• Direct port of servlet proved too difficult Dependencies on mysql, OpenJMS, Tomcat Versioning nightmare

• Architecture converted to 3-tier• New Grid service adapter constructed• Adapter forwards to old Web service• Client adaptation costed time

Experience of NEReSC CGM team: notification service


Experience of NEReSC CGM team: conclusions

Courtesy of Saras Parastatidis and Paul Watson,

http://www.neresc.ac.uk/projects/CGM/web • Web and Grid services can coexist• Adapter pattern is one solution• Alternative to OGSI: Grid Application

Framework based on Web Services Specifications and Practices document (http://www.neresc.ac.uk/ws-gaf)


Choose the best approach

• Writing Grid services from scratch• Redesigning legacy system in Grid

service architecture• Refactoring communication layer• Façade pattern approach


High or low abstraction level

• Adapt to lowest necessary abstraction Web services? OGSI? (what features needed?) OGSA? (what services needed?)

• Reuse highest possible abstraction track GGF progress track Globus plans (announcements) Ongoing work: OGSA-DAI, OGSA-DQP,

workflows, OGSA itself, WS-Agreement


Data access & integration

• Global Grid Forum Data Access and Integration Services (DAIS) working group

• Reference implementation OGSA-DAI by Edinburgh Parallel Computing Center (EPCC)

• “middleware to assist with access and integration of data from separate data sources via the grid”

• First stage ended July 2003; software available: http://www.ogsadai.org.uk/


Hints

• Grid services can be small grained One idea for the next GridFTP server is to

represent each file as a Grid service Existence of a handle does not imply existence

of a service! It should apear as a service, but implementation can be custom

• Represent abstractions as services Agreement, transaction

• Method or SDE? Get/set functionality can be faster to implement

as methods, but more elegant as service data


Experience with working systems

• NeesGRID• Quertermaster• NEReSC Core Grid Middleware• other


Stay tuned

• Quarterly GGF: gridforum.org• Globus World, January 20-23 2004, San

Francisco• GridToday, May 2004 to focus on

commercial applications• Perfringo events

Training events, tutorials, newsletter: [email protected]

• Questions and comments? [email protected]


Your feedback

• Slides available for download http://perfringo.com/events

• Please send feedback to [email protected] What did you learn? What did you expect? What could be added / improved?


Thanks!

• Laura Pearlman (NeesGRID, Globus)• Bill Spencer (NeesGRID, UIUC)• Savas Parastatidis (NEReSC CGM)• Lisa Childers, Jarek Gawor, Gregor

von Laszewski (Globus team)

Porting applications to Globus Toolkit 3.0 and designing an OGSI-based architecture Pawel Plaszczak...

Documents

Transcript of Porting applications to Globus Toolkit 3.0 and designing an OGSI-based architecture Pawel Plaszczak...