APPLICATIONS OF SOA AND WEB SERVICES IN GRID

COMPUTINGThanh H. Pham – Quynh Le

19 May 2010

2

31.900.000

3

• Service-oriented Architecture (SOA)1

• Web Services (WS)2

• Open Grid Service Architecture (OGSA)3

• Web Services Resource Framework (WSRF)4

Today’s Overview

4

Objectives

• SOA – An overview• WS & Related Technologies• OSGA & WSRF Overview• Applications of

SOA and WSin Grid Computing

5

SERVICE-ORIENTED ARCHITECTURE

6Image from Dr. Dang Tran Khanh slide

A Case Study

7

Key terms[1]

• An architecture:–Formal description of a system–Defines system purpose, functions–Defines system externally visible

properties, and interfaces.

[1] OGSA Glossary of Terms v1.0: http://www.gridforum.org/documents/GFD.44.pdf

8

Key terms (cont.)

• A service–Is a software component –Can be accessed via a network–Provides functionality to a requester

9

Key terms (cont.)• Service-oriented architecture refers to a

style of building reliable distributed systems that deliver functionality as loose coupling services

• Technically, then, the term SOA refers to the design of a system, not to its implementation

10

Services Characteristics

• May be individual useful, or can be integrated

• Communicate with clients by exchanging messages

• Can participate in a workflow

11

Services Characteristics (cont.)

• May be completely self-contained, or may depend on other services, resources

• Advertise details such as their capabilities, interfaces, policies, and supported communications protocols

• How’s about implementation details? Programing language, hosting platform…

12

Service interaction

13

A Real Life Example

14

Loose Coupling

• This term implies that the interacting software components minimize their in-built knowledge of each other

• They discover the information they need at the time they need it.

15

State and Statelessness

• Client can choose to go to any service that is capable of fulfilling its need: loose coupling

• Loose coupling vs. tight coupling

• Ex: A calculator service

16

Stateless vs. Stateful• Ex: a transaction that requires several steps

• The service retains some information (“state”) about each step

• This might result in a delay

17

Stateless Service Design• Service must hand back state information to

client

• Client must hand the state information to whichever service it selects to process the next step

• The selected service must be able to accept and handle the state information

18

A multi-step client/service interaction

19

WEB SERVICES

20

A Case Study

21

Step 1: A customer chooses to pay with a Credit Card.

22

Step 2: They enter shipping info on your site

23

Step 3: They enter their credit card details

24

Step 4: They review their payment info

25

Step 5: Their payment is processed.

26

Step 6: You get paid. (The fun part.)

27

1

2 3

As easy as

Right?

2828

What is Web Service?• Web Service Essential

Book: A web service is any

service that is available over the Internet, uses a standardized XML messaging system, and is not tied to any one operating system or programming language

XMLXML

Computer A:Language: Perl

OS: Windows 2000

Computer B:Language: Java

OS: Linux

A basic web service

2929

What is Web Service? (cont.)• W3C:

– A software system designed to support interoperable machine-to-machine interaction over a network

• W3schools.com:– Application components– Communicate using open protocols– Self-contained and self-describing– Can be used by other applications

30

Key specifications used by Web services

• XML: eXtensible Markup Language

• SOAP: Simple Object Access Protocol

• WSDL : Web Services Description Language

• UDDI: Universal Description Discovery and Integration

31

Key specifications used by Web services

• XML: eXtensible Markup Language

• SOAP: Simple Object Access Protocol

• WSDL : Web Services Description Language

• UDDI: Universal Description Discovery and Integration

3232

About…

• XML does not do anything• With XML, you invent your own tags• XML just plain text

<note> <to>Tove</to> <from>Jani</from> <heading>Reminder</heading> <body>Don't forget me this weekend!</body></note>

33

SOAP• Protocol for messaging and remote

procedure calling between distributed applications

• SOAP is for exchange information between applications

• SOAP is a format for sending messages

34

SOAP Message Structure

3535

A SOAP request• POST /InStock HTTP/1.1Host: www.example.orgContent-Type: application/soap+xml;

<?xml version="1.0"?><soap:Envelope><soap:Body xmlns:m="http://www.example.org/stock">  <m:GetStockPrice>    <m:StockName>IBM</m:StockName>  </m:GetStockPrice></soap:Body></soap:Envelope>

3636

The SOAP response• HTTP/1.1 200 OKContent-Type: application/soap+xml;

<?xml version="1.0"?><soap:Envelope><soap:Body xmlns:m="http://www.example.org/stock">  <m:GetStockPriceResponse>

    <m:Price>34.5</m:Price>  </m:GetStockPriceResponse></soap:Body></soap:Envelope>

37

WSDL

• Used to define Web Services and describe how to access them

• Describes location of the service, the function calls

38

UDDI

39

SOA AND WEB SERVICES: STYLE VS. IMPLEMENTATION

40

SOA Is Not Web Services

• Service-orientation is an architectural style

• Web services are an implementation technology

• The two can be used together, and they frequently are, but they are not mutually dependent.

41

Demonstration

OPEN GRID SERVICE ARCHITECTURE (OGSA)

Abstract

• The Open Grid Services Architecture (OGSA), set to become the standard architecture for most grid applications, depends on Web Services as the underlying middleware.

• OGSA first spawned the Open Grid Services Infrastructure which, despite improving Web Services in several ways, failed to converge with existing Web Services standards.

• The Web Services Resource Framework (WSRF), introduced last January, improves on OGSI and will eventually replace it. The presentation will cover the evolution and current state of OGSA, OGSI, WSRF, and the next version of the Globus Toolkit (GT4), which will be the first implementation of the WSRF specification.

44

Building an Open Grid

OpenStandards

OpenSource

OpenInfrastructure

OpenGrid

45

Grids and Open StandardsIn

crea

sed

func

tiona

lity,

stan

dard

izatio

n

Time

Customsolutions

OGSA

GGF: OGSI, …(+ OASIS, W3C)

Multiple implementations,including Globus Toolkit

Web services

Globus Toolkit

Defacto standardsGGF: GridFTP, GSI

X.509,LDAP,FTP, …

App-specificServices

46

OGSA Overview (1/2)

• OGSA = Open Grid Services Architecture Open: communicated extensibility, vendor neutrality. Services: OGSA is based on SOA manifested in WS Architecture: well-defined set of basic interfaces… Grid Services = Grid + WS; service semantics

• aims to standardize the different services that are commonly found in a Grid application Job management, security, VO management, data

management, workflow, deployment, etc.

47

OGSA Overview (2/2)

• Addressing the challenges of integrating services across distributed, heterogeneous, dynamic VO formed from the disparate resources

• OGSA was introduced in “Physiology of the Grid” (I.Foster, C.Kesselman, J.Nick, y S.Tuecke)

• The actual standardization process is being carried out by the Global Grid Forum (GGF) http://www.ggf.org/

48

OGSA Evolution

Autonomy ofthe Grid

Physiology ofthe Grid

OGSA V1.0

OGSA Glossary

OGSAProfile

Basic ExecutionService

Naming

JSDL

Open GridServices

Infrastructure

Web ServicesResource

Framework

Web ServicesNotification

informational

Normative

49

What does OGSA define?

• “A set of core interfaces and behaviours that address key concerns in Grid systems.”– Each particular service can be accessed in the same fashion,

regardless of vendor, organization, internal implementation, etc.

• What underlies this?– How are the interfaces defined? – How does a client request a particular operation from a

service?

• OGSA relies on (&aims to improve) WS technologies– Broader context: Service-Oriented Architectures (SOA)– WS & GS

50

Web Services (1)

• A simple but powerful distributed system paradigm, that allows one to:– Describe a service (WSDL)– Invoke a service (SOAP)– Discover a service (various)

• What sets it apart?– Clean separation between interface (what the service does)

and implementation (how it does it)– Based on standard interoperable languages (XML)– Widespread use and abundant software available

• We can use web services to build SOAs, improving interoperability and reusability.

Web Services (2)

• Why are WS ideal for OGSA and for Grid systems?– Can easily cope with heterogeneous systems (different

programming languages, platforms, etc.)– Common interface language (WSDL) allows:

• Virtualization: from a pool of services (with the same interface) I can access any service in the same fashion regardless of vendor, platform, etc.

• Dynamic service indexing and discovery.• Dynamic access to services

51

Grid Services (1)

• Despite being the best option for OGSA, WSs still have important disadvantages:– Stateless (…)– Non-transient ('Persistent')– No supporting services (notifications, lifecycle

management, etc.)

• OGSA defines the concept of a Grid Service to overcome these limitations:– Grid Service = improved Web Service– Grid Services are compatible with WS– Note that Grids need access to stateful resource

52

Grid Services (2)• Grid services

– can maintain internal state for the lifetime of the service– can be created and destroyed dynamically (via a Factory)– have a globally unique name, the Grid service handle (GSH)– each instance has a set of service data elements (a

collection of XML elements)

• OGSA is based on Grid Services– Grid Services are specified in a separate spec spawned by OGSA: Open Grid Services Infrastructure (OGSI).– All the services in OGSA (Job management, security, etc.) will be based on (and implemented with) Grid Services

53

Grid Services (3)

54

55

OGSA Model: At a Glance

GT3 capabilities – e.g. GRMS, GridFTP, GRAM..

GT3 Security

Web services

OGSA – Grid Enabled Web Services

Other services – data services etc

56

The OGSA Platform

OGSI

TransportProtocolHosting EnvironmentHosting Environment

Host. Env. & Protocol Bindings

OGSA Platform services: registry,authorization, monitoring, data

access, etc., etc.

More specialized &domain-specific

services

Models for resources &

other entitiesO

therm

odels

Environment-specificprofiles

Domain-specificprofiles

OGSAPlatform

57

OGSI (1)

Web Service Interface (WSDL)

SOAPServer

Service Implementation

Web Services

A Grid ServiceGWSDL

Extends ..

58

OGSI (2)

Implementation

Hosting environment/runtime(“C”, J2EE, .NET, …)

Dataaccess

Grid ServiceHandle

Grid ServiceReference

handleresolution

Servicedata

element

Servicedata

element

GridService(required)

Introspection:• What port types?• What policy?• What state?

Client

Servicedata

element

Other standard interfaces:factory,

notification,collections

59

60

Globus Toolkit 3 (1)

• The Globus Toolkit is probably the best-known software package for Grid programming

• However, there seems to be an ongoing confusion regarding the relationship between GT3 and OGSA + OGSI...– GT3 == OGSA?– GT3 == OGSI?– Is GT3 a magical toolkit which will allow me to program

Grid systems out-of-the-box?

61

Globus Toolkit 3 (2)

• The Globus Toolkit 3 (GT3) is a complete implementation, in Java, of the OGSI spec

• Very important: GT3 isn't only an OGSI implementation.– Includes OGSA-defined services– Includes its own non-OGSA services (which have become

a de facto standard)– Also includes non-WS components (GT2.4)

62

GT3 Architecture

63

Non-GT3 services based on the GT3 architecture

Management of large volumes of data (Replica Management)

MMJFS, MDS, RFT

GSI (Grid Security Infrastructure), SSL, WS-Security, SOAP, ...

OGSI: Grid Services, Service Data, Notifications, Transient Services (Factory/Instance), Service Groups

OGSA, OGSI, and GT3(1)

64

OGSA, OGSI, and GT3 (2)

65

In standardization process (GGF)- VO management- Security- Job management- Data management- Workflow- etc.

Standardized (GGF) and implemented (GT3)

Standardized (W3C) and implemented (Apache Axis)

OGSA: summing up

• OGSA: standards-based Grid technology– From Web services: standard IDL, discovery, binding

independence, other desirable features– From Grid: naming, state, lifetime management, etc., etc.

66

WEB SERVICES RESOURCE FRAMEWORK (WSRF)

68

Grid and WS: Convergence?

• Despite being generally accepted as a good change within the Grid community, OGSI has not caught the eye of the Web Service community.– GGF had hoped that Web Services standards and OGSI

would converge, but that convergence is not happening.

Started far apart in apps & tech

OGSI

GT2GT1

HTTPWSDL,

WS-*

WSDL 2,

WSDM

Have beenconverging ?

OGSA Review

69

Web Services Messaging, Security, Etc.

Open Grid Services Infrastructure

Domain-Specific Services

Core Services

ProgramExecution Data Services

Sta

ndar

diza

tion

70

Three Major WS Concerns about OGSI

• “Dense specification”• “Does not work well with existing WS tooling”

– Extended WSDL & therefore not compatible with other Web service standards ..

– Grid services need to be stateful and dynamic– Therefore, WSDL had to be extended (how) to add state &

notification• Too “object oriented”

– Web Services are not supposed to be object-oriented, despite the fact that most Web Services are implemented using object-oriented languages. This specially refers to the fact that OGSI Services are stateful (Web Services are supposed to be stateless)

– Grid services extended Web services – means that Grid services are ON THEIR OWN…

WSRF (1)

• To achieve Web Service & Grid Service convergence, a new standard was announced during GlobusWORLD 2004 (January 2004)

• This new standard will supersede OGSI• WSRF – Web Services Resource Framework• OASIS, from 2006

71

WSRF (2)

72

• The definition of WSRF means that Grid and Web communities can move forward on a common base

WSRF (3)

73

•OGSA Services can be defined and implemented asWeb services instead of working w/ a diff “improved version” of WS

•OSGA can take advantage of other Web services standards

•OGSA can be implemented using standard Web services development tools

•Grid applications will NOT require special Web services infrastructure

• WSRF would be part of the WS standard, instead of being a “patch” over existing one (like OGSI)

WSRF (4)

74

Web Services Messaging, Security, Etc.

Open Grid Services Infrastructure

Domain-Specific Services

Core Services

ProgramExecution Data Services

Stan

dard

izatio

n

WS-Resource Framework

WSRF (5)• WS-Resource: Web Service + stateful resource

– Put state in the resource, not WS, each has a unique key– instruct the WS to use a particular resource, if we want a stateful

interaction with a WS– Pairing a WS with a resource is called a WS-Resource

• WS-Resource is a construct used to model stateful resources using WS architectural framework

• Unique (for the Web Service) - Has a distinguishable identity and lifetime

• Stateful - Maintains a specific state that can be materialized using XML

• May be accessed through one or more Web Services

75

Example of stateful Web services

76

Example of stateful Web services

77

Note: There are three resources, and the service interface is instructed to use Resource C

78

79

WSRF specification• WSRF is a collection of four different specifications

related to the management of WS-Resource1. WS-ResourceProperties – Supplies a set of interfaces

that will allow us to access, modify, and query resource properties

2. WS-ResourceLifetime: Resources have non-trivial life cycles. In other words, they are not static entity that is created when our server stops. Resources can be created and destroyed at anytime. The WS-ResourceLifetime supplies some basic mechanisms to manage lifecycle of resources.

80

WSRF specification (2)

• WSRF is a collection of four different specifications related to the management of WS-Resource3. WS-ServiceGroup – This is useful in managing groups of

WS or group of WS-Resources and performing operations such as “add new service to group”, “remove this service from group”, and (more importantly), “find a service in the group that meets the condition FOOBAR”, very basic but is the base of more powerful discovery services (such as GT4’s Index Service)

4. WS-BaseFaults – Aims to provide a standard way of representing faults when something goes wrong during WS service invocation

81

Related Specifications

1. WS-Notification – This is another collection of specification that, although not part of a WSRF, is closely related to it.

2. WS-Addressing: Provides mechanism for addressing WS. More versatile than URI. Can be used to address WS-Resource pair

82

83

Concerns Addressed

How WSRF deals with OGSI's problems:* “Dense specification” WSRF is divided in five documents plus a complementary

specification (WS-Notification) * “Doesn’t work well w/ existing WS tooling” WSRF harmonizes with WS: less aggressive use of XML

Schema, use of pure WSDL 1.1 for interface description (instead of GWSDL, a 'patched' version of WSDL fit for OGSI)

* “Too object oriented” WSRF makes an explicit distinction between the “service”

and the stateful “resources” acted upon by that service

Relation from WSRF to ...

84

• OGSA: WSRF mechanisms will enable OGSA• OGSI: WSRF restates OGSI concepts in WS terms

OGSI WSRF

Grid Service Reference (GSR) WS-Addressing Endpoint Reference

Grid Service Handle (GSH) WS-Addressing Endpoint Reference

HandleResolver portType WS-RenewableReferences

Service data elements (SDE) WS-ResourceProperties

GridService lifetime managementt

WS-ResourceLifeCycle

Notification portTypes WS-Notification

Factory portType Treated as a pattern

ServiceGroup portTypes WS-ServiceGroup

Base fault type WS-BaseFaults

WSRF and GT4

85

WSRF – summing up• Why is WSRF important?

– WSRF completes Grid/Web convergence

• How does WSRF relate to OGSI?– WSRF restates OGSI concepts in WS terms

• How does WSRF relate to OGSA?– WSRF mechanisms will enable OGSA

• Why is WSRF more likely to succeed? – Harmonizes with Web Services– Backed by IBM and HP

• First, full, free implementation: GT4– IBM will include support for WSRF in some of its tools.

• More WSRF details: – http://www.globus.org/wsrf

86

Summary

• “Grid Computing” is undergoing an intense standardization process.– OGSA: Still work-in-progress (some parts finished)– OGSI: Promising start, but didn't quite make it– WSRF: Will it achieve true Grid/Web convergence?

• Until OGSA is truly stable, the Globus Toolkit remains the de facto standard, and one of the most important pieces of software in Grid system development.

87

88

Resources[1] The Physiology of the Grid, Ian Foster, Carl Kesselman, Jeffrey M. Nick, Steven Tuecke, 2002[2] Towards a service-oriented Grid, Borja Sotomayor, University of Deusto, 2004[3] SOA, Web Service In Grid Computing, Phạm Hùng Tiến, Đặng Hoài Đức, K2008[4] Open Grid Services Architecture, Ian Foster, University of Chicago, 2003[5] WS-Resource Framework: Globus Alliance Perspectives, Ian Foster, University of Chicago, 2003[6] OGSA, WSRF, and the Foundations of Grid, David Snelling, Fujitsu Laboratories of Europe, 2005[7] Global Globus Forum http://www.globus.org[8] IBM: http://www.ibm.com/developerworks/library/ws-resource/

89

Q & A

Phạm Hồng Thanhthanhphh@gmail.com

Lê H.N. Quỳnhlhnquynh@gmail.com

90

THANK YOU!

91

APPENDIX

WSRF 101: Stateful Resource• A Resource:

– A specific set of state data expressible as an XML document• This is not typically all of the resource’s state!

– Has a well-defined identity and lifecycle – Known to, and acted upon, by one or more Web services.

• Many Possible Instances– Files, Database tables, EJB Entities, XML documents, Compositions of

multiple data sources, Virtualized executions of applications, etc.• A WS-Resource has:

– Identity: Can be uniquely identified/referenced– Lifetime: Often created & destroyed by clients– State: Part of the state can be projected as XML– Type: Its Web service interface

92

Lifetime• Defines:

– Immediate, synchronous destruction operation– Time-based, scheduled destruction operation– “Soft-state” or “leased” lifetime management– Termination time not required to increase monotonically

• Resource properties:– CurrentTime: Can be used to determine clock skew– TerminationTime: Current scheduled termination time

• Notification of resource termination• Why:

– Define clear means by which resources can be destroyed– Allow the Grid to “Garbage Collect” itself automatically

93

Notification

• WS-Notification– Brings publish and subscribe messaging to Web services– Loosely coupled, asynchronous messaging in WSs– WS Notification composes with WSRF and other WSs

• Brokered notification– Support for intermediates, queuing, aggregation, distribution,

filtering, ...

• Topics and Topic Spaces– Define a mechanism to advertise topics for subscription.

• Use of WS-Notification in WS-RF– Receive notification changes to the Resource Properties

94

WSDL: Web Services Description Language

Define expected messages for a service, and their (input or output parameters)

An interface groups together a number of messages (operations)

Bind an Interface via a definition to a specific transport (e.g. HTTP) and messaging (e.g. SOAP) protocol

The network location where the service is implemented , e.g. http://localhost:8080

Web Services Server Applications• Web service – software that exposes a

set of operations• SOAP Engine – handle SOAP requests

and responses (Apache Axis)• Application Server – provides “living

space” for applications that must be accessed by different clients (Tomcat)

• HTTP server- also called a Web server, handles http messages

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s02.html

Container

Let’s talk about state

• Plain Web services are stateless

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html

However, Many GridApplications Require State

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html

Keep the Web Serviceand the State Separate

• Instead of putting state in a Web service, we keep it in a resource

• Each resource has a unique key

Borja Sotomayor , http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s03.html

Web Service+

Resource=

WS-Resource

Address of a WS-resource is called an end-point reference

Resources Can Be Anything Stored

“Stateless” vs. “Stateful” Services

• Without state, how does client:– Determine what happened (success/failure)?– Find out how many files completed?– Receive updates when interesting events arise?– Terminate a request?

• Few useful services are truly “stateless”, but WS interfaces alone do not provide built-in support for state

Client

FileTransferService

move (A to B)move

GT3/GT4 Evolution

102

Although Grid services are implemented using WS technology, the following are the differences btw them

Grid services vs. Web Services

Web service• Addresses the issue of discovery and invocation of persistent services

•A WS is often presumed available if its corresponding WSDL file is accessible to its client

•WS typically outlive all their clients

Grid service• Addresses the issue of a virtual resource and its state management

•A grid is a dynamic environment hence, Grid services can be transient rather than persistent

•A Grid service can be dynamically created and destroyed.

104

OGSA Specs LandscapeSYSTEMSMANAGEMENT

UTILITYCOMPUTING

GRIDCOMPUTING

Core Services

Basic Profile WS-Addressing

Privacy

WSRF-RAP

WSDM

WS-Security

Naming

OGSA-EMSOGSA Self Mgmt

Others ...

GGF-UR Data Model

HTTP(S)/SOAP

Discovery

SAML/XACML

WSDL WSRF-RL

Trust

WS-DAI

VO Management

Information

Distributed query processing

ASP

Data CentreUse Cases &Applications Collaboration Multi MediaPersistent Archive

WSRF-RP

X.509

NotificationService Groups WS-I BP