CMS LNL OGSA INFRASTRUCTURE EVALUATION Specification Applications Giuliano Rorato INFN Laboratori...

CMS

LNL

OGSA INFRASTRUCTURE OGSA INFRASTRUCTURE EVALUATIONEVALUATION

Specification & ApplicationsSpecification & Applications

Giuliano RoratoGiuliano Rorato

INFN – Laboratori Nazionali di LegnaroINFN – Laboratori Nazionali di Legnaro

Giuliano Rorato - INFN - LNL 19/11/2002

2

CMS

LNLWhy OGSA

Our interest in OGSA (Open Grid Services Architecture) technology is about a possible application in the online software of CMS. In particular for RCMS.

OGSA: defines mechanisms for managing distribuited entities.

RCMS: distribuited services system for the run control of CMS.

OGSA framework

RCMS Services


3

CMS

LNLOpen Grid Services Architecture

OGSA: Grid system architecture based on an integration of GRID and Web Services concepts and technologies.

Initial proposed technical specification: Globus project and IBM

• Put forward at the Globus Grid Forum: for discussion, refinement, and eventual standardization

2002-2003: Globus Toolkit OGSA-compliant Globus Toolkit 3.0

Now there is the Preview Release 5

• www.globus.org/ogsa/releases/TechPreview/


4

CMS

LNLWeb Services vs Grid Services

WEB SERVICE: interface that describes a collection of operations that are network accessible through standardized XML messaging

WSDL (Web Services Description Language): language XML-based that describe the web service (accessible operations and location)

GRID SERVICE: web service that conforms to a set of conventions (interfaces and behaviors) that define how a client interacts with a Grid Service


5

CMS

LNLOGSA - Definition The Open Grid Services Architecture (OGSA) defines mechanisms for creating,

managing, and exchange information among entities called Grid Services

Grid Service instance: (persistent or transient) service that conforms to a set of conventions (expressed as WSDL interfaces, extensions, and behaviors) for such purposes as lifetime management, discovery of characteristics, notification, and so forth, described from a WSDL PortType

PortType: a WSDL element that describes a collection of message signatures (operation elements) that define the abstract interface of the Web Service

Grid Service instances are made accessible to client applications through the use of a GSH and a GSR

• GSH (Grid Service Handle): URI (uniform resource idetifier) that identifies in univocal way a instance, directs to a GSR

• GSR (Grid Service Reference): WSDL document that contains informations about the location of the instance


6

CMS

LNLGSH e GSR

URIwww.pc2.it/gsr1

GSH1

WSDL Document

1

GSR1

Grid ServiceInstance

1

URIwww.pc3.it/gsr2

GSH2

WSDL Document

2

GSR2

Grid ServiceInstance

2

Client


7

CMS

LNLGrid Service PortTypes (1)

A Grid Service can implement following PortType:

GridService: encapsulates the root behavior of the component model

HandleResolver: mapping form a GSH to a GSR

NotificationSource: allows clients to subscribe to notification messages

NotificationSubscription: defines the relationship between a single NotificationSource and NotificationSink pair

NotificationSink: defines a single operation for delivering a notification message to the service instance that implements the operation

Factory: standard operation for creation of Grid Service instances

Registration: allows clients to register and unregister registry contents


8

CMS

LNLGrid Service PortTypes (2)

Grid Service

(required)

…other interfaces…

(opzional)

Implementation

Service data access

Explicit destruction

Soft-state lifetime

Standard:

-Notification

-Authorization

-Service creation

-Service registry

-Manageability

-Concurrency

+ application-specific interfaces

Hosting environment/runtime(“C”, J2EE, .NET, …)


9

CMS

LNLWSDL Extensions

OGSA architecture introduced WSDL extensions to allow Grid Service implementation

New WSDL elements:

• serviceData: represents properties of the service’s state that may be externally queried

• serviceDataDescription: formal description of serviceData elements

• Convention on portType name: naming conventions and immutability of portType names

• Grid Service Reference

• Grid Service Handle


10

CMS

LNLServiceData Concept (1)

Service data: to support discovery, introspection, and monitoring of Grid Service instanceò it refers to descriptive information about Grid Service instance, including

Meta-Data: information about the service instance

State Data: runtime properties of the service instance

Each Grid Service instance has an associated set of SDEs (each one is represented in XML by a serviceData element)


11

CMS


serviceData elements samples:

Instance’s data which value can change at runtime


12

CMS


Each Grid Service instance is associated with a set of SDEs. This set is modelled as a serviceDataSet:

serviceData cpuSpeed with relative description: type, value

serviceData configuration with relative description


13

CMS

LNLGrid Service =

Interfaces + Service Data

cpuLoad Counter value


14

CMS

LNLExample of Grid Service : DB Service

A DBaccess Grid Service has at least two portType

GridService

DBaccess

Each has any serviceData

GridService: base informations, lifetime informations,…

DBaccess: kind of database, query languages supported, current load,…

CMS

LNL

15Giuliano Rorato - INFN - LNL 19/11/2002

Example of Grid Service : DB Service

The UK e-Science Institute collaborate in the OGSA Project, and it implemented a Grid Data Service, a Grid Service with portType to work with many database (relational, XML-DB, …)

http://umbriel.dcs.gla.ac.uk/NeSC/general/

Grid Service

XML-DB portType

Relational-DB portType

…Grid Data Service

http://umbriel.dcs.gla.ac.uk/NeSC/general/


16

CMS

LNLGridService portType: operations

FindServiceData: query the service data

RequestTerminationAfter: request that the termination time of this service be changed

RequestTerminationBefore: request that the termination time of the service be changed

Destroy: explicity request destruction of this service

ClientFindServiceData(lifetime)


17

CMS

LNLFactory portType

It is used by a client to create an instance of a Grid Service

It is a Grid Service that must implement the Factory portType, which provides a standard WSDL operation for creation of Grid Service instances

The used operation is: CreateService

User Application

Service Provider

Grid Service

Counter Factory


18

CMS

LNLFactory portType




User Application

Service Provider

Grid Service

Counter Factory

CreateService


19

CMS

LNLFactory portType




User Application

Service Provider

Grid Service

Counter Factory

CreateService

Counter


20

CMS

LNLRegistration

A registry is a Grid Service that maintains a collection of GSH, with policies associated with that collection. Clients may query the registry to discover what services are available

The registration portType allows clients to register and unregister registry content

The registry makes available a WS-Inspection document (defined with the Web Services Inspection Language – WSIL) to aid in discovery of the services in that registry


21

CMS

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType


22

CMS

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

RegisterService(locator 1)


23

CMS

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

RegisterService(locator 1)

GSH 1


24

CMS

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

GSH 1


25

CMS

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

UnregisterService(GSH 1)

GSH 1


26

CMS

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType


27

CMS

LNLTransients Database Services


28

CMS

LNLNotification

The purpose of the notification is to deliver interesting messages from a notification source to a notification sink

Notification source: sender of the notification messages

Notification sink: Grid Service instance that receives notification messages from any number of sources

Notification message: XML element sent from a notification source to a notification sink

Subscription expression: XML element that describes what messages should be sent from the notification source to the notification sink


29

CMS

LNLNotification portTypes

NotificationSource portType

Subscribe

NotificationSubscription portType: used from Clients to manage the subscription (lifetime and others properties)

NotificationSink portType

deliverNotification


30

CMS

LNLUSE CASE OF NOTIFICATION

User Application

Service Provider

Grid Service

Notification portType

X=0

serviceData


31

CMS


User Application

Service Provider

Grid Service


X=0

serviceData

Subscribe(x=1, sink, exp time)


32

CMS


User Application

Service Provider

Grid Service


X=0

serviceData


33

CMS


User Application

Service Provider

Grid Service


X=1

serviceData


34

CMS


User Application

Service Provider

Grid Service


X=1

serviceData

DeliverNotification(x=1)


35

CMS

LNLHandleResolver portType

A Handle Resolver is a Grid Service instance that implements the HandleResolver portType

FindByHandle: returns a serviceLocator, which contains one or more GSR (Grid Service Reference) for a GSH (Grid Service Handle)


36

CMS

LNLExample: Data Mining for

Bioinformatics

User Application

CommunityRegistry

“I want to create a personal database containing data on e.coli metabolism”

Mining Factory

Compute Service Provider

Database Factory

Storage Service Provider

DatabaseService

DB 1

Database Service

DB n


37

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB n

“Find me a data mining service, and somewhere to store data”


38

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB n

GSHs for Mining and Database factories


39

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB n

“Create a data mining service with initial lifetime 10”

“Create a database with initial lifetime 1000”


40

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB n

“Create a data mining service with initial lifetime 10”

“Create a database with initial lifetime 1000”

Miner

Database


41

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB n

Miner

Database

Query

QueryKeepalive

Keepalive


42

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB n

Miner

Database

ResultsKeepalive

Keepalive

Results


43

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB n

Miner

Database

Keepalive


44

CMS


Bioinformatics

User Application

CommunityRegistry

Mining Factory


Database Factory


DatabaseService

DB 1

Database Service

DB nDatabase

Keepalive


45

CMS

LNLCode Preview

Grid Service Container Frameworks:

Java

.NET

“C”

In this moment there are many java hosting environment:

1. Servlet: OGSA container is put into a standard Java Servlet Engine

2. EJB: OGSA container is put into a EJB Application Server


46

CMS

LNLJava Container Framework

OGSA Core endorsed Java Specifications:

Apache Axis (Web Services Container)

JAXP (XML Parsing)

JAX-RPC

J2EE

• EJB

• JMS

• JNLP (Java Network Launch Protocol, used for the mobile Java code)

Documentation: still poor and with not much examples


47

CMS

LNLCounter Example

CounterClient

“I want to create a Counter Grid Service”


Grid Service

Counter Factory


48

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

CreateService


49

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

CreateService

Counter

X=0


50

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

Counter

Subscribe(X)

X=0


51

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

CounterI want to Add “10” and then Subtract “5” X=0


52

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

Counter

Add (10)

X=0


53

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

Counter

X=10


54

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

Counter

X=10

Notify (10)

Counter State = 10


55

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

Counter

X=10

Subtract (4)


56

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

Counter

X=6


57

CMS

LNLCounter Example

CounterClient


Grid Service

Counter Factory

Counter

X=6

Notify (6)

Counter State = 6


58

CMS

LNLLog Grid Service Example

CreateServiceClient

Grid Service

Log Factory

Registry


59

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

CreateS

ervice


60

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

CreateS

ervice

LOG

GSH 1


61

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

Register GSH 1

LOG


62

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1


63

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Producer


64

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Find(LOG)

Message Producer


65

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1 GSH 1

Message Producer


66

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Producer

GSH1


67

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Subscriber

Message Producer

GSH1


68

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Producer

GSH1

Subs

cribe

Message Subscriber


69

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Producer

GSH1

Sink

Source

Message Subscriber


70

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Producer

GSH1

Sink

Source

Add(“ciao”)

Message Subscriber


71

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Producer

GSH1

Sink

Source

Add(“ciao”)

Notify

Message Subscriber


72

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG

GSH1

Message Producer

GSH1

Sink

Source

CiaoMessage Subscriber


73

CMS

LNL“Balanced” Log Grid Service (in progress)

CreateServiceClient

Grid Service

Log Factory

Registry

LOG 1

GSH1

Message Producer

LOG n…

GSHn


74

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG 1

GSH1

Message Producer

LOG n…

GSHn

Find(“LOG”)


75

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG 1

GSH1

Message Producer

LOG n…

GSHn

LOG 1


76

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG 1

GSH1

Message Producer

LOG n…

GSHn

UseService


77

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG 1

GSH1

Message Producer

LOG n…

GSHn

Find(“LOG”)


78

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG 1

GSH1

Message Producer

LOG n…

GSHn

LOG n


79

CMS


CreateServiceClient

Grid Service

Log Factory

Registry

LOG 1

GSH1

Message Producer

LOG n…

GSHn

UseService


80

CMS

LNLResources Discovery

REGISTRYGSH STATE

Grid Service

ResourceFactory

CreateXDaqClient

Grid Service

Registration portType


81

CMS


CreateXDaqClient

Crea

teSer

vice

REGISTRYGSH STATE

Grid Service

ResourceFactory

Grid Service



82

CMS


CreateXDaqClient

GSH 1

REGISTRYGSH STATE

Grid Service

ResourceFactory

Grid Service


Grid Service Resource

1


83

CMS


CreateXDaqClient

Regist

er(GSH

1)

REGISTRYGSH STATE

Grid Service

ResourceFactory

Grid Service



1


84

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service


Regist

er(GSH

1)


1


85

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service




1 2


86

CMS


CreateXDaqClient

Reg

iste

r(G

SH2)

REGISTRYGSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service




1 2


87

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service




Reg

iste

r(G

SH2)

1 2


88

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service




1 2


89

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service





1 2 3

Keepalive


90

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service





1 2 3


91

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service





1 2 3

GSH 2 = “OFF”


92

CMS


CreateXDaqClient

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3


93

CMS


CreateXDaqClient

Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3


94

CMS


CreateXDaqClient

Client

Find(“Xdaq”)

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3


95

CMS


CreateXDaqClient

Client

GSH 1

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3


96

CMS


CreateXDaqClient

Client

UseService

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3


97

CMS


CreateXDaqClient

Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3


98

CMS


CreateXDaqClient

Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3


99

CMS


CreateXDaqClient

Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service





1 2 3

GSH 2 = “ON”


100

CMS


CreateXDaqClient

Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service





1 2 3


101

CMS


CreateXDaqClient

Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service


Find(“Xdaq”)




1 2 3


102

CMS


CreateXDaqClient

Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service


GSH 1, GSH 2




1 2 3


103

CMS


CreateXDaqClient

Client

UseService

UseService

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service





1 2 3


104

CMS

LNLResources AutoDiscovery

Client

REGISTRYGSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service



1


105

CMS


Client

REGISTRYGSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service



1


2


106

CMS


Client

REGISTRYGSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service



1


2

Reg

iste

r(G

SH2)


107

CMS


Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service



1


2

Reg

iste

r(G

SH2)


108

CMS


Client

REGISTRYGSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service



1


2


109

CMS

LNLProblems

The problems to use this architecture are in the lack of documentation and in the presence of little bug, although they are correct in slightly time

FORUM

• [email protected]


110

CMS

LNL

All the RCMS Services can be developed (or re-implemented) under the OGSA framework

Some OGSA features can be exploited to design the services (e.g. publish/subscribe in the IMS)

Benefits for the overall RCMS infrastructure Services instantiation / Job Control (Factory) Services location (Registry) Uniform development platform Built-in security mechanism (at the service level) Common mechanism to access the services (api for clients):

• SOAP• Wsdl• portTypes• Software State (e.g. to monitor the service itself)

Cons Dependence from the Globus/IBM OGSA implementation

OGSA for RCMS?


111

CMS

LNL

RCMSRS

Resource Discovery

RCMSIMS

System Info/Alarm(CPU, Mem,etc.)

RCMSJC

XDAQExec

XDAQAppl 1

XDAQAppl 2

XDAQAppl n

RCMSFM

GenericDAQ Node

GRID DAQ Service

XDAQ Control (start / kill / inspection)

Benefits:• homogenous control infrastructure

• built-in resource discovery

• Notify mechanism in case of hardware errors (e.g. disk full)

• Node Services (e.g. node monitor)

• Application Job Control (e.g. XDAQ) XDAQ OGSA Compliant?

• very usefull a XDAQ service description in WSDL• very usefull a notify mechanism OGSA compliant • Registry/factory/discovery mechanisms can be done by the GRID DAQ Service

OGSA enabled DAQ resources?


112

CMS

LNL

RCMS SERVICES

DAQ SERVICES

DA

TAB

AS

E S

ER

VIC

ES

RE

GIS

TRIE

S, F

AC

TOR

IESPortTypes

SQLXML:DB

OGSA BASED RCMS INFRASTRUCTURE


113

CMS

LNLConclusions

OGSA evaluation in progress based on a preview code

First feedback positive based on the working examples implemented

Performance tests needed, wait for alfa version implementation (January 2003)

Discussion started on the use of OGSA in the RCMS context


114

CMS

LNLFuture Development

Gennaio 2003 OGSA ALPHA-Version

More documentation and examples

Correction of Bugs

New hosting environments (“C”)

...


115

CMS

LNLBIBLIOGRAPHY

[1] – www.globus.org/ogsa

[2] – www.globus.org/ogsa/releases/TechPreview/ - OGSA Release Preview 4

[3] – Open Grid Services Architecture: An Introduction – Ian Foster

[4] – Grid Service Specification (Draft 4, October 4, 2002)

[5] – Ogsa Physiology Paper

[6] – Building Web Services with Java – Steve Graham

http://www.globus.org/ogsa




http://www.globus.org/ogsa/releases/TechPreview/

CMS LNL OGSA INFRASTRUCTURE EVALUATION Specification Applications Giuliano Rorato INFN Laboratori...

Documents

Transcript of CMS LNL OGSA INFRASTRUCTURE EVALUATION Specification Applications Giuliano Rorato INFN Laboratori...