OGSA INFRASTRUCTURE EVALUATION

OGSA INFRASTRUCTURE OGSA INFRASTRUCTURE EVALUATIONEVALUATION

Specification & ApplicationsSpecification & Applications

Giuliano RoratoGiuliano Rorato

INFN – Laboratori Nazionali di LegnaroINFN – Laboratori Nazionali di Legnaro

Giuliano Rorato - INFN - LNL 19/11/2002

LNLWhy OGSA

Our interest in OGSA (Open Grid Services Architecture) technology is about a possible application in the online software of CMS. In particular for RCMS.

OGSA: defines mechanisms for managing distribuited entities.

RCMS: distribuited services system for the run control of CMS.

OGSA framework

RCMS Services

LNLOpen Grid Services Architecture

OGSA: Grid system architecture based on an integration of GRID and Web Services concepts and technologies.

Initial proposed technical specification: Globus project and IBM

• Put forward at the Globus Grid Forum: for discussion, refinement, and eventual standardization

2002-2003: Globus Toolkit OGSA-compliant Globus Toolkit 3.0

Now there is the Preview Release 5

• www.globus.org/ogsa/releases/TechPreview/

LNLWeb Services vs Grid Services

WEB SERVICE: interface that describes a collection of operations that are network accessible through standardized XML messaging

WSDL (Web Services Description Language): language XML-based that describe the web service (accessible operations and location)

GRID SERVICE: web service that conforms to a set of conventions (interfaces and behaviors) that define how a client interacts with a Grid Service

LNLOGSA - Definition The Open Grid Services Architecture (OGSA) defines mechanisms for creating,

managing, and exchange information among entities called Grid Services

Grid Service instance: (persistent or transient) service that conforms to a set of conventions (expressed as WSDL interfaces, extensions, and behaviors) for such purposes as lifetime management, discovery of characteristics, notification, and so forth, described from a WSDL PortType

PortType: a WSDL element that describes a collection of message signatures (operation elements) that define the abstract interface of the Web Service

Grid Service instances are made accessible to client applications through the use of a GSH and a GSR

• GSH (Grid Service Handle): URI (uniform resource idetifier) that identifies in univocal way a instance, directs to a GSR

• GSR (Grid Service Reference): WSDL document that contains informations about the location of the instance

LNLGSH e GSR

URIwww.pc2.it/gsr1

WSDL Document

Grid ServiceInstance

URIwww.pc3.it/gsr2

WSDL Document

Grid ServiceInstance

Client

LNLGrid Service PortTypes (1)

A Grid Service can implement following PortType:

GridService: encapsulates the root behavior of the component model

HandleResolver: mapping form a GSH to a GSR

NotificationSource: allows clients to subscribe to notification messages

NotificationSubscription: defines the relationship between a single NotificationSource and NotificationSink pair

NotificationSink: defines a single operation for delivering a notification message to the service instance that implements the operation

Factory: standard operation for creation of Grid Service instances

Registration: allows clients to register and unregister registry contents

LNLGrid Service PortTypes (2)

Grid Service

(required)

…other interfaces…

(opzional)

Implementation

Service data access

Explicit destruction

Soft-state lifetime

Standard:

-Notification

-Authorization

-Service creation

-Service registry

-Manageability

-Concurrency

+ application-specific interfaces

Hosting environment/runtime(“C”, J2EE, .NET, …)

LNLWSDL Extensions

OGSA architecture introduced WSDL extensions to allow Grid Service implementation

New WSDL elements:

• serviceData: represents properties of the service’s state that may be externally queried

• serviceDataDescription: formal description of serviceData elements

• Convention on portType name: naming conventions and immutability of portType names

• Grid Service Reference

• Grid Service Handle

LNLServiceData Concept (1)

Service data: to support discovery, introspection, and monitoring of Grid Service instanceò it refers to descriptive information about Grid Service instance, including

Meta-Data: information about the service instance

State Data: runtime properties of the service instance

Each Grid Service instance has an associated set of SDEs (each one is represented in XML by a serviceData element)

serviceData elements samples:

Instance’s data which value can change at runtime

Each Grid Service instance is associated with a set of SDEs. This set is modelled as a serviceDataSet:

serviceData cpuSpeed with relative description: type, value

serviceData configuration with relative description

LNLGrid Service =

Interfaces + Service Data

cpuLoad Counter value

LNLExample of Grid Service : DB Service

A DBaccess Grid Service has at least two portType

GridService

DBaccess

Each has any serviceData

GridService: base informations, lifetime informations,…

DBaccess: kind of database, query languages supported, current load,…

15Giuliano Rorato - INFN - LNL 19/11/2002

Example of Grid Service : DB Service

The UK e-Science Institute collaborate in the OGSA Project, and it implemented a Grid Data Service, a Grid Service with portType to work with many database (relational, XML-DB, …)

http://umbriel.dcs.gla.ac.uk/NeSC/general/

Grid Service

XML-DB portType

Relational-DB portType

…Grid Data Service

LNLGridService portType: operations

FindServiceData: query the service data

RequestTerminationAfter: request that the termination time of this service be changed

RequestTerminationBefore: request that the termination time of the service be changed

Destroy: explicity request destruction of this service

Client

FindServiceData(lifetime)

LNLFactory portType

It is used by a client to create an instance of a Grid Service

It is a Grid Service that must implement the Factory portType, which provides a standard WSDL operation for creation of Grid Service instances

The used operation is: CreateService

User Application

Service Provider

Grid Service

Counter Factory

LNLFactory portType

User Application

Service Provider

Grid Service

Counter Factory

CreateService

LNLFactory portType

User Application

Service Provider

Grid Service

Counter Factory

CreateService

Counter

LNLRegistration

A registry is a Grid Service that maintains a collection of GSH, with policies associated with that collection. Clients may query the registry to discover what services are available

The registration portType allows clients to register and unregister registry content

The registry makes available a WS-Inspection document (defined with the Web Services Inspection Language – WSIL) to aid in discovery of the services in that registry

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

RegisterService(locator 1)

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

RegisterService(locator 1)

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

UnregisterService(GSH 1)

LNLRegistration

User Application

Service Provider

Grid Service

Registry portType

LNLTransients Database Services

LNLNotification

The purpose of the notification is to deliver interesting messages from a notification source to a notification sink

Notification source: sender of the notification messages

Notification sink: Grid Service instance that receives notification messages from any number of sources

Notification message: XML element sent from a notification source to a notification sink

Subscription expression: XML element that describes what messages should be sent from the notification source to the notification sink

LNLNotification portTypes

NotificationSource portType

Subscribe

NotificationSubscription portType: used from Clients to manage the subscription (lifetime and others properties)

NotificationSink portType

deliverNotification

LNLUSE CASE OF NOTIFICATION

User Application

Service Provider

Grid Service

Notification portType

serviceData

User Application

Service Provider

Grid Service

serviceData

Subscribe(x=1, sink, exp time)

User Application

Service Provider

Grid Service

serviceData

User Application

Service Provider

Grid Service

serviceData

User Application

Service Provider

Grid Service

serviceData

DeliverNotification(x=1)

LNLHandleResolver portType

A Handle Resolver is a Grid Service instance that implements the HandleResolver portType

FindByHandle: returns a serviceLocator, which contains one or more GSR (Grid Service Reference) for a GSH (Grid Service Handle)

LNLExample: Data Mining for

Bioinformatics

User Application

CommunityRegistry

“I want to create a personal database containing data on e.coli metabolism”

Mining Factory

Compute Service Provider

Database Factory

Storage Service Provider

DatabaseService

Database Service

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

“Find me a data mining service, and somewhere to store data”

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

GSHs for Mining and Database factories

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

“Create a data mining service with initial lifetime 10”

“Create a database with initial lifetime 1000”

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

“Create a data mining service with initial lifetime 10”

“Create a database with initial lifetime 1000”

Database

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

Database

QueryKeepalive

Keepalive

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

Database

ResultsKeepalive

Keepalive

Results

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

Database

Keepalive

Bioinformatics

User Application

CommunityRegistry

Mining Factory

Database Factory

DatabaseService

Database Service

DB nDatabase

Keepalive

LNLCode Preview

Grid Service Container Frameworks:

“C”

In this moment there are many java hosting environment:

1. Servlet: OGSA container is put into a standard Java Servlet Engine

2. EJB: OGSA container is put into a EJB Application Server

LNLJava Container Framework

OGSA Core endorsed Java Specifications:

Apache Axis (Web Services Container)

JAXP (XML Parsing)

JAX-RPC

• EJB

• JMS

• JNLP (Java Network Launch Protocol, used for the mobile Java code)

Documentation: still poor and with not much examples

LNLCounter Example

CounterClient

“I want to create a Counter Grid Service”

Grid Service

Counter Factory

LNLCounter Example

CounterClient

Grid Service

Counter Factory

CreateService

LNLCounter Example

CounterClient

Grid Service

Counter Factory

CreateService

Counter

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

Subscribe(X)

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

I want to Add “10” and then Subtract “5” X=0

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

Add (10)

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

Notify (10)

Counter State = 10

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

Subtract (4)

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

LNLCounter Example

CounterClient

Grid Service

Counter Factory

Counter

Notify (6)

Counter State = 6

LNLLog Grid Service Example

CreateServiceClient

Grid Service

Log Factory

Registry

CreateServiceClient

Grid Service

Log Factory

Registry

CreateS

ervice

CreateServiceClient

Grid Service

Log Factory

Registry

CreateS

ervice

CreateServiceClient

Grid Service

Log Factory

Registry

Register GSH 1

CreateServiceClient

Grid Service

Log Factory

Registry

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

CreateServiceClient

Grid Service

Log Factory

Registry

Find(LOG

Message Producer

CreateServiceClient

Grid Service

Log Factory

Registry

GSH1 GSH 1

Message Producer

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

CreateServiceClient

Grid Service

Log Factory

Registry

Message Subscriber

Message Producer

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

Message Subscriber

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

Source

Message Subscriber

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

Source

Add(“ciao”)

Message Subscriber

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

Source

Add(“ciao”)

Message Subscriber

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

Source

CiaoMessage Subscriber

LNL“Balanced” Log Grid Service (in progress)

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

LOG n…

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

LOG n…

Find(“LOG

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

LOG n…

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

LOG n…

UseService

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

LOG n…

Find(“LOG

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

LOG n…

CreateServiceClient

Grid Service

Log Factory

Registry

Message Producer

LOG n…

UseService

LNLResources Discovery

REGISTRY

GSH STATE

Grid Service

ResourceFactory

CreateXDaqClient

Grid Service

Registration portType

CreateXDaqClient

REGISTRY

GSH STATE

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

REGISTRY

GSH STATE

Grid Service

ResourceFactory

Grid Service

Grid Service Resource

CreateXDaqClient

Regist

REGISTRY

GSH STATE

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service

Regist

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

Keepalive

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

GSH 2 = “OFF”

CreateXDaqClient

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

Find(“Xdaq”)

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

UseService

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 OFF

Grid Service

ResourceFactory

Grid Service

GSH 2 = “ON”

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

Find(“Xdaq”)

CreateXDaqClient

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

GSH 1, GSH 2

CreateXDaqClient

Client

UseService

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

LNLResources AutoDiscovery

Client

REGISTRY

GSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service

Client

REGISTRY

GSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service

Client

REGISTRY

GSH STATE

GSH 1 ON

Grid Service

ResourceFactory

Grid Service

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

Client

REGISTRY

GSH STATE

GSH 1 ON

GSH 2 ON

Grid Service

ResourceFactory

Grid Service

LNLProblems

The problems to use this architecture are in the lack of documentation and in the presence of little bug, although they are correct in slightly time

• OGSA-ALPHA@GLOBUS.ORG

All the RCMS Services can be developed (or re-implemented) under the OGSA framework

Some OGSA features can be exploited to design the services (e.g. publish/subscribe in the IMS)

Benefits for the overall RCMS infrastructure Services instantiation / Job Control (Factory) Services location (Registry) Uniform development platform Built-in security mechanism (at the service level) Common mechanism to access the services (api for clients):

• SOAP• Wsdl• portTypes• Software State (e.g. to monitor the service itself)

Cons Dependence from the Globus/IBM OGSA implementation

OGSA for RCMS?

RCMSRS

Resource Discovery

RCMSIMS

System Info/Alarm(CPU, Mem,etc.)

RCMSJC

XDAQExec

XDAQAppl 1

XDAQAppl 2

XDAQAppl n

RCMSFM

GenericDAQ Node

GRID DAQ Service

XDAQ Control (start / kill / inspection)

Benefits:• homogenous control infrastructure

• built-in resource discovery

• Notify mechanism in case of hardware errors (e.g. disk full)

• Node Services (e.g. node monitor)

• Application Job Control (e.g. XDAQ) XDAQ OGSA Compliant?

• very usefull a XDAQ service description in WSDL• very usefull a notify mechanism OGSA compliant • Registry/factory/discovery mechanisms can be done by the GRID DAQ Service

OGSA enabled DAQ resources?

RCMS SERVICES

DAQ SERVICES

PortTypes

XML:DB

OGSA BASED RCMS INFRASTRUCTURE

LNLConclusions

OGSA evaluation in progress based on a preview code

First feedback positive based on the working examples implemented

Performance tests needed, wait for alfa version implementation (January 2003)

Discussion started on the use of OGSA in the RCMS context

LNLFuture Development

Gennaio 2003 OGSA ALPHA-Version

OGSA INFRASTRUCTURE EVALUATION

Documents

Transcript of OGSA INFRASTRUCTURE EVALUATION

Mike Jackson EPCC michaelj@epcc.ed.ac.uk@epcc.ed.ac.uk OGSA-DAI Architecture + Extensibility OGSA-DAI Tutorial GGF17, Tokyo.

Data access and integration with OGSA-DAI: OGSA-DQP

Http:// Introduction to OGSA-DAI The OGSA-DAI Team info@ogsadai.org.uk.

Open Grid System Architecture (OGSA)

Defining the Grid: A Roadmap for OGSA Standardsxml.coverpages.org/OGSA-StandardsRoadmapV11-2007.pdf · OGSA is registered trademark and service mark of the Open Grid Forum. Obsoletes

UK OGSA Evaluation Project Report 1.0 Evaluation of Globus …sse.cs.ucl.ac.uk/UK-OGSA/Report1.pdf · 2005-03-22 · The scope of this report (1.0) is limited to the installation,

Distributed Query Processing with OGSA-DQP

Grid, Globus Toolkit, and OGSA

OGSA™ EMS Architecture Scenarios Version 1 18 April 2007 ogsa-wg@ogf.org 3 1 Introduction The OGSA EMS (execution management services) architecture ([GFD.80] 3.4) has been under

CMS LNL OGSA INFRASTRUCTURE EVALUATION Specification Applications Giuliano Rorato INFN Laboratori Nazionali di Legnaro.

OGSA-DAI Architecture + Extensibility

OGSA Resource Selection Services WG

OGSA WSRF Basic Profile 1 - Open Grid Forum · The OGSA WSRF Basic Profile 1.0 described in this document is an OGSA Recommended Profile as Proposed Recommendation as defined in the

7. the grid ogsa

INFRASTRUCTURE CONDITION EVALUATION

Http:// The OGSA-DAI Client Toolkit The OGSA-DAI Team info@ogsadai.org.uk.

OGSA/GT3 evaluation at CERN

OGSA-DAI Lectures Part 2

OGSA-DAI Client Toolkit

Introduction to OGSA-DAI - unina.itmurli/ISSGC06/ogsadai/public_html/Slides/OGSA... · 2 17 July 2006 ISSGC06, Ischia, Italy 3 OGSA-DAI OGSA-DAI OGSA-DAI In One Slide • An extensible