GRASPGRASP

GRid based Application Service GRid based Application Service ProvisionProvision

http://www.eu-grasp.net

Microsoft Academic DaysMicrosoft Academic Days

Segrate 22-24 November 2004Segrate 22-24 November 2004

Pierluigi RitrovatoResearch and Technology Director

ritrovato@crmpa.unisa.it

Giuseppe LariaHead of GRID Unitlaria@crmpa.unisa.it

2

Brief Grid overview Introduction to the GRASP project GRASP infrastructure description .NET and GRASP subsystems

Agenda

3

The evolution of the Grid concept

Metacomputing: A “new” computational approach is

introduced To use geographically distributed

supercomputing resources as they were a single high powerful computer

Clearly arising from need of high performance computer in scientific research world

Middleware Development: Identification of common building blocks Development of several middleware based on

proprietary protocols Heavy customisation was neded in terms of

development for setting up a Grid system

Grid computing & SOA for enabling scalable Virtual Organization: The Grid meets the Web Services world The definition of the “physiology of the grid” concept

and the emerging of OGSA Flexible secure coordinated resource sharing among

dynamic collections of individuals, institutions, and resources arranged as a Virtual Organisation

Grid computing has becoming synonymous of high scale distributed resource sharing for execution of innovative applications oriented to business and, in same cases, with high performance orientation

Guaranteeing interoperability inside the VO and across VOs

The visionary idea of the NGG expert group http://www.cordis.lu/ist/grids

4

EU funded projects in 5FP

IST Grid Projects

EGSOAVO

GRIA

CrossGridGridLab

GRIP

DataTAG

EuroGrid

DAMIEN

DataGrid

GEMSS

MammoGrid

BioGridSeLeNe

OpenMolGrid

COG

FlowGrid

GRACEMOSES

GRASP

1-10-2000 1-10-2001 1-10-2002

GRIDSTART cluster

Wave 2 Projects

5

The Grid meets the Web Services world (1)

At the origin the marriage between web services and Grid led to the Grid Service: “A WSDL-defined service that conforms to a set of conventions relating to its interface definitions and behaviors. Thus, every Grid service is a Web service, though the converse of this statement is not true”

The Grid Services basilar characteristics have been defined in the OGSI specifications (started at the end 2001 and released in 2003)

6

The Grid meets the Web Services world (2)

Grid

Web

Started far apart in apps & tech

OGSI

GT2

GT1

HTTPWSDL,

WS-*

WSDL 2

Have beenconverging ?

Source: globus alliance CrossGrids’04

7

The Grid meets the Web Services world (2)

Grid

Web

Started far apart in apps & tech

OGSI

GT2

GT1

HTTPWSDL,

WS-*

WSDL 2

Have beenconverging

Source: globus alliance CrossGrids’04

WSRF

WSRF

8

Web Service Resource Framework

WSRF is a framework consisting of a number of specifications WS-Resource Properties WS-Resource Lifetime WS-Service Groups WS-Notification WS-BaseFaults WS-Renewable References (unpublished)

Other WS specifications : WS-Addressing

9

From OGSI to WSRF:Refactoring and Evolution

OGSI WSRF

Grid Service Reference WS-Addressing Endpoint Reference

Grid Service Handle WS-Addressing Endpoint Reference

HandleResolver portType WS-RenewableReferences

Service data defn & access WS-ResourceProperties

GridService lifetime mgmt WS-ResourceLifeCycle

Notification portTypes WS-Notification

Factory portType Treated as a pattern: WS-Resource

ServiceGroup portTypes WS-ServiceGroup

Base fault type WS-BaseFaults

Source: globus alliance CrossGrids’04

10

Context Services Info

Services

InfraServices

SecurityServices

Rsrc Mgmt Services

Execution Mgmt

Services

DataServices

PolicyMgmt

VOMgmt

Access

Integration

Provisioning

Cataloging

BoundaryTraversal

Integrity

Authorization

Authentication

WSRF WSN WSDM

EventMgmt

Trouble-shooting

Discovery

JobMgmt

Logging

ExecutionPlanning

WorkflowMgmt

WorkloadMgmt

Provisioning

ApplicationMgmt

DeploymentConfigurationReservation

Naming

Self MgmtServices

HeterogeneityMgmt

Service LevelAttainment

QoSMgmt

Optimization

Information Services

Infrastructure Services

SelfMgmtServices

SecurityServices

Resource Mgmt Services

Execution Mgmt Services

DataServices

Context Services

Release date June 2004Release date June 2004

11

Basic capabilities

Infrastructure WS-Resource Framework (WSRF), WS-Notification (WSN) and other WS-

Standards Execution Management

concerned with the problems of instantiating and managing tasks, that are defined as single units of work to be managed

Data concerned with the movement, access and update of data resources

Resource Management performs several forms of management on resources in a Grid

Security to facilitate the enforcement of the security-related policy within a VO, e.g.

message integrity and confidentiality, authentication of interacting entities, minimum authentication strength, secure logging and audit, separation of responsibilities

Self-Management reduce the cost and complexity of owning and operating an IT infrastructure, e.g.

self-configuring, self-healing and self-optimizing Information

to efficiently access and manipulate information about applications, resources and services in the Grid environment

12

OGSA Framework

services are built on Web service standards, with semantics, additions, extensions and modifications that are relevant to Grids, e.g. WSRF

services are loosely coupled peers that, either singly or as part of an interacting group of services, realize the capabilities of OGSA through implementation, composition, or interaction with other services

services may be part of, or participate in, virtual collections called virtual domains

13

Brief overview of Grid Introduction to the GRASP project GRASP infrastructure description .NET and GRASP subsystems

14

Project Goals

GRASP is an RTD project started in 2002 and funded by the EC in the frame of IST-FP5 with 3 major goals:

Overcome weakness of current ASP solutions enabling Effective access to potentially distributed resources Delivery of services respecting negotiable service level agreements Charging services on the basis of effective use and delivered quality Managed security across different administrative domains

Design, implement and validate a layered architecture by: Analysis of existing GRID middleware and COTS for applicability

for critical aspects of an ASP infrastructure Utilising the concepts of the OGSA for creating Virtual

Organisations Test beds in the medical attention management and e-learning

domains as driver of requirements and validation scenarios Innovative ASP collaboration models

Federated model: The ASP consists itself of many actors that combine their resources in order to provide services to their clients

Many to many model: allows the integration of provider and user resources

15

Evolution of ASP

Internet Based Service Provider Provision of network centric

applications Target economies of scale Run services in a multi-tenancy model Support thousands of customers on a

single code basis Reduce hardware and admin cost by

partitioning and distributing resources Reduce of client-side development

investment and maintenance use standard Web technology as their

client-side interface

Utility computing shared infrastructure provided on

demand to multiple customers. lets companies pay for IT services as

needed Charge customers the same way an

electric company does: When usage spikes, so does the bill.

Pay-per-use or metered use Available on demand Dynamic allocation and integration

GRASP brings together elements of GRASP brings together elements of IBSP and Utility computing to enable IBSP and Utility computing to enable the next Generation Grid based ASPthe next Generation Grid based ASP

GRASP brings together elements of GRASP brings together elements of IBSP and Utility computing to enable IBSP and Utility computing to enable the next Generation Grid based ASPthe next Generation Grid based ASP

16

Technical Choices

GRASP platform main characteristics: The Open Grid Service Architecture is the reference model Microsoft (.NET platform) is the chosen deployment

infrastructure Compliance with OGSI specification Based on the OGSI.NET framework implementing the OGSI

v1.0 (released on July 2003) specification on Microsoft platform

GRASP platform leverages on standards coming from the Web Services world: SOAP, UDDI, BPEL, WS-Security, WS-Manageability, WSLA

GRASP platform integrates COTS components and commodity technologies: BizTalk for orchestration of Grid Services UDDI for searching Grid Services WSE: Microsoft implementation of WS standards MS Performance Counter for resource monitoring

17

MS Operating System

GRASP platform anatomy

OGSI.NET (Virginia University)

OGSI extensions

DestroyManageability

NotificationSecurity

WSRF.NET (Virginia University)

SL

A

Service

Instatiatio

n

Secu

rity

Acco

un

ting

&

Ch

argin

g

Microsoft .NET Framework

BizTalk 2004, UDDI directory

Service

Lo

cation

OrchestrationService

18

HE2

HE3 HEn

GRASP Physiology

HE1

INTE

RNET

End use

r

Host ASP application

VO

HE1 HE2

HE3HEn

19

Brief overview of Grid Introduction to the GRASP project GRASP infrastructure description .NET & GRASP subsystems

20

Basic GRASP Interactions

ServiceLocator

ServiceRequestor

ServiceProvider

ServiceInstantiator

Request

Service

Loca

tion

Return

SP

ServiceHost

Request Istance with specified

SLA

Create Instances or reuse idle Instance

(Invoking factory)

Provide access details

ServiceHostService

Host

Request Services/negotiation

SLAMonitoring

AS (SLA)

Acc (SLA)

Notify violation

SLA

Choose the host

ServiceHost GS

Collect data

HE

Gateway

21

The VHE concept

Grasp

ISA

PI F

ilter

OGSI.NET container

GRASPHandler

[Http Handler]

Service Registry

Service Instantiator

ServiceFactory

ServiceInstance1.2

ServiceInstance1.1

ServiceFactory

ServiceInstance2.2

Gateway Server Hosting Server(s)

New

OGSI.NET

OGSI.NET

Container PUC [Windows Service]

Handler

22

The VHE concept

Grasp

ISA

PI F

ilter

Service Registry

Service Instantiator

ServiceFactory

ServiceInstance1.2

ServiceInstance1.1

ServiceFactory

ServiceInstance2.2

Gateway Server Hosting Server(s)

Invoke

OGSI.NET container

OGSI.NET

OGSI.NETGRASPHandler

[Http Handler]

Container PUC [Windows Service]

23

Locator Subsystem

General scenario

VO 1

SP1

SP2

Service Locator

HE1

publish

Look up

HE2

publ

ish

SP3 deploy

deploy

MS UDDI Registry

24

Orchestration: 1st phase deployment

BizTalk Server 2004

Componentinvokers

Location and instantiation

Locator

HE

Instantiator

HE

Instantiator

HE

Instantiator

HE

Instantiator

ProcessDescription

AdditionalMetadata

GRASPcomposer

EnhancedProcess

Description

Deployment

ComposedService

Orchestration 2nd phase: invocation

25

Accounting & Manageability

OGSI.NET

•SDE management•Lifetime Management

ReferenceReference Grid Service

Manageability

•Metrics value SDEs•Service Shutdown

•Other OGSI port types•GRASP related port types

GridService PortType

(mandatory)

Other Port Type(optional)

26

Starting service accounting - Dynamic view

Accounting Service

Agreement Service

Instantiator Service

Charging Service

Business Service

accountService

chargeService

Retrieve SLA document (based on WSLA)

Subscribe to metric values changes

Creates records for informations related to: User, Service Account,User-Service relationships and then starts charging

Searches for Charge and Penaltiesin the SLA document; initializes internal logic with these policies

Subscribe to be informed about violation

Mng

Mng

27

Charging service usage – Dynamic view

Accounting Service

Charging Service

deliverNotification(…)

addServiceCharges(…)

Applies the Charge policy (defined in the SLA) andcalculates a new cost for theservice instance

MNG

Manageability detects metric value changes

Metric Provider

Updates metric changes

WMI

28

GRASP SOAP pipeline related to ASP.NET pipeline

IIS GRASPGRASPHandlerHandler

OGSI Handler

OGSI dispatcher

GRASP SOAP Handler

WSERoleCheck

SOAP Grid ServiceGrid Service

IIS ASP .NET WSE Web ServiceWeb Service

Configures WSE pipeline in order to allow it the management of GRASP related security aspects

Redirect incoming message towards

OGSI Handler

29

Secure group infrastructure

TT

AT

AT AT

ATAT

GM

GT

HE 1

HE 2

LSM1

LSM2

GT

30

Brief overview of Grid Introduction to the GRASP project GRASP infrastructure description .NET & GRASP subsystems

31

GRASP and MS.NET features

Instantiator Service: MS ISAPI Filter MS HTTP Handler MS Windows Service

Locator Service: MS UDDI registries

Orchestrator Service: MS BizTalk Server 2004 Orchestrator engine

Accounting: Windows Management Instrumentation (WMI)

SLA Monitoring: Performance counters

Security: Web Service Enhancement SOAP Handler

32

Contact Details

GrASP Scientific coordinator: Pierluigi Ritrovato (CRMPA)

Urls: www.eu-grasp.net www.crmpa.it

Contact persons: Pierluigi Ritrovato, Giuseppe LariaE-mail: ritrovato@crmpa.unisa.it,

laria@crmpa.unisa.it

33

Thank you… and now questions and

the second part

34

Charging service level violation – Dynamic view

Accounting Service

Charging Service

Agreement Service

deliverNotification(…)addServiceCharges(…)

Detects a servicelevel violation

Applies the Penalty policy (defined inthe SLA) and calculates the implied discount for the service instance

35

Instantiator Subsystem Instantiator extends OGSI standard factory

functionalities: Manages the creation of service instance within the

HE Interacts with the Service Level Agreement

monitoring subsystem to choose the machine (within the HE) that addresses the required QoS

Instantiates the service on the chosen machine Implements Handler Resolver capabilities:

The instance requestor isn’t provided with the real endpoint of the created service instance

It invokes the instance using a virtual GSH (but this is transparent to the requestor)

A subcomponent of the instantiator forwards the incoming request to the right instance

Implementation Status: Implementation is finished Possible improvements:

solving potential bottleneck issues Support migration mechanism Implement a WSRF version

36

Locator Subsystem Locator extends classical UDDI directory

functionalities: Allows to publish factory services against the directory

through an automatic mechanism Allows to classify factory on the basis of metadata that

describes features of instances created with this factory Locator subsystem architecture:

It has a hierarchical architecture with registries at three different levels:

Host Hosting environment Virtual organization

The deployment of a factory on a host automatically starts a process that publishes info about the new available service across three different levels of the locator subsystem

Based on following MS components: MS UDDI registry

Implementation Status: Implementation can be considered finished Possible improvements:

To introduce more advanced research criteria (at the moment they are based on the provision of a SLA document).

37

The orchestrator uses BPEL4WS as business process definition language (with some extensions to manage grid services)

It is exposed as a service (currently a Grid Service) that:

Takes as input parameters an extended BPEL script, including some ad hoc metadata for allowing the orchestrator to manage the grid service invocation

Performs “on the fly” location and instantiation of grid service during the business process execution

Exposes the composite service as a grid service Provides a simple mechanism for event handling based on

policy definitions

Orchestrator subsystem

38

Charging service level violation – Dynamic view

Accounting Service

Charging Service

Agreement Service

deliverNotification(…)addServiceCharges(…)

Detects a servicelevel violation

Applies the Penalty policy (defined inthe SLA) and calculates the implied discount for the service instance

39

It supports use-based and instance level-based charging; It uses policies (supported by WSLA, plus business extensions) for

providing a flexible, dynamically-configurable accounting (taking in account possible violation of SLA;

It supports multi-purpose accounting: we can identify two basic accounting threads: the first one will account costs to the

client (financial accounting), the second one will trace the costs that the provider of the service supports for providing services (cost accounting); Currently the only available type of accounting is financial accounting

It supports accounting of compound services Implementation status:

The implementation is finished It is based on the manageability concepts It is based on the Windows Management Instrumentation to store metric values At the moment a limited number of metrics are available:

Total Nr of invocation Nr of invocation for each operation Nr of invocation for each SDE Response time Size of passed parameters

Accounting subsystem

40

In GRASP we have two security contexts: Between ASP and their clients Within the Virtual Organization, that involves:

Creation of secure groups for grid services collaboration within the group

Provision of mechanism to enforce the instance access restriction within groups and across groups

GRASP focuses on the second context providing the following functionalities:

Security across organizational boundaries Manages secure service collaboration through:

Group membership management Encryption schema Management of policies based security mechanism

Security subsystem