INFSO-RI-508833 Enabling Grids for E-sciencE Grid Services for Resource Reservation and Allocation...
-
Upload
stuart-barrett -
Category
Documents
-
view
219 -
download
0
description
Transcript of INFSO-RI-508833 Enabling Grids for E-sciencE Grid Services for Resource Reservation and Allocation...
INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Grid Services for Resource Reservation and AllocationTiziana FerrariIstituto Nazionale di Fisica Nucleare, INFN CNAF
Terena Networking Conference, Poznan, June 6 2005
Enabling Grids for E-sciencE
INFSO-RI-508833
Outline
• Use cases• The Grid Resource Allocation and Agreement Protocol• The EGEE gLite Agreement Service:
– Architecture– Components
• Workload Management System instrumentation• Implementation: the space reservation Agreement
Service prototype• Future work
Enabling Grids for E-sciencE
INFSO-RI-508833
Grid Computing (2)
“A computational Grid is a hardware and software infrastructure that provides
dependable, consistent, pervasive and inexpensive access to high-end
computational capabilities”
Ian Foster, Carl Kesselman
• Purpose of Grid Computing: flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources
From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”
Enabling Grids for E-sciencE
INFSO-RI-508833
Use Cases
• NETWORK: Data replication:
to optimize performance of a data transfer session (that otherwise would compete with other streams and would be subject to variable throughput)
to support file transfer with deadline (to synchronize job execution with input file transfer)
• COMPUTING: to reserve computing resources (e.g. worker nodes, CPU cycles) in
presence of a large number of other competing jobs• STORAGE:
to guarantee that a sufficient amount of space is present in a Storage Element to save the output data
the Storage Element is close to the Computing Element on which computing resources have been reserved
Enabling Grids for E-sciencE
INFSO-RI-508833
• Use cases• The Grid Resource Allocation and Agreement Protocol• The gLite Agreement Service:
– Architecture– Components
• Workload Management System instrumentation• Implementation: the space reservation Agreement
Service prototype• Future work
Enabling Grids for E-sciencE
INFSO-RI-508833
• WS-Agreement defines a language and a protocol for– Advertising the capabilities of providers– Checking compliance to pre-defined templates– Creating agreements based on creational offers
• Agreement Layer:– Provides a web service based interface– represents and monitors agreements with respect to provisioning
of services implemented in the service layer• Service Layer (out of the GRAAP scope):
– is an resource-specific layer of a provided service– The interface to this layer is domain-specific: may or may not be
exposed as a web service interface
Grid Resource Allocation and Agreement Protocol – GGF GRAAP 2/2
Enabling Grids for E-sciencE
INFSO-RI-508833
Grid Resource Allocation and Agreement Protocol – GGF GRAAP 1/2
Consumer Provider
create()
foo()Application Instance
Factory
Manager
create()Factory Agreement
Ops:terminate(limits)inspect(query)...SDEs:
Terms RelatedStatusAgrmts
inspect()
Agreement Layer
Service Layer
Consumer Provider
create()
foo()Application Instance
Factory
Manager
create()Factory Agreement
Ops:terminate(limits)inspect(query)...SDEs:
Terms RelatedStatusAgrmts
inspect()
Agreement Layer
Service Layer
WS-Agreement specification Conceptual Layered Service Model [2]
gLite Agreement Service
allocation and reservation service provider
Enabling Grids for E-sciencE
INFSO-RI-508833
Agreement: XML structure
Agreement
Terms
Service Description Terms
Guarantee Terms
Context
Name
Name: identificator
Context: participants’ names, lifetime, links to otheragreements related to this (co-allocation)
Terms:
Service Description Terms:- provide information needed to instantiate and identify a service to which this agreement pertains- describes the functionality that will be delivered under an agreement
Guaranteed Terms:specify the service level that theparties are agreeing to
Terms are usage-domain independent.
Enabling Grids for E-sciencE
INFSO-RI-508833
• Use cases• The Grid Resource Allocation and Agreement Protocol• The gLite Agreement Service:
– Architecture– Components
• Workload Management System instrumentation• Implementation: the space reservation Agreement
Service prototype• Future work
Enabling Grids for E-sciencE
INFSO-RI-508833
Agreement InitiatorAgreement Initiators
Agreement Offer
Storage/Computing/Network Agreement Service
Reservation and AllocationService Providers
The gLite Agreement Service 2/2
Enabling Grids for E-sciencE
INFSO-RI-508833
The gLite Agreement Service 1/2• Agreement offer
– the “description” of the requested reservation• Agreement Initiator
– WMS Reservation Manager (reservation manager)– Data Scheduler– User
• Agreement Service– Checks offer compliance to one of theinternal templates– Can interact with several types of resource reservation service
providers (network – JRA4 bandwidth advance reservation, storage – StoRM, computing)
• Resource reservation Service Provider Enforces the reservation on a specific resource instance
• --> Full architecture described in [3]
Enabling Grids for E-sciencE
INFSO-RI-508833
Agreement Service deployment
Reservation Manager (simple and co-allocation)
Agreement Offer_1
Logging &Bookkeeping
Agreement Service_1
Monitor_1
Reservation usageSensor_1
Service Provider_1
Workload Manager System
Agreement Offer
Agreement Service_n-1 Agreement Service_n
Agreement Offer_n-1 Agreement Offer_n
Resource
Monitor_1
Reservation usageSensor_1
Service Provider_1
Resource
Enabling Grids for E-sciencE
INFSO-RI-508833
• One Service Provider instance per “resource” instance (e.g. CE and SE).
• manages resource availability information over time through a local “slot table”
• enforces resource reservation through resource-specific interfaces/for example:– Network:
the set of agents responsible of configuring switches/routers on a given path
– Storage: srmReserveSpace / srmReleaseSpace
• Monitoring of resource usage
Component 1. Service Provider
Enabling Grids for E-sciencE
INFSO-RI-508833
Example: the JRA4 network Service Provider
“The Network Element should be seen as a set of services rather than a single component in the architecture . It provides user-oriented network services to the Higher Layer Middleware by abstracting the physical network services. Specifically it will allow reservation of network bandwidth and the definition of the desidered network QoS. This may involve mapping the the higher level service requests into network specific service requests.” [1]
Network Agreement Service
Network Service Provider
Enabling Grids for E-sciencE
INFSO-RI-508833
Component 2. Agreement Service • Functionality:
– Interacts with one or more service providers– Translates high-level service description terms (from initiator) to low level service-specific
terms– Advertises the Service Provider capabilities through agreement templates. The template is
an XML document that describes the contract skeleton. – Handles the agreement negotiation (agreement offer attributes from the initiator are tuned
during the negotiation phase) – Provides information about: status of the agreement negotiation process and attributes of a specific
agreement instance
• Benefits:1. It hides the complexity of the service providers’ interfaces from the agreement initiator (the
client).2. It exposes a single operation which can be used independently of the nature of the
agreement type (createAgreement). 3. Scalability: new Service Providers can be easily integrated by advertising the
corresponding templates (clients can be notified when new templates are added).4. No need of a-priori knowledge about services available to users.
Enabling Grids for E-sciencE
INFSO-RI-508833
Component 3. Agreement Initiator: the WMS Reservation Manager
• Reservation Manager: – a proposed component of the Workload Management
System (WMS) [4] which handles: Simple agreements (to start with) Compound agreements (next in development roadmap)
• Why integrated in the WMS?– Agreement offer submission: similar to job submission– Agreement status: information from Logging and Bookkeeping– Agreement Service discovery: a list of Agreement Services is
tried (more powerful than submission to a single Agreement Service)
based on matchmaking library takes advantage of the WMS internal cache
– Periodic re-submission in case of failure
Enabling Grids for E-sciencE
INFSO-RI-508833
• Use cases• The Grid Resource Allocation and Agreement Protocol• The gLite Agreement Service:
– Architecture– Components
• Workload Management System instrumentation• Implementation: the space reservation Agreement
Service prototype• Future work
Enabling Grids for E-sciencE
INFSO-RI-508833
WMS instrumentation
UserInterface
AgreementOffer
Taskqueu
e
Match
Maker
ReservationManager
Cache (ISM)Cacheupdate
r
Logging and Bookkeeping
Reserv.Helper
Agr.offer + Resource list + Agr. Service EPR list
Web Service interface
Reserv.Cleaner
Enabling Grids for E-sciencE
INFSO-RI-508833
WMS Agreement Service Discovery
• Problem: given an agreement offer including1. Resorce requirements2. Resource preferences (optional)3. Service functionalityfind Agreement Services that support service functionality (3) and can(indirectly) do reservation on resource instances satisfying (1) and (2)
• Matchmaking finds a list of resources: – on which user is authorised to do reservation and– satisfying resource requirements (1), preferences (2), and supporting (3)
• find Agreement Services:– authorized to talk to the resources in the list identified above– Supporting the corresponding allocation and reservation service provider
interfaces
Enabling Grids for E-sciencE
INFSO-RI-508833
• Use cases• The Grid Resource Allocation and Agreement Protocol• The gLite Agreement Service:
– Architecture– Components
• Workload Management System instrumentation• Implementation: the storage space reservation
Agreement Service prototype• Future work
Enabling Grids for E-sciencE
INFSO-RI-508833
Implementation 2/2
• Agreement service:– based on WS-Agreement XML Schema Definitions (GGF,
GRAAP WG)– ... but operations exposed are different (no Web Services
Resource Framework - WSRF)– gSOAP 2.7.0– agreement service tested with StoRM (SRM v 2.1)
Enabling Grids for E-sciencE
INFSO-RI-508833
• Use cases• The Grid Resource Allocation and Agreement Protocol• The gLite Agreement Service:
– Architecture– Components
• Extension of the Workload Management System• Implementation: the storage space reservation
Agreement Service prototype• Future work
Enabling Grids for E-sciencE
INFSO-RI-508833
Future work• Extension of the prototype's functionality set
– Improvement of error handling– Authorization– Study of deployment model– Implementation of new operations– Full integration with WMS– Agreement monitoring through LB
• Performance testing• Support of more Service Providers:
– Network– computing
Enabling Grids for E-sciencE
INFSO-RI-508833
References• [1] Specification of interfaces for bandwidth reservation
service, EGEE deliverable DJRA4.1, May 2005 (http://edms.cern.ch/document/501154)
• [2] The Grid Resource Allocation and Agreement Protocol Working Group; Global Grid Forum (https://forge.gridforum.org/projects/graap-wg).
• [3] gLite Allocation and Reservation Architecture, EGEE JRA1 technical report (http://edms.cern.ch/document/508055)
• [4] EGEE Middleware Architecture; EGEE Deliverable JRA1.1(http://edms.cern.ch/document/476451)