Grid Access Service Predrag Buncic

EGEE is a project funded by the European Union under contract IST-2003-508833

Grid Access Service

Predrag Buncic

JRA1 Meeting, 28-30 Jun 2004

www.eu-egee.org

JRA1 Meeting, 28-30 June 2004 - 2

Introduction

• JRA1 Clusters Integration Testing Information and Monitoring Data Management Workload Management Security Etc…


Current gLite Prototype

• A Prototype Middleware on a testbed consists of AliEn “shell” Job submission:

• Alien CE->Condor-G->blaph->PBS/Condor• Globus Gatekeeper

Data Management• AliEn File & Metadata catalog • AliEn SE with Castor & D-Cache SE with SRM• gridFTP for transfers• AliEn FTD• Aiod/GFal • RLS (EDG)

Security• VOMS for certificate handling/SE gridmap files

(NIKHEF)• MyProxy for certificate delegation in GAS

GAS (Grid Access Service)• Prototype with a few file cataloging functions

R-GMA & EDG WMS (not integrated yet)

ExtraTerrestrialCluster

[looking for help!]


Talk Outline

1) API and Grid Access Service (GAS) Why and how? Service Controller (or Controller Service)?

2) GAS & Prototype (=>Pablo)

3) Package Manager Why and how?


API and GAS


Starting from AliEn…

User Interface Factory

Auditing

DBD/RDBMS

Registry/Lookup/Config

V.O. directory

Authentication

Storage Element

Gatekeeper

Job Manager

Transfer Manager

File Transfer

Process Monitor

Transfer Broker

Job Broker

Job Optimizer

Transfer Optimizer

Catalogue Optimiser

User Interface

Grid Monitoring

CE

1

1..n1

1

0..n

1..n

1

1

1

1

11

1

1

1

0..n

0..n

0..n

0..n

0..n

1. lookup

2. authenticate

3. register

4. bindAuthorisation

File Catalogue

Metadata Catalogue

Task Queue

DB PRoxy

1

1

1

1

Package Manager

Job Provenance

1

Authorisation

Accounting

111111111111111

1

1

API


Stepping stone: ARDA…

Information Service

Authentication

Authorisation

Auditing

Grid Monitoring

Workload Management

Metadata Catalogue

File Catalogue

Data Management

Computing ElementStorage

Element

Job Monitor

Job Provenance

Package Manager

Grid Access Service

Accounting

Grid Access Service

User Application

Site Gatekeeper

7: 12:

5:

13:

8: 15:

11:

9: 10:

1:

4:

2:

3:

6:

14:


Design team working document..


DJRA1.1


Client (application) side: API

API

(from User Application)+ Authentication

+ Data Management+ Grid Service Management

+ Job Control+ Metadata Management

+ NewInterface+ Posix I/O

SOAP

(from API)

Grid File Access

(from API)

User Application

POOL/ROOT/...(from User Application)

API (OGSI User Interface Factory)

SE (POSIX I/O service)

An API would be a library of functions used for building client applications, graphical user interfaces or even Grid Web portals (e.g. AliEn, Genius or Clarens).

The API is used also to authenticate user to the Grid, let them submit jobs inquire job status and manage jobs access the files available on the Grid and put users files onto the Grid

The application should be able to gain access files on the Grid by issuing requests to copy files to local temporary storage or via POSIX like interface to a near Storage Element.


Server side: GAS

• The Grid Access Service (GAS) is the counterpart of the user API on service side and represents the user entry point to a set of core services.

Many of the User Interface API functions are simply delegated to the methods of the GAS. In turn many of the GAS functions are delegated to the appropriate service.

GAS service will be constructed out of Service Components that will in turn present a uniform public interface to underlying service.

The components from which the interface is constructed can be defined by the VO preferences at run time

Grid Service Components

Grid Service Component

Library

Construction specification

Composition Logic

Composition TypeMessage Dependency

Interface specification

Messages Operations

Grid Applications

The Service Components are realized as a pluggable library with each component providing an interface to the specific middleware service.

The intention was to define end user interface that allows them to interface their application to the Grid and keep this interface reasonably stable and protected from

inevitable changes in the middleware.


Grid Login Use Case

The application then connects to the GAS Factory and passes over the secure line user name and password needed to get delegation of user credentials from the credential wallet. If this operation is successful, the GAS Factory will start a GAS service for the user and return the service endpoint. The application then connects to its endpoint and gains access to other Grid service.

Many of the User Interface API functions are simply delegated to the methods of the GAS. In turn many of the GAS functions are delegated to the appropriate service.GAS service will be constructed out of Service Components that will in turn present a uniform public interface to underlying service. During its creation the user is authenticated

and his rights for various Grid operations are checked against the Authorization Service. The GAS keeps the user credentials and authorization information.

Before attempting to connect to the Grid, a user is expected to register his or her temporary credentials with VO independentcredential wallet (like the NERSC secure MyProxy).

As a first step in connecting to the Grid, the user application uses the API to connect to a configurable list of Configuration Services. These are the public services that can exist per VO or serve multiple VOs. They inquire VO configuration and Information Services as well as use DNS information to deliver initial configuration back to the user API


Controller Service

GAS lifetime will be restricted to the lifetime of delegated proxy credentials and will be managed by the Controller Service and user who will be able to destroy his own GAS instance.

An instance of GAS should be created in a service environmentin the proximity of the user (local site) where proper containerhas been located

The GAS factory will ask Controller Service for appropriate service endpoint

The Controller can decide to create local service or can contact another

Controller


GAS: Summary

• The GAS model of accessing the Grid is in many ways (authentication, proxy service) similar to various Grid Portals but it is meant to be distributed (the GAS Factory can start GAS in a service environment close to the user in network terms) and is therefore more scalable and resilient.

• As opposed to a traditional Web Portal, the GAS interface is more dynamics and reflects the role of the user in the system.

• The GAS is intended to be used by the application and not by the interactive user therefore it offers no presentation layer.

• The traditional Grid Portal can actually be easily constructed by specializing GAS into a Portal Service that will provide necessary content to a presentation layer provided by the Web Server.

• Similarly, the specialized application services can be constructed by extending GAS or providing appropriate Service Components.

• Grid services also have to be accessible directly using their respective mechanisms (i.e. not via the GAS).

Package Manager Service


Package Manager Service

• A Package Management Service is a helper service that automates the process of installing, upgrading, configuring, and removing software packages from a shared area (software cache) on a grid site.

• Explicitly requested by the users of a prototype

• This service represents an extension of a traditional package managment systems to distributed computational environment and it should use one of the estabilished package management systems as a backend.

• Some well-known examples of such systems include

RPM, Red Hat's package manager, used not only by Red Hat Linux but by several other Linux distributions.

dpkg/APT (used originally by Debian GNU/Linux, now ported to other systems). Portage, used by Gentoo Linux and inspired by the BSDc ports system. The ``ports tree'' system used by FreeBS NetBSD,OpenBSD and the like. Pacman, package manager developed at Boston Univerity and used by several Grid projects

(International Virtual Data Toolkit - iVDGL, Grid3)


Basic assumptions

• The software is distributed in packages, usually encapsulated into a single file that contains metadata that describes the package's details, including its name, checksums, and dependencies on any other packages that it needs to work.

• It may also include information on how to configure the package for use and how to remove the package cleanly when it is no longer required.

• The package manager then uses this information to install, configure, and remove packages as requested by the user.

• The PM Service operates in the context of a VO and understands and resolves possible dependencies between the package versions provided by the V.O. administrator.

• This service is not responsible for the maintenance and deployment of middleware or system software components, unless the VO takes the responsibility to provide and maintain the middleware and/or system software as a part of the VO contributed software.


Use case scenario

• In a typical scenario, the VO package administrator creates the binary package caches for one or more computing platforms, verifies and possibly digitally signs them. These caches are then published and made available for download via the PM Service.

• On the execution site, a local instance of PM Service will, on request from CE or JW, fetch and install binary packages into the local package cache. This local package cache should reside on a file system managed by the PM Service assuring that unused old packages are removed if disk space is needed to install newer versions.

• The existence of binaries can be advertised, thus minimizing download of packages from multiple locations. In this way, the PM Service could maintain the hierarchy of package caches to assure scalability and provide a fail-over capability.

• Access to VO packages should be controlled and possibly restricted and audited.

• The easiest way to achieve that is to treat the packages as any other File Catalogue entry and to apply common Authentication, Authorization and Auditing mechanisms. The integrity of individual packages should be verified by appropriate checksums.

• The package metadata information (including checksum information) should be retrieved from a trusted and certified VO site, independently from the package itself.


Package Manager: Summary

• Service urgently required by users

• The software components needed for realization of such service exist

• Possible implementation scenario Reworked AliEn Packman component exposed as a service using

one (or more) of the package managers as a backend Try to extract minimal package manager interface to allow

alternative package manager backends Personal preference: start with Portage

• Some prototyping needed


Issues


Issues

Issues Configuration Service

• Discover VO services• Bootstrap client application

Alternative transport and messaging protocols• SOAP over IM protocol (Jabber) • No need for incoming IP connectivity• Service presence information as bonus• Scalable asynchronous system logging (syslog)

Grid Access Service Predrag Buncic

Documents

Transcript of Grid Access Service Predrag Buncic