17 March 2008Standards for Interoperable Grids 1 Job Execution Standards for Interoperable Grids:...

17 March 2008 Standards for Interoperable Grids 1

Job Execution

Standards for Interoperable Grids: Experience from NextGRID and OMII-Europe

Clive Davenhall

National e-Science Centre, University of Edinburgh

Standards for Interoperable Grids 2

Job Execution: Introduction Running jobs on a Grid.

Topics:

Naming and addressing (brief introduction; needed by this talk and the next)

OGSA-EMS

OGSA-BES

JSDL


NextGRID Naming and Addressing NextGRID uses WS-Addressing and WS-Naming.

WS-Addressing W3C standard. Way of addressing components, services, data items etc. De facto standard in the Web services community.

WS-Naming OGF Standard. Extends WS-Addressing. By adding optional naming elements.

NextGRID Has some additional extensions and restrictions.


Job Execution

Topics:

Naming and addressing

OGSA-EMS

OGSA-BES

JSDL


OGSA-EMS EMS: Execution Management Services

An architecture for initiating, managing and controlling the execution of ‘activities’ or ‘units of work.’

Units of work might be: OGSA applications. Legacy applications running in an OGSA harness.

An architecture not a standard So in the OGSA ‘informational’ rather than ‘recommendation’

track.


Role and Status of OGSA-EMS OGSA uses EMS to identify and position standards that

address part of the ‘activity management’ problem. Tool or mechanism for identifying necessary standards and their

interrelations.

This effort continues.

Not all the necessary standards are yet in place.

Initial focus on fundamental job execution.

Where possible OGSA adopts or adapts existing standards, Rather than inventing new ones ab initio.


Problems addressed by EMS Finding possible locations where activities can

execute.

Selecting the execution location for an activity.

Preparing for execution.

Initiating the execution.

Etc…


EMS Standards BES

Basic Execution Service

JSDL Job Submission Description Language

CDDLM Deployment API Configuration, Description, Deployment and Lifecycle Management

Deployment API

CDDLM CDL Configuration, Description, Deployment and Lifecycle Management

Configuration Description Language

ACS Applications Content Service

RSS Resource Selection Services


Job Execution

Topics:


OGSA-EMS

OGSA-BES

JSDL


OGSA-BES BES: Basic Execution Service

Defines Web service interfaces for creating, monitoring and controlling computational entities: Units of work. Activities.

Individual activities to be performed are defined in JSDL.

Activities execute on a suitable computational resource. Agnostic about the underlying environment on which the activity

executes.

BES interface is extensible.


BES Port-types

BES-Management Allows clients to monitor and control a BES instance. Intended for system administrators.

BES-Factory Allows clients to create, monitor and control sets of

activities.

BES-Activity Allows clients to create, monitor and control individual

activities.


Extensibility State Model

A set of defined states and the transitions between them. BES defines a basic mode and allows extensions.

Information Model Allows clients to examine aspects of a BES instance and its

applications. Allows mandatory, optional and user-defined attributes.

Resource Model Specifies basic operations, but allows additional ones. Specifically allows various OGSA interaction modes.


Basic State Model

Pending Finished

Failed

Running

Terminated

System error/failure event

Request to terminate activity

Activity terminatessuccessfully


Basic State Model

Allows extensibility by specialisation.

Pending Finished

Failed

Running

Terminated

System error/failure event

Request to terminate activity

Activity terminatessuccessfully


BES-Factory Attributes IsAcceptingNewActivities CommonName LongDescription TotalNumberOfActivities ActivityReference TotalNumberOfContainedResources ContainedResource NamingProfile BESExtension LocalResourceManagerType ResourceName OperatingSystem CPUArchitecture CPUCount CPUSpeed PhysicalMemory VirtualMemory


Operations Provided by Port-types

BES-Management StopAcceptingNewActivities StartAcceptingNewActivities

BES-Factory CreateActivity GetActivityStatuses TerminateActivities GetActivityDocuments GetFactoryAttributesDocument


BES-Factory CreateActivity Used to request the creation of a new activity.

In its simplest form:

<bes:ActivityDocument><jsdl:JobDefinition>

…</jsdl:JobDefinition><xsd:any/>*

</bes:ActivityDocument>


Job Execution

Topics:


OGSA-EMS

OGSA-BES

JSDL


JSDL JSDL: Job Submission Description Language.

Not an OGSA standard; but adopted by OGSA.

Abstract description of the requirements for a job, activity or unit of work.

Can contain: Generic job identification information. Application description. Resource requirements (main focus is on computational jobs). Description of any data files required.

Describes what the job is to do, not how to do it.


JSDL and BES JSDL is an abstract description of a job or activity.

It deliberately does not define a job submission interface.

And it is therefore independent of any submission interface.

BES provides a submission interface.

BES

Container JSDL


Extensions

An extension for applications executing on a POSIX-compliant operating system is included in the basic specification.

Two additional extensions have subsequently been defined: JSDL HPC (High Performance Computing) extension. JSDL SPMD (Single Program Multiple Data) extension.

Additional extensions are anticipated.


JSDL Document Structure

<JobDefinition><JobDescription>

<JobIdentification ... />?<Application ... />?<Resources... />?<DataStaging ...

/>*</JobDescription><xsd:any##other/>

</JobDefinition>

Note: None [1..1] ? [0..1] * [0..n] + [1..n]

Note: None [1..1] ? [0..1] * [0..n] + [1..n]


JSDL Document Structure



/>*</JobDescription> <xsd:any##other/>

</JobDefinition>

Job Identification

Application description

Resource requirements

Data files required

Note: None [1..1] ? [0..1] * [0..n] + [1..n]

Note: None [1..1] ? [0..1] * [0..n] + [1..n]


Job Identification Element




</JobDefinition>

<JobIdentification><JobName ... />?<Description ... />?

<JobAnnotation ... />?<JobProject ... />? <xsd:any##other/>

</JobIdentification>


Job Identification Example




</JobDefinition>

<JobIdentification><JobName>

My Job</JobName><Description>

Simple application</Description>…

</JobIdentification>


Hello World in JSDL

<?xml version="1.0" encoding="UTF-8"?><jsdl:JobDefinition xmlns:jsdl=“http://schemas.ggf.org/2005/11/jsdl” xmlns:jsdl-posix= “http://schemas.ggf.org/jsdl/2005/11/jsdl-posix”><jsdl:JobDescription> <jsdl:Application> <jsdl-posix:POSIXApplication> <jsdl-posix:Executable> /bin/echo <jsdl-posix:Executable> <jsdl-posix:Argument>hello</jsdl-posix:Argument> <jsdl-posix:Argument>world</jsdl-posix:Argument> </jsdl-posix:POSIXApplication> </jsdl:Application> </jsdl:JobDescription></jsdl:JobDefinition>


NextGRID Recommendations

BESReiterates that extensions to WS-Naming are

permitted, specifically the ones required by NextGRID naming.

JSDLDefines extensions to support data-staging

which is compatible with NextGRID data transfer.

17 March 2008Standards for Interoperable Grids 1 Job Execution Standards for Interoperable Grids:...

Documents

Transcript of 17 March 2008Standards for Interoperable Grids 1 Job Execution Standards for Interoperable Grids:...