Session 42 - GridSAM

Web: www.omii.ac.uk Email: [email protected]

GridSAM Introduction & Practical

Steve Crouch, [email protected]

14/07/09


Overview

• An example scenario - CPOSS• Introduction to GridSAM• How and where to get it, future

developments• The basic practical

o Client installationo Basic job submission and monitoringo A job with data – using GILDA GridFTP

service


Example Scenario - CPOSS ENGAGE Project

• Users and their applications are all important

• Grid is an enabler

CondorCondor …

Legion(PBS Torque)

Legion(PBS Torque)

…

UK NGSUK NGS

…

Job Managemen

tSystem

Job Managemen

tSystem

Workflow

WorkflowPortalPortal

GridSAM

GridSAM

GridSAMGridSAM

ActiveBPEL

ActiveBPEL


GridSAM Overview

• What is GridSAM to the resource owners?o A Web Service to expose heterogeneous execution

resources uniformly• Single machine through Forking or SSH• Condor• Portable Batch Scheduler (PBS)• Platform Load Sharing Facility (LSF)• Sun GridEngine• Globus

o Acts as a client to these resources• What is GridSAM to end-users?

o A set of end-user command-line tools and client-side APIs to interact with GridSAM Web Services

• Submit, monitor and terminate compute jobs• Cross-protocol file transfer (gsiftp, ftp, sftp, WebDav, http,

https, soon SRB, iRODS) via Commons-VFS (http://sourceforge.net/projects/commonsvfsgrid)


GridSAM and OGF Open Standards

• Two key standards for two key elements:o The Basic Execution Service interface (OGSA-BES)

• Handles basic job lifecycle management• Defines simple (but extendable) job state model

– Pending, running, cancelled, failed or finishedo The Job Submission Description Language (JSDL)

• Specify job executable, data staging and resource requirements

• Partial support for (across some DRMs):o HPC File Staging Profile – fine-grained security for

data stagingo JSDL Single Process Multiple Data (SPMD)

Application Extension


GridSAM Architecture

• A staged event-driven architectureo Submission pipeline is constructed as a network of

stages connected by event queueso Each stage performs a specific action upon incoming

events

Example Pipeline: Condor


GridSAM – Publications & Enabled Activities


OMII Development Kit: Server

• Currently version 3.4.4; comprised of:o Web Services Container

• Tomcat/Axis - Web Services provider/container with optional WS-Security - handles authentication

o Data persistence managed by readily-installable PostgreSQL or HyperSonic databases, or pre-installed MySQL

o Optionally downloadable and installable Commissioned Software server components:

• GridSAM: Job submission and monitoring• AHE: Application Hosting Environment• Grimoires: UDDI + metadata registry• …

• Development Kit Client – that’s the practical


CGT – The Campus Grid Toolkit

• Client/server solution that containso Apache Tomcat/Axis/WSS4j (WS-Security)o GridSAMo Optional - Application Hosting Environment (AHE)o PostgreSQL or HyperSonic database

• Easy installation for quick deployment via autoconfiguration

o Initial security setup for testingo Hooks into existing computational resources

• End-to-end, ‘out of the box’ working examples

• Documentation• Version 1.1.3 (with GridSAM 2.1.6) out now


Open Development

• GridSAM is Open Source, Open Development

• GridSAM SourceForge project:

o 99.03% activity, 1 release/month

o SVN source code repositoryo Developer & User mailing lists

• You are more than welcome to contribute!

http://sourceforge.net/projects/gridsam/


Future Developments

• Consolidation, extensive testing towards public 2.2.0 release end-Q3 2009

o Stress testing, improvement of support for standards• Deployable as a standalone, vanilla web

service• PBS-based deployment on NGS• JSDL Parameter Sweep Extension

o Single job submission that parameterises inputs• Policy-based brokering to other GridSAM

instances• Publication of roadmap by end-July for

comment and feedback• ...what our users want!


The Practical

• Go to http://www.ecs.soton.ac.uk/~stc/ISSGC09

• Use issgc-ui.polytech.unice.fro Username: issgc<xx> o Password: GridISS<xx>

• Included in the Integrated Practical• Good luck!• If you want more information:

o http://www.omii.ac.uk/wiki/Documentation - for docso http://www.omii.ac.uk/ - for general OMII-UK info

Session 42 - GridSAM

Education

Transcript of Session 42 - GridSAM