EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 New Generation...

EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005

WP2-3New Generation Environment

for Grid Interactive MPI Applications

Migrating Desktop / Roaming Access Server

Miroslaw Kupczyk -Rafal Lichwala - Bartek Palakmiron - syriusz -bartek[@man.poznan.pl]

Marcin Plociennik - Pawel Wolniewicz marcinp - pawelw[@man.poznan.pl]

Stefano Beco - Marco Sottilaro stefano.beco-marco.sottilaro[@datamat.it]

PSNC (Poland) - Datamat S.p.A. (Italy)


WP2-3Outline

Objectives

What is interactivity ?

CrossGrid architecture for interactive services

CrossGrid key components: Migrating Desktop & Roaming

Access Server

Interactivity schema

at the end …. some conclusions


WP2-3Objectives !

To provide user-friendly framework for grid interactive and data intensive applications

To support parallel and real-time computation

To hide the complexity of data and resources handling

To deliver one uniform GUI that supports different HPC infrastructures on different platforms

Application developers

requirementsReal userrequirements

Grid without real users is useless!!!


WP2-3Batch Jobs vs Interactive Jobs

“I submit my job over the grid, and ... ………… I wonder what I want ….”

1) „I want it to be executed now!”(priority and advance reservation)

2) „I want to monitor the execution” (checking the status)

3) „I want to know its registered events” (logging information)

BATCH JOBSBATCH JOBS

I want to see its final output

I wait for the end of the job !!!

INTERACTIVE JOBSINTERACTIVE JOBS

While the ones are running I want to:1) to receive any outputs2) send my input data to it


WP2-3Requirements for interactivity

When I want to submit an interactive job I need:

the allocation of grid resources throughout the whole “interactive session” ;

a bi-directional channel between my client and the application program running on a remote machine;

the input and output streams are exchanged with my client via this channel (I can send my input data to the job and receive its intermediate output and error

messages).


WP2-3Architecture

JobSubmissionServices

CrossBroker

EDG 2 LB

Roaming Access Server

ComputingElement

WN Node

Java Visualisation/VNCplug-ins

Migrating Desktop

Interactive Session Services

File Management Services

Profile Management Services

Application Management Services

Virtual Directory

II / LDAP

SE


WP2-3Migrating Desktop

Key CrossGrid component

Integration point of most CrossGrid tools and application

Open system that integrates access to many grids all over the world

It provides work environment to users in any location, independently of a hardware platform and the operating system

A flexible system structure enables defining and adaptation requirements of individual groups of users

Advanced user-friendly Java application based on web services and Java CoG technology

Interactive Application oriented framework giving the entire work environment anywhere in the grid


WP2-3How does it look like?


WP2-3Migrating Desktop tools

Desktop tools- Job Wizard- Job Monitor- Application Container and Application Plugin- GridFTP Commander- User Profile Manager- Private Storage Management

- VNC/SSH console


WP2-3Migrating Desktop functionality

Main Features: Single sign-on / authorisation Platform independent Batch jobs MPI jobs Running interactive

applications using java plugins or VNC

Monitoring grid applications Flexible Application

framework User profile management Easy application add on Local and grid file

management


WP2-3Supported applications and tools

CrossGrid Applications:• Medical application• Air pollution• HEP• ...

CrossGrid Tools:• GridBench• PPC• Marmot• OCM-G• ...


WP2-3Roaming Access Server

Well-defined set of web-services An interface for accessing HPC

systems and services (based on various technologies) in a common and standardised way

Interconnection between various grid middleware and applications

Additional features:• Virtual Directory support• Plug-in for various grid

middleware








WP2-3Job Submission Services (JSS)

a set of Web Services, which allows performing job submission and job monitoring:• Job-Submission • Job-Cancel (cancelling of one or more

submitted jobs); • Job-Status (bookkeeping information on

the submitted jobs); • Job-Logging-Info (logging information on

the submitted jobs);• GetUserJobs (list of the jobs “recently”

submitted by users) ;• Job-List-Match (list of resources fulfilling

the user job requirements)








WP2-3Job Submission Services (JSS)

advanced services for MPI and interactive jobs batch jobs :

• sequential and parallel (MPI) ? ok ! interactive jobs :

• sequential and parallel (MPI) ? ok !

two types of parallel jobs based on the MPICH library:• “MPICH-P4” (single cluster)• “MPICH-G2” (multiple clusters)

Interactive services based on Condor ByPass technology (to handle i/o streams between jobs on remote machine and user applications)


WP2-3Interactivity for

MD-integrated applications

Job Submission Services CrossBroker

Logging&

Bookkeeping


Computing Element

LRMS

Gatekeeper

JDLMigrating Desktop

RAS shadow port,

RAS shadow host

Process Launched

Interactive data

Control data

Submission flow

In/Out/Err job data

010011000

WorkerNode

Job Shadow

stdin – stdout -stderr

Console Agent

JobJob

Condor ByPass SystemCondor ByPass System

Job Shadow

Java Visualisation

plug-in


WP2-3Conclusions

Supporting real grid interactive MPI application Extending EDG interactivity approach –

interactive application at user desktop Providing trasparency of grid files and resource

management Integration of CrossGrid tools and applications Easy to use in production environment

• not only grids• easy to adopt for special purposes (computing on-

demand) MD/RAS already used by other grid projects (e.g.

Progress project)


WP2-3Product availability- contact

Released under open source licence Migrating Desktop website:

http://ras.man.poznan.pl/crossgrid

You can plug&play there the last MD/RAS server version


WP2-3

Thanks for your attention

l

http://www.datamat.ithttp://www.datamat.it www.man.poznan.plwww.man.poznan.pl

if you need further information :

[email protected]

[email protected]

EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 New Generation...

Documents

Transcript of EGC 2005, CrossGrid technical achievements, Amsterdam, Feb. 16th, 2005 WP2-3 New Generation...