Globus Grid TutorialPart 2:

Running Programs Across Multiple Resources

Using Multiple Resources 2

Learn how to couple resources at multiple sites and use them for a single application Required by very large applications Also by applications that need a heterogeneous

mix of resources Learn how to run existing parallel codes under

Globus By using MPICH-G, a grid-enabled MPI

Other application examples include SF-Express, climate models, etc.

Goals of this Tutorial

Using Multiple Resources 3

Gas Dynamics (PPM)

BrickManager

BrickUpdater

BrickUpdater

BrickUpdater

TaskManager

BrickManager

Archiver

Woodward, U Minn.

Tightly coupled CFD problem Needs large

computational power

Mask latency by overlapping communication and computation Move data a brick at a

time Size bricks to CPU and

network

Using Multiple Resources 4

Problems

How do we start a program running across multiple machines? Co-allocation and scheduling Different schedulers and security systems

What programming model should be used? Can we run existing applications?

Using Multiple Resources 5

Globus Advantages Resource management architecture provides

co-allocation tools Can mix communication methods

Nexus multimethod communication MPI, sockets, etc.

Uniform access to local services Security, resource management, etc.

Architecture promotes building high-level programming tools E.g., MPICH-G, a grid-enabled MPI

Using Multiple Resources 6

Programming Tools: Approaches

“Hand coded” applications, combining existing tools with Globus calls Use sockets, MPI, threads, SHM, etc. Globus security and resource management

still provide added value Grid-enabled libraries

Manage both communication and resource management

Provide uniform programming environment across resources

Using Multiple Resources 7

MPICH-G

A complete implementation of the Message Passing Interface (MPI) Passes MPICH regression test without change

MPI is the defacto standard for message-passing, parallel programs.

Enables existing MPI programs to run within a grid environment without change. 

Documentation http://www.globus.org/mpi

Using Multiple Resources 8

Running a Program Goal: Run a Message Passing Interface (MPI)

program on multiple computers

MPICH-G uses Globus for authentication, resource allocation, executable staging, output redirection, etc.

% mpirun -np 4 my_app

4 32

1

Using Multiple Resources 9

Running an MPICH-G Program

Create a file named “machines” A list of Globus managers and counts

sp2.sdsc.edu-loadleveler 4

neptune.cacr.caltech.edu-lsf 4

jupiter.isi.edu-fork

% mpirun -np 12 my_app Creates a total of 12 tasks allocated in a

round-robin fashion with “count” tasks per allocation request

(sp2 4) (neptune 4) (jupiter 1) (sp2 3)

Using Multiple Resources 10

How MPICH-G Works mpirun:

Locates complete globus resource manager information for specified resources (MDS)

Creates resource specification request Calls globusrun to execute the program

Uses Nexus for communication Delivers enhanced performance by using

multiple communication protocols

Using Multiple Resources 11

Starting Multiple Jobs

The globusrun command: Submits multiple simultaneous job requests Stages executables (GASS) Waits for termination (GRAM/DUROC) Forwards stdout/stderr (GASS)

Convenient wrapper around several Globus services: DUROC, GASS, GRAM, GSI, MDS

Using Multiple Resources 12

Globus Resource Managers

Every resource is controlled by a resource manager called a GRAM Interfaces to local resource management

system, e.g., LoadLeveler, NQE, LSF. Every resource manage has a unique

distinguished name, or DN DN is a sequence of attribute-value pairs:

/C=US/O=Globus/O=USC/OU=ISI/CN=jupiter.isi.edu-fork

The MDS stores information about each resource manage

Using Multiple Resources 13

Limitations of Simple mpirun Limitations of “machines” file

Executable staging only for homogeneous sets of machines

For heterogeneous sets, executables must be placed in the same location on every machine

More general MPICH-G startup is possible Dynamic discovery of resources Specify name of the executable at each site Specify location of executables and data files

Currently achieved by passing RSL string

Using Multiple Resources 14

Exercise 2Introduction to MPI

Use mpirun to run an MPI program% mpirun -np 2 program

Use globus-rcp to copy files remotely% globus-rcp filename host:filename

Using Multiple Resources 15

Globus Components in Action

DUROC

fork

P1 P2

GRAM

LSF

P1 P2

GRAM

LoadLeveler

P1 P2

GRAM

Nexus

globusrun

mpirun

Using Multiple Resources 16

Summary

Using multiple resources located in multiple domains is a basic grid operation

Globus supports this operation via core services and high-level tools

Standard MPI programming environment provides a convenient way of building grid applications Must be careful about configuration and

latency