Dynamic Grid Computing:The Cactus Worm

The Egrid CollaborationRepresented by: Ed Seidel

Albert Einstein InstituteEgrid@egrid.org

cactus@cactuscode.org

Grid Computing: a new paradigm Computational Resources Scattered Across the

World Compute servers Handhelds File servers Networks Playstations, etc…

How to take advantage of this for scientific/engineering simulations?

Harness multiple sites and devices Simulations at new level of complexityand scale

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

Components for Grid Computing Resources: Egrid (www.egrid.org)

A “Virtual Organization” in Europe for Grid Computing Over a dozen sites across Europe Many different machines

Infrastructure: Globus Metacomputing Toolkit Develops fundamental technologies needed to build

computational grids.  Security: logins, data transfer Communication Information (GRIS, GIIS)

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

Components for Grid Computing

Application: Cactus Computational Toolkit Modular Toolkit for Parallel Computation Numerical/Computational Infrastructure to

solve PDE’s Enables Grid applications of many types… www.cactuscode.org

Grid Computing Scenarios: The Vision Distributed Computing: Sit here, compute there,

monitor and steer… Managing intelligent parameter surveys

jobs to new machines, e.g. analysis tasks Dynamic staging … seeking out and moving to

faster/larger/cheaper machines as they become available

Scripting capabilities (management, launching new jobs, checking out new code, etc)

Dynamic load balancing

Application Code as Information Server/Gatherer

Code should be aware of its environment What resources are out there? What is their current state? What is my allocation? What is the bandwidth and latency between sites? How can I adjust myself to take advantage of the current state?

Code should be able to make decisions on its own A slow part of my simulation can run asynchronously…spawn it off! New, more powerful resources just became available…migrate there!

Code should be able to publish this information to central server for tracking, monitoring, steering...

Cactus has modules to enable this for any application

Cactus Worm: Illustration of basic scenario Cactus simulation starts, launched from a portal Queries a Grid Information Server, finds available resources Migrates itself to next site

Uses some logic to choose next resource Securely starts up remote simulation Transfers memory contents to remote simulation (using streaming

HDF5, scp, GASS, whatever…) Registers new location to server, terminates previous

simulation User tracks and monitors with continuous remote viz and

control using thorn http, streaming data, etc...… Continues around Europe, and so on…

If we can do this, much of what we want can be done!

Grand Picture: we are very close!Remote steering and monitoring

from airport

Origin: NCSA

Remote Viz in St Louis

T3E: Garching

Simulations launched from Cactus Portal

Grid enabled Cactus runs on

distributed machines

Remote Viz and steering from Berlin

Viz of data from previous simulations in SF café

DataGrid/DPSSDownsampling

Globus

http

Streaming Data

IsoSurfaces

Next…

Tonight: Global Grid Forum BOF Tomorrow: manchester Booth at 10:30

Thanks to Sun for sponsoring us!