Real-World Distributed Computing with Ibis Henri Bal [email protected] Vrije Universiteit Amsterdam.

Real-World Distributed Computing with Ibis

Henri [email protected]

Vrije Universiteit Amsterdam

Outline

● Distributed systems

● Distributed applications

● The Ibis distributed programming system

● Ibis in the mobile networked world

Distributed Systems: 1980s

● Networks of Workstations (NOWs)● Collections of Workstations (COWs)● Processor pools (Amoeba)● Condor pools● (Beowulf) clusters


● Metacomputing (Smarr & Catlett, CACM)● Flocking Condor (Epema)● DAS (Distributed ASCI Supercomputer)● Grid Blueprint (Foster & Kesselman)● Desktop grids, SETI@home

DAS-3


● Cloud computing● Infrastructure as a service● Virtualization

● Mobile computing● Sensor networks● Smart phones

● The Networked World

Problem

● How to write high-performance applications for real-world distributed systems?

● How to integrate many different resources?

Our approach

● Study fundamental underlying problems

● … hand-in-hand with realistic applications

● … integrate solutions in one system: Ibis

Distributed SystemsUseUserr

!

Fundamental problems

● Performance – efficiency on wide-area systems

● Heterogeneity – different systems & APIs● Malleability – resources come and go● Fault tolerance - crashes● Connectivity – firewalls, NAT, etc.

Applications

● Scientific applications● Imaging (VU Medical Center, AMOLF)● Bioinformatics (sequence analysis, cell

modeling)● Astronomy (data analysis challenge)

● Multimedia content analysis● Games and model checking● Semantic web (distributed reasoning)

Awards

SCALE 2008

(CCGrid’08)

DACH 2008 – BS DACH 2008 - FT

AAAI-VC 2007

ISWC 2008 Multimedia

Computing

Astronomy

Semantic Web (van Harmelen et al.)

(Cluster/Grid’08)

Multimedia content analysis

● Automatically extract information from images & video

● Extract feature vectors from images● Describe properties (color, shape)● Data-parallel task on a cluster

● Compute on consecutive images● Task-parallelism on a grid

MMCA

‘Most Visionary Research’ award at AAAI 2007, (Frank Seinstra et al.)

Games and Model Checking

● Can solve entire Awari game on wide-area DAS-3● Needs 10G private optical network

(StarPlane)

● Distributed model checking has very similar communication pattern● Search huge state spaces, random

work distribution, bulk asynchronous transfers

● Can efficiently run DeVinE model checker on wide-area DAS-3, use up to 1 TB memory● See IPDPS’09 (May 2009)

Distributed reasoning

● MaRVIN (Frank van Harmelen et al, VU):● a distributed platform for massive RDF

inferencing (deductive closure)● ``a brain the size of a planet’’

● Uses Ibis to run on heterogeneous systems (clusters, desktop grids)

● 3rd prize at Billion Triple trackof Semantic Web Challenge 2008

European users

● D-Grid: Workflow engine for astronomy● U. Erlangen: grid file system● INRIA: ProActive on Ibis RMI● U. Patras: Jylab scientific computing● UPC Barcelona: Grid Superscalar● HITACHI: Peta-scale data management

Grid’5000

Outline





Ibis Philosophy● Real-world distributed applications should

be developed and compiled on a local workstation, and simply be launched from there

Ibis Approach● Virtual Machines (Java) deal with

heterogeneity● Provide range of programming

abstractions● Designed for dynamic/faulty environments● Easy deployment through middleware-

independent programming interfaces● Modular and flexible: can replace Ibis

components by external ones

Ibis Design

● Functionality from programming languages● High-Performance

Application Programming System

● Functionality fromoperating systems● Distributed Application

Deployment System

Ibis System

Programming system

● Programming models:● Message passing (RMI, MPJ)

● Divide-and-conquer (Satin)

● Jorus: (multimedia applications)

● Dataflow framework (Maestro) see HPDC09 Sat 14.30

● IPL (Ibis Portability Layer)● Java-centric “run-anywhere” library● Point-to-point, multicast, streaming, ….● Simple model (Join-Elect-Leave ) for tracking resources,

supports malleability & fault-tolerance

SmartSockets library

● Detects connectivity problems● Tries to solve them automatically

● With as little help from the user as possible● Integrates existing and several new

solutions● Reverse connection setup, STUN, TCP splicing,

SSH tunneling, smart addressing, etc.● Uses network of hubs as a side channel

Example

Deployment system

● IbisDeploy GUI

● JavaGAT:● Java Grid Application Toolkit

● Make applications independent of underlying middleware

● Zorilla P2P system● Job management, gossiping,

clustering, flood scheduling

JavaGAT

Grid ApplicationFile.copy(...) submitJob(...)

cpftpgridftpscphttp

forkpbs

condorunicoreglobus

● For simple grid operations there are many ways of implementing them ● different middleware available on different

sites

JavaGAT


cpftpgridftpscphttp

forkpbs

condorunicoreglobus??

● Which should you use ?● Some may not be available on all sites ● Many combinations

JavaGAT


GAT Engine

RemoteFiles

MonitoringInfo

serviceResource

Management

GridLab Globus Unicore SSH P2P LocalGATgridftp globus

Multimedia Content Analysis

Client

Broker

Servers IbisIbis

(Java)

● Runs simultaneously on clusters (DAS-3, Japan, Australia), Desktop Grid, Amazon EC2 Cloud

● Connectivity problems solved automatically by Ibis SmartSockets

Connection management

With SmartSockets: run everywhere

Standard sockets: only local VU machines can be reached due to firewalls problems

Ibis movie (part 1)

Performance on 1 DAS-3 cluster

● Relative speedups of Java/Ibis and C++/MPI● Using TCP or Myricom’s MX protocol

● Sequential performance Java: 80% of C++

Speedup (wide-area)● Homogeneous wide-area systems (DAS-

3):● Frame rate increases linearly with #clusters

● World-wide experiment:

● 24 frames per second● Speed limited by camera,

not computing infrastructure

Outline





Smart Phones

● GSM + PC + GPS + camera + networks + ….

● Location-aware● What if everyone always carries a smart

phone (like a GSM now)?● Next wave in computing?

Ibis on Smart Phones● Our focus: distributed smart phone

applications● Applications running on multiple phones● Integration with distributed computing

backbone

● Use Android for development● Google’s open-source platform● Java-based

Distributed applications● Disaster management (Katrina)

● Use ad-hoc Wifi network when GSM network fails

● Finding nearby people with certain skills● Bus drivers, CPR

● Distributed decision support● Moving people to shelters (logistics)

● Social networks● Similar issues

● Find nearby friends, decide on restaurant

Wild example

● Track position => automatic diary of your life

● Cross-comparisons between diariesHaven’t we met before?

Yes, on 23 Oct 2010, 3.48 pm atN 52°22.688´ E 004°53.990´

eyeDentify● Object recognition on a G1 smartphone● Smartphone is a limited device:

● Can run only 64 x 48 pixels (memory bound)● 1024 x 768 pixels would take 5 minutes

● Distributed Ibis version:

+ =+1024 x 768 pixels

2.0 seconds

Ibis movie (part 2)

Interdroid

Distributed Communication

Data Management

Novel Mobile Distributed Applications

Context SensitiveProgrammingModels

Current work● Raven: API for Viable Episodic Networking

● Decentralized synchronization API● Fine grained control over data sharing

● Bluetooth support for ad-hoc communication● Discovery of devices using multiple networks● Context Aware Programming Models

● Supporting distributed decision making● Representing and using context (location etc.)● Exploiting social relationships (Hyves, Facebook)

Summary● Ibis provides integrated solutions for many

hard problems● performance, heterogeneity, malleability, fault

tolerance, connectivity

● It combines functionality from programming languages and operating systems

● Used for many applications on real-world distributed systems

● Download from http://www.cs.vu.nl/ibis/

Acknowledgements

Niels DrostCeriel JacobsRoelof Kemp

Timo van KesselThilo KielmannJason Maassen

Rob van NieuwpoortNick Palmer

Kees van ReeuwijkFrank J. SeinstraKees Verstoep

Gosia Wrzesinska

Questions?

Real-World Distributed Computing with Ibis Henri Bal [email protected] Vrije Universiteit Amsterdam.

Documents

Transcript of Real-World Distributed Computing with Ibis Henri Bal [email protected] Vrije Universiteit Amsterdam.