The Ibis model as a paradigm for

programming distributed systems

Henri Balbal@cs.vu.nl

Vrije Universiteit Amsterdam

(from Grids and Clouds to Smartphones)

Outline

● History of distributed systems

● Clusters, grids, clouds, networked world

● Programming distributed systems

● Driving applications

● The Ibis system

● Ibis on smartphones

History of the Distributed World – Part

I (1980s)● Networks of Workstations (NOWs)● Collections of Workstations (COWs)● Processor pools

(Amoeba)● Condor pools● (Beowulf) clusters

Amoeba processor pool (Zoo, 1994)

History of the (more) Distributed World – Part

II● Metacomputing (Smarr & Catlett, CACM

1992)● Flocking Condor (Epema, FGCS 1996)● Distributed ASCI Supercomputer (1996 –

?)● Grid Blueprint (Foster & Kesselmann

1998)● Desktop grids, SETI@home (1999)

Design of DAS (1996)

DAS-1 DAS-2 DAS-3 DAS-4

(slide from Andy’s ASCI’97 presentation)

DAS-3

A real (heterogeneous) Grid

History of the (Modern) Distributed World – Part

III● Cloud computing

● Infrastructure as a service● Virtualization

● Mobile computing● Sensor networks● Smart phones

● The Networked World

Problem

● How to write high-performance applications for real-world distributed systems?

● How to integrate many different resources?

Our approach

● Study fundamental underlying problems

● … hand-in-hand with realistic applications

● … integrate solutions in one system: Ibis

Distributed SystemsUseUserr

!

Fundamental problems

● Performance – efficiency on wide-area systems

● Heterogeneity – different systems & APIs● Malleability – resources come and go● Fault tolerance - crashes● Connectivity – firewalls, NAT, etc.● Security – very hard

Case study: spam filters

#1 Dutch Computer scientist

Top security-expert

CONGRATULATIONS!! YOU HAVE WON2.5 MEURO!!

Dear prof. Tanenbaum,

You may not know me,but I’d like to give you2.5 Million Euro to do research. Please come to Brussels to collect the money.

Yours sincerely,

Mr. V.I. Person

Applications

● Scientific applications● Imaging (VUMC, AMOLF)● Bioinformatics (sequence analysis, cell

modeling)● Astronomy (data analysis challenge)

● Multimedia content analysis● Games and model checking● Semantic web (distributed reasoning)

Multimedia content analysis

● Automatically extract information from images & video

● Extract feature vectors from images● Describe properties (color, shape)● Data-parallel task on a cluster

● Compute on consecutive images● Task-parallelism on a grid

MMCA

‘Most Visionary Research’ award at AAAI 2007, (Frank Seinstra et al.)

Games and Model Checking

● Can solve entire Awari game on wide-area DAS-3● Needs 10G private optical network (StarPlane)

● Distributed model checking has very similar communication pattern● Search huge state spaces, random work

distribution, bulk asynchronous transfers

● Can efficiently run DeVinE model checker on wide-area DAS-3, use up to 1 TB memory

Awards

SCALE 2008

DACH 2008 - BS DACH 2008 - FT

AAAI-VC 2007

ISWC 2008 Multimedia

Computing

Astronomy

Semantic Web (van Harmelen et al.)

Outline

● History of distributed systems

● Clusters, grids, clouds, networked world

● Programming distributed systems

● Driving applications

● The Ibis system

● Ibis on smartphones

Ibis Philosophy● Real-world distributed applications should

be developed and compiled on a local workstation, and simply be launched from there

Ibis Approach● Virtual Machines (Java) deal with

heterogeneity● Provide range of programming

abstractions● Designed for dynamic/faulty environments● Easy deployment through middleware-

independent programming interfaces● Modular and flexible: can replace Ibis

components by external ones

Ibis Design

● Functionality from programming languages● High-Performance

Application Programming System

● Functionality fromoperating systems● Distributed Application

Deployment System

Ibis System

Programming system

● Programming models:● Message passing (RMI, MPJ)

● Divide-and-conquer (Satin)

● Master-worker (Maestro)

● Jorus: (multimedia applications)

● IPL (Ibis Portability Layer)● Java-centric “run-anywhere” library● Point-to-point, multicast, streaming, ….● Simple model (Join-Elect-Leave ) for tracking

resources, supports malleability & fault-tolerance

SmartSockets library

● Detects connectivity problems● Tries to solve them automatically

● With as little help from the user as possible● Integrates existing and several new

solutions● Reverse connection setup, STUN, TCP splicing,

SSH tunneling, smart addressing, etc.● Uses network of hubs as a side channel

Example

Example

Deployment system

● IbisDeploy GUI

● JavaGAT:● Java Grid Application Toolkit

● Make applications independent of underlying middleware

● Zorilla P2P system● Jobs management, gossiping,

clustering, flood scheduling

● Runs simultaneously on clusters (DAS-3, Japan, Australia), Desktop Grid, Amazon EC2 Cloud

● Connectivity problems solved automatically by Ibis SmartSockets

Multimedia Content Analysis

Client

Broker

Servers IbisIbis

(Java)

Connection management

With SmartSockets: run everywhere

Standard sockets: only local VU machines can be reached due to firewalls problems

Ibis movie (part 1)

Performance on 1 DAS-3 cluster

● Relative speedups of Java/Ibis and C++/MPI● Using TCP or Myricom’s MX protocol

● Sequential performance Java: 80% of C++

Ibis Performance (wide area)

● Wide-area DAS:● Frame-rate increases linearly with

#clusters from 1 frame/sec to 4 frames/sec

● World-wide experiments: 22 frames/sec

Smart Phones

● GSM + PC + GPS + camera + networks + ….

● Location-aware● What if everyone always carries a smart

phone (like a GSM now)?● Next wave in computing?

Ibis on Smart Phones● Our focus: distributed smart phone

applications● Applications running on multiple phones● Integration with distributed computing

backbone

● Use Android for development● Google’s open-source platform● Java-based

Distributed applications● Disaster management (Katrina)

● Use ad-hoc Wifi network when GSM network fails

● Finding nearby people with certain skills● Bus drivers, CPR

● Distributed decision support● Moving people to shelters (logistics)

● Social networks● Similar issues

● Find nearby friends, decide on restaurant

Wild example

● Track position => automatic diary of your life

● Cross-comparisons between diariesHaven’t we met before?

Yes, on 23 Oct 2010, 3.48 pm atN 52°22.688´ E 004°53.990´

eyeDentify● Object recognition on a G1 smartphone● Smartphone is a limited device:

● Can run only 64 x 48 pixels (memory bound)● 1024 x 768 pixels would take 5 minutes

● Distributed Ibis version:

+ =+1024 x 768 pixels

2.0 seconds

Ibis movie (part 2)

Interdroid

Distributed Communication

Data Management

Novel Mobile Distributed Applications

Context SensitiveProgrammingModels

Current work● Raven: API for Viable Episodic Networking

● Decentralized synchronization API● Fine grained control over data sharing

● Bluetooth support for ad-hoc communication● Discovery of devices using multiple networks● Context Aware Programming Models

● Supporting distributed decision making● Representing and using context (location etc.)● Exploiting social relationships (Hyves, Facebook)

Summary

● It’s a wild (distributed) world

Acknowledgements

Niels DrostCeriel JacobsRoelof Kemp

Timo van KesselThilo KielmannJason Maassen

Rob van NieuwpoortNick Palmer

Kees van ReeuwijkFrank J. SeinstraKees Verstoep

Gosia Wrzesinska

Big Acknowledgement

Andy

Questions?