JuxMem: An Adaptive Supportive Platform

for Data Sharing on the Grid

Gabriel Antoniu, Luc Bougé, Mathieu Jan

IRISA / INRIA & ENS Cachan, France

Workshop on Adaptive Grid MiddlewareNew Orleans, September 2003

2

Context: Data Management on the Grid

Distributed numerical simulations (code coupling)

Problem: data management

Solid mechanics

Thermodynamics

Optics

Dynamics

Satellite design

3

Existing Data Management Systems

Non-transparent large scale data management

GridFTP (Globus) and MPI-IO Security, heterogeneity

Internet Backplane Protocol (IBP) Control

Explicit transfer No consistency guarantee

4

Existing Data Management Systems

Transparent small-scale data management Distributed shared memory (DSM) Transparent access Transparent data localization Consistency models and protocols

Static, homogeneous architecture

5

Another Approach: Peer-to-Peer Systems

Peer-to-peer systems (P2P) Distributed (large-scale) Volatile peers Peers have the same capacities and responsibilities

Sharing immutable data Centralized (Napster) Flooding (Gnutella, KaZaA) Distributed hash table (CFS, PAST)

Sharing mutable data One writer per data, static assumptions (OceanStore) Manual conflict resolution (Ivy)

6

DSM systems and P2P systems

Comparing basic hypotheses

DSM P2P

Scale 101-102 105-106

Dynamicity Null High

Resource homogeneit

y

Homogeneous (clusters)

Heterogeneous (Internet)

Control and trust

High Low

Topology Flat Flat

Data type Mutable Immutable

Typical applications

Scientific computation

File sharing and storage

7

Idea: Data Sharing Service Proposal: hybrid approach

DSM systems: consistency and transparent access P2P systems: scalability and high dynamicity

DSM Grid Data Service P2P

Scale 101-102 103 - 104 105-106

Dynamicity Null Medium High

Resource homogeneit

y

Homogeneous (clusters)

Rather heterogeneous (clusters of clusters)

Heterogeneous (Internet)

Control and trust

High Medium Low

Topology Flat Hierarchical Flat

Data type Mutable Mutable Immutable

Typical applications

Scientific computation

Scientific computation and data storage

File sharing and storage

8

A Data Sharing Service for the Grid

Internet

Persistence

9

A Data Sharing Service for the Grid

Internet

Data transfer

?

Transparent data location

10

A Data Sharing Service for the Grid

Internet

Scalability

Internet

11

A Data Sharing Service for the Grid

InternetInternet

Volatility tolerance

12

JXTA: a Framework for P2P Services

Open-source platform for programming P2P applications

http://www.jxta.org A peer

Uniquely identified (ID) Address independent of physical location Multiple network access points (TCP, HTTP, etc)

Peer

Peer

Peer Peer

Peer

PeerPeer

Peer

PeerPeer

Peer

Peer

FirewallPeer

PeerTCP/IP

HTTP

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

Firewall

13

JXTA: Peer Groups

Set of peers that share a common set of interests

Specific management policy Peer group services

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

Peer ID

NetPeerGroup

PeerGroupA

PeerGroupB

14

JXTA: Advertisements

Every resource is described by an advertisement

Peers Peer groups Communicationchannels Services …

PeerGroup Advertisement:

<?xml version="1.0"?><!DOCTYPE jxta:PGA><jxta:PGA>

<GID>urn:jxta: uuid-

BCBCDEABDBBBABEABBBABA000000</GID><MSID>

urn:jxta:uuid-BFEFDEDFBABAFRUDBACE00000001</MSID><Name>

My Group</Name><Desc>

This group is to be used for my own testing</Desc>

</jxta:PGA>

15

JuxMem: an Architecture Proposal

juxmem group

cluster A group

cluster B group

cluster C group

data group

Physical architecture

Logical architecture

16

JuxMem API

Alloc (size, attribs)

Map (id, attribs)

Put (id, value)

Get (id)

Lock (id)

Unlock (id)

17

Managing Memory Resources

cluster group

juxmem group

Size: 8 MB

Size: 8 MB

Memory provided

Provider advertisements: cluster group

Cluster advertisements: juxmem group

18

Allocation: How Does It Work?

21

3a

3a

3b

3b

4

5

6

8 MB?

19

Managing Shared Data Blocks

Allocate a memory block = create a data group

Data blocks identified by the ID of the peer group Transparent access for clients via data ID

Consistency Data blocks replicated on providers Simultaneous updates (logical multicast) Clients are not notified of updates

Synchronization One lock per data block Other mechanisms: in progress

20

Handling Peer Volatility

Provider volatility A manager per peer group (cluster and data) Dynamic monitoring of available peers

(cluster) Automatic replication of data blocks (data)

Manager volatility Periodic exchange of heartbeats Dynamic replication of managers if needed

on other peers

21

Implementation and Preliminary Evaluation

Implementation JXTA service, 5000 Java code lines

Experimental setup PentiumII: 450 Mhz and 256 MB of RAM FastEthernet 100 Mb/s Linux 2.4 Number of nodes: 20

Experiment Study provider volatility

22

Study: Provider Volatility (1)

juxmem group

cluster group

data group

Data size: one byte

Replication degree = 3

Data manager not killed

1 client: 100 iterations lock-put-unlock

16 providers

23

Study: Provider Volatility (2)

juxmem group

cluster group

data group

1 client: 100 iterations lock-put-unlock

16 providers

Data size: one byte

Replication degree = 3

Data manager not killed

24

Study: Provider Volatility (3)

Internal locking during replication Guarantee consistency during replica

creation Client is blocked

juxmem group

cluster group

data group

25

Study: Provider Volatility (4)

JXTA/Java Expensive underlying JXTA-level dynamic channel management

0

20

40

60

80

100

160 140 120 100 80 60 50 40 30

Provider volatility (seconds)

Rela

tive o

verh

ead (

%))

Reconfiguration time

11 seconds

Targeted volatility is weaker ( >> 80

seconds)

26

Conclusion

A hierarchical architecture for a data sharing service for the grid Hybrid approach: DSM and P2P systems Transparent access to data blocks Persistent storage Mutable data: consistency guarantees Active support for peer volatility

27

Ongoing Work

Studies Replication strategies for fault tolerance Consistency protocols in a dynamic

environment Co-scheduling computation and data

distribution Manage data-data affinity Integrate high-speed networks: Myrinet, SCI.

Goal: build a Grid Data Service GDS project: http://www.irisa.fr/GDS Extensive evaluation on realistic codes