Post on 05-Jan-2016
description
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 1
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Dynamic Load Balancing in Distributed Hash Tables
Marcin Bieńkowski
Mirosław Korzeniowski
Friedhelm Meyer auf der Heide
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 2
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Ring / Hypercube Topologies
• Chord, Tapestry, Pastry, de Bruijn• Space: ring [0,1]• Arithmetics mod 1• Links to:
– predecessor and successor– shortcuts
• Degree: log n• Diameter: log n (or log n / log log n)• Balance: (log n) • Smoothness (ratio of the longest and shortest interval): (n¢log n)
.12
.2
.23
.25
.3
.32
.4.45.5
Server Key
.1
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 3
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Balls into Bins
If we throw n balls into n bins uniformly at random, the fullest bin will have (log n / log log n) balls.
If we throw n¢ log n balls into n bins, each bin will have (log n) balls
If we throw n balls sequentially into n bins and each ball chooses the emptiest of log n randomly chosen bins, each bin will have (1) balls
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 4
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Previous Results
Recent publications concerning balancing in P2P:– Moni Naor, Udi Wieder „Novel Architectures for P2P
Applications: the Continous-Discrete Approach” SPAA 2003– Micah Adler, Eran Halperin, Richard Karp, Vijay Vazirani „A
stochastic process on the hypercube with appications to peer-to-peer networks” STOC 2003
• Both optimal • Both sequential
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 5
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Previous Results
Recent publications concerning balancing in P2P:– David Karger, Matthias Ruhl, Simple Efficient Load
Balancing Algorithms for Peer-to-Peer Systems, SPAA 2004• each join / leave operation makes log log n nodes
migrate– Gurmeet Singh Manku, Balanced Binary Trees for ID
Management and Load Balance in Distributed Hash Tables, PODC 2004
• a general algorithm, needs locally centralized decisions
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 6
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Assumptions
• We know:– n (the number of nodes) def. := 1/n– or at least log n– or at least an upper bound on log n
• No interaction during rebalancing• Any initial state of the network• The network organises itself and implements:
– join(where)– leave– insert(hash)
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 7
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Lazy nodes leave
First phase:if (length < 1/10¢) leave
Minimizing balance = minimizing smoothness
Balance = the longest intervalSmoothness = ratio between the longest and the
shortest interval
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 8
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Continous scheme
All the next phases:if (length < 4¢) either propose(random) or notif (length > 10¢) halve using the first proposal else forward (log n) times
Analysis (sketch)• There are (n) proposals • Interval longer than (¢log n) receives enough
proposals in one step (w.h.p.)• Interval shorter than (¢log n) receives enough
proposals from its predecessors
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 9
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
To Propose or Not to Propose
Many consecutive short intervals:
L R L R L R L R L R L R L R
L L R L L R R R L L R L L L
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 10
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Continous scheme
All the next phases:if (length < 4¢) either propose(random) or notif (length > 10¢) accept(length / 10)if (length < 10¢) forward(log n)
Analysis (sketch)• There are (n) proposals • Interval longer than (¢log n) receives enough
proposals in one step (w.h.p.)• Interval shorter than (¢log n) receives enough
proposals from its predecessors
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 11
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Managing the volunteers – 1st phase
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 12
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Managing the volunteers – 2nd phase
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 13
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Estimating n and
• Choose global k > (log n)• Each node inserts k markers into the network (n¢log n) markers are distributed uniformly• An interval of length contains (k) items w.h.p.• Instead of lengths we consider the number of
markers (weights)
• If a node should insert a normal item it replaces a marker with it ) load increased only if it was low
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 14
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Results
• A scheme that– copes with insertions and deletions– works in constant number of rounds– each round takes logarithmic time– has optimal number of migrations(up to a constant
factor)– each node migrates at most once– has bounded communication per half-life
IPTPS 2005IPTPS 2005 Mirek Korzeniowski: Page 15
International Graduate School of Dynamic Intelligent Systems
HEINZ NIXDORF INSTITUTUniversity of Paderborn
Algorithms and Complexity
Dynamic Load Balancing in DHTsDynamic Load Balancing in DHTs
Dynamic Load Balancing in Distributed Hash Tables
Thank you for your attention