Locality Aware Network Solutions
description
Transcript of Locality Aware Network Solutions
Locality Aware Network Solutions
Dahlia MalkhiThe Hebrew University of Jerusalem
2
A Brief Overview of Distributed Computing
The 90’s: – Internet activity: Web browsing
– Paradigm: Client-server
– Techniques: cluster computing, Paxos, group communication
3
A Brief Overview of Distributed Computing
The 90’s: 2000-
– Internet activity: File sharing
– Paradigm: P2P, grid, web-services
– Techniques: overlay networks, content distribution networks, resource location
4
Application: IPv6 Routing over IPv4[van Renesse 02]
AF3S:::3FF1:43E4
0001:::3BBB:5555
1111:::7777:7754
2222:::2222:2222
EEE0:::EEEE:EEEE
5151:::6161:6666
567A:::0202:0202
8888:::0909:9999
Distribute Hash Tables (DHT)
5
Application: Content Delivery / Finding Nearest Copies of Data
?
?
?
6
Application: Hyperencryption[Maurer 92, Ding & Rabin 02]
Random bits
Alice BobKey
Adversary bits
7
Application: A Hyperencryption P2P Network
[Rabin 03]Distributed Hash Table
(DHT)
8
Application: A Distributed Google?
?
9
Scalable Network Solutions
Overlay networks provide added functionality at the application level– Search, routing, location services
Network theory provides the foundations– Possibilities, impossibilities, lower/upper bounds
Practical solutions require flexible deployment
10
Distributed Data Structures (DDS)
Peers jointly implement a data structure, e.g., hash table
Route queries based on data-name (key)
11
DDS Problem Reduced to Routing?? 00001111
Responsible for 00001111
12
Why classic routing network designs don’t help
Static # of nodes a priori
known Node labels
designated by network designer
000
111
110
101001
100
011
010
13
DDS Reduced to Routing
The problem: Overlay routing network– Variants: labeled routing,
name-independent routing, finding nearest copies
Dynamic emulation
14
Distributed Hash Tables
[Malkhi, Naor, Ratacjzak, PODC 2002]
SchemeDegreeRoute
Chord, Tapestry, Pastry [2001]
Log nLog n
CAN [2001]
dd*n(1/d)
Viceroy [2002]
5Log n
Koorde, D2B, DH, generic
[2003]
2Log n
[Abraham, Awerbuch, Azar, Bartal, Malkhi, Pavlov, IPDPS 2003]
15
Tree View of Dynamic Graphs
Leafs of the tree represent current nodes
Inner nodes in the tree represent nodes that were split
000
111
110
101
00
110 111
001
100
011
010
000 001 010 011 100 101
00
Example: merge of 000, 001 into 00
16
Locality awareness
source
target
17
Locality awareness
source
target
18
Locality Awareness in Overlay Networks
Model the network as a weighted undirected graph– c(x, y): cost of shortest path from x to y– c() is a metric
An overlay network is a sub-graph Let x=x0 , x1, …, xt=y be a route in the overlay
network Stretch: Ratio between overlay route cost and
shortest path cost:( c(x, x1) + c(x1,x2) + … c(xt-1, y) ) / c(x,y)
19
Overlay Networks inGrowth-Bounded Metrics
Previous work:– [Plaxton, Rajaraman, Rica 1997], Tapestry (Berkeley),
Pastry (MS UK)
– Expected (large) constant stretch– Logarithmic node degree
LAND [Abraham, Malkhi, Dobzinski, SODA 2004]:– Guaranteed stretch (1+ε)– Expected logarithmic node degree, constant
depends on growth-bound– Simple, intuitive construction and proofs
20
Overlay Networks in Geometric Spaces
Modeling the Internet as a geometric space is practical– Ubiquitous GPS devices– Successful embeddings in virtual
coordinate-space Problem 1: Locate nodes Problem 2: Route to known coordinates
21
Location Services and Routingin Geometric Spaces
LLS: First fully-locality aware location service [Abraham Dolev Malkhi 2004]
– bounded stretch lookup– bounded stretch update
First constant-degree routing scheme (to known coordinates)[Abraham Malkhi, PODC 2004]
– constant node degree, logarithmic hops, 1+ε stretch
22
Routing in Arbitrary Graphs: Lower and upper bounds
Name-independent routing: node names are independent of routing scheme [Awerbuch, Bar Noy, Linial, Peleg 1989]
Lower bounds: [Gavoille Gengler 2001] – Stretch < 3 O(n) routing information– Stretch < 5 √n routing information
Upper bound: [Abraham, Gavoille, Malkhi, Nisan, Thorup, SPAA 2004]
– stretch-3 routing with O(√n ) routing information– Stretch 3 is indeed attainable!
General upper bound: [Abraham Gavoille, Malkhi, DISC 2004]
– Stretch-k routing with memory O(k2 k√n )
23
ever-growing global scale-free networks, their provisioning, repair and unique functions
EVERGROW
The Vision
ultimate RAID
ultimate GNUTELLA
ultimate GOOGLE
ultimate AKAMAI
infrastructure and new methods and systems devoted to measurement, mock-up and and analysis of present and future network traffic, topology and logical structure, to bridge the gap in theory, protocols and understanding to what the Internet can be in 2025.
An EC project. Coordinators: SICS (Sweden) and HUJI (Jerusalem)
24
Locality-Aware, Robust Overlay for
Information Lookup and Content Delivery Degree O(√n) Locality awareness:
– Formally stretch 3– For far-apart nodes, lower stretch
Mostly two-hop– Whenever full connectivity exists
Flexibility – Estimate √n roughly– Cache information on many vicinity nodes– Store information about any known node of same color
Fault tolerance:– Multiple route choices– Quick repair– Maintain QoS in face of churn
25
26
Large Scale Content Delivery
Initially split the content Then cross-exchange data pieces Solutions build on top of overlay routing
networks
27
FastReplica Cherkasova,Lee -HP Labs - 2003
Phase 1 Source
Clients
28
FastReplica Cherkasova,Lee -HP Labs - 2003
Phase 2 Source
Clients
29
Locality motivation – Tree example
30
Julia algorithm motivation: “Divide and conquer”
First phase
31
2nd phase 2nd phase
Julia algorithm motivation: “Divide and conquer”
32
Network nodes
33
Nodes’ random identifiers
0111010
0011110 1111110
0001000
10010010101001
34
Coloring and Vicinities
35
Coloring and Vicinities
?
?
?
?
36
Stretch 3
?
d
≤ d
≤ 2d
37
The Full Routing Scheme
?
12
345
?a
b
c
d