Overlay Network Creation and Maintenance with Selfish Users Georgios Smaragdakis Dissertation...
-
date post
20-Dec-2015 -
Category
Documents
-
view
218 -
download
0
Transcript of Overlay Network Creation and Maintenance with Selfish Users Georgios Smaragdakis Dissertation...
Overlay Network Creation and Maintenance
with Selfish Users
Georgios Smaragdakis
Dissertation committee members: Azer Bestavros,
Nikolaos Laoutaris, John Byers
2
Overlay applications:overlay routing,p2p file sharing,content distribution..
Access ISP
Access ISP
Transit ISP
Overlays & Neighbor Selection
Internet Overlay links
Transit ISP
Access ISP
Overlay node
Focus on service quality!
3
Challenges
v1v2
v3
v4
v5 v6
v7
v8
v9
p1=[v2v3v4v5v6v7v8v9]
p9=[v1v2v3v4v5v6v7v8]
p3=[v1v2v4v5v6v7v8v9]
p8=[v1v2v3v4v5v6v7v9]
Selfish node
What is the performance gain that can be achieved by a selfish node?
What is the impact of selfish neighbor selection to overlay network performance?
What are the implications of selfish neighbor selection to system design?
4
Selfish Neighbor Selection
Implications
to
Overlay Routing
Implications to File SharingImplications to
Service Provisioning
Outline
5
Selfish Neighbor Selection
Implications
to
Overlay Routing
Implications to File SharingImplications to
Service Provisioning
6
Selfish Neighbor Selection (SNS)
Constraints that need to be addressed in a realistic model for overlay networks:
Bounded degree Preference vectors Realistic network distance Link directionality
Fundamentally different from other models that have been proposed for other networks.
[Fabrikant et al.,PODC’03; Chun et al., Infocom’04 …]
7
Optimal Neighbor Selection
vi: choose k neighbors, s.t.
vi
G-i=( V-i , S-i )
u
w
ij Vv
jiSiji vvdpSC ),()(min
over all siSi
vi’s residual network
Set of residual nodes
Set of residual wiring
8
SNS & Facility Location
Uniform link weights, and uniform preference k-median on asymmetric distances
9
k-mediank-median: Find a subset I of F and a function σ:CI
to min ( Σi,j sjcij ) such that |I| ≤ k
F: set of
facilities
C: set of clients,
cij: cost connecting
client jfacility I
sj: demand of node j
10
Uncapacitated Facility LocationUncapacitated Facility Location (UFL): Find a subset I of F and a function σ:CI
to min ( Σi fi + Σi,j sjcij )F: set of facilities
fi: cost to
openfacility
C: set of clients,
cij: cost connecting
client jfacility I
sj: demand of node j
11
Non-uniform link weights, and uniform preference
ILP formulation
SNS & Facility Location
Uniform link weights, and uniform preference k-median on asymmetric distances
u
w
w,u can be obtained from k-median on
reversed distances
w
u
vi
ij Vv
jiSiji vvdpSC ),()(min
Since the wiring cost is the same
12
Local Search (LS)
vi: choose k neighbors
viu
w
ij Vv
jiSiji vvdpSC ),()(min
over all siSi
vi’s residual network
[Arya et al,STOC’01]
G-i=( V-i , S-i )
Set of residual nodes
Set of residual wiring
13
SNS : the GameGame <V,{si},{Ci}>
V : set of n players (nodes) {si}: strategies available to vi (wirings),
choose k out of n to connect {Ci}: set of costs for vi
min
Best response of a node: node’s optimal wiring
Outcome: S, the global wiring A stable wiring is a pure Nash equilibium Using iterative best response
Fundamentally different from selfish routing
ij Vv
jiSiji vvdpSC ),()(
14
SNS : Equilibria
n=15 k=2
k=3
k=8
k=11
Uniform Preference Skewness of preference
k (Link density)
In-degrees are highly skewed even under uniform preference ! Quality-based
“preferential attachment”
15
Performance of ILP & LS is close to Utopian!
Theoretical results showed in the worst case the cosial cost can be bad
[Laoutaris, Poplawsi, Rajaraman, Sundaram, Teng,PODC’08]
SNS : Efficiency
Link density Skewness of preference Link density
Skewness of preference
16
SNS : Trace-Driven Evaluation How we assign the distance:
Synthetically using BRITE Empirically from PlanetLab Empirically from AS-level maps [Routeviews]
Neighbor Selection Strategies: k-Random heuristic k-Closest heuristic k-Regular heuristic k-Best Response
Control parameter: Bound on out-degree k (link density)
17
Connecting on a k-Random graph
k kk
AS-Level (n=50)PlanetLab (n=50)BRITE (n=50)
If your neighbors are naïve, it pays to be selfish!
0 2 3 5 11 22 0 2 3 5 11 22 0 2 3 5 11 22
18
Connecting on a k-Closest graph
kkk
0 2 3 5 11 22 0 2 3 5 11 22 0 2 3 5 11 22
If your neighbors are greedy, it pays to be selfish!
“Greed is not good”
AS-Level (n=50)PlanetLab (n=50)BRITE (n=50)
19
Connecting on a k-Regular graph
k kk
0 2 3 5 11 22 0 2 3 5 11 22 0 2 3 5 11 22
If your neighbors have the same wiring pattern, it pays to be selfish!
“Common pattern is not good”
AS-Level (n=50)PlanetLab (n=50)BRITE (n=50)
20
Connecting on a Best Response graph
The BR graph is highly optimized!
kkk
0 2 3 5 11 22 0 2 3 5 11 22 0 2 3 5 11 22
AS-Level (n=50)PlanetLab (n=50)BRITE (n=50)
If your neighbors are selfish, it is OK to be naïve!
21
SNS vs. Heuristics: Social Cost
Macroscopic view: Focusing on the social welfare
The network is better off with selfish nodes!
(k=2) k-Random/BR k-Closest/BR
k-Regular/BR
BRITE 1.44 1.53 3.61
PlanetLab 2.23 1.48 3.84
AS 2.04 1.90 4.78
23
SNS with Variable Degree
Real-time applications Variable degree through LS:
Swap 1 link Add 1 link Drop 1 link
Application requirement
(Performance when k=5, n=50 i.e. 250 links)
100 links
120 links
24
Selfish Neighbor Selection
Implications
to
Overlay Routing
Implications to File SharingImplications to
Service Provisioning
25
Basic design of EGOIST:Link state protocolMeasurements of distance to candidate
neighborsWirings according to chosen strategy Re-wirings every T second
A newcomer bootstraps by connecting to arbitrary neighbors
27
EGOIST: Passive Measurements
Passive measurements based on virtual coordinates (pyxida system) with minimal cost
28
EGOIST: Other Metrics
End-to-end available bandwidth (pathchirp) with minimal measurement overhead CPU load (loadavg)
29
EGOIST: Marginal Utility of Rewiring
There exists a performance knee (k=3 or 4) Re-wirings could be reduced with lazy BR
BR Lazy BR (threshold = 10%)
30
EGOIST: Effect of Churn
Connectivity is guaranteed (in T/n time) HybridBR (a connected ring is maintained) delivers much of the efficiency of BR
Effi
ciency
Index
Connect
ivit
y
qualit
y
31
EGOIST: Effect of Churn
BR and Hybrid BR dominate all the other heuristics HybridBR pays off at high churn
Effi
ciency
Index
Connect
ivit
y
qualit
y
32
EGOIST : Other Work CPU and memory load is very low
Robust to cheating
Scalability via topological sampling via layered architecture
Applications including multi-player P2P games, real-time traffic over IP etc.
33
Selfish Neighbor Selection
Implications
to
Overlay Routing
Implications to File SharingImplications to
Service Provisioning
34
Access ISP
Access ISP
Transit ISP
Modern File Sharing Systems
Parallel upload/ download- Swarming
Local scheduling - Local Rarest First
Flat connectivity- Choke/unchoke
Internet
Transit ISP
Access ISP
Overlay node
Seeder
Leecher
35
n-way Broadcast
Internet Synchronization- Distributed databases - Backups
Batch parallel processing
- The files have to be received by all nodes before the next step
of processing begins
36
Preliminary Solutions
n co-existing swarms (-) Stress of physical links
(-) Exchange of multiple chunks in parallel overpartitions
the uplink capacity [Tian et al., ICPP’06]
End-system multicast (mesh) [SplitStream, Bullet] (-) Creates an overlay for each swarm
(-) No coordination among swarms
(-) Monitor overhead
37
Design Strategies for n-way Broadcast
Joint optimization of upload/download while participating in many swarms
Data Agnostic - Keeps swarming and local scheduling
Bandwidth-Centric - Max-flow to approximate swarming behavior
[Massoulie et al., Infocom’07]
Bounded Degree
38
Reducing the Average Download Time
Objective: Minimize the average download time
Max-Sum: Neighbor selection strategy of node vi:
max (sum (MaxFlow(vi, vj)), for all vj
39
Reducing the Download Time
Objective: Minimize the total download time
Max-Min: Neighbor selection strategy of node vi:
max (min (MaxFlow(vi, vj)), for all vj
40
Optimized Graphs and Swarming
Formation of stable graphs
Each node strives to improve both the upload and download flow
Performance of swarming on optimized graphs- Max flow might not be realizable
41
Performance Evaluation
File ID
Nod
e ID
Deliv
ery
Tim
e
Naive Max-Sum Max-Min
File ID File ID
Flattens distribution time! Guarantees synchronization! Comparable average
download time
Selfish Upload:
Protects the uplinkcapacity of the slow
node
Improves the download time in the
system
42
Other Work: File Searching
Best response: max #nodes reached
Bootstrap
Server 1
2
3
4
5
6
TTL of scoped flooding is 2
Maximum Coverage Problem
selfishly
43
Selfish Neighbor Selection
Implications
to
Overlay Routing
Implications to File SharingImplications to
Service Provisioning
45
Centralized Deployment
Generic Service Host
Software server
Demand changee.g. Flash crowd, time-of-dayeffect
46
Dynamic Service Deployment
Generic Service Host
Software server
Demand changee.g. Flash crowd, time-of-dayeffect
47
r-ball (r=2)
Distributed Service Migration (DSM)
Solve k-median or UFL in an r-ball ..BUT nodes outside the r-ball are totally neglected
“ring” nodes
Iterate until
convergence
48
DSM: Properties
Convergence: Migration only if the cost of facilitating the
demand decreases at least be a%, converges in O(log1+a n) steps We can control the speed of convergence by
tuning a
Limited horizon view requirement: r regulates the trade-off between scalability and
performance
49
Similar results for UFL under different cost functions to open and maintain the server
DSM: Evaluation
51
Conclusions What is the performance gain that
can be achieved by a selfish node?
Selfish nodes can reap substantial performance gain.
What is the impact of selfish neighbor selection to overlay network performance?
Surprisingly, the evolving graphs have also good performance!
52
Conclusions What are the implications of selfish
neighbor selection to system design?
Selfish wiring strategies are easily realizable
Selfish wiring behavior can be used towards distributed overlay network creation and maintenance
Selfish wiring must be a component of any system to protect it from abuse
Selfish wiring behavior can be used for efficient dynamic service provisioning