Network delay-aware load balancing in sel sh and cooperative

Problem presentationCooperative organizations

Selfish organizations

Network delay-awareload balancing

in selfish and cooperativedistributed systems

Piotr Skowron, Krzysztof [email protected]

University of Warsaw

March 17, 2013

Piotr Skowron, Krzysztof Rzadca Network delay-aware load balancing

Abstract

We show a mathematical model for divisible load balancing ingeographically distributed systems.

We prove that such expressed problem is polynomiallysolvable.

We present a distributed algorithm that is robust and, onaverage, efficient.

In case when organizations have conflicting goals:

We prove that in homogeneous networks the worst-case lossof performance (the price of anarchy) is low.

For other configurations we experimentally assess the loss ofperformance due to the selfishness and show it is low.

Abstract

Problem presentation

Geographically distributed computing system

Each organization has its own load

Servers should balance their load

Load balancing should consider communication latency

Each organization i has its own load – ni

-�n1

-�n2

-� n3

-�n4

-�n5

-� n7

ni – the own load of organization i

Each organization i has its own load – ni

-�n1

-�n2

-� n3

-�n4

-�n5

-� n7

ni – the own load of organization i

Remark: the model is suitable for continuously running systems

In continuously running systems ni can be also viewed as theaverage size of the queue (this is related to the queuing theory).

Communication delays: latencies between pairs of servers

-�n1

-�n2

-� n3

-�n4

-�n5

-� n7

AAAAAAAU

��

BBBBMc3,1

ci ,j – the communication delay between server i and j

Communication delays: latencies between pairs of servers

-�n1

-�n2

-� n3

-�n4

-�n5

-� n7

AAAAAAAU

��

BBBBMc3,1

ci ,j – the communication delay between server i and j

Latencies are independent on the amount of sent data

-�n1

-�n2

-� n3

-�n4

-�n5

-� n7

AAAAAAAU

��

BBBBMc3,1

Usually communication delays are constant.

Communication delays are constant

We run the experiments on PlanetLab with 50 servers

and fordifferent values of background load bt we measured the averageRTT (round trip time) of the single packet – RTT (bt). Wechecked the ratio:

loss(bt) =RTT (bt)− RTT (400Kb/s)

RTT (400Kb/s)

loss(b

0.5 1 2 5 10 20 50 100

background load bt [Mb/s]

We run the experiments on PlanetLab with 50 servers and fordifferent values of background load bt we measured the averageRTT (round trip time) of the single packet – RTT (bt).

Wechecked the ratio:

RTT (400Kb/s)

loss(b

0.5 1 2 5 10 20 50 100

We run the experiments on PlanetLab with 50 servers and fordifferent values of background load bt we measured the averageRTT (round trip time) of the single packet – RTT (bt). Wechecked the ratio:

RTT (400Kb/s)

loss(b

0.5 1 2 5 10 20 50 100

We run the experiments on PlanetLab with 50 servers and fordifferent values of background load bt we measured the averageRTT (round trip time) of the single packet – RTT (bt). Wechecked the ratio:

RTT (400Kb/s)

loss(b

0.5 1 2 5 10 20 50 100

Delegating tasks

Delegating tasks – relay fractions ρi ,j

ρ3,6ρ3,4=0

ρ7,5ρ7,6

��

ρi ,j – relay fraction (fraction of i-th load relayed to j-th server)

Delegating tasks – relay fractions ρi ,j

ρ3,6ρ3,4=0

ρ7,5ρ7,6

��

ρi ,j – relay fraction (fraction of i-th load relayed to j-th server)

Current load on server i

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

Example – 2nd server:

The load is a sum of loads relayed by all organizations

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

li – current load

Load is composed of large number of small tasks

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

li – current load (number of tasks executed on i)

The tasks are of the same sizes

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

si – speed (running single task takes 1si

We are interested in optimizing each task performance

Individual tasks may correspond to:

single work units in a BOINC-type application,

single invocations of the map functions in a map-reducecomputational model,

individual requests for data in Content Delivery Networks,

consecutive reads when accessing remote streams of data.

The goal: average response time

In each such case we are interested in the average response timefor the individual request.

The tasks are of the same sizes

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

We assume random execution order

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

We assume random execution order:

For cooperative organizations the order of processing taskshave no impact on average completion time.

In case of selfish organizations, where servers are commongood, any other order would favor some organization.

In a continuously running distributed systems:

we have no control over the order in which requests areproduced,

we have no control over the order in which the requests aredelivered by the network.

The usual FIFO policy results in an arbitrary order.

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

JJJJJJJJJJJJJJJ

the expected completion time of each task is li2si

Total processing time of i -th organization’s tasks

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

the expected total completion time of i-th organization’s tasks is:

Ci =∑m

2sj+ cij

)ρijni

Total completion time of i -th organization’s tasks

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

Ci =∑m

2sj+ cij

)ρijni

the expected processing time on server

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

Ci =∑m

2sj+ cij

)ρijni

the communication delay

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

Ci =∑m

2sj+ cij

)ρijni

the number of tasks relayed to j-th server

-�ρ2,2n2

-�ρ3,2n3

-�ρ4,2n4

-�l2 =

∑mj=1 ρj,2nj

Ci =∑m

2sj+ cij

)ρijni =

∑mj=1

((cij +

∑mk=1

ρkjnk2si

)ρijni

The problem

Cooperative organizations: Find the relay fractions ρij thatminimize

∑Ci .

Selfish organizations: We consider a game:

players: organizations,actions: i-th organization can modify its relay fractions ρi,j ,utility of organization i : minimizing the total completion timeof its own requests Ci ,the servers are common good.

The problem

∑Ci .

The problem

∑Ci .

Related work

congestion games (no network topology),

divisible load model with constant-cost communication andmultiple source (makespan),

selfish job model (makespan).

Related work

The problem

∑Ci .

Cooperative organizations

∑Ci =

((cij +

ρkjnk2si

)ρijni

Good news

The optimization problem can be presented as an instance ofquadratic programming with a positive definite matrix. Thus it ispolynomially solvable!

Cooperative organizations – quadratic programming

∑Ci = ρTQρ+ bTρ.

ρ = 〈ρ1,1, ρ1,2, ρ1,3, . . . , ρ2,1, ρ2,2, . . . ρm,m〉b = 〈b1,1, b1,2, b1,3, . . . , b2,1, b2,2, . . . bm,m〉 b(i ,j) = cijni

ρ = 〈ρ1,1, ρ1,2, ρ1,3, . . . , ρ2,1, ρ2,2, . . . ρm,m〉

b = 〈b1,1, b1,2, b1,3, . . . , b2,1, b2,2, . . . bm,m〉 b(i ,j) = cijni

-� -�

��)nink/sj

��)

nink/2sj

-� -�

��)nink/sj

��)

nink/2sj

-upper triangular matrix(positive-definite)

Cooperative organizations

Bad news

The complexity of the standard solvers is O(m6), where mdenotes the number of machines.

The central algorithm must collect the information about thewhole network (cij) which is of the size O(m2).

The central algorithm is vulnerable to failures.

Cooperative organizations – distributed algorithm

Each server chooseslocally optimal partner

The servers optimallyexchange their loads

Remark: no requests are transferred

This is only the way of distributed calculating relay fractions.During communication no requests are transferred over thenetwork. Adjusting the loads can be viewed as a simulation.

They exchange tasksof each owner

Distributed algorithm – what we did

In our paper we show (here we skip the description of the technicalparts):

how to choose the locally optimal partner (nontrivial),

how to optimally exchange the load of all owners between thepair of servers (nontrivial),

how to estimate the distance to the optimal solution based onthe amount of exchanged information (nontrivial).

Simulations: convergence of the distributed algorithm

totalprocessingtime(∑

0 5 10 15 20

iteration

#servers = 500

#servers = 1000

#servers = 2000

#servers = 3000

#servers = 5000

Initial distribution – whole load on a single server (worst case)

Selfish organizations – the problem

∑Ci .

Selfish servers

We simplified the model:

all the servers have equal processing power (∀i si = 1);

all the connections between servers have the samecommunication delay (∀ij cij = c).

Theorem 1

The price of anarchy in the homogeneous network is:PoA = 1 + c

lav+ O(( c

lav)2).

Here, lav denotes the average load on the server, thuslav = 1

∑i=mi=1 li . The Theorem 1 gives accurate estimation for the

case when c << lav .

Price of anarchy – sketch of proof

Main idea – organizations have incentives to unbalance the system,but we show that the difference between loads of saervers is small.

-c1,2 = 1

l1 = 10 l2 = 10

C1 = 12

∑10i=1 i = 55

-c1,2 = 1

l1 = 10 l2 = 10

C1 = 12

∑9i=1 i + 6 + 1 = 45 + 6 + 1 = 52 < 55

-c1,2 = 1

l1 = 10 l2 = 10

C1 = 12

∑8i=1 i + 2 · 6.5 + 2 = 36 + 13 + 2 = 51 < 52

-c1,2 = 1

l1 = 10 l2 = 10

C1 = 12

∑7i=1 i + 3 · 7 + 3 = 28 + 21 + 3 = 52 > 51

Selfish organizations – experimental results

Ratioavg. max st. dev.

lav ≤ 30cij = 20 1.041 1.098 0.029

PL 1.014 1.049 0.007

lav = 50cij = 20 1.114 1.150 0.031

PL 1.011 1.033 0.006

lav ≥ 200cij = 20 1.024 1.055 0.018

PL 1.003 1.022 0.003

lav ≤ 30cij = 20 1.000 1.022 0.001

PL 1.000 1.000 0.000

lav = 50cij = 20 1.041 1.062 0.018

PL 1.000 1.000 0.000

lav ≥ 200cij = 20 1.001 1.029 0.006

PL 1.000 1.000 0.000

Table: Experimental assesment of ratio:∑

Ci for selfish organizations∑Ci for cooperative organizations .

Here PL means that we took communication delays from PlanetLab.

lav ≤ 30cij = 20 1.041 1.098 0.029

PL 1.014 1.049 0.007

lav = 50cij = 20 1.114 1.150 0.031

PL 1.011 1.033 0.006

lav ≥ 200cij = 20 1.024 1.055 0.018

PL 1.003 1.022 0.003

lav ≤ 30cij = 20 1.000 1.022 0.001

PL 1.000 1.000 0.000

lav = 50cij = 20 1.041 1.062 0.018

PL 1.000 1.000 0.000

lav ≥ 200cij = 20 1.001 1.029 0.006

PL 1.000 1.000 0.000

lav ≤ 30cij = 20 1.041 1.098 0.029

PL 1.014 1.049 0.007

lav = 50cij = 20 1.114 1.150 0.031

PL 1.011 1.033 0.006

lav ≥ 200cij = 20 1.024 1.055 0.018

PL 1.003 1.022 0.003

lav ≤ 30cij = 20 1.000 1.022 0.001

PL 1.000 1.000 0.000

lav = 50cij = 20 1.041 1.062 0.018

PL 1.000 1.000 0.000

lav ≥ 200cij = 20 1.001 1.029 0.006

PL 1.000 1.000 0.000

Extensions

The original model can be generalized by:

considering different sizes of the tasks: Ji = {Ji (k)}. Ji (k)has processing time pi (k). Then we take ni =

∑k pi (k) and

we look for the relay sets Si (j) that minimize∑

err(Si (j)):

err(Si (j)) = |∑

k:Ji (k)∈Si (j)

pi (k)− ρijni |.

(multiple subset problem with different knapsack capacities –FPTAS exists)

adding redundancy : ∀i ,j ρij ≤ 1R .

Extensions

considering different sizes of the tasks: Ji = {Ji (k)}. Ji (k)has processing time pi (k).

Then we take ni =∑

k pi (k) andwe look for the relay sets Si (j) that minimize

∑err(Si (j)):

err(Si (j)) = |∑

k:Ji (k)∈Si (j)

pi (k)− ρijni |.

Extensions

∑k pi (k) and

err(Si (j)):

err(Si (j)) = |∑

k:Ji (k)∈Si (j)

pi (k)− ρijni |.

Extensions

∑k pi (k) and

err(Si (j)):

err(Si (j)) = |∑

k:Ji (k)∈Si (j)

pi (k)− ρijni |.

Extensions

∑k pi (k) and

err(Si (j)):

err(Si (j)) = |∑

k:Ji (k)∈Si (j)

pi (k)− ρijni |.

Conclusions and future work

Conclusions:

the system can be managed efficiently ,(the problem ispolynomial solvable, an efficient distributed algorithm),

even without central management, when the organizationshave conflicting goals ,(low price of anarchy, low average lossof performance due to selfishness).

Future work:

what if the servers are not a common good?

Conclusions:

the system can be managed efficiently ,

(the problem ispolynomial solvable, an efficient distributed algorithm),

Future work:

Conclusions:

Future work:

Conclusions:

even without central management, when the organizationshave conflicting goals ,

(low price of anarchy, low average lossof performance due to selfishness).

Future work:

Conclusions:

Future work:

Conclusions:

Future work:

Conclusions:

Future work:

Questions?Also, feel free to send any questions to:[email protected].

Network delay-aware load balancing in sel sh and cooperative

Documents

Transcript of Network delay-aware load balancing in sel sh and cooperative