1

Best Response Best Response Dynamics in Dynamics in

Multicast Cost Multicast Cost SharingSharing

Seffi NaorSeffi Naor

Microsoft Research and TechnionMicrosoft Research and Technion

Based on papers with:Based on papers with:

C. Chekuri, J. Chuzhoy, L. Lewin-Eytan, and A. OrdaC. Chekuri, J. Chuzhoy, L. Lewin-Eytan, and A. Orda [EC 2006] [EC 2006]

M. Charikar, C. Mattieu, H. Karloff, M. Saks [2007]M. Charikar, C. Mattieu, H. Karloff, M. Saks [2007]

2

MotivationMotivation

Traditional networks – single entity, single control objective.Traditional networks – single entity, single control objective.

Modern networking – many entities, different parties.Modern networking – many entities, different parties.

Users act selfishly, maximizing their objective function.Users act selfishly, maximizing their objective function.

Decisions of each user are based on the state of the network, Decisions of each user are based on the state of the network, which depends on the behavior of the other users.which depends on the behavior of the other users.

non-cooperative network games.non-cooperative network games.

3

Our FrameworkOur Framework

A network shared by a finite number of users.A network shared by a finite number of users. Each edge has a Each edge has a fixedfixed cost. cost. Cost sharingCost sharing method defines the rules of the game: method defines the rules of the game:

determines the determines the mutual influencemutual influence between players. between players. Performance of a user is total payment = Performance of a user is total payment = sum of payments for all the edges it usessum of payments for all the edges it uses..

Two fundamental models:Two fundamental models: TheThe congestion modelcongestion model.. TheThe cost sharing modelcost sharing model..

4

CongestionCongestion ModelModel Cost Sharing ModelCost Sharing Model

CommonCommon inin unicast routing.unicast routing. Edge cost:Edge cost:

Modeled by a load dependent Modeled by a load dependent function.function. Non-decreasing in the total Non-decreasing in the total flow of the edge.flow of the edge.

Each user has a negative Each user has a negative effect on the performance of effect on the performance of other users.other users.

CommonCommon inin multicast routing.multicast routing. Edge cost:Edge cost:

Fixed costFixed cost Cost sharing mechanism Cost sharing mechanism determines how the cost is shared determines how the cost is shared by the users.by the users.

Each user has a favorable Each user has a favorable effect on the performance of effect on the performance of other users (cross monotonicty).other users (cross monotonicty).

In both models:In both models:• Each user routes its traffic over a minimum-cost path.Each user routes its traffic over a minimum-cost path.• SplittableSplittable routing model vs. routing model vs. unsplittableunsplittable routing model.routing model.

5

The Multicast GameThe Multicast Game

A special A special rootroot node node rr, and a set of , and a set of nn receiversreceivers ( (playersplayers).). A player’s strategy isA player’s strategy is a a routingrouting decision – decision –

the choice of a single path to the choice of a single path to rr.. EgalitarianEgalitarian ((ShapleyShapley(( cost sharing mechanism cost sharing mechanism: the cost of : the cost of

each edge is each edge is evenly splitevenly split among the players using it. among the players using it.Each player on edge Each player on edge ee with with nnee players pays: players pays:

cce e / / nnee

The goal of the players is to connect to the root by making The goal of the players is to connect to the root by making a routing decision minimizing their payment.a routing decision minimizing their payment.

6

The Multicast GameThe Multicast Game

Two different models:Two different models:

1.1. The The integral modelintegral model: each player connects : each player connects to the root through a to the root through a singlesingle path. path.

1.1. The The fractional modelfractional model: each player is : each player is allowed to split its connection to the root allowed to split its connection to the root into into several several paths (fractions add up to 1).paths (fractions add up to 1).

7

Nash EquilibriumNash Equilibrium

Players are Players are rational.rational. Each player knows the rules of the underlying Each player knows the rules of the underlying

game.game.

Nash EquilibriumNash Equilibrium:: no player can unilaterally no player can unilaterally improve its cost by changing its path to the root. improve its cost by changing its path to the root. Cost of a path takes into account Cost of a path takes into account cost sharingcost sharing..

Nash equilibrium solutions are stable operating Nash equilibrium solutions are stable operating points.points.

8

The Price of AnarchyThe Price of Anarchy

Nash equilibrium outcomes do not necessarily optimize Nash equilibrium outcomes do not necessarily optimize the overall network performance.the overall network performance.

Price of AnarchyPrice of Anarchy:: The ratio between the cost of the The ratio between the cost of the worst Nash equilibrium and the (social) optimum.worst Nash equilibrium and the (social) optimum.

Quantifies the “penalty” incurred by lack of Quantifies the “penalty” incurred by lack of cooperation.cooperation.

9

The Integral Multicast GameThe Integral Multicast Game

Potential function Potential function ΦΦ of a solution of a solution T T [Rosenthal `73][Rosenthal `73]::

Exact potential: Exact potential: changechange in cost of a connection of in cost of a connection of player player ii to the root is equal to the to the root is equal to the changechange in the in the potentialpotential ΦΦ..

If edge If edge ee is deleted from is deleted from TT: : ΦΦ == ΦΦ - - cce e / / nnee((TT))

If edge If edge ee is added to is added to TT: : ΦΦ == ΦΦ + + cce e / (/ (nnee((TT)+1))+1)

Te

Tn

k

ee

k

cT

)(

1

)(

10

The Integral Multicast GameThe Integral Multicast Game Finite strategy space Finite strategy space ΦΦ has an optimal value. has an optimal value.

ΦΦ Nash equilibrium existence. Nash equilibrium existence. Global / Local optima of Global / Local optima of ΦΦ correspond to a NE. correspond to a NE.

A Nash equilibrium solution is a tree rooted at A Nash equilibrium solution is a tree rooted at rr spanning the players.spanning the players.

Special case of a congestion game.Special case of a congestion game.

11

Price of Anarchy vs. Price of StabilityPrice of Anarchy vs. Price of Stability

Price of anarchyPrice of anarchy can be as bad as can be as bad as ((nn).).

Price of stabilityPrice of stability – ratio between the cost of best Nash solution to the cost – ratio between the cost of best Nash solution to the cost of OPT.of OPT.

Outcome of scenarios in the ‘middle ground’ between centrally enforced Outcome of scenarios in the ‘middle ground’ between centrally enforced solutions and non-cooperative games.solutions and non-cooperative games. E.g.: central entity can enforce the initial operating point.E.g.: central entity can enforce the initial operating point.

[Anshelevich [Anshelevich et al., FOCS 2004et al., FOCS 2004] ] Directed graphs - price of stability is Directed graphs - price of stability is θθ(log (log nn).). Undirected graphs – upper bound on the price of stability is O(log Undirected graphs – upper bound on the price of stability is O(log nn). ).

PoS can be reached from a 2-approximate Steiner tree configuration:PoS can be reached from a 2-approximate Steiner tree configuration:

CC((TTNashNash) ) ΦΦ((TTNashNash) ) ΦΦ((TT2-Seiner2-Seiner) ) log log n ∙C n ∙C ((TT2-Steiner2-Steiner) )

r t

12

Each player, in its turn, selects a path minimizing its cost (Each player, in its turn, selects a path minimizing its cost (best responsebest response).). Eventually, an equilibrium point is reached.Eventually, an equilibrium point is reached. PoA strongly depends on the choice of the initial configuration.PoA strongly depends on the choice of the initial configuration. Starting from a near-optimal solution may be hard to enforce: requires Starting from a near-optimal solution may be hard to enforce: requires

relying on a central trusted authority.relying on a central trusted authority.

Question: Question: What happens if we start from an What happens if we start from an emptyempty configuration?configuration? [Chekuri, Chuzhoy, Lewin-Eytan, Naor, and Orda, EC 2006][Chekuri, Chuzhoy, Lewin-Eytan, Naor, and Orda, EC 2006]

Round 1Round 1: Players arrive : Players arrive one by oneone by one, each player plays best response., each player plays best response. Round 2Round 2: Best response dynamics continue in : Best response dynamics continue in arbitrary orderarbitrary order till NE. till NE.

Question: Question: Can a good equilibriumCan a good equilibrium always always be achieved as a be achieved as a consequence of best-response dynamics in this model?consequence of best-response dynamics in this model?

Best Response DynamicsBest Response Dynamics

13

r

21 3 n…

11 1 1

11

¼ + ε¼ + ε

¼ + ε¼ + ε

3/4

11

x

r

1

x

r

1

x

r

1

x

r

1

x

2

r

1

x

2

r

1

x

2

r

1

x

2

r

1

x

32

r

1

x

321

x

321

x

321

x

321

x

321

x

Cost of user 1:c (r, x, 1) = 1+εc (r , 1) = 1

Cost of user 2:c (r , x, 2) = 1+εc (r, 1, x, 2 ) = 1+2εc (r, 2) = 1

Greedy cost of 3, … ,n = 1

Price of anarchy = 4

Can a good equilibrium be achieved as a consequenceCan a good equilibrium be achieved as a consequence of best-response dynamics?of best-response dynamics?

n-2 n-1

14

ResultsResults

The integral multicast game for undirected graphs:The integral multicast game for undirected graphs:

Upper bound of Upper bound of OO(log(log33nn) on the PoA of best-response ) on the PoA of best-response dynamics in the dynamics in the two-roundtwo-round gamegame starting from an “empty” starting from an “empty” configuration. (Improving over the bound of configuration. (Improving over the bound of [CCLNO-EC06] of[CCLNO-EC06] of .).)

Upper bound of Upper bound of OO(log(log2+2+ nn) on the cost of the solution at the ) on the cost of the solution at the end of the end of the first roundfirst round..

Lower bound of Lower bound of (log (log nn) on the PoA of this game.) on the PoA of this game.

Computing a Nash equilibrium minimizing Rosenthal’s Computing a Nash equilibrium minimizing Rosenthal’s potential function is NP-hard. potential function is NP-hard.

)log( 2 nnO

15

Theorem:Theorem: Price of anarchy of our game Price of anarchy of our game ¸̧ (log(lognn))..

Proof:Proof: Adaptation of lower bound proof for the online Adaptation of lower bound proof for the online Steiner problem [Imaze&Waxman] Steiner problem [Imaze&Waxman]

Online Steiner problem: used edges have cost 0 Online Steiner problem: used edges have cost 0 Take hard instance [IW] and replace each terminal Take hard instance [IW] and replace each terminal

by a star of by a star of nn22 + 1 terminals at zero distance + 1 terminals at zero distance The The nn22 + 1 terminals choose the same path to root + 1 terminals choose the same path to root cost of used edge becomes negligiblecost of used edge becomes negligible

Price of Anarchy: Lower Price of Anarchy: Lower BoundBound

16

PoA in Undirected Graphs:PoA in Undirected Graphs: Upper Bound Upper Bound

Analysis is performed in two steps:Analysis is performed in two steps:

• Round 1Round 1: players connect one by one to the root via best response.: players connect one by one to the root via best response.

Solution Solution TT is reached after a sequence of arrivals is reached after a sequence of arrivals tt1 1 , t, t2 2 ,…, t,…, tnn . We . We show:show:

O(logO(log33n )∙ c(Tn )∙ c(TOPTOPT ) )

c(c(TT) ) ·· O(log O(log2+2+ n )∙ c(Tn )∙ c(TOPTOPT ) )

• Round 2Round 2: players play in arbitrary order till NE is reached.: players play in arbitrary order till NE is reached.

cc((TTNashNash) ) ΦΦ((TTNashNash) ) ΦΦ((TT) )

( )

1

( ) cos t( )en T

e

e T k i

cT i

k

17

The First RoundThe First Round

Choose a threshold Choose a threshold 22 (0,1). (0,1). Terminal Terminal t t is is -good-good if cost of next terminal if cost of next terminal

using the same path as using the same path as tt ·· (1- (1-) ) ¢¢ cost( cost(tt). ). OtherwiseOtherwise, , terminal terminal t t is is -bad-bad.. IdeaIdea:: bound bound separatelyseparately the contribution to the contribution to ΦΦ of of

the the -good terminals and the -good terminals and the -bad terminals.-bad terminals.

18

Charging -good Terminals

• tt and and t’ t’ areare -good-good terminalsterminals• tt arrives first, then arrives first, then t t’ arrives’ arrives

Suppose there is a tree Suppose there is a tree T’T’ spanning the spanning the -good terminals.-good terminals.

Upon arrival Upon arrival tt pays: cost( pays: cost(tt) )

Upon arrival, t’ pays at most: Upon arrival, t’ pays at most:

cost(t’) cost(t’) d + (1- d + (1-) ) ¢¢ cost(t) cost(t)

r

t

t’d

cost(t)

19

Charging -good Terminals (contd.)

• tt11,…, ,…, ttkk areare -good-good

terminalsterminals• arrival order: arrival order: tt11,…,,…, t tkk

Charges decay at an exponential rate Charges decay at an exponential rate along a root – leaf path in along a root – leaf path in TT’’

Upon arrival of Upon arrival of ttii it pays at most: it pays at most:

cost(cost(ttii) ) ·· ddii + (1- + (1-))¢¢cost(cost(tti-i-11))

cost(cost(ttkk) ) · · ddkk + d + dkk-1-1(1(1--))

+ d+ dk-2k-2((1-1-))22 + + … + d … + d11(1(1--))kk-1-1

r

t4

tk

dk

t3

t1

t2

d1

d2

d3

d4

The charge to each edge The charge to each edge ee 22 T’T’::·· dd((ee) ) ¢¢ ii (1-(1-))ii ¢¢ nnee((ii))

20

Auxiliary TreeAuxiliary Tree

TTOPTOPT is transformed to an is transformed to an auxiliary treeauxiliary tree T’T’ defined defined

on the set of on the set of -good terminals:-good terminals:

The descendants of terminal The descendants of terminal tt in in T’T’ are terminals are terminals which have arrived after which have arrived after tt..

c(c(T’T’) ) ·· O(1/ O(1/ ¢¢ logn) logn) ¢¢ c(Tc(TOPTOPT ) ) The depth ofThe depth of T’ T’ ·· O(1/O(1/ ¢¢ logn) logn)

21

Contribution of Contribution of -good Terminals-good Terminals

The cost of the °-good paths is bounded as follows:

X

i2X g

c(i) ·O(logn)

°2 ¢TOPT

Theorem:Theorem: The contribution of the The contribution of the -good terminals to -good terminals to ΦΦ in the first phase is bounded as follows: in the first phase is bounded as follows:

Proof:Proof: Follows from the properties of Follows from the properties of T’T’ together with together with the exponential decay of the charges to the edges of the exponential decay of the charges to the edges of T’T’ of of the the -good terminals. -good terminals.

OPT2

(log )cos t( ) ( )

i

O ni c T

22

Contribution of Contribution of -bad Terminals-bad Terminals

The cost of the °-good paths is bounded as follows:

X

i2X g

c(i) ·O(logn)

°2 ¢TOPT

Theorem:Theorem: The contribution of the The contribution of the -bad terminals to -bad terminals to ΦΦ in in the first phase is bounded as follows:the first phase is bounded as follows:

Intuition:Intuition: The cost of the edges “opened” for the first time The cost of the edges “opened” for the first time by by -bad terminals constitutes only a small part of the sum -bad terminals constitutes only a small part of the sum of the costs of the of the costs of the -bad paths.-bad paths.

• Setting Setting = O(1/log = O(1/lognn) O(log) O(log44nn ) upper bound on PoA ) upper bound on PoA• Setting Setting cleverly O(log cleverly O(log33nn ) upper bound on PoA ) upper bound on PoA

:bad : ood

1cos ( ) cos ( )

i i g

t i O t i

23

The Fractional Multicast GameThe Fractional Multicast Game

Players split their connection to the source.Players split their connection to the source. A splittable multicast model.A splittable multicast model.

r

1 2

3/4

1/4

3/4

1/4x y

Flow on (r, x):

• ¼ unit of flow is shared by users 1 & 2.

• ½ unit of flow is used only by user 1.

Flow on (r, y):

• ¼ unit of flow is shared by users 1 & 2.

• ½ unit of flow is used only by user 2.

24

The Fractional Multicast GameThe Fractional Multicast Game

The cost of each flow fraction is split evenly between its users.The cost of each flow fraction is split evenly between its users.

ccee·f·fe,n_ee,n_e is the total cost of edge is the total cost of edge ee..

3

4/1 ec

1/4 3/4 7/8

2

2/1 ecec8/1

fe,1 = 1/4fe,2 = 3/4fe,3 = 7/8

Total cost of user 3: ce· (1/12 + 1/4 + 1/8).

i

ke

kekee

ie kn

ffcc

1

1,,

1

25

The Fractional Multicast GameThe Fractional Multicast Game

The potential function The potential function ΦΦ of the fractional model: of the fractional model:

ΦΦ is an exact potential. is an exact potential. A fractional flow configuration defining a local A fractional flow configuration defining a local

minimum of minimum of ΦΦ corresponds to a NE. corresponds to a NE.

se

sn

j

jn

i

jejee

e e

i

ffc

)(

1

1

1

1,,

26

Results: Fractional Multicast GameResults: Fractional Multicast Game Nash equilibrium existence.Nash equilibrium existence.

NE - minimizing the potential function:NE - minimizing the potential function:

Can be computed in polynomial time (using LP).Can be computed in polynomial time (using LP). It is NP-hard in the case of an It is NP-hard in the case of an integralintegral Nash. Nash.

PoA of the computed NE is O(log PoA of the computed NE is O(log nn).).

27

Create a new graph Create a new graph G’ G’ = (= (VV, , E’E’) by replacing each ) by replacing each edge edge ee by by nn copies copies ee11, , ee22, …, , …, eenn..

Copy Copy eej j : “should” be used if : “should” be used if jj players use edge players use edge ee..

The cost of a unit flow on copy The cost of a unit flow on copy jj of edge of edge ee is is ccee / / jj.. The undirected graph is replaced by a directed flow The undirected graph is replaced by a directed flow

network.network.

Computing a Minimum Potential NE

28

Computing a Minimum Potential NEComputing a Minimum Potential NE

The linear program:The linear program: Objective function = potential function:Objective function = potential function: The capacity of edge The capacity of edge eejj is 0 is 0 xxe_je_j 1. 1. Variables of the LP:Variables of the LP:

Flows of the users on the edges Flows of the users on the edges eejjE’.E’. Capacities of the edges in Capacities of the edges in E’E’..

Constraints:Constraints: Non-aggregatingNon-aggregating flow constraint: (flow of user flow constraint: (flow of user ii on on eej j ) ) xxe_j e_j ..

Aggregating Aggregating flow constraint: (total flow on flow constraint: (total flow on eejj ) = ) = j j ∙ ∙ xxe_je_j ..

1 1

j

jne e

e E j i

c x

i

29

The Linear ProgramThe Linear Program

Theorem 5:Theorem 5: There exists an optimal solution to the linear program There exists an optimal solution to the linear program that corresponds to a fractional multicast flow.that corresponds to a fractional multicast flow.

Heavily depends on the non-increasing property of the Heavily depends on the non-increasing property of the cost function. cost function.

LP can be used:LP can be used: For any cost sharing mechanism that is cross-monotonic.For any cost sharing mechanism that is cross-monotonic. In case players are not restricted to have a common source.In case players are not restricted to have a common source.

PoA of a minimum potential fractional NE is PoA of a minimum potential fractional NE is OO(log (log nn).).

30

Integral vs. Fractional Potential Integral vs. Fractional Potential MinimizationMinimization

There exists an instance where the gap There exists an instance where the gap between the integral and fractional between the integral and fractional minimum potential solutions is a minimum potential solutions is a smallsmall constant.constant.

Finding an Finding an integralintegral Nash equilibrium that Nash equilibrium that minimizes the potential function is NP-hard. minimizes the potential function is NP-hard. Building block: a variation of the Lund-Building block: a variation of the Lund-

Yannakakis hardness proof of approximating Yannakakis hardness proof of approximating the set cover problem.the set cover problem.

31

The Weighted Multicast GameThe Weighted Multicast Game Each player Each player ii has a positive weight has a positive weight wwii

.. The cost share of each player is proportional to The cost share of each player is proportional to

its weight.its weight. Integral: Integral: cost share of player cost share of player ii = c = cee· (· (wwii / W / Wee)) ((WWee – weight of players currently using – weight of players currently using ee))

Fractional: Fractional: weighted sharing on each fraction.weighted sharing on each fraction.

Theorem:Theorem: A NE always exists for the weighted A NE always exists for the weighted fractional modelfractional model..

Note:Note: NE does not necessarily exists for the weighted integral NE does not necessarily exists for the weighted integral model [Chen-Roughgarden, SPAA 06].model [Chen-Roughgarden, SPAA 06].

32

Thank You!Thank You!