1

Algorithmic Game Algorithmic Game Theoretic Perspectives Theoretic Perspectives

in Networkingin Networking

Dr. Liane Lewin-EytanDr. Liane Lewin-Eytan

2

Prisoner’s DilemmaPrisoner’s Dilemma

Scenario:Scenario:Two suspects in a crime are put into separate cells. If they Two suspects in a crime are put into separate cells. If they both confess, each will be sentenced into 3 years in prison. If both confess, each will be sentenced into 3 years in prison. If only one of them confesses, he will be freed and used as a only one of them confesses, he will be freed and used as a witness against the other, who will receive a sentence of 4 witness against the other, who will receive a sentence of 4 years. If neither confesses, they will both be convicted of a years. If neither confesses, they will both be convicted of a minor offense and spend one year in prison.minor offense and spend one year in prison.

1,11,1 4,04,0

0,40,4 3,33,3confess

don’t confess

don’t confess confess 1 equilibrium: (confess, confess)

3

Games in NetworksGames in Networks

Users with a multitude of Users with a multitude of diverse economic diverse economic interests sharing a interests sharing a Network (Network (InternetInternet))

browsersbrowsers routersrouters serversservers

Selfishness:Selfishness:Parties deviate from their Parties deviate from their

protocol if it is in their protocol if it is in their interestinterest

4

MotivationMotivation

1. Networking1. Networking

Traditional networks – single entity with Traditional networks – single entity with single control objective.single control objective.

Modern networking – interaction of many Modern networking – interaction of many entities controlled by different parties.entities controlled by different parties.

5

MotivationMotivation

2. Games2. Games

Users act according to their individual interests so Users act according to their individual interests so as to maximize their own objective functions.as to maximize their own objective functions.

A user makes selfish decisions based on the state A user makes selfish decisions based on the state of the network, which depends on the behavior of of the network, which depends on the behavior of the other users.the other users.

non-cooperative network games.non-cooperative network games.

6

MotivationMotivation

3. Algorithms3. Algorithms

Computational and algorithmic issues Computational and algorithmic issues arise in large and complex games arise in large and complex games motivated by large decentralized motivated by large decentralized computer networks computer networks (the Internet).(the Internet).

7

Networking

Games Algorithms

Non-Cooperative Network Games

Algorithmic Game Theory

Algorithmic Perspectives of Game Theory

in (Large Scale) Networks

8

A Simple Game: Load A Simple Game: Load BalancingBalancing

Each job Each job wantswants to be on a lightly loaded to be on a lightly loaded machine.machine.

2

2

1

3

machine 1 machine 2machine 1 machine 2

With coordination we can arrange them to minimize load

Example: load of 4

9

A Simple Game: Load A Simple Game: Load BalancingBalancing

Each job Each job wantswants to be on a lightly loaded to be on a lightly loaded machine.machine.

2

2

1

3

• Without coordination?

• Stable arrangement: No job has incentive to switch

• Example: some have load of 5

10

Games: SetupGames: Setup

A set of players A set of players (in example: jobs)(in example: jobs) for each player, a set of strategies for each player, a set of strategies

(which machine to choose)(which machine to choose)

Game: each player picks a strategyGame: each player picks a strategy

For each strategy profile (a strategy For each strategy profile (a strategy for each player) for each player) a payoff to each a payoff to each playerplayer (load on selected (load on selected machine)machine)

11

Nash EquilibriumNash Equilibrium

A set of actions (strategy choices), one A set of actions (strategy choices), one per player, where no player can per player, where no player can unilaterally improve its performance by unilaterally improve its performance by changing its strategy.changing its strategy.

The Nash equilibrium solutions of a game The Nash equilibrium solutions of a game are its stable operating points (stable are its stable operating points (stable strategy profile).strategy profile).

12

Quality of Outcome:Quality of Outcome: Goal of the GameGoal of the Game

Personal objective for player i:min load Li

Overall objective?

• Social Welfare: i Li

• Makespan: maxi Li

13

Routing network:Routing network:

ℓe(x) = x

s t

Delay as a function of load:

x unit of load causes delay ℓe(x)

Load Balancing and RoutingLoad Balancing and Routing

Load balancing:

jobs

machines

ℓe(x) = x

Allow more complex networks

s tx 1

x10

14

FrameworkFramework

A network shared by selfish users.A network shared by selfish users. Each resource has a cost to be paid by its Each resource has a cost to be paid by its

users. users. Performance of a user = its total payment = Performance of a user = its total payment =

sum of payments for all the resources it uses.sum of payments for all the resources it uses.

Two fundamental models:Two fundamental models: TheThe congestion modelcongestion model.. TheThe cost sharing modelcost sharing model..

15

Resource cost is fixed.Resource cost is fixed. Cost sharing Cost sharing mechanism determines mechanism determines how the cost is shared how the cost is shared by the users.by the users.

Each user has a Each user has a favorable effect on the favorable effect on the performance of other performance of other users.users.

Resource cost:Resource cost: Modeled by a load Modeled by a load dependent function.dependent function. Non-decreasing in Non-decreasing in the load of the the load of the resource.resource.

Each user has a Each user has a negative effect on the negative effect on the performance of other performance of other users.users.

Cost Sharing ModelCost Sharing ModelCongestionCongestion ModelModel

16

The Game PerspectiveThe Game Perspective

Strategy space of each player: subsets of Strategy space of each player: subsets of resources.resources.

Cost allocation method defines the rules of the Cost allocation method defines the rules of the game: game: determines the mutual influence among the determines the mutual influence among the players.players.

Each player knows the rules of the underlying Each player knows the rules of the underlying game.game.

Players are rational: a player chooses a strategy Players are rational: a player chooses a strategy that minimizes its total payment.that minimizes its total payment.

17

The Congestion ModelThe Congestion Model

18

Routing with DelayRouting with Delay

Edge-delay is a function ℓe(•) of the load on the edge e

Assume Assume ℓℓee(x)(x) continuous and non-continuous and non-decreasing in load x on edge e.decreasing in load x on edge e.

s t

x 1

x1

0

19

One unit of flow sent from s to tOne unit of flow sent from s to t

A stable solution:

Users control an infinitesimally small amount of flow.

x

s t1

Flow = .5

Flow = .5 Traffic on upper edge is envious.

Example on two linksExample on two links

No-one is better off

x

s t1

Flow = 1

Flow = 0

20

Model of Routing GameModel of Routing Game

A directed graph G = A directed graph G = (V,E)(V,E)

source–sink pairs ssource–sink pairs sii,t,tii for for i=1,..,ki=1,..,k

rate rrate ri i 0 of traffic 0 of traffic between sbetween sii and t and tii for for each i=1,..,keach i=1,..,k

r1 =1

s tx 1

.5

x1 .5

.5

.5

• Load-balancing jobs wanted min load• Here want minimum delay:

delay adds along path

21

Goal of the GameGoal of the Game

Personal objective: choose a path minimizing

ℓP(f) = sum of latencies of edges along path P

Overall objective: minimize

C(f) = total latency of a flow f = e fe•ℓe(f)

=social welfare

22

Network Routing GameNetwork Routing Game

Flow representsFlow represents cars on highways cars on highways packets on the Internetpackets on the Internet

User goal: Find a path minimizing delay

true for cars, packets?: users do not choose paths on the Internet: routers do!

With delay as primary metric router protocols choose shortest path!

23

Braess’s ParadoxBraess’s Paradox

Original Network

Cost of Nash flow =2(1.5*0.5)=1.5

s tx 1.5

x1.5

.5

.5

Added edge:

Effect?

s tx 1

x1

.5 .5.5 .50

24

Braess’s ParadoxBraess’s Paradox

Original Network

Added edge:

Cost of Nash flow = 2

All the flow has increased delay!

s tx 1

x1

11 10

Cost of Nash flow = 2(1.5*0.5)=1.5

s tx 1.5

x1.5

.5

.5

25

Some ResultsSome Results

Theorem Theorem (Roughgarden-Tardos’00)(Roughgarden-Tardos’00) In a network with linear latency functionsIn a network with linear latency functions

i.e., of the formi.e., of the form ℓℓee(x)=a(x)=aeex+bx+bee

the cost of a Nash flow is at most the cost of a Nash flow is at most 4/3 4/3 times that of the minimum-latency flowtimes that of the minimum-latency flow

Price of Anarchy = Price of Anarchy = 3/43/4

26

Some ResultsSome Results

Theorem 1Theorem 1 (Roughgarden-Tardos’00)(Roughgarden-Tardos’00) In a network with linear latency functionsIn a network with linear latency functions

i.e., of the formi.e., of the form ℓℓee(x)=a(x)=aeex+bx+bee

the cost of a Nash flow is at most the cost of a Nash flow is at most 4/3 4/3 times that of the minimum-latency flowtimes that of the minimum-latency flow

s tx 1

r=1

x10

x

s t1

Flow = .5

Flow = .5

Nash cost 1 optimum 3/4 Nash cost 2 optimum 1.5

27

The Price of AnarchyThe Price of Anarchy

Typically,Typically, Nash equilibrium outcomes do not Nash equilibrium outcomes do not optimize the overall network performance.optimize the overall network performance.

Price of AnarchyPrice of Anarchy:: The ratio between the cost of The ratio between the cost of the worst Nash equilibrium and the (social) the worst Nash equilibrium and the (social) optimum.optimum.

Quantifies the penalty incurred by lack of Quantifies the penalty incurred by lack of cooperation.cooperation.

28

The Cost Sharing ModelThe Cost Sharing Model

29

MulticastMulticast

• A source simultaneouslyA source simultaneously transmits the same data to a group of transmits the same data to a group of destinations.destinations.• Messages are transmitted over each link of the network only once.Messages are transmitted over each link of the network only once.• Multicast nodes create copies when the links to the destinations Multicast nodes create copies when the links to the destinations split.split.

• Multicast routing increases network efficiency.Multicast routing increases network efficiency.r

t1

t2 t4 t5 t6

t3

30

A Cost Sharing Multicast GameA Cost Sharing Multicast Game

A special source node (A special source node (rootroot) ) rr, and a set , and a set NN of of n n receivers (receivers (playersplayers).).

A player’s strategy isA player’s strategy is a routing decision – a routing decision – the choice of a route from its terminal to the choice of a route from its terminal to rr..

Egalitarian cost sharing mechanismEgalitarian cost sharing mechanism: the : the cost of each edge is evenly split among its cost of each edge is evenly split among its downstream receivers.downstream receivers.

cceeii((ss)) = = cce e / / nnee((ss))

31

Egalitarian Cost Sharing Egalitarian Cost Sharing MechanismMechanism

Payment of t1: c1/4

Payment of t2: c1/4 + c2

Payment of t3: c1/4 + c3/2

Payment of t4: c1/4 + c3/2 + c4

r

t1

t2 t3 t5 t6

t4

c1

c2 c3

c4

32

Goal of the GameGoal of the Game

Personal objective: choose a path to the root minimizing payment.

Overall objective: minimize

C(T) = total cost of T = eT ce

= social welfare

= Steiner tree !

33

Potential FunctionPotential Function

Multicast game admits a potential function.Multicast game admits a potential function. Potential function Potential function ΦΦ of a solution of a solution T T [Rosenthal `73][Rosenthal `73]::

Exact potential:Exact potential:

Change in potential = change in payoff of player Change in potential = change in payoff of player making a movemaking a move

Global / Local optimum of Global / Local optimum of ΦΦ corresponds to a NE. corresponds to a NE.

Te

Tn

k

ee

k

cT

)(

1

)(

34

Price of anarchyPrice of anarchy can be as bad as can be as bad as ((nn).).

1

s tn

OPT (= Best NE)all players use cheap edgeeach pays 1/ntotal cost = 1

1

s tn

Worst NE all players use expensive edgeeach pays n/n=1total cost = n

Price of AnarchyPrice of Anarchy

35

The Price of StabilityThe Price of Stability

Price of anarchy: Price of anarchy: Can be unbounded.Can be unbounded. Also captures “non-interesting” equilibria.Also captures “non-interesting” equilibria.

Price of StabilityPrice of Stability:: The ratio between the cost of The ratio between the cost of the best Nash solution and the cost of OPT.the best Nash solution and the cost of OPT.

Outcome of scenarios in the ‘middle ground’ Outcome of scenarios in the ‘middle ground’ between centrally enforced solutions and selfish between centrally enforced solutions and selfish behavior.behavior. E.g.: central entity can enforce the initial E.g.: central entity can enforce the initial

operating point.operating point.

36

Price of stabilityPrice of stability – upper bound is O(log – upper bound is O(log nn). ).

cc((TTNashNash) ) ΦΦ((TTNashNash) ) ΦΦ((TTinitialinitial) ) log log n ∙cn ∙c((TTinitialinitial) )

proof:proof: edge costedge cost ccee withwith n nee > 0 > 0 usersusers

edge potentialedge potential withwith n nee > 0 > 0 usersusers

e e =c=cee·(1+1/2+1/3+…+1/n·(1+1/2+1/3+…+1/nee))

Ratio at most HRatio at most Hnn=O(log n)=O(log n)

Price of StabilityPrice of Stability

37

Example: Bound is TightExample: Bound is Tight

1 1n

12

13

1 2 3 n

t

0 0 0 0

1+ . . . n-1

0

1n-1

38

Example: Bound is TightExample: Bound is Tight

1 1n

12

13

1 2 3 n

t

0 0 0 0

1+ . . . n-1

0

1n-1

cost(OPT) = 1+ε

39

Example: Bound is TightExample: Bound is Tight

1 1n

12

13

1 2 3 n

t

0 0 0 0

1+ . . . n-1

0

1n-1

cost(OPT) = 1+ε…but not a NE: player n pays (1+ε)/n, could pay 1/n

40

Example: Bound is TightExample: Bound is Tight

1 1n

12

13

1 2 3 n

t

0 0 0 0

1+ . . . n-1

0

1n-1

so player n would deviate

41

Example: Bound is TightExample: Bound is Tight

1 1n

12

13

1 2 3 n

t

0 0 0 0

1+ . . . n-1

0

1n-1

now player n-1 pays (1+ε)/(n-

1), could pay 1/(n-

1)

42

Example: Bound is TightExample: Bound is Tight

1 1n

12

13

1 2 3 n

t

0 0 0 0

1+ . . . n-1

0

1n-1

so player n-1 deviates too

43

Example: Bound is TightExample: Bound is Tight

1 1n

12

13

1 2 3 n

t

0 0 0 0

1+ . . . n-1

0

1n-1

Continuing this process, all players defect.

This is a NE!(the only Nash)cost = 1 + +

… +

Price of Stability is Hn = Θ(log n) !

1 12 n

44

Best response dynamics Best response dynamics :: each player, in its turn, each player, in its turn, selects a strategy minimizing its cost (or selects a strategy minimizing its cost (or maximizing its profit).maximizing its profit).

Natural game course continues until a NE is Natural game course continues until a NE is reached.reached.

PoA may depend on the initial game PoA may depend on the initial game configuration. configuration.

A natural starting point: empty configuration.A natural starting point: empty configuration.

Best Response DynamicsBest Response Dynamics

45

r

21 3 n…

11 1 1

11

¼ + ε¼ + ε

¼ + ε¼ + ε

3/4

11

x

r

1

x

r

1

x

r

1

x

r

1

x

2

r

1

x

2

r

1

x

2

r

1

x

2

r

1

x

32

r

1

x

321

x

321

x

321

x

321

x

321

x

Cost of user 1:c (r, x, 1) = 1+εc (r , 1) = 1

Cost of user 2:c (r , x, 2) = 1+εc (r, 1, x, 2 ) = 1+2εc (r, 2) = 1

Greedy cost of 3, … ,n = 1

Price of anarchy = 4

Can a good equilibrium be achieved as a consequence of Can a good equilibrium be achieved as a consequence of best-response dynamics, starting from an empty configuration?best-response dynamics, starting from an empty configuration?

n-2 n-1

NE OPT

46

Some Results Some Results (Chuzhoy (Chuzhoy et al.et al. ‘06) ‘06)

Upper bound of on the PoA of Upper bound of on the PoA of best-response dynamics in case players best-response dynamics in case players join the game sequentially starting from join the game sequentially starting from an ‘empty’ configuration.an ‘empty’ configuration. was improved to O(log was improved to O(log33 nn) by Charikar ) by Charikar et alet al. .

Lower bound of on the PoA of Lower bound of on the PoA of this game.this game.

Computing a NE minimizing Rosenthal’s Computing a NE minimizing Rosenthal’s potential function is NP-hard.potential function is NP-hard.

)log( 2 nnO

n

n

loglog

log

47

Thank You Thank You