Potential games, Congestion gamesComputational game theory

Spring 2010

Adapting slides by Michal Feldman

Potential games

• Definition: a game is an ordinal potential game if there exists :S1×…×Sn R, s.t. i,si,s-i,si’,ci(si,s-i) > ci(si’,s-i) IFF (si,s-i) > (si’,s-i)

• Note: G is an exact potential game if ci(si,s-i) - ci(si’,s-i) = (si,s-i) - (si’,s-i)

• Example: max-cut is an exact potential game, where is the cut size– Unfortunately, is not always so natural

Potential games• Lemma: a game is a potential game IFF local

improvements always terminate• proof:

– Define a directed graph with a node for each possible pure strategy profile

– Directed edge (u,v) means v (which differs from u only in the strategy of a single player, i) is a (strictly) better action for i, given the strategies of the other players

– A potential function exists IFF graph does not contains cycles• If cycle exists, no potential function; e.g., (a,b,c,a) means

f(a)<f(b)<f(c)<f(a)• If no cycles exist, can easily define a ordinal potential function WHY?

Examples direction of local improvement

-1,11,-1

1,-1-1,1

3,30,5

5,01,1

2,20,0

0,03,3

Matching penniesPrisoner’s dilemma

Coordination game

C

D

DC

col

col

rowrow

Which are potential games?Exact potential games?

Are the potential functions unique ?

2,10,0

0,01,2

Battle of the sexes

Properties of potential games

• Admit a pure strategy Nash equilibrium• Best-response dynamics converge to NE• Price of stability is bounded

Existence of a pure NE

• Theorem: every potential game admits a pure NE• Proof: we show that the profile minimizing is a NE

– Let s be pure profile minimizing – Suppose it is not a NE, so i can improve by deviating to a

new profile s’– (s’) - (s) = ci(s’) – ci(s) < 0– Thus(s’) < (s) , contradicting s minimizes

• More generally, the set of pure-strategy Nash equilibria is exactly the set of local minima of the potential function– Local minimum = no player can improve the potential function by

herself

Best-response dynamics converge to a NE

• Best-response dynamics: – Start with any strategy profile– If a player is not best-responding, switch that player’s

strategy to a better response (must decrease potential)– Terminate when no player can improve (thus a NE)

– Alas, no guarantee on the convergence rate

8

Multicast (and non-multicast) Routing

• Multicast routing: Given a directed graph G = (V, E) with edge costs ce 0, a source node s, and k agents located at terminal nodes t1, …, tk. Agent j must construct a path Pj from node s to its terminal tj.

• Routing: Given a directed graph G = (V, E) with edge costs ce 0, and k agents seeking to connect sj,tj pairs, Agent j must construct a path Pj from node sj to its terminal tj.

• Fair share: If x agents use edge e, they each pay ce / x.

Slides on cost sharing based on slides by Kevin Wayne.Copyright @ 2005 Pearson-Addison Wesley.

All rights reserved.

9

Multicast Routing : Shapley price sharing (fair cost sharing)

outer

2

outer

middle

4

1 pays

5 + 1

5/2 + 1

middle 4

1

outer

middle

middle

outer

8

2 pays

8

5/2 + 1

5 + 1

s

t1

v

t2

4 8

1 1

5

10

Nash Equilibrium• Example:

– Two agents start with outer paths.– Agent 1 has no incentive to switch

paths (since 4 < 5 + 1), but agent 2 does (since 8 > 5 + 1).

– Once this happens, agent 1 prefers middle path (since 4 > 5/2 + 1).

– Both agents using middle path is a Nash equilibrium.

s

t1

v

t2

4 8

1 1

5

Recall price of anarchy and stability

• Price of anarchy (poa)=cost of worst NE / cost of OPT

• Price of stability (pos)=cost of best NE / cost of OPT

Socially Optimum• Social optimum: Minimizes total costs of all agents.• Observation: In general, there can be many Nash

equilibria. Even when it is unique, it does not necessarily equal the social optimum.

s

t1

v

t2

3 5 5

1 1

Social optimum = 7Unique Nash equilibrium = 8

s

t

k1 +

Social optimum = 1 + Nash equilibrium A = 1 + Nash equilibrium B = k

k agents

pos=1, poa=k pos=poa=8/7

Price of anarchy

• Claim: poa ≤ k• Proof:

– Let N be the worst NE– Suppose by contradiction c(N) > k OPT– Then, there exists a player i s.t. ci(N) > OPT

– But i can deviate to OPT (by paying OPT alone), contradicting that N is a NE

• Note: bound is tight (lower bound in prev. slide)

14

Price of Stability• What is price of stability in multicast routing?

• Lower bound of log k:

s

t2 t3 tkt1. . .

1 1/2 1/3 1/k

0 0 0 0

1 +

1 + 1/2 + … + 1/k

Social optimum: Everyone Takes bottom paths.

Unique Nash equilibrium: Everyone takes top paths.

Price of stability: H(k) / (1 + ).

• upper bound will follow..

16

Finding a potential functionConsider a set of paths P1, …, Pk.

– Let xe denote the number of paths that use edge e.

– Let (P1, …, Pk) = eE ce· H(xe) be a potential function.

– Consider agent j switching from path Pj to path Pj'.

– Change in agent j’s cost:

H(0) = 0 ,

1

1( )

k

ii

H k

c f

x f 1f Pj ' Pj

newly incurred cost

ce

xee Pj Pj '

cost saved

17

Potential function– increases by

– decreases by

– Thus, net change in is identical to net change in player j’s cost

c f H(x f 1) H(x f ) f Pj ' Pj

c f

x f 1 f Pj ' Pj

ce H(xe ) H(xe 1) e Pj Pj '

ce

xe

e Pj Pj '

18

Bounding the Price of StabilityClaim: Let C(P1, …, Pk) denote the total cost of selecting

paths P1, …, Pk.

For any set of paths P1, …, Pk , we have

Proof: Let xe denote the number of paths containing edge e.

– Let E+ denote set of edges that belong to at least one of the paths.

C(P1,, Pk ) cee E ce H(xe )

e E

(P1,, Pk )

ce H(k) H(k)e E C(P1,, Pk )

),,()( ),,( ),,( 111 kkk PPCkHPPPPC

19

Bounding the Price of StabilityTheorem: There is a Nash equilibrium for which the total cost to

all agents exceeds that of the social optimum by at most a factor of H(k) (i.e., price of stability ≤ H(k)).

Proof:– Let (P1

*, …, Pk*) denote set of socially optimal paths.

– Run best-response dyn algorithm starting from P*.– Since is monotone decreasing (P1, …, Pk) (P1

*, …, Pk

*).

C(P1,, Pk ) (P1,, Pk ) (P1*,, Pk *) H(k) C(P1*,, Pk *)

previous claimapplied to P

previous claimapplied to P*

Congestion games [Rosenthal 73]

• There is a set of resources R

• Agent i’s set of actions (pure strategies) Ai is a subset of 2R, representing which subsets of resources would meet her needs – Note: different agents may need different resources

• There exist cost functions cr: {1, 2, 3, …} → such that agent i’s cost for a = (ai, a-i) is Σr ai

cr(nr(a)) – nr(a) is the number of agents that chose r as one of their resources in

the profile a

Example: multicast routing• Resources = edges• Each resource r has a cost cr

• Player 1’s action set: {{A}, {C,D}}• Player 2’s action set: {{B}, {C,E}}• For all resources r, cr(nr(a)) = cr / nr(a)

s

t1

v

t2

E

8

1 1

5A

4 C

D

B

Every congestion game is an exact potential game

• Use potential (a) = Σr Σ1 ≤ i ≤ nr(a) cr(i)– One interpretation: the sum of the costs that the agents would

have received if each agent were unaffected by all later agents • Why is this a correct potential function?

• Suppose an agent changes action: stop using some resources (R-), start using others (R+)

• increase in the agent’s cost equals Σr R+ cr(nr(a) + 1) - Σr R- cr(nr(a))

This is exactly the change in the potential function above

– Conclusion: congestion games are exact potential games

Other Congestion games

• (si,ti) connectivity, atomic flow, cost = latency, all flow one unit.

Computational Game Theory:Network Creation Game

Arbitrary Payments (Not a congestion game)

Credit to Slides

To Eva TardosModified/Corrupted/Added to

by Michal Feldman and Amos Fiat

Network Creation Game – Arbitrary Cost partition

G = (V,E) is an undirected graph with edge costs c(e).

There are k players.

Each player i has a source si and a sink ti he wants to have connected.

s1 t3

t1

t2s2

s3

Model (cont’)

Player i picks payment pi(e) for each edge e.

e is bought if total payments ≥ c(e).

Note: any player can use bought edges

s1 t3

t1

t2s2

s3

The Game

Each player i has only 2 concerns :

1 (Must be a bought path from si to ti

s1 t3

t1

t2s2

s3

boughtedges

The Game

Each player i has only 2 concerns :

1 (Must be a bought path from si to ti

2 (Given this requirement, i wants to pays as little as possible.

s1 t3

t1

t2s2

s3

Nash Equilibrium

A Nash Equilibium (NE) is set of payments for players such that no player wants to deviate .

Note: player i doesn’t care whether other players connect.

s1 t3

t1

t2s2

s3

An Example

One NE: Each player pays 1/k to top edge.

Another NE: Each player pays 1 to bottom edge.

Note: No notion of “fairness”; many NE that pay unevenly for the cheap edge.

s1…sk t1…tk

c(e) = 1

c(e) = k

Three Observations

1) The bought edges in a NE form a forest.

2) Players only contribute to edges on their si-ti path in this forest.

3) The total payment for any edge e is either c(e) or 0.

Example 2: No Nash

s1

t1

t2

s2

all edges cost 1

ab

cd

Example 2: No Nash

s1

t1

t2

s2

We know that any NE must be a tree: WLOG assume the tree is a,b,c.

all edges cost 1

ab

cd

Example 2: No Pure Nash

s1

t1

t2

s2

We know that any NE must be a tree: WLOG assume the tree is a,b,c.

• Only player 1 can contribute to a.

all edges cost 1

ab

cd

Example 2: No Pure Nash

s1

t1

t2

s2

We know that any NE must be a tree: WLOG assume the tree is a,b,c.

• Only player 1 can contribute to a.

• Only player 2 can contribute to c.

all edges cost 1

ab

cd

Example 2: No Pure Nash

s1

t1

t2

s2

We know that any NE must be a tree: WLOG assume the tree is a,b,c.

• Only player 1 can contribute to a.

• Only player 2 can contribute to c.

• Neither player can contribute to b, since d is tempting deviation.

all edges cost 1

ab

cd