Potential Functions and the Inefficiency of Equilibria

Tim RoughgardenStanford University

2

Pigou's Example

Example: one unit of traffic wants to go from s to t

Question: what will selfish network users do?• assume everyone wants smallest-possible cost• [Pigou 1920]

s t

c(x)=x

c(x)=1

cost depends on congestion

no congestion effects

3

Motivating Example

Claim: all traffic will take the top link.

Reason:• Є > 0 traffic on bottom is envious• Є = 0 equilibrium

– all traffic incurs one unit of cost

s t

c(x)=x

c(x)=1

Flow = 1-Є

Flow = Єthis flow is envious!

4

Can We Do Better?

Consider instead: traffic split equally

Improvement:• half of traffic has cost 1 (same as before)• half of traffic has cost ½ (much improved!)

s t

c(x)=x

c(x)=1

Flow = ½

Flow = ½

5

Braess’s Paradox

Initial Network:

s tx 1

½

x1½

½

½

Cost = 1.5

6

Braess’s Paradox

Initial Network: Augmented Network:

s tx 1

½

x1½

½

½

Cost = 1.5

s tx 1

½

x1½

½

½0

Now what?

7

Braess’s Paradox

Initial Network: Augmented Network:

s tx 1

½

x1½

½

½

Cost = 1.5 Cost = 2

s t

x 1

x10

8

Braess’s Paradox

Initial Network: Augmented Network:

All traffic incurs more cost! [Braess 68]

• also has physical analogs [Cohen/Horowitz 91]

s tx 1

½

x1½

½

½

Cost = 1.5 Cost = 2

s t

x 1

x10

9

High-Level Overview

Motivation: equilibria of noncooperative network games typically inefficient

• e.g., Pigou's example + Braess's Paradox• don't optimize natural objective functions

Price of anarchy: quantify inefficiency w.r.t some objective function

Our goal: when is the price of anarchy small?– when does competition approximate cooperation?– benefit of centralized control is small

10

Selfish Routing Games

• directed graph G = (V,E)

• source-destination pairs (s1,t1), …, (sk,tk)

• ri = amount of traffic going from si to ti

• for each edge e, a cost function ce(•)– assumed continuous and nondecreasing

Examples: (r,k=1)

s1 t1

c(x)=x c(x)=1

c(x)=xc(x)=1

½

½s1 t1

c(x)=x

c(x)=1

½

½c(x)=0

11

Outcomes = Network Flows

Possible outcomes of a selfish routing game:

• fP = amount of traffic choosing si-ti path P

• outcomes of game flow vectors f– flow vector: nonnegative and total flow fP

on si-ti paths equals traffic rate ri (for all i)s t

12

Outcomes = Network Flows

Possible outcomes of a selfish routing game:

• fP = amount of traffic choosing si-ti path P

• outcomes of game flow vectors f– flow vector: nonnegative and total flow fP on si-ti

paths equals traffic rate ri (for all i)

Question: What are the equilibria (natural selfish outcomes) of this game?

s t

13

Nash Flows

Def: [Wardrop 52] A flow is at Nash equilibrium (or is a Nash flow) if no one can switch to a path of smaller cost. I.e., all flow is routed on min-cost paths. [given current edge congestion]

xs t

1s t

1

x

Examples:½

½

1

s tx 1

x10 1s t

x 1

x10

½

½

14

Our Objective Function

Definition of social cost: total cost C(f) incurred by the traffic in a flow f.

Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:

C(f) = P fP • cP(f)

s t

15

Our Objective Function

Definition of social cost: total cost C(f) incurred by the traffic in a flow f.

Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:

C(f) = P fP • cP(f)

Example:

s t

s tx

1½½

Cost = ½•½ +½•1 = ¾

16

The Price of Anarchy

Defn:

– definition from [Koutsoupias/Papadimitriou 99]

price ofanarchy of a game

=obj fn value of selfish outcome

optimal obj fn value

xs t

1s t

1

x

Example: POA = 4/3 in Pigou's example½

½

1

Cost = 1 Cost = 3/4

17

A Nonlinear Pigou Network

Bad Example: (d large)

equilibrium has cost 1, min cost 0

s t

xd

10

1 1-Є

Є

18

A Nonlinear Pigou Network

Bad Example: (d large)

equilibrium has cost 1, min cost 0

price of anarchy unbounded as d -> infinity

Goal: weakest-possible conditions under which P.O.A. is small.

s t

xd

10

1 1-Є

Є

19

When Is the Price of Anarchy Bounded?

Examples so far:

Hope: imposing additional structure on the cost functions helps– worry: bad things happen in larger networks

s tx

1s t

xd

1s t

x 1

x10

20

Polynomial Cost Functions

Def: linear cost fn is of form ce(x)=aex+be

Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:

≤ 4/3 × cost of Nash flow

cost of opt flow

s tx

1

21

Polynomial Cost Functions

Def: linear cost fn is of form ce(x)=aex+be

Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:

≤ 4/3 ×

Bounded-deg polys: (w/nonneg coeffs) replace 4/3 by Θ(d/log d)

cost of Nash flow

cost of opt flow

s txd

1

tightexample

s tx

1

22

A General Theorem

Thm: [Roughgarden 02], [Correa/Schulz/Stier

Moses 03] fix any set of cost fns. Then, a Pigou-like example 2 nodes, 2 links, 1 link w/constant cost fn) achieves worst POA

s txd

1

tightexample

23

Interpretation

Bad news: inefficiency of selfish routing grows as cost functions become "more nonlinear".– think of "nonlinear" as "heavily congested"– recall nonlinear Pigou's example

Good news: inefficiency does not grow with network size or # of source-destination pairs.– in lightly loaded networks, no matter how

large, selfish routing is nearly optimals txd

1

tightexample

24

Benefit of Overprovisioning

Suppose: network is overprovisioned by β > 0 (β fraction of each edge unused).

Then: Price of anarchy is at most ½(1+1/√β).

• arbitrary network size/topology, traffic matrix

Moral: Even modest (10%) over-provisioning sufficient for near-optimal routing.

25

Potential Functions

• potential games: equilibria are actually optima of a related optimization problem

– has immediate consequences for existence, uniqueness, and inefficiency of equilibria

– see [Beckmann/McGuire/Winsten 56], [Rosenthal 73], [Monderer/Shapley 96], for original references

– see [Roughgarden ICM 06] for survey

26

The Potential Function

Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f

ce(x)dx (over all flows).

ce(fe)

00 fe

0e

27

The Potential Function

Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f

ce(x)dx (over all flows).

Lemma 1: locally optimal solutions are precisely the Nash flows (derivative test).

Lemma 2: all locally optimal solutions are also globally optimal (convexity).

Corollary: Nash flows exist, are unique.

ce(fe)

00 fe

0e

28

Consequences for the Price of Anarchy

Example: linear cost functions.

Compare cost + potential function:

C(f) = e fe • ce(fe) = e [ae fe + be fe]

PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe]

2

0e

2

29

Consequences for the Price of Anarchy

Example: linear cost functions.

Compare cost + potential function:

C(f) = e fe • ce(fe) = e [ae fe + be fe]

PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe]

• cost, potential fn differ by factor of ≤ 2• gives upper bound of 2 on price on anarchy

– C(f) ≤ 2×PF(f) ≤ 2×PF(f*) ≤ 2×C(f*)

2

0e

2

30

Better Bounds?

Similarly: proves bound of d+1 for degree-d polynomials (w/nonnegative coefficients).

• not tight, but qualitatively accurate – e.g., price of anarchy goes to infinity with

degree bound, but only linearly

• to get tight bounds, need "variational inequalities"– see my ICM survey for details

31

Variational Inequality

Claim: • if f is a Nash flow and f* is feasible, then

e fe • ce(fe) ≤ e f* • ce(fe)

• proof: use that Nash flow routes flow on shortest paths (w.r.t. costs ce(fe))

e

32

Pigou Bound

Recall goal: want to show Pigou-like examples are always worst cases.

Pigou bound: given set of cost functions (e.g., degree-d polys), largest POA in a network:

• two nodes, two links• one function in given set• one constant function

– constant = cost of fully congested top edge

s txd

1

33

Pigou Bound (Formally)

Let S = a set of cost functions.– e.g., polynomials with degree at most d,

nonnegative coefficients

Definition: the Pigou bound α(S) for S is:

max

• max is over all choices of cost fns c in S, traffic rate r 0, flow y 0

s txd

1

r • c(r)

y • c(y) + (r-y) • c(r)

34

Pigou Bound (Example)

Let S = { c : c(x) = ax +b } [linear functions]

Recall: the Pigou bound α(S) for S is:

max

• max is over all choices of cost fns c in S, traffic rate r 0, flow y 0

• choose c(x) = x; r = 1; y = 1/2 get 4/3• calculus: α(S) = 4/3 [d/ln d for deg-d polynomials]

s tx

1

r • c(r)

y • c(y) + (r-y) • c(r)

35

Main Theorem (Formally)

Theorem: [Roughgarden 02, Correa/Schulz/Stier Moses 03]: For every set S, for every selfish routing network G with cost functions in C, the POA in G is at most α(S).– POA always maximized by Pigou-like examples

That is, if f and f* are Nash + optimal flows in G, then C(f)/C(f*) ≤ α(S).

– example: POA ≤ 4/3 if G has affine cost fns

36

Proof of General Thm

Let f and f* are Nash + optimal flows in G.

37

Proof of General Thm

Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound

with c = ce, y = f*, r = fe:

α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*

) • ce(fe)]eee

e

38

Proof of General Thm

Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound

with c = ce, y = f*, r = fe:

α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*

) • ce(fe)]

Step 2: rearrange and sum over e: C(f*) = e f*

• ce(f*)

eee

e e

e

39

Proof of General Thm

Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with

c = ce, y = f*, r = fe:

α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*

) • ce(fe)]

Step 2: rearrange and sum over e: C(f*) = e f*

• ce(f*) [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)]

eee

e e

e

e

40

Proof of General Thm

Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with c

= ce, y = f*, r = fe:

α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*

) • ce(fe)]

Step 2: rearrange and sum over e: C(f*) = e f*

• ce(f*) [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)]

Step 3: apply VI

eee

e

0

e e

e

41

Proof of General Thm

Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with

c = ce, y = f*, r = fe:

α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*

) • ce(fe)]

Step 2: rearrange and sum over e: C(f*) = e f*

• ce(f*) [e fe • ce(fe)]/α(S)

Step 3: apply VI, done!

eee

=C(f)

e e

e

42

Recap

• selfish routing: simple, basic routing game– inefficient equilibria: Pigou + Braess examples

• price of anarchy: ratio of objective fn values of selfish + optimal outcomes

• potential functions: equilibria actually solving a related optimization problem– immediate consequence for existence,

uniqueness, and inefficiency of equilibria

43

Recap

• variational inequality: inequality based on "first-order condition" satisfied by equilibria

• Pigou bound: given a set of cost functions, largest POA in a Pigou-like example

• main result: for every set of cost fns, Pigou bound is tight (all multicommodity networks)– POA depends only on complexity of cost functions,

not on complexity of network structure

44

Outline

Part I: The Price of Anarchy in Selfish Routing Games

Part II: The Price of Stability in Network Connectivity Games

45

Selfish Network Design

Given: G = (V,E), fixed costs ce for all e є E,

k vertex pairs (si,ti)

Each player wants to build a network in which its nodes are connected.

Player strategy: select a path connecting si to ti.

• [Anshelevich et al 04]

46

Shapley Cost Sharing

How should multiple players on a single edge split costs?

Natural choice is fair sharing, or Shapley cost sharing:

Players using e pay for it evenly: ci(P) = Σ ce/ke

Each player tries to minimize its cost.

e є P

47

Comparison to Selfish Routing

Note: like selfish routing, except:• finite number of outcomes

– in selfish routing, outcomes = fractional flows

• positive (not negative) externalities– cost function (per player) = ce/ke

Objective: C = Σi ci(Pi) = Σ ce

• where S = union of Pi's

e є S

48

What's the POA?

Example:

t

s

1+ k

t1, t2, … tk

s1, s2, … sk

49

What's the POA?

Example:

t

s

1+ k

t1, t2, … tk

s1, s2, … sk

t

s

1+ k

OPT(also Nash eq)

50

What's the POA?

Example:

t

s

1+ k

t1, t2, … tk

s1, s2, … sk

t

s

1+ k

OPT(also Nash eq)

t

s

1+ k

anotherNash eq

51

Multiple Equilibria

Moral: in Shapley network design games, different Nash eq can have different costs.

Recall:

Note: not well defined if Nash eq not unique.• which one do we look at?

POA of a game

=obj fn value of selfish outcomeoptimal obj fn value

52

The Price of Stability

General definition of POA: [KP99]

• POA = k in last example, uninteresting

Price of Anarchy = cost(worst NE)

cost(OPT)

53

The Price of Stability

General definition of POA: [KP99]

• POA = k in last example, uninteresting

Alternative:

• POS = 1 in last example

Price of Anarchy = cost(worst NE)

cost(OPT)

Price of Stability = cost(best NE)

cost(OPT)

54

The Price of Stability

Note: small price of stability only guarantees that some Nash eq has low cost.

• much weaker guarantee than small POA

Interpretation: best solution consistent with self-interested players

• natural outcome for centralized planner to suggest [e.g., network protocol designer]

55

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

56

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

cost(OPT) = 1+ε

57

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

cost(OPT) = 1+ε

…but not a NE:

player k

pays (1+ε)/k,

could pay 1/k

58

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

so player k

would deviate

59

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

now player k-1

pays (1+ε)/(k-1),

could pay 1/(k-1)

60

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

so player k-1

deviates too

61

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

Continuing this process, all players defect.

This is a NE!

(the only Nash)

cost = 1 + + … +

Price of Stability is Hk = Θ(log k)!

1 12 k

62

The Price of Stability of Selfish Network Design

Thus: the price of stability of selfish network design can be as high as ln k. [k = # players]

Our goals: in all such games,• there is at least one pure-strategy Nash eq• one of them has cost ≤ ln k • OPT

– i.e. price of stability always ≤ ln k– [Anshelevich et al 04]

Technique: potential function method.

63

Potential Functions

Recall: potential function Փ of a game = function optimized by selfish players– not necessarily a natural objective function

Defn: Փ (fn from outcomes to reals) is a potential function if for all outcomes S, players i, and deviations by i from S:

ΔՓ = Δci

64

Potential Functions

So: potential fn tracks deviations by players

Thus: equilibria of game = local optima of Փ• so finite potential games have pure-strategy

Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]– precursors: [Rosenthal 73], [Beckmann et al 56]

65

Potential Functions

So: potential fn tracks deviations by players

Thus: equilibria of game = local optima of Փ• so finite potential games have pure-strategy

Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]– precursors: [Rosenthal 73], [Beckmann et al 56]

Claim: every Shapley network design game has a potential function.

66

Proof of Potential Function

Define Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]

where ke is # players using e in S. Hk

Let Ф(S) = Σ Фe(S)

Consider some solution S (a path for each player).

Suppose player i is unhappy and decides to deviate.

What happens to Ф(S)?

e є S

e

67

Proof of Potential Function

Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]

Suppose player i’s new path includes e.i pays ce/(ke+1) to use e.

Фe(S) increases by the same amount.

If player i leaves an edge e ’, Фe ’(S) exactly reflects the

change in i’s payment.

e

e’

ce[1+ 1/2 +… +1/ke]

ce’[1+ 1/2 +… +1/ke’]

i

68

Proof of Potential Function

e

e’

ce[1+ 1/2 +… +1/ke]+ce/(ke+1)

ce’[1+ 1/2 +… +1/ke’] -ce’/ke’

i

Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]

Suppose player i’s new path includes e.i pays ce/(ke+1) to use e.

Фe(S) increases by the same amount.

If player i leaves an edge e ’, Фe ’(S) exactly reflects the

change in i’s payment.

69

Bound on Price of StabilityCompare cost + potential function:

C(S) = e ce

PF(S) = e ce[1+ 1/2 + 1/3 + … 1/ke]

• cost, potential fn differ by factor of ≤ Hk

• gives upper bound of Hk on price on stability– let S = min-potential soln [note: also a Nash eq]– let S* = opt solution

C(S) ≤ PF(S) ≤ PF(S*) ≤ Hk • C(S*)

70

Undirected Networks

Open Question: what is the POS in undirected graphs?

• best known lower bound = 12/7• [Fiat et al 06]: O(log log k) for special case

1 1k12 13

= =

t

0 0 0 0

1+ . . .0

1k-1

71

Shapley Cost-Sharing

Summary: with Shapley cost sharing,• POA = k, even in undirected graphs

• POS = Hk in directed graphs– (unknown in undirected graphs)

Question #1: can we do better?

Question #2: subject to what?

72

In Defense of Shapley

Essential properties: (non-negotiable)• "budget-balanced" (total cost shares = cost)• "local" (cost shares computed edge-by-edge)• pure-strategy Nash equilibria exist

Bonus good properties: (negotiable)• "uniform" (same definition for all networks)• "fair" (characterizes Shapley)

73

Other Cost Shares?

Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.– Shapley justified on efficiency grounds!– non-uniform schemes not well understood

74

Other Cost Shares?

Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.– Shapley justified on efficiency grounds!– non-uniform schemes not well understood

Theorem: [Chen/Roughgarden/Valiant 07] Can do much better in undirected graphs.– can get POA = O(log2 k)– better for special cases or non-uniform protocols

75

Wrap-Up

• network games arise in many CS applications

• price of anarchy/stability/etc a flexible tool to measure inefficiency of selfish behavior– future direction: inform protocol design

• potential functions are an easy-to-use, versatile techniques to bound POA/POS

• many open questions...– looking forward to future theorems from you!