Potential Functions and the Inefficiency of Equilibria Tim Roughgarden Stanford University.

Potential Functions and the Inefficiency of Equilibria

Tim RoughgardenStanford University

2

Pigou's Example

Example: one unit of traffic wants to go from s to t

Question: what will selfish network users do?• assume everyone wants smallest-possible cost• [Pigou 1920]

s t

c(x)=x

c(x)=1

cost depends on congestion

no congestion effects

3

Motivating Example

Claim: all traffic will take the top link.

Reason:• Є > 0 traffic on bottom is envious• Є = 0 equilibrium

– all traffic incurs one unit of cost

s t

c(x)=x

c(x)=1

Flow = 1-Є

Flow = Єthis flow is envious!

4

Can We Do Better?

Consider instead: traffic split equally

Improvement:• half of traffic has cost 1 (same as before)• half of traffic has cost ½ (much improved!)

s t

c(x)=x

c(x)=1

Flow = ½

Flow = ½

5

Braess’s Paradox

Initial Network:

s tx 1

½

x1½

½

½

Cost = 1.5

6

Braess’s Paradox

Initial Network: Augmented Network:

s tx 1

½

x1½

½

½

Cost = 1.5

s tx 1

½

x1½

½

½0

Now what?

7

Braess’s Paradox


s tx 1

½

x1½

½

½

Cost = 1.5 Cost = 2

s t

x 1

x10

8

Braess’s Paradox


All traffic incurs more cost! [Braess 68]

• also has physical analogs [Cohen/Horowitz 91]

s tx 1

½

x1½

½

½

Cost = 1.5 Cost = 2

s t

x 1

x10

9

High-Level Overview

Motivation: equilibria of noncooperative network games typically inefficient

• e.g., Pigou's example + Braess's Paradox• don't optimize natural objective functions

Price of anarchy: quantify inefficiency w.r.t some objective function

Our goal: when is the price of anarchy small?– when does competition approximate cooperation?– benefit of centralized control is small

10

Selfish Routing Games

• directed graph G = (V,E)

• source-destination pairs (s1,t1), …, (sk,tk)

• ri = amount of traffic going from si to ti

• for each edge e, a cost function ce(•)– assumed continuous and nondecreasing

Examples: (r,k=1)

s1 t1

c(x)=x c(x)=1

c(x)=xc(x)=1

½

½s1 t1

c(x)=x

c(x)=1

½

½c(x)=0

11

Outcomes = Network Flows

Possible outcomes of a selfish routing game:

• fP = amount of traffic choosing si-ti path P

• outcomes of game flow vectors f– flow vector: nonnegative and total flow fP

on si-ti paths equals traffic rate ri (for all i)s t

12

Outcomes = Network Flows

Possible outcomes of a selfish routing game:

• fP = amount of traffic choosing si-ti path P

• outcomes of game flow vectors f– flow vector: nonnegative and total flow fP on si-ti

paths equals traffic rate ri (for all i)

Question: What are the equilibria (natural selfish outcomes) of this game?

s t

13

Nash Flows

Def: [Wardrop 52] A flow is at Nash equilibrium (or is a Nash flow) if no one can switch to a path of smaller cost. I.e., all flow is routed on min-cost paths. [given current edge congestion]

xs t

1s t

1

x

Examples:½

½

1

s tx 1

x10 1s t

x 1

x10

½

½

14

Our Objective Function

Definition of social cost: total cost C(f) incurred by the traffic in a flow f.

Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:

C(f) = P fP • cP(f)

s t

15

Our Objective Function

Definition of social cost: total cost C(f) incurred by the traffic in a flow f.

Formally: if cP(f) = sum of costs of edges of P (w.r.t. flow f), then:

C(f) = P fP • cP(f)

Example:

s t

s tx

1½½

Cost = ½•½ +½•1 = ¾

16

The Price of Anarchy

Defn:

– definition from [Koutsoupias/Papadimitriou 99]

price ofanarchy of a game

=obj fn value of selfish outcome

optimal obj fn value

xs t

1s t

1

x

Example: POA = 4/3 in Pigou's example½

½

1

Cost = 1 Cost = 3/4

17

A Nonlinear Pigou Network

Bad Example: (d large)

equilibrium has cost 1, min cost 0

s t

xd

10

1 1-Є

Є

18

A Nonlinear Pigou Network

Bad Example: (d large)

equilibrium has cost 1, min cost 0

price of anarchy unbounded as d -> infinity

Goal: weakest-possible conditions under which P.O.A. is small.

s t

xd

10

1 1-Є

Є

19

When Is the Price of Anarchy Bounded?

Examples so far:

Hope: imposing additional structure on the cost functions helps– worry: bad things happen in larger networks

s tx

1s t

xd

1s t

x 1

x10

20

Polynomial Cost Functions

Def: linear cost fn is of form ce(x)=aex+be

Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:

≤ 4/3 × cost of Nash flow

cost of opt flow

s tx

1

21

Polynomial Cost Functions

Def: linear cost fn is of form ce(x)=aex+be

Theorem: [Roughgarden/Tardos 00] for every network with linear cost functions:

≤ 4/3 ×

Bounded-deg polys: (w/nonneg coeffs) replace 4/3 by Θ(d/log d)

cost of Nash flow

cost of opt flow

s txd

1

tightexample

s tx

1

22

A General Theorem

Thm: [Roughgarden 02], [Correa/Schulz/Stier

Moses 03] fix any set of cost fns. Then, a Pigou-like example 2 nodes, 2 links, 1 link w/constant cost fn) achieves worst POA

s txd

1

tightexample

23

Interpretation

Bad news: inefficiency of selfish routing grows as cost functions become "more nonlinear".– think of "nonlinear" as "heavily congested"– recall nonlinear Pigou's example

Good news: inefficiency does not grow with network size or # of source-destination pairs.– in lightly loaded networks, no matter how

large, selfish routing is nearly optimals txd

1

tightexample

24

Benefit of Overprovisioning

Suppose: network is overprovisioned by β > 0 (β fraction of each edge unused).

Then: Price of anarchy is at most ½(1+1/√β).

• arbitrary network size/topology, traffic matrix

Moral: Even modest (10%) over-provisioning sufficient for near-optimal routing.

25

Potential Functions

• potential games: equilibria are actually optima of a related optimization problem

– has immediate consequences for existence, uniqueness, and inefficiency of equilibria

– see [Beckmann/McGuire/Winsten 56], [Rosenthal 73], [Monderer/Shapley 96], for original references

– see [Roughgarden ICM 06] for survey

26

The Potential Function

Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f

ce(x)dx (over all flows).

ce(fe)

00 fe

0e

27

The Potential Function

Key fact: [BMV 56] Nash flows minimize “potential function” e ∫f

ce(x)dx (over all flows).

Lemma 1: locally optimal solutions are precisely the Nash flows (derivative test).

Lemma 2: all locally optimal solutions are also globally optimal (convexity).

Corollary: Nash flows exist, are unique.

ce(fe)

00 fe

0e

28

Consequences for the Price of Anarchy

Example: linear cost functions.

Compare cost + potential function:

C(f) = e fe • ce(fe) = e [ae fe + be fe]

PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe]

2

0e

2

29

Consequences for the Price of Anarchy

Example: linear cost functions.

Compare cost + potential function:

C(f) = e fe • ce(fe) = e [ae fe + be fe]

PF(f) = e ∫f ce(x)dx = e [(ae fe)/2 + be fe]

• cost, potential fn differ by factor of ≤ 2• gives upper bound of 2 on price on anarchy

– C(f) ≤ 2×PF(f) ≤ 2×PF(f*) ≤ 2×C(f*)

2

0e

2

30

Better Bounds?

Similarly: proves bound of d+1 for degree-d polynomials (w/nonnegative coefficients).

• not tight, but qualitatively accurate – e.g., price of anarchy goes to infinity with

degree bound, but only linearly

• to get tight bounds, need "variational inequalities"– see my ICM survey for details

31

Variational Inequality

Claim: • if f is a Nash flow and f* is feasible, then

e fe • ce(fe) ≤ e f* • ce(fe)

• proof: use that Nash flow routes flow on shortest paths (w.r.t. costs ce(fe))

e

32

Pigou Bound

Recall goal: want to show Pigou-like examples are always worst cases.

Pigou bound: given set of cost functions (e.g., degree-d polys), largest POA in a network:

• two nodes, two links• one function in given set• one constant function

– constant = cost of fully congested top edge

s txd

1

33

Pigou Bound (Formally)

Let S = a set of cost functions.– e.g., polynomials with degree at most d,

nonnegative coefficients

Definition: the Pigou bound α(S) for S is:

max

• max is over all choices of cost fns c in S, traffic rate r 0, flow y 0

s txd

1

r • c(r)

y • c(y) + (r-y) • c(r)

34

Pigou Bound (Example)

Let S = { c : c(x) = ax +b } [linear functions]

Recall: the Pigou bound α(S) for S is:

max

• max is over all choices of cost fns c in S, traffic rate r 0, flow y 0

• choose c(x) = x; r = 1; y = 1/2 get 4/3• calculus: α(S) = 4/3 [d/ln d for deg-d polynomials]

s tx

1

r • c(r)

y • c(y) + (r-y) • c(r)

35

Main Theorem (Formally)

Theorem: [Roughgarden 02, Correa/Schulz/Stier Moses 03]: For every set S, for every selfish routing network G with cost functions in C, the POA in G is at most α(S).– POA always maximized by Pigou-like examples

That is, if f and f* are Nash + optimal flows in G, then C(f)/C(f*) ≤ α(S).

– example: POA ≤ 4/3 if G has affine cost fns

36

Proof of General Thm

Let f and f* are Nash + optimal flows in G.

37


Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound

with c = ce, y = f*, r = fe:

α(S) fe • ce(fe)/[f* • ce(f*) + (fe -f*

) • ce(fe)]eee

e

38


Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound

with c = ce, y = f*, r = fe:


) • ce(fe)]

Step 2: rearrange and sum over e: C(f*) = e f*

• ce(f*)

eee

e e

e

39


Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with

c = ce, y = f*, r = fe:


) • ce(fe)]


• ce(f*) [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)]

eee

e e

e

e

40


Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with c

= ce, y = f*, r = fe:


) • ce(fe)]


• ce(f*) [e fe • ce(fe)]/α(S) + [e (f* - fe) • ce(fe)]

Step 3: apply VI

eee

e

0

e e

e

41


Let f and f* are Nash + optimal flows in G.Step 1: for each e, invoke Pigou bound with

c = ce, y = f*, r = fe:


) • ce(fe)]


• ce(f*) [e fe • ce(fe)]/α(S)

Step 3: apply VI, done!

eee

=C(f)

e e

e

42

Recap

• selfish routing: simple, basic routing game– inefficient equilibria: Pigou + Braess examples

• price of anarchy: ratio of objective fn values of selfish + optimal outcomes

• potential functions: equilibria actually solving a related optimization problem– immediate consequence for existence,

uniqueness, and inefficiency of equilibria

43

Recap

• variational inequality: inequality based on "first-order condition" satisfied by equilibria

• Pigou bound: given a set of cost functions, largest POA in a Pigou-like example

• main result: for every set of cost fns, Pigou bound is tight (all multicommodity networks)– POA depends only on complexity of cost functions,

not on complexity of network structure

44

Outline

Part I: The Price of Anarchy in Selfish Routing Games

Part II: The Price of Stability in Network Connectivity Games

45

Selfish Network Design

Given: G = (V,E), fixed costs ce for all e є E,

k vertex pairs (si,ti)

Each player wants to build a network in which its nodes are connected.

Player strategy: select a path connecting si to ti.

• [Anshelevich et al 04]

46

Shapley Cost Sharing

How should multiple players on a single edge split costs?

Natural choice is fair sharing, or Shapley cost sharing:

Players using e pay for it evenly: ci(P) = Σ ce/ke

Each player tries to minimize its cost.

e є P

47

Comparison to Selfish Routing

Note: like selfish routing, except:• finite number of outcomes

– in selfish routing, outcomes = fractional flows

• positive (not negative) externalities– cost function (per player) = ce/ke

Objective: C = Σi ci(Pi) = Σ ce

• where S = union of Pi's

e є S

48

What's the POA?

Example:

t

s

1+ k

t1, t2, … tk

s1, s2, … sk

49

What's the POA?

Example:

t

s

1+ k

t1, t2, … tk

s1, s2, … sk

t

s

1+ k

OPT(also Nash eq)

50

What's the POA?

Example:

t

s

1+ k

t1, t2, … tk

s1, s2, … sk

t

s

1+ k

OPT(also Nash eq)

t

s

1+ k

anotherNash eq

51

Multiple Equilibria

Moral: in Shapley network design games, different Nash eq can have different costs.

Recall:

Note: not well defined if Nash eq not unique.• which one do we look at?

POA of a game

=obj fn value of selfish outcomeoptimal obj fn value

52

The Price of Stability

General definition of POA: [KP99]

• POA = k in last example, uninteresting

Price of Anarchy = cost(worst NE)

cost(OPT)

53


General definition of POA: [KP99]

• POA = k in last example, uninteresting

Alternative:

• POS = 1 in last example

Price of Anarchy = cost(worst NE)

cost(OPT)

Price of Stability = cost(best NE)

cost(OPT)

54


Note: small price of stability only guarantees that some Nash eq has low cost.

• much weaker guarantee than small POA

Interpretation: best solution consistent with self-interested players

• natural outcome for centralized planner to suggest [e.g., network protocol designer]

55

Example: High Price of Stability

1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

56


1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

cost(OPT) = 1+ε

57


1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

cost(OPT) = 1+ε

…but not a NE:

player k

pays (1+ε)/k,

could pay 1/k

58


1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

so player k

would deviate

59


1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

now player k-1

pays (1+ε)/(k-1),

could pay 1/(k-1)

60


1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

so player k-1

deviates too

61


1 1k

12

13

1 2 3 k

t

0 0 0 0

1+ . . . k-1

0

1k-1

Continuing this process, all players defect.

This is a NE!

(the only Nash)

cost = 1 + + … +

Price of Stability is Hk = Θ(log k)!

1 12 k

62

The Price of Stability of Selfish Network Design

Thus: the price of stability of selfish network design can be as high as ln k. [k = # players]

Our goals: in all such games,• there is at least one pure-strategy Nash eq• one of them has cost ≤ ln k • OPT

– i.e. price of stability always ≤ ln k– [Anshelevich et al 04]

Technique: potential function method.

63

Potential Functions

Recall: potential function Փ of a game = function optimized by selfish players– not necessarily a natural objective function

Defn: Փ (fn from outcomes to reals) is a potential function if for all outcomes S, players i, and deviations by i from S:

ΔՓ = Δci

64

Potential Functions

So: potential fn tracks deviations by players

Thus: equilibria of game = local optima of Փ• so finite potential games have pure-strategy

Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]– precursors: [Rosenthal 73], [Beckmann et al 56]

65

Potential Functions

So: potential fn tracks deviations by players

Thus: equilibria of game = local optima of Փ• so finite potential games have pure-strategy

Nash equilibria (proof: just do "best-response dynamics") [Monderer/Shapley 96]– precursors: [Rosenthal 73], [Beckmann et al 56]

Claim: every Shapley network design game has a potential function.

66

Proof of Potential Function

Define Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]

where ke is # players using e in S. Hk

Let Ф(S) = Σ Фe(S)

Consider some solution S (a path for each player).

Suppose player i is unhappy and decides to deviate.

What happens to Ф(S)?

e є S

e

67


Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]

Suppose player i’s new path includes e.i pays ce/(ke+1) to use e.

Фe(S) increases by the same amount.

If player i leaves an edge e ’, Фe ’(S) exactly reflects the

change in i’s payment.

e

e’

ce[1+ 1/2 +… +1/ke]

ce’[1+ 1/2 +… +1/ke’]

i

68


e

e’

ce[1+ 1/2 +… +1/ke]+ce/(ke+1)

ce’[1+ 1/2 +… +1/ke’] -ce’/ke’

i

Фe(S) = ce[1+ 1/2 + 1/3 + … 1/ke]

Suppose player i’s new path includes e.i pays ce/(ke+1) to use e.

Фe(S) increases by the same amount.

If player i leaves an edge e ’, Фe ’(S) exactly reflects the

change in i’s payment.

69

Bound on Price of StabilityCompare cost + potential function:

C(S) = e ce

PF(S) = e ce[1+ 1/2 + 1/3 + … 1/ke]

• cost, potential fn differ by factor of ≤ Hk

• gives upper bound of Hk on price on stability– let S = min-potential soln [note: also a Nash eq]– let S* = opt solution

C(S) ≤ PF(S) ≤ PF(S*) ≤ Hk • C(S*)

70

Undirected Networks

Open Question: what is the POS in undirected graphs?

• best known lower bound = 12/7• [Fiat et al 06]: O(log log k) for special case

1 1k12 13

= =

t

0 0 0 0

1+ . . .0

1k-1

71

Shapley Cost-Sharing

Summary: with Shapley cost sharing,• POA = k, even in undirected graphs

• POS = Hk in directed graphs– (unknown in undirected graphs)

Question #1: can we do better?

Question #2: subject to what?

72

In Defense of Shapley

Essential properties: (non-negotiable)• "budget-balanced" (total cost shares = cost)• "local" (cost shares computed edge-by-edge)• pure-strategy Nash equilibria exist

Bonus good properties: (negotiable)• "uniform" (same definition for all networks)• "fair" (characterizes Shapley)

73

Other Cost Shares?

Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.– Shapley justified on efficiency grounds!– non-uniform schemes not well understood

74

Other Cost Shares?

Theorem: [Chen/Roughgarden/Valiant 07] Shapley minimizes POS among all uniform protocols in directed graphs.– Shapley justified on efficiency grounds!– non-uniform schemes not well understood

Theorem: [Chen/Roughgarden/Valiant 07] Can do much better in undirected graphs.– can get POA = O(log2 k)– better for special cases or non-uniform protocols

75

Wrap-Up

• network games arise in many CS applications

• price of anarchy/stability/etc a flexible tool to measure inefficiency of selfish behavior– future direction: inform protocol design

• potential functions are an easy-to-use, versatile techniques to bound POA/POS

• many open questions...– looking forward to future theorems from you!

Potential Functions and the Inefficiency of Equilibria Tim Roughgarden Stanford University.

Documents

Transcript of Potential Functions and the Inefficiency of Equilibria Tim Roughgarden Stanford University.