1

Robust Price of Anarchy Bounds via

Smoothness Arguments

Tim RoughgardenStanford University

2

Price of Anarchy

Definition: price of anarchy (POA) of a game (w.r.t. some objective function, eq concept):

optimal obj fn value

equilibrium objective fn value

the closer to 1 the better

3

Price of Anarchy

Definition: price of anarchy (POA) of a game (w.r.t. some objective function, eq concept):

optimal obj fn value

equilibrium objective fn value

the closer to 1 the better

s t

2x 12

5x5

cost = 14+10 = 24

cost = 14+14 = 28

s t

2x 12

5x5

00

4

The Need for Robustness

Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal.

5

The Need for Robustness

Meaning of a POA bound: if the game is at an equilibrium, then outcome is near-optimal.

Problem: what if can’t reach equilibrium?• (pure) equilibrium might not exist• might be hard to compute, even

centrally– [Fabrikant/Papadimitriou/Talwar], [Daskalakis/

Goldbeg/Papadimitriou], [Chen/Deng/Teng], etc.

• might be hard to learn in distributed way

Worry: are our POA bounds “meaningless”?

6

Fix #1: Provable Convergence

One Good Research Agenda: prove that “natural learning dynamics” converge (quickly) to equilibrium.

• [Hart/Mas-Collell], [Even-Dar/Kesselman/Mansour], [Fisher/ Racke/Voecking], [Chien/Sinclair], [Awerbuch et al], [Even-Dar/Mansour/Nadav], [Kleinberg/Piliouras/Tardos], etc.

Problem: this “best-case scenario” has limited reach

• non-existence/complexity issues• lower bounds on convergence time of natural

dynamics (e.g., [Skopalik/Voecking STOC 08])

7

Fix #2: Robust POA Bounds

High-Level Goal: worst-case bounds that apply even to non-equilibrium outcomes!

• best-response dynamics, pre-convergence– [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05],

[Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08]

• correlated equilibria– [Christodoulou/Koutsoupias 05]

• coarse correlated equilibria aka “price of total anarchy” aka “no-regret players”– [Blum/Even-Dar/Ligett 06],

[Blum/Hajiaghayi/Ligett/Roth 08]

8

Abstract Setup

• n players, each picks a strategy si

• player i incurs a cost Ci(s)

Important Assumption: objective function is cost(s) := i Ci(s)

Key Definition: A game is (λ,μ)-smooth if, for every pair s,s* outcomes (λ > 0; μ < 1):

i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)

[(*)]

9

Smooth => POA Bound

Next: “canonical” way to upper bound POA (via a smoothness argument).

• notation: s = a Nash eq; s* = optimal

Assuming (λ,μ)-smooth:

cost(s) = i Ci(s) [defn of cost]

≤ i Ci(s*i,s-i) [s a Nash

eq] ≤ λ●cost(s*) + μ●cost(s)

[(*)]

Then: POA (of pure Nash eq) ≤ λ/(1-μ).

10

Why Is Smoothness Stronger?

Key point: to derive POA bound, only needed

i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)

[(*)]

to hold in special case where s = a Nash eq and s* = optimal.

Smoothness: requires (*) for every pair s,s* outcomes.– even if s is not a pure Nash equilibrium

11

Some Smoothness Bounds• atomic (unweighted) selfish routing

[Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05], [Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09]

• nonatomic selfish routing [Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05]

• weighted congestion games [Aland/Dumrauf/Gairing/Monien/Schoppmann 06],

[Bhawalkar/Gairing/Roughgarden 10]

• submodular maximization games [Vetta 02], [Marden/Roughgarden 10]

• coordination mechanisms [Cole/Gkatzelis/Mirrokni 10]

Application: Out-of-Equilibria

Definition: a sequence s1,s2,...,sT of outcomes is no-regret if:

• for each player i, each fixed action qi:– average cost player i incurs over sequence

no worse than playing action qi every time– simple hedging strategies can be used by

players to enforce this (for suff large T)

Theorem: [Roughgarden STOC 09] in a (λ,μ)-smooth game, average cost of every no-regret sequence at most [λ/(1-μ)] x cost of optimal outcome. 12

13

Smooth => POTA Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

14

Smooth => POTA Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

= t i [Ci(s*i,st

-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st

-

i)]

15

Smooth => POTA Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

= t i [Ci(s*i,st

-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st

-

i)]

≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]

16

Smooth => POTA Bound

• notation: s1,s2,...,sT = no regret; s* = optimal

Assuming (λ,μ)-smooth:

t cost(st) = t i Ci(st) [defn of cost]

= t i [Ci(s*i,st

-i) + ∆i,t] [∆i,t:= Ci(st)- Ci(s*i,st

-

i)]

≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]

No regret: t ∆i,t ≤ 0 for each i.

To finish proof: divide through by T.

17

Why Important?

pureNash

mixed Nash

correlated eq

no regret• bound on no-regret

sequences implies bound on inefficiency of mixed and correlated equilibria

• bound applies even to sequences that don’t converge in any sense• no regret much weaker than reaching

equilibrium• e.g., if every player uses “multiplicative

weights” then get o(1) regret in poly-time

Tight Game Classes

Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight:

maximum [pure POA] = minimum [λ/(1-μ)]congestion games

w/cost functions in C(λ ,μ): all such gamesare (λ ,μ)-smooth

18

Tight Game Classes

Theorem: [Roughgarden STOC 09] for every set C, unweighted congestion games with cost functions restricted to C are tight:

maximum [pure POA] = minimum [λ/(1-μ)]

• weighted congestion games [Bhawalkar/ Gairing/Roughgarden ESA 10] and submodular maximization games [Marden/Roughgarden CDC 10] are also tight in this sense

congestion gamesw/cost functions in C

(λ ,μ): all such gamesare (λ ,μ)-smooth

19

20

Further Applications

pureNash

mixed Nash

correlated eq

no regret

best-responsedynamics

approximate Nash

Theorem: in a (λ,μ)-smooth game, everything in these sets costs (essentially) λ/(1-μ) x OPT.

21

More Applications?

Theorem: [Nadav/Roughgarden 10] Consider a (λ,μ)-smooth game for optimal choices of λ,μ. Then precisely the “aggregate” coarse correlated equilibria have cost ≤ λ/(1-μ) x OPT.• essentially, bound holds if and only if the

average (rather than per-player) regret is non-positive

Proof: convex duality.

When Is a POA Bound a Smoothness Proof?

Need to show: for every pair s,s* outcomes:

i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)

Generally sufficient: prove POA bound for pure Nash equilibria such that:

• invoke best-response condition once/player

• hypothetical deviation by i independent of s-i

Non-example: network creation games [Fabrikant et al], [Albers et al], [Demaine et al], etc.

22

Application: POA of Non-Truthful Mechanisms

Mechanism Design: design protocol with desirable outcome despite selfish participants

• example: Vickrey (second-price) auction• focus thus far on "truthful" mechanisms

Non-truthful mechanisms: motivated by• simplicity (e.g., sponsored search

auctions)• low communication complexity• low computational complexity 23

Application: POA of Non-Truthful Mechanisms

Fact: plausible outcomes of non-truthful mechanisms = Bayes-Nash equilibria

• POA results for simple non-truthful auctions in [Christodoulou/Kovacs/Schapira ICALP 08] and [Borodin/Lucier SODA 10]

• first bound POA of Nash equilibria, then "by magic" same bound holds for Bayes-Nash

Fact: these are automatic consequences of smoothness. 24

POA of Non-Truthful Mechanisms (continued)

[Paes Leme/Tardos FOCS 10]• POA upper bounds for welfare of

Generalized Sponsored Search auction• different bounds for pure (1.618), mixed

(4), and Bayes-Nash eq (8) (without smoothness)

Open Questions: • compute best smoothness bound• prove a lower bound separating the best-

possible pure vs. Bayes-Nash POA 25

POA of Non-Truthful Mechanisms (continued)

[Bhawalkar/Roughgarden 10]• POA upper bounds for welfare of

subadditive combinatorial auctions with “item bidding”

– generalizes [Christodoulou/Kovacs/Schapira 08]

• pure Nash: non-smooth upper bound of 2• Bayes-Nash: lower bound of 2.01, upper

bound of 2 ln m [m = # goods]

Open Questions: • is Bayes-Nash POA = O(1)?

26

Local Smoothness and Splittable Congestion

GamesLocal smoothness: require smoothness

condition only for outcomes that are “close”

– assume strategy sets = convex subset of Rn

• can only decrease optimal value of λ/(1-μ) • [Harks 08]: local smoothness gives

improved POA bounds for splittable congestion games

• [Roughgarden/Schoppmann 10]: matching lower bounds => first tight bounds in this model

• [RS10]: local smoothness bounds extend to correlated eq but not to no-regret outcomes!

27

Splittable Congestion Games: Open Questions

Open Question #1: determine optimal POA bounds for no-regret sequences.

Open Question #2: is (the distribution of) every no-regret sequence a convex combination of pure Nash equilibria?

Open Question #3: prove something non-trivial about the POA of symmetric splittable congestion games.

28