Game Theory: Strategic games and equilibria concepts

TAAC Oct-2010

Summary

2 General setting

2 Strategic Games and strategies

2 Best response

2 Pure Nash Equilibria

2 Mixed Nash Equilibria

Basic Reference

2 Osborne. An Introduction to Game Theory, Oxford University Press, 2004

2 Ad-hoc references.

2 Nisan, Roughgarden, Tardos, Vazirani, Eds. Algorithmic game theory,

Cambridge Univ. Press, 2007

What is Game Theory?

Game theory is often described as a branch of applied mathematics and

economics that studies situations where players choose different actions in

an attempt to maximize their returns.

The essential feature, however, is that it provides a formal modelling

approach to social situations in which decision makers interact with other

minds.

Game theory extends the simpler optimization approach developed in

neoclassical economics.

Where to use game theory?

Game theory studies decisions made in an environment in which players interact.

In other words, game theory studies choice of optimal behavior when costs and

benefits of each option depend upon the choices of other individuals.

What for?

Game theory looks for states of equilibrium sometimes called solutions

History

The field of game theory came into being with the 1944 classic Theory of Games

and Economic Behavior by John von Neumann and Oskar Morgenstern.

1713 The first known discussion of game theory occurred in a letter written by

James Waldegrave.

Provides a minimax mixed strategy solution to a two-person version of the

card game.

1838 Antoine Augustin Cournot’s Researches into the Mathematical Principles of

the Theory of Wealth.

Considers a duopoly and presents a solution that is a restricted version of the

Nash equilibrium.

1928 John von Neumann published a series of papers.

1944 The field of game theory came into being with the classic Theory of Games

and Economic Behavior by John von Neumann and Oskar Morgenstern.

During this initial time period, work focus on cooperative game theory.

Which analyzes optimal strategies for groups of individuals.

1950s the first discussion of the prisoner’s dilemma appeared, and an experiment

was undertaken on this game at the RAND corporation.

John Nash developed a definition of an ”optimum” strategy for multiplayer

games where no such optimum was previously defined, known as Nash

equilibrium.

Analysis of non-cooperative games in addition to cooperative ones.

Concepts: the extensive form game, fictitious play, repeated games, and the

Shapley value were developed.

Applications of Game theory to philosophy and political science.

1960s and 1970s Development of new concepts:

Subgame perfect equilibria. Complete information and Bayesian games.

Correlated equilibria, trembling hand perfection, common knowledge.

Evolutionary game theory.

Applications of Game theory to biology an many other fields.

90s 00s Popular acknowledgement.

John Nash and Reinhard Selten, won the Nobel Prize in 1994.

Thomas Schelling and Robert Aumann, won the Nobel Prize in 2005

In addition to its academic interest, game theory receives attention in popular

culture.

John Nash, was the subject of the 1998 biography by Sylvia Nasar and the

2001 film A Beautiful Mind.

Game theory was also a theme in the 1983 film WarGames.

Several game shows have adopted game theoretic situations, at the end of the

course you might recognize them!

What about Internet?

Christos Papadimitriou (STOC 2001)

“The internet is unique among all the computer systems in that it is build,

operated and used by multitude of diverse economic interests, in varing

relationships of collaboration and competition with each other. This suggest that

the mathematical tools and insights most appropriate for understanding the

Internet may come from the fusion of algorithmic ideas with concepts and

techniques from Mathematical Economics and Game Theory.”

http://www.cs.berkeley.edu/∼christos/games/cs294.html

Game Theory for CS?

2 Framework to analyze equilibrium states of protocols used by rational agents.

Price of anarchy/stability.

2 Tool to design protocols for internet with guarantees.

Mechanism design.

2 New concepts to analyze/justify behavior of on-line algorithms

Give guarantees of stability to dynamic network algorithms.

2 Source of new computational problems to study.

Algorithmic game theory

Non-cooperative games

2 Strategic games

2 Extensive games

2 Repeated games

2 Bayesian games

Complete information versus partial information

One shot?

Strategic game

A strategic game Γ (with ordinal preferences) consists of:

2 A finite set N = {1, . . . , n} of players.

2 For each player i ∈ N , a nonempty set of actions Ai.

2 Each player chooses his action once. Players choose actions simultaneously.

No player is informed, when he chooses his action, of the actions chosen by

others.

2 For each player i ∈ N , a preference relation (a complete, transitive, reflexive

binary relation) �i over the set A = A1 × · · · × An.

It is frequent to specify the players’ preferences by giving utility functions

ui(a1, . . . an). Also called pay-off functions.

Example: Prisoner’s Dilemma

The story

2 Two suspects in a major crime are held in separate cells.

2 Evidence to convict each of them of a minor crime.

2 No evidence to convict either of them of a major crime unless one of them

finks.

The penalties

2 If both stay quiet, be convicted for a minor offense (one year prison).

2 If only one finks, he will be will be freed (and used as a witness) and the other

will be convicted for a major offense (four years in prison).

2 If both fink, each one will be convicted for a major offense with a reward for

coperation (three years each).

The Prisoner’s Dilemma models a situation in which

2 there is a gain from cooperation,

2 but each player has an incentive to free ride.

Game representation

2 Players N = {Suspect 1, Suspect 2} = {1, 2}.

2 Actions A1 = A2 = {Quiet, Fink}.

2 Action profiles

A = A1 × A2 = {(Quiet, Quiet), (Quiet, Fink), (Fink, Quiet), (Fink, Fink)}

2 Preferences

⋆ Player 1

(Fink, Quiet) �1 (Quiet, Quiet) �1 (Fink, Fink) �1 (Quiet, Fink)

⋆ Player 2

(Quiet, Fink),�2 (Quiet, Quiet) �2 (Fink, Fink) �2 (Fink, Quiet)

2 Utilities

u1(Fink, Quiet) = 3, u1(Quiet, Quiet) = 2, u1(Fink, Fink) = 1, u1(Quiet, Fink) = 0

u2(Quiet, Fink) = 3, u2(Quiet, Quiet) = 2, u2(Fink, Fink) = 1, u2(Fink, Quiet) = 0

We can represent pay-offs in a compact way on a bi-matrix.

Suspect 1

Suspect 2

Quiet Fink

Quiet 2,2 0,3

Fink 3,0 1,1

Example: Matching Pennies

2 Two people choose, simultaneously, whether to show the head or tail of a coin.

2 If they both show the same side, person 2 pays 1 euro to person 1, otherwise

person 1 pays 1 euro to person 2.

2 Payoffs are equal to the amounts of money involved.

Person 1

Person 2

Head Tail

Head 1,-1 -1,1

Tail -1,1 1,-1

This is an example of a zero-sum game that form part of a more general class of

constant-sum games.

Example: Stag Hunt

Jean-Jacques Rousseau. Discourse on the origin and foundations of inequality

among men.

2 A goup of hunters who wish to catch a stag.

2 They will succed if they all remain attentive, but each is tempted to desert his

post and catch a hare.

Example: Stag Hunt

Jean-Jacques Rousseau. Discourse on the origin and foundations of inequality

among men.

2 A goup of hunters who wish to catch a stag.

2 They will succed if they all remain attentive, but each is tempted to desert his

post and catch a hare.

Stag Hare

Stag 2,2 0,1

Hare 1,0 1,1

Example: BoS, Bach or Stravinky

2 Two persons want to go out together.

2 Two concerts are available, Bach or Stravinsky.

2 One person prefers Bach, the other Stravinsky.

2 They agree that cooperation is better but disagree about the best outcome.

Example: BoS, Bach or Stravinky

2 Two persons want to go out together.

2 Two concerts are available, Bach or Stravinsky.

2 One person prefers Bach, the other Stravinsky.

2 They agree that cooperation is better but disagree about the best outcome.

Bach Stravinsky

Bach 2,1 0,0

Stravinsky 0,0 1,2

Example: BoS, Bach or Stravinky

2 Two persons want to go out together.

2 Two concerts are available, Bach or Stravinsky.

2 One person prefers Bach, the other Stravinsky.

2 They agree that cooperation is better but disagree about the best outcome.

Bach Stravinsky

Bach 2,1 0,0

Stravinsky 0,0 1,2

The game is also referred as a Battle of Sexes:

Husband

Wife

Fights Ballet

Fights 2,1 0,0

Ballet 0,0 1,2

Example: Sending from s to t

2 We have a graph G = (V, E) and two vertices s, t.

2 There is one player for each vertex v ∈ V , v 6= t.

2 The set of actions for player u is NG(u).

2 A strategy profile is a set of vertices (v1, . . . , vn−1).

2 Pay-offs are defined as follows:

player u gets 1 if the shortest path joining s to t in the digraph induced by

v1, . . . , vn−1 contains (u, vu), otherwise gets 0.

Players are selfish but the system can get a different reward/cost. For example

the cost of the shortest path.

Sending from s to t: example

s tb b

b

b

b

b b

Sending from s to t: strategies

s tb b

b

b

b

b b

Sending from s to t: pay-offs

s tb b

b

b

b

b b

Red nodes get pay-off 1, blue nodes get pay-off 0.

Strategies

A strategy of player i ∈ N in a strategic game is an action ai ∈ Ai.

A strategy profile s = (s1, . . . , sn) consists of a strategy for each player.

For each s = (s1, . . . sn) we denote by

s−i = (s1, . . . , si−1, si+1, . . . , sn)

(s−i, s′

i) = (s1, . . . , si−1, s′

i, si+1, . . . , sn)

Best response

Let Γ be a strategic game (N, (Ai)i∈N , (ui)i∈N ).

The set of best responses, for player i, to strategy si is

BR(s−i) = {ai ∈ Ai | ui(s−i, ai) = maxa′

i∈Ai

ui(s−i, a′

i)}

Those are the actions that give maximum pay-off provided the other players do not

change their strategies.

Best response: examples

Quiet Fink

Quiet 2,2 0,3

Fink 3,0 1,1

Bach Stravinsky

Bach 2,1 0,0

Stravinsky 0,0 1,2

Stag Hare

Stag 2,2 0,1

Hare 1,0 1,1

Head Tail

Head 1,-1 -1,1

Tail -1,1 1,-1

What is the best response of player 1 to

game strategy profile best response

PriDil (F,F)

BoS (B,S)

Stag Hunt (H,H)

Mat Penn (H,T)

Pure Nash equilibrium

A pure Nash equilibrium is a strategy profile a∗ = (a∗

1, . . . , a∗

n) such that

no player i can be better off choosing an action different from a∗

i ,

given that every other player j adheres to a∗

j

for every player i and for every action ai ∈ Ai it holds

ui(a∗

−i, a∗

i ) > ui(a∗

−i, ai).

Equivalently, for every player i and for every action ai ∈ Ai it holds

a∗

i ∈ BR(a∗

−i).

Pure Nash Equilibrium

2 Is a strategy profile in which all players are happy.

2 Identified with a fixed point of an iterative process of computing a best

response.

2 However, the game is played only once!

2 GT deals with the existence and analysis of equilibria assuming rational

behavior.

players try to maximize their benefit

2 GT does not provide algorithmic tools for computing such equilibrium if one

exists.

Pure Nash equilibria, examples

Quiet Fink

Quiet 2,2 0,3

Fink 3,0 1,1

Bach Stravinsky

Bach 2,1 0,0

Stravinsky 0,0 1,2

Stag Hare

Stag 2,2 0,1

Hare 1,0 1,1

Head Tail

Head 1,-1 -1,1

Tail -1,1 1,-1

Pure Nash equilibria, examples

Quiet Fink

Quiet 2,2 0,3

Fink 3,0 1,1

Bach Stravinsky

Bach 2,1 0,0

Stravinsky 0,0 1,2

Stag Hare

Stag 2,2 0,1

Hare 1,0 1,1

Head Tail

Head 1,-1 -1,1

Tail -1,1 1,-1

2 Prisoner’s Dilemma, (Fink, Fink).

2 Bach or Stravinsky, (Bach, Bach), (Stravinsky, Stravinsky).

2 Stag Hunt, (Stag, Stag), (Hunt, Hunt).

2 Matching Pennies, none.

Example: Sending from s to t

2 We have a graph G = (V, E) and two vertices s, t ∈ V .

2 There is one player for each vertex v ∈ V , v 6= t.

2 The set of actions for player u is NG(u).

2 A strategy profile is a set of vertices (v1, . . . , vn−1).

2 Pay-offs are defined as follows:

player u gets 1 if the shortest path joining s to t in the digraph induced by

v1, . . . , vn−1 contains (u, vu), otherwise gets 0.

Sending from s to t: pay-offs

s tb b

b

b

b

b b

Red nodes get pay-off 1, blue nodes get pay-off 0.

Is a Nash equilibrium?

Pure Nash equilibrium

2 First notion of equilibria for non-cooperative games.

2 There are strategic games with no pure Nash equilibrium.

2 There are games with more than one pure Nash equilibrium.

2 How to compute a Nash equilibrium if there is one?

Exercise

Consider the game of major difference:

Two players select, simultaneously a number in the interval [0, 1], what they get

depends on the difference between the selected numbers as follows:

u1(s1, s2) = u2(s1, s2) = (s1 − s2)2.

Analyse the best response for player two to player’s one selection:

1. s1 = 0.4

2. s1 = 0.5

3. s1 = 0.7

Has the game a Nash equilibrium?

Mixed strategies

We have a strategic game Γ = (N, (Ai)i∈N , (ui)i∈N ).

Until now players are selecting as strategy an action.

When each player i is allowed to select as strategy a distribution σi on the set of

actions Ai we have a mixed strategy.

A mixed strategy profile is a combination σ = (σ1, . . . , σn) of mixed strategy

profiles.

In the case that the number of actions for player i is m a mixed strategy profile is

a tuple (p1, . . . , pm) for whichm∑

i=1

pi = 1

The utility of a mixed strategy

The utility function for player i of mixed strategy profile σ is the expected utility

under the joint distribution described by σi.

Ui(σ) =∑

s∈A1×···×An

σ1(s1) . . . σn(sn)ui(s)

Mixed Nash equilibrium

A mixed Nash equilibrium is a profile σ∗ = (σ∗

1 , . . . , σ∗

n) such that no player i can

get better utility choosing a distribution different from σ∗

i , given that every other

player j adheres to σ∗

j .

Theorem (Nash): Every strategic game has a mixed Nash equilibrium.

A Nash equilibrium for the matching pennies game

What are the best responses for player two to the mixed strategies for player 1:

2 (1/4, 3/4)

2 (1, 0)

2 (1/2, 1/2)

Person 1

Person 2

Head Tail

Head 1,-1 -1,1

Tail -1,1 1,-1

From a computational point of view, mixed strategies present an additional

representation problem.

In CS we can store only rational numbers. It is known

2 For two player game there are always mixed Nash equilibrium with rational

probabilities.

2 There are three player games without rational mixed Nash equilibrium.

[Schoenebeck and Vadhan: eccc 51, 2005]

Different notions of approximate Nash equilibria

A characterization of Nash equilibria

Theorem: Strategy σi is the best response to σ−i iff any pure strategy to which σi

asigns positive probability is a best response to σ−i.

Exercice: Resource allocation game

2 We have 3 players each one of them has a job to print. Player one wants to

print a 4Mb job, player two a 6Mb job and player 3 a 10Mb job.

2 The system has two identical printers A and B.

2 Each player has to select the printer for its job

2 A printer process the received jobs in time proportional to the total load.

2 The printed pages are accesible only after the printer stops.

2 The cost for a user is the total load in the selected printer.

Formalize the problem as a 3-player strategic game and find all its Nash equilibria.