Network Routing Problem

Input: network topology, link metrics, and traffic matrix

Output: set of routes to carry traffic

AB

C

D

E

S1

R1

R3

R2

S3

S2

Network Routing: Classical Approach

Routing as optimization problem e.g., minimum total delay in network focus on global network performance (social

optimal) performance of individual user not important

Centralized or distributed algorithms e.g., link state or distance vector

Passive users users are oblivious to routing decisions

Network Routing: Game-Theoretic Approach

Routing as game between users users determine route decision based solely on individual performance (selfish routing) strongly dependent on other users’ decisions

Non-cooperative game (non-zero sum) users compete for network resources

Equilibrium point of operation Nash equilibrium point (NEP)

Selfish Network Routing

Advantages no need of centralized control or global

agreement on routing algorithm individual user’s performance considered greater adaptability

• changes in user demands or changes in network conditions

Disadvantages multiple equilibria (eq. selection problem) convergence to equilibrium no network-wide optimality at equilibrium

• cost of “selfish routing” user’s must have detailed knowledge of network

Applications of Game Theory to Network Routing

Competitive routing in multiuser communication networksA. Orda, R. Rom and N. ShimkinIEEE/ACM Transactions on Networking, 1 (5) 1993

How bad is selfish routing?T. Roughgarden and E. TardosJournal of the ACM, 49 (2) 2002

Selfish routing with atomic playersT. RoughgardenACM/SIAM Symp. on Discrete Algorithms (SODA) 2005

Simple Model: Network of Parallel Links

set of users share a set of parallel links each user has fixed demand (data rate) users decide how to split demand across links

minimize individual cost link has a load dependent cost (e.g., delay)

A

S1

B

R1

S2

S3

S4

R2

R3

R4

set of parallel links: set of users: each user has a fixed demand (data rate): user splits its demand across links

flow of user i on link l: flow configuration of user i:

system flow configuration:

feasible configurations satisfy nonnegative and demand constraints

Network of Parallel Links

),,( I1 fff

}L,,1{ L}I,,1{ I

ir

ilf

),,( L1iii fff

User’s Cost Function Cost function of user i:

cost depends on flow configuration of all users

Assumptions on cost function sum of user-link cost function: can be infinite convex in when finite, continuously differentiable in at least one user with infinite cost can change its flow

configuration to have finite cost• aggregate demand must be less than aggregate link capacity

Ll

lil

i fCfC )()(

ilC

ilf

)( fC i

Very mild conditions on cost function

The Game Users individually decide their flow configuration

goal is to minimize its own cost

Nash Equilibrium Point (NEP) system flow configuration such that no user can

reduce their cost by changing its flow allocation is a NEP if for all i, the following

holds:Ffff I )

~,...,

~(

~ 1

)}~

,...,,...,~({)

~( 1min Iii

Ff

i fffCfCii

No user can reduce their cost by rerouting their own flow

The Issues

Existence of NEP is at least one NEP guaranteed to always exist

Uniqueness of NEP under which conditions (if any) do we have a

single NEP Convergence to (and stability of) NEP

play dynamics that lead to a NEP

System properties at the NEP e.g., how does users divide allocate their flows

Existence of NEP

N-person convex game [Rosen65] joint strategy set is convex, closed and

bounded each player’s payoff function is convex in their

own strategy existence of NEP proven by Katutani fixed point

theorem Can also show using Kuhn-Tucker

conditions necessary and sufficient for system flow

configuration to be a NEP

Uniqueness of NEP

Uniqueness of NEP only under a type of cost functions (type-A functions) cost function has two parameters: user’s i and

aggregate of all others monotonically increasing in each parameter still very general (e.g., M/M/1 delay function)

Proof by contradiction using Kuhn-Tucker conditions

System has a single operating point

System Properties at NEP

Assume all users share same type-A cost function but users can have different demands

Monotonicity of link usage user with higher demand uses more of each

and every link used a user with higher demand uses more links

Higher capacity links receive more users does not hold in general, only under yet

another type of cost function (which still captures M/M/1)

Intuitive but assuring properties

Simple case study two-users sharing two parallel links

Dynamical model: Elementary Stepwise System Users take turns in updating their flow

configuration• measure load on links, adjust its flow to minimize cost

flow of user i on link l at step n

Dynamical System

A

S1

BR1

S2 R2

)(nf il

Convergence to NEP

Let denote unique NEP of gameInitialize system with any feasible

flow configuration: f(0)Convergence to NEP guaranteed

*f

*)(lim fnfn

Framework used in proof not aplicable in general limited to two link, two user structure

General Topology

Users decide how to split their demands over possible paths users know network topology (directed graph)

AB

C

D

E

S1

R1

R3

R2

S3

S2

Existence and Uniqueness of NEP

Existence of NEP same argument as before (N-person convex

game) No unique NEP for type-A cost functions

shown by counterexample Uniqueness shown only under very strict

conditions for cost function not very interesting networking scenarios

Analysis of general network in this modeling framework is much harder

The “Price of Anarchy” Equilibria of non-cooperative games usually

inefficient e.g., prisoner’s dilemma Pareto optimal usually not a NEP

Quantify inefficiency in terms of a global objective “price of anarchy” (coordination versus

competition)

Price of Anarchy of a Game

objective function value at NEP

optimal objective function value=

if multiple NEP exists, take sup (or inf) over NEP set

Cost of Selfish Routing

How does total cost compare? flow allocation at a NEP optimal flow allocation

Total cost of flow configuration:

where is load dependent link cost function e.g., link delay

lLl

ll ffcfC

)()(

(.)lc

Example (1/4)

flow configuration cost

optimal flow allocation

can be realized with

A

S1

BR1

S2 R2

r1 = 0.5

r2 = 0.5

xxcl )(1

1)(2

xcl

)25.0,25.0(,)25.0,25.0( 21 ff

21222111

2)()()( llllllll ffffcffcfC

)5.0,5.0(),( ***

21 ll fff

4/375.0)( * fC

Example (2/4)

But this is not NEP… Cost of a flow configuration to user i

A

S1

BR1

S2 R2

r1 = 0.5

r2 = 0.5

xxcl )(1

1)(2

xcl

il

ill

illl

illl

i fffffcffcfC211222111

)()()(

375.025.025.0*5.0)( * fC i

By rerouting traffic user 1 (or 2) can reduce its cost: )2.0,3.0('1f

365.02.03.0*55.0)'(1 fClower cost!

higher cost

Example (3/4)

NEP given by

link 1 is a dominant strategy (link 2 never used) Cost to user i at NEP

Total cost of NEP configuration

A

S1

BR1

S2 R2

r1 = 0.5

r2 = 0.5

xxcl )(1

1)(2

xcl

)0,5.0(,)0,5.0( 21 ff

5.0)( fC i

1)( fC

higher cost!

Example (4/4)

Optimal cost: NEP cost:

Price of Anarchy:

A

S1

BR1

S2 R2

r1 = 0.5

r2 = 0.5

xxcl )(1

1)(2

xcl

1)( fC4/3)( * fC

3

4

4/3

1

)(

)(*

fC

fC

Thm:[Roughgarden/Tardos00] POA of selfish routing w/affine cost functions is at most 4/3 for any network topology and traffic matrix!

Another example (non-linear cost)…

NEP: both users only use link 1 cost is 1

Optimal: 1-ε for link 1 and ε for link 2 ε depends on d, but is small for large d cost ≈ 0

Price of anarchy can be arbitrarily large goes to infinity as d goes to infinity

A

S1

BR1

S2 R2

r1 = 0.5

r2 = 0.5

dl xxc )(1

1)(2

xcl

So how bad is selfish routing? It depends...

cost functions, network topology, traffic matrix, user demands, etc.

In reality, not so bad achieves close to optimal cost in Internet-like

environments (simulation study) Another positive (and nice) result:

Thm:[Roughgarden/Tardos00] selfish routing is no worst than the optimal routing of twice as much traffic for any cost function, network topology and traffic matrix!

Title

Congestion Control Problem

Input: network topology, routes, link characteristics, traffic matrix

Output: set of data rates to be used

AB

C

D

E

S1

R1

R3

R2

S3

S2

Congestion Control: Classical Approach

Congestion control as optimization problem match user’s demand to network capacity and

achieve some fairness among users focus on global network performance (social

optimal) performance of individual user not important

Centralized or distributed algorithms e.g., TCP, max-min fairness

Passive users users are oblivious to congestion decisions

Congestion Control: Game-Theoretic Approach Congestion control as game between users

users determine their own data rates decision based solely on individual performance

Non-cooperative game (non-zero sum) users compete for network resources

Equilibrium point of operation Nash equilibrium point (NEP)

Key Assumption: A higher sending rate do not necessarily yields better performance for user

Routing Games vs Congestion Control Games

Routing games users determine

network routes multi-path routing

and traffic splitting is possible

users’ data rates are given and must be routed

Congestion games users determine

their data rate network routes

are given (single path)

Applications of Game Theory to Congestion Control

Making greed work in networks: a game-theoretic analysis of switch service disciplinesS. ShenkerIEEE/ACM Transactions on Networking, 3 (6) 1995

An evolutionary game-theoretic approach to congestion controlD. Menasché, D. Figueiredo, E. de Souza e Silva Performance Evaluation, 62 (1-4) 2005

Simple Model: Single Bottleneck Link

S1R1

S2

S3

S4

R2

R3

R4

set of users share a bottleneck link users decide their data rates

maximize individual performance user’s performance depends on link load

e.g., quality of service provided by link

Single Bottleneck Link Users determine sending rate: Link modeled as M/M/1 queue

unit capacity packet scheduling policy

Scheduling policy induces average queue length for each user : avg. queue length of user i

User’s utility function

strictly increasing in strictly decreasing in convex and derivable everywhere

ir

ic

),( iii crU

icir

Scheduling Policy Determined by system operator Allocation function

scheduling policy P induces an avg. queue length for each user given all user’s data rate

FIFO example

Must satisfy some constraints aggregate average queue size same as M/M/1

Allocation function can be realized by different service disciplines

)(rC Pi

j j

iFIFOi r

rrC

1)(

Fair Share Allocation Allocate service capacity fairly among user’s

demand user’s requesting less obtain higher priority

Implemented through a priority queueing algorithm

Assume: r1 < … < rN

UserPriority Level

A B C D

1 r1 - - -

2 r1 r2 - r1 - -

3 r1 r2 - r1 r3 - r2 -

4 r1 r2 - r1 r3 - r2 r4 - r3

fraction of traffic gets lower priority

MAC: Set of Monotonic Allocation Functions

Consider a set of possible allocation functions

: increases, increases

: increases, does not decrease

Includes all typical service disciplines FIFO, LIFO, PS, fair share allocation

0

i

i

r

C

0

j

i

r

C

or

0

j

i

r

Cat for all with rk ro

k0

j

i

r

C r

ir ic

jr ic

The Problem Investigated

Relationship between NEP and service disciplines (MAC functions)

Which service disciplines yield good NEP? Properties of NEP of a given MAC

efficiency fairness convergence to equilibrium user protection

System designer can select service discipline that yields good equilibrium

Efficiency of NEP

Efficiency in terms of Pareto optimal no global objective function of system outcome

Pareto optimal outcome: no other outcome is preferred by all users

Thm:[Shenker95] There is no allocation function in MAC such that every NEP is Pareto optimal

Under some additional constraints fair share is always efficient constrained users’ utility function symmetric rate vector

Uniqueness of NEP

Allocation functions can induce multiple NEP undesirable since users cannot coordinate

Thm:[Shenker95] Fair share mechanism always has a unique NEP Fair share is the only allocation function that always yields a unique NEP

Convergence to Equilibrium Dynamics through a generalized hill

climbing algorithm users eliminate strategies that always perform

worst system converges to a reduced set of strategies

Different from best-response dynamics

Thm:[Shenker95] With the fair share mechanism, all generalized hill climbing

algorithm converges to the NEP

Convergence is also fast (superlinear) and stable

Title

Application to Multimedia Traffic

Users share common bottleneck link User’s choose data rate to be sent by source

only few data rates available Utility given by “perceived” quality

S1

R1

R2

R3

R4

Investigate dynamics and convergence using evolutionary game theory

1r2r3r

Why Evolutionary Game Theory

Model how users change their strategy

Users are not perfect: stochastic dynamics, myopic, etc

Which NEP will be achieved (if more than one exists)

Efficiency of selected NEP

Evolutionary Game Theory

Entities of Model and Interactions

Users(strategy set)

QoS Model(E-model or other)

Link model(M/M/1/k or other)

choice of strategiescauses impact on

performancemetric feeds

yields perceivedquality to

Two-layer Markovian Model

users’actions 0, 3 1, 2 2, 1 3, 0

link perf.

layer 1

layer 2

QoS ofeach user

QoS ofeach user

QoS ofeach user

QoS ofeach user

States and Users Utility

)()()()(1 ,,,,, i

Mi

mi

li nnnn

: number of users selecting strategy l in state)(i

lnM : number of data rates available to users

is

: state is

),( islU : utility function of strategy l in state is

No constraints on users’ utility function should be defined for every state

Transition Matrix

)()()()(1 ,,,,, i

Mi

mi

li nnnn

)()()()(1 ,1,,1,, j

Mj

mj

lj nnnn

otherwisen

slUslUifslUsmUni

l

ijiji

l

)(

)( ),(),()),(),((

Transitions determined by QoS in each state rate of change proportional to gain transitions can reduce QoS (users make errors) Markov chain is ergodic

Main Problem Investigated

System in steady state Users make no mistakes

Assume: 0, t

States that have non-negligible steady state probability

States that correspond to

NEP

What is the relationship?

Proposition 1

under the condition that this state is contained in a quasi-absorbing set

If a state has non-negligible steady state probability

This state is also a NEP

Proposition 2

proof via simple counter-example

This state also has non-

negligible steady state probability

If a state is a NEP

Summary of ResultsStates with non-negligible SS probability are NEP

correspond to “stable” statesSome NEP are not stable

system dynamics cannot converge on themStill possible to have multiple stable NEP

not clear where system will convergestate with highest probability?

Title

Title