Achieving Network Optima Using Stackelberg Routing Strategies

Yannis A. Korilis, Member, IEEE

Aurel A. Lazar, Fellow, IEEE

&

Ariel Orda, Member IEEEIEEE/ACM transactions on networking, vol. 5, No. 1, February 1997

Sanjeev Kohli

EE 228A

Presentation Outline

Introduction to non cooperative networks Overview of approach Model and Problem Formulation Non cooperative User & Manager Single Follower Stackelberg Routing game Multi Follower Stackelberg Routing game Issues

Non-cooperative Networks

Non-cooperative Networks

Users take control decisions individually to max own performance

Non-cooperative Networks

Users take control decisions individually to max own performance

Similar to non cooperative games

Non-cooperative Networks

Users take control decisions individually to max own performance

Similar to non cooperative games Operating points of such networks are determined by Nash equilibria

Non-cooperative Networks

Users take control decisions individually to max own performance

Similar to non cooperative games Operating points of such networks are determined by Nash equilibria Nash Equilibria – Unilateral deviation doesn’t

help any user

Non-cooperative Networks

Users take control decisions individually to max own performance

Similar to non cooperative games Operating points of such networks are determined by Nash equilibria Nash Equilibria – Unilateral deviation doesn’t

help any user Inefficient, leads to sub optimal performance

Non-cooperative Networks

Users take control decisions individually to max own performance

Similar to non cooperative games Operating points of such networks are determined by Nash equilibria Nash Equilibria – Unilateral deviation doesn’t

help any user Inefficient, leads to sub optimal performance Better solution needed !

Network Manager

Network Manager

Architects the n/w to achieve efficient equilibria

Network Manager

Architects the n/w to achieve efficient equilibria Run time phase

Network Manager

Architects the n/w to achieve efficient equilibria Run time phase Awareness of users behavior

Network Manager

Architects the n/w to achieve efficient equilibria Run time phase Awareness of users behavior Aims to improve overall system performance

through maximally efficient strategies

Network Manager

Architects the n/w to achieve efficient equilibria Run time phase Awareness of users behavior Aims to improve overall system performance

through maximally efficient strategies Maximally efficient strategy

Optimizes overall performance

Network Manager

Architects the n/w to achieve efficient equilibria Run time phase Awareness of users behavior Aims to improve overall system performance

through maximally efficient strategies Maximally efficient strategy

Optimizes overall performance Individual users are well off at this operating point [Pareto Efficient]

Presentation Outline

Introduction to non cooperative networks Overview of approach Model and Problem Formulation Non cooperative User & Manager Single Follower Stackelberg Routing game Multi Follower Stackelberg Routing game Issues

Overview of this approach

Overview of this approach

Total flow: Flow of users + Flow of manager

Overview of this approach

Total flow: Flow of users + Flow of manager Example of manager’s flow

• Traffic generated by signaling/control mechanism

Overview of this approach

Total flow: Flow of users + Flow of manager Example of manager’s flow

• Traffic generated by signaling/control mechanism

• Users traffic that belongs to virtual networks

Overview of this approach

Total flow: Flow of users + Flow of manager Example of manager’s flow

• Traffic generated by signaling/control mechanism

• Users traffic that belongs to virtual networks

Manager optimizes system performance by controlling its portion of flow

Overview of this approach

Total flow: Flow of users + Flow of manager Example of manager’s flow

• Traffic generated by signaling/control mechanism

• User traffic that belongs to virtual networks

Manager optimizes system performance by controlling its portion of flow Investigates manager’s role using routing as a

control paradigm

Non Cooperative Routing Scenario

IPv4/IPv6 allow source routing

• User determines the path its flow follows from source-destination

Goal of Manager

Optimize overall network performance according to some system wide efficiency criterion

Capability of Manager

It is aware of non cooperative behavior of users and performs its routing based on this information

Central Idea

Central Idea

Manager can predict user responses to its routing strategies

Central Idea

Manager can predict user responses to its routing strategies

Allows manager to choose a strategy that leads of optimal operating point

Central Idea

Manager can predict user responses to its routing strategies

Allows manager to choose a strategy that leads of optimal operating point

Example of Leader-Follower Game [Stackelberg]

MAN

Org1

Org2 Org n

VP’s k

VP’s kVP’s k

User 1

User 2

User 3

User p

Need to derive

A necessary and sufficient condition that guarantees that the manager can enforce an equilibrium that coincides with the network optimum

Above condition requires –Manager’s flow Control > Threshold

Need to derive

A necessary and sufficient condition that guarantees that the manager can enforce an equilibrium that coincides with the network optimum

Above condition requires –Manager’s flow Control > Threshold

If the above criterion is met, we can show that the maximally efficient strategy of manager is unique and we will specify its structure explicitly

Presentation Outline

Introduction to non cooperative networks Overview of approach Model and Problem Formulation Non cooperative User & Manager Single Follower Stackelberg Routing game Multi Follower Stackelberg Routing game Issues

Model and Problem Formulation

User set I = {1,…..,I} Communication Links L= {1,.....,L}

Source Destination

1

2

L

Model and Problem Formulation (contd)

Manager is referred at user 0 I0 = I U {0}

cl = capacity of link l

c = (c1,….cL) : capacity configuration

C = lL cl : total capacity of the systemof parallel links

c1 >= c2 >= …. >= cL

Each i I0 has a throughput demand ri > 0

r1 >= r2 >= …. >= rI r = iI ri

R = r + r0 Demand is less than capacity of links R < C

Model and Problem Formulation (contd)

User i I0 splits its demand ri over the set of parallel links to send its flow

Expected flow of user i on link l is fli

Routing strategy of user i fi = (f1i,….fL

i)

Strategy space of user i Fi = {fi IRL : 0 <= fl

i <= cl, l L; lL fl

i = ri}

Routing strategy profile f = {f0, f1,….,fI) System strategy space F = iIo Fi

Model and Problem Formulation (contd)

Cost function quantifying GoS of user i’s flow isJi : F IR i I0

Cost of user i under strategy profile f is Ji(f) Ji(f) = lL fl

iTl(fl); Tl(fl) is the average delay on link l, depends only on the total flow fl = iIo fl

i on that link

Tl(fl) = (cl - fl)-1, fl < cl

= , fl >= cl

Total cost J(f) = iIo Ji(f) = lL fl / (cl - fl)

Higher cost lower GoS provided to the user’s flow, higher average delay

Model and Problem Formulation (contd)

is a convex function of (f1, …, fL)

a unique link flow configuration exists – min cost

(f1*,….fL

*) ;

Above is solution to classical routing opt problem, routing of all flow (users+manager) is centrally

controlled; referred to as network optimum.

1)( lll l fcf

Rffl ll ** &0

Kuhn – Tucker Optimality conditions

(f1*,….fL

*) is the network optimum if and only if there exists a Lagrange Multiplier , such that for every link l L*

0 if )(

*2

*

lll

l ffc

c

0 if 1 ** l

l

fc

Presentation Outline

Introduction to non cooperative networks Overview of approach Model and Problem Formulation Non cooperative User & Manager Single Follower Stackelberg Routing game Multi Follower Stackelberg Routing game Issues

Non cooperative users

Non cooperative users

Each user tries to find a routing strategy fi Fi that minimizes its cost Ji (average time delay)

Non cooperative users

Each user tries to find a routing strategy fi Fi that minimizes its cost Ji (average time delay) This minimization depends on strategies of the manager and

other users, described by strategy profilef-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Non cooperative users

Each user tries to find a routing strategy fi Fi that minimizes its cost Ji (average time delay) This minimization depends on strategies of the manager and

other users, described by strategy profilef-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED f0

Non cooperative users

Each user tries to find a routing strategy fi Fi that minimizes its cost Ji (average time delay) This minimization depends on strategies of the manager and

other users, described by strategy profilef-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED f0

Each user adjusts its strategy to other users actions

Non cooperative users

Each user tries to find a routing strategy fi Fi that minimizes its cost Ji (average time delay) This minimization depends on strategies of the manager and

other users, described by strategy profilef-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED f0

Each user adjusts its strategy to other users actions Can be modeled as a non cooperative game, any operating

point is Nash Equilibrium; dependent on f0 !

Non cooperative users

Each user tries to find a routing strategy fi Fi that minimizes its cost Ji (average time delay) This minimization depends on strategies of the manager and

other users, described by strategy profilef-i = (f0 , f1 ,… fi-1, fi+1 ,… fI )

Routing strategy of manger is FIXED f0

Each user adjusts its strategy to other users actions Can be modeled as a non cooperative game, any operating

point is Nash Equilibrium; dependent on f0 ! From users view point, manager reduces capacity on each

link l by fl0 , the system reduces to a set of parallel links

with capacity configuration c – f0 has a unique Nash Equilibrium

f0 f -0 ……. N0(f0)

Non cooperative users

For a given strategy profile f-i of other users in I0, the cost of i, Ji(f) = lL fl

iTl(fl), is a convex fn of its strategy fi , hence the following min problem has a unique solution

IifgJf iii

F

i

i

),,(min arg

ig

Kuhn – Tucker Optimality conditions

fi is the optimal response of user i if and only if there exists a (Lagrange Multiplier) , such that for every link l L,

we have

i

0 if , )(

1

0 if , )( 2

il

ll

i

il

ll

illli

ffc

ffc

ffc

Non cooperative users

f-0 F-0 is a Nash Equilibrium of the self optimizing users induced by strategy f0 of the manger.

The function N0 : F0 F-0 that assigns the induced equilibrium of the user routing game (to each strategy of the manger) is called the Nash Mapping. It is continuous.

Role of the Manager

It has knowledge of non cooperative behavior of users; determines the Nash Equilibrium N0(f0) induced by any routing strategy it f0 chosen by him

Acts as Stackelberg leader, that imposes its strategy on the self optimizing users that behave as followers

Aims to optimize the overall network performance, plays a social rather than selfish role

To find f0 such that if f-0 = N0(f0), then iIo fli = fl

* for all lThis f0 is called maximally efficient strategy of

manager

It is Pareto efficient !

Outline of Results

In case of a single user, the manager can always enforce network optimum; its MES is specified explicitly

In case of any no of users, the manager can enforce the network optimum iff its demand is higher that some

threshold r0, in which case the MES is specified explicitly r0 is feasible if total demand of users plus r0 is less than C It is easy for manager to optimize heavily loaded networks

as r0 is small As the no of user increases, threshold increases i.e. harder

for manager to enforce network optimum The higher the difference in throughput demands of any two

users, the easier it is for manager to enforce network optimum

Network optimum: (f1*,….fL

*)

Flow on link l, fl* is decreasing in link no l L

There exists some link L*, such that fl* > 0 for l <= L* and

fl* = 0 for l > L* ; L* is determined by (from [1] & [2]),

where

and G1=0, GL+1=ln=1cn = C

cl >= cl+1 Gl <= Gl+1

1**

LLGRG

LlcccGl

nnl

l

nnl ,...,2

1

1

1

1

Using Lagrange Multiplier’s equations, we get,

Network Optimum is given by [2]

1,.....,1 *11

* Llfcfc llll

*

2

* ,...,1 ,)(

LAfc

c

Al ll

Al l

**

*

1

1

*

0

, *

*

Llf

Llc

cRccf

l

L

n n

lL

n nll

Best reply fi of user i I0 to the strategies of manager and other users, described by f-i, can be determined as network optimum for a system of parallel links with capacity

configuration (c1i,…, cL

i)

Assuming cli >= cl+1

i , l=1,…,L-1

the flow fli is decreasing in the link no l L

There exists some link Li, such that fli > 0 for l <= Li and

fli = 0 for l > Li ; The threshold Li is determined by

LlcccG

GrG

l

n

in

il

l

n

in

il

i

L

ii

L ii

,...,2 ,

where1

1

1

1

1

LlGG

rRCcGGil

il

iL

n

in

iL

i

allfor

)(,0 and

1

111

Best reply fi of user i to strategy profile f-i of the other users in I0 is given by

Best reply doesn’t depend on detailed description of f-i but only on residual capacity cl

i seen by user on every link l L

In practice, residual capacity info can be acquired by measuring the link delays using an appropriate estimation technique

iil

i

L

m

im

ilL

m

iim

il

il

Llf

Llc

crccf i

i

,0

, )(

1

1

Presentation Outline

Introduction to non cooperative networks Overview of approach Model and Problem Formulation Non cooperative User & Manager Single Follower Stackelberg Routing game Multi Follower Stackelberg Routing game Issues

Single Follower Stackelberg Routing Game

Single Follower Stackelberg Routing Game

In this game, there exists a MES of the manager then it is unique and is given by

LlHHthus

LlRH

LlcGH

RfHH

Llcc

ffH

ll

l

lll

L

n nL

l

n

l

nn

l

lnl

,...,1 ,

,

,/

,0 and

,...,2 ,

1

*

**

1

*11

1

1

1

1

**

1

1

11*0

1

1

1

1*0

11

1

1

by determined is where ,

,

LL

ll

L

n n

L

n nll

HrH

LLlff

Llc

rfcf

Single Follower Stackelberg Routing Game

The best reply f1 of the follower is

Therefore, {1,…,L1} is the set of links over which the follower sends its flow when manager implements f0. For manager: Send flow fl

* on every link l that will not receive any flow from the follower

Split the rest of its flow among the links that willreceive user flow proportional to their capacities

1

1

1

1

1**0

1*1

,0

,1

1

Ll

Llc

rfcffff

L

n n

L

n nllll

Presentation Outline

Introduction to non cooperative networks Overview of approach Model and Problem Formulation Non cooperative User & Manager Single Follower Stackelberg Routing game Multi Follower Stackelberg Routing game Issues

Multi Follower Stackelberg Routing Game

Multi Follower Stackelberg Routing Game

An arbitrary number I of self optimizing users share the system of parallel links

Multi Follower Stackelberg Routing Game

An arbitrary number I of self optimizing users share the system of parallel links

Maximally Efficient Strategy of manager (if it exists) and the corresponding Nash Equilibrium of non cooperative users is:

lli

l

L

i

L

i

llL

n n

L

n

in

ll

IILlIiILl

HrH

LIi

LlfIc

rfcf

ii

i

i

and }:{,link every for and

by determined is ,user every for where

,)1(

1

*

1

1

*0

Multi Follower Stackelberg Routing Game

Equilibrium strategy fi of user i I is described by

If a MES exists, then the induced Nash equilibrium of the followers has precisely the same structure with the best reply follower in the single follower case

i

i

L

n

in

lli

l

Ll

Llrf

cff

i

,0

,1

**

Remarks - M F Stackelberg Routing Game

Remarks - M F Stackelberg Routing Game

{1,…., Li} is the set of links that receive flow from follower i I

Remarks - M F Stackelberg Routing Game

{1,…., Li} is the set of links that receive flow from follower i I

Il is the set of followers that send flow on link l. Since H1 = 0 < ri, i I, all users send flow on link 1 I1 = I

Remarks - M F Stackelberg Routing Game

{1,…., Li} is the set of links that receive flow from follower i I

Il is the set of followers that send flow on link l. Since H1 = 0 < ri, i I, all users send flow on link 1 I1 = I

For f0 to be admissible, fl0 >= 0, for all l L

Remarks - M F Stackelberg Routing Game

{1,…., Li} is the set of links that receive flow from follower i I

Il is the set of followers that send flow on link l. Since H1 = 0 < ri, i I, all users send flow on link 1 I1 = I

For f0 to be admissible, fl0 >= 0, for all l L

If fl0 < 0 fl-1

0 < 0

Remarks - M F Stackelberg Routing Game

{1,…., Li} is the set of links that receive flow from follower i I

Il is the set of followers that send flow on link l. Since H1 = 0 < ri, i I, all users send flow on link 1 I1 = I

For f0 to be admissible, fl0 >= 0, for all l L

If fl0 < 0 fl-1

0 < 0

Admissible condition reduces to f10 >= 0

Remarks - M F Stackelberg Routing Game

{1,…., Li} is the set of links that receive flow from follower i I

Il is the set of followers that send flow on link l. Since H1 = 0 < ri, i I, all users send flow on link 1 I1 = I

For f0 to be admissible, fl0 >= 0, for all l L

If fl0 < 0 fl-1

0 < 0

Admissible condition reduces to f10 >= 0

f10 is an increasing function of the throughput demand r0 of

leader, r0 [0, C - r] ………. [3]

Theorem

There exists some r0, with 0 < r0 < C – r, such that the leader in multi follower Stackelberg routing game can enforce the network optimum if and only if its throughput demand r0

satisfies r0 < r0 < C – r. The maximally efficient strategy of leader is given by

lli

l

L

i

L

i

llL

n n

L

n

in

ll

IILlIiILl

HrH

LIi

LlfIc

rfcf

ii

i

i

and }:{,link every for and

by determined is ,user every for where

,)1(

1

*

1

1

*0

Presentation Outline

Introduction to non cooperative networks Overview of approach Model and Problem Formulation Non cooperative User & Manager Single Follower Stackelberg Routing game Multi Follower Stackelberg Routing game Issues

Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation

“f10(r0) = 0” in r0 [0, C - r]

Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation

“f10(r0) = 0” in r0 [0, C - r]

When r C, r0 0 i.e. in heavily loaded networks, controlling a small portion of flow can drive the system into the network optimum

Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation

“f10(r0) = 0” in r0 [0, C - r]

When r C, r0 0 i.e. in heavily loaded networks, controlling a small portion of flow can drive the system into the network optimum

With throughput demand r fixed, the leader threshold r0 increases with increase in no of users.

Properties of Leader Threshold r0

r0 of the leader is a unique solution of the equation

“f10(r0) = 0” in r0 [0, C - r]

When r C, r0 0 i.e. in heavily loaded networks, controlling a small portion of flow can drive the system into the network optimum

With throughput demand r fixed, the leader threshold r0 increases with increase in no of users.

Leader threshold r0 decreases with increase in difference in user demands

Properties of Leader Threshold r0

Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by I identical followers with total demand r

Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager

r0 = r0

Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager

r0 = r0

Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager

r0 = r0

Set of Parallel links with capacity conf (12,7,5,3,2,1), shared by 100 identical self optimizing users with total demand r and the manager

r0 = r0

Scalability

To determine maximally efficient strategy, manager needs throughput demand ri of every user.

Scalability

To determine maximally efficient strategy, manager needs throughput demand ri of every user.

In many networks, user declare average rate ri during negotiation phase

Scalability

To determine maximally efficient strategy, manager needs throughput demand ri of every user.

In many networks, user declare average rate ri during negotiation phase

Alternatively, the manager can estimate average rates by monitoring the behavior of users

Scalability

To determine maximally efficient strategy, manager needs throughput demand ri of every user.

In many networks, user declare average rate ri during negotiation phase

Alternatively, the manager can estimate average rates by monitoring the behavior of users

Manager can adjust its strategy to maximally efficient one whenever a user departs or a new one joins the network

Scalability

To determine maximally efficient strategy, manager needs throughput demand ri of every user.

In many networks, user declare average rate ri during negotiation phase

Alternatively, the manager can estimate average rates by monitoring the behavior of users

Manager can adjust its strategy to maximally efficient one whenever a user departs or a new one joins the network

User not necessarily mean a single user, it can be a group of users joining the network as an organization. It also reduces threshold r0

Scalability

References

[1] A. Orda, R. Rom, and N. Shimkin, “Competitive routing in multi-user communication networks,” IEEE/ACM Trans. Networking, vol. 1, pp. 510-521, Oct. 1993.

[2] Y.A. Korilis, A.A. Lazar, and A. Orda, “Capacity allocation under non cooperative routing,” IEEE Trans. Automat. Contr.

[3] Y.A. Korilis, A.A. Lazar, and A. Orda, “Achieving network optima using Stackelberg routing strategies,” Center for

Telecommunications Research, Columbia University, NY, CTR Tech. Rep. 384-94-31, 1994.

THANK YOU