Oblivious AQM and Nash Equilibria D. Dutta, A. Goel and J. Heidemann USC/ISI USC USC/ISI IEEE...

Oblivious AQM and Oblivious AQM and Nash EquilibriaNash EquilibriaD. Dutta, A. Goel and J. Heidemann

USC/ISI USC USC/ISI

IEEE INFOCOM 2003 - The 22nd Annual Joint Conference of the IEEE Computer and Communications Societies

Presented By Sharon MendelGame Theory in Networks Seminar 25/01/2006

Before we Begin…

Today’s Internet - Motivation

Oblivious AQM & Nash Equilibria

Oblivious AQM & Nash Equilibria 3

Active Queue Management ??

A congestion control protocol (e.g. TCP) operates at the end-points and uses the drops or marks received from the AActivective Q Queueueue MManagementanagement policies (e.g. Drop-tail, RED) at routers as feedback signals to adaptively modify the sending rate in order to maximize its own goodput.

Before we Begin…


Transport Control ProtocolTCP is the dominating transport layer protocol in the Internet and accounts for over 90% of the total traffic. The TCP Protocol is well defined, robust and congestion-reactive (thus stable).The end-to-end congestion control mechanisms of TCP have been a critical factor in the robustness of the Internet.It is widely believed that if all users deployed TCP, networks will rarely see congestion collapses and the overall utilization of the network will be high.

Before we Begin…


Today’s InternetThere are indications that the amount of non-congestion-reactive traffic is on the rise. Most of this misbehaving traffic does not use

TCP. e.g. Real media, network games, other real time

multimedia applications.

The unresponsive behavior can result in both unfairness and congestion collapse for the Internet.The network itself must now participate in controlling its own resource utilization.

Before we Begin…

* Some of the previous slides:“Promoting the Use of End-to-End Congestion Control in the

Internet”,S. Floyd and K. Fall, IEEE/ACM Transactions on Networking Vol. 7

1999.


The Paper’s MotivationTCP (and in fact, any transport protocol) does not guarantee good performance in the face of aggressive, greedy users (who are willing to violate the protocol to obtain better performance). Protocol Equilibrium – A protocol which leads to an efficient utilization and a somewhat fair distribution of network resources (like TCP does), and also ensure that no user can obtain better performance by deviating from the protocol.If protocol equilibrium is achievable, then it would be a useful tool in designing robust networks.

Before we Begin…

More Introduction

Oblivious AQM and Nash Equilibria



Oblivious AQMOblivious (stateless) AQM scheme – a router strategy that does not differentiate between packets belonging to different flows.Stateful schemes – e.g. Fair-Queuing.Stateful Schemes offer good performance, but oblivious schemes are easier to implement.

More Introduction


Popular AQM Schemes (1)Congestion avoidance is achieved through packet dropping.

1. Drop-Tail – Buffers as many packets as it can and drops the ones it can't buffer:

Distributes buffer space unfairly among traffic flows.

Can lead to global synchronization as all TCP connections "hold back" simultaneously, hence networks become under-utilized.

More Introduction

DDTT


Popular AQM Schemes (2)2. RED – Random Early Dedication - Monitors

the average queue size and drops packets based on statistical probabilities:

If the buffer is almost empty, all incoming packets are accepted; As the queue grows, the probability for dropping an incoming packet grows; When the buffer is full, the probability has reached 1 and all incoming packets are dropped.

Considered more fair than tail drop - The more a host transmits, the more likely it is that its packets are dropped.

Prevents global synchronization and achieves lower average buffer occupancies.

More Introduction

RREEDD


Oblivious AQM and Nash Equilibria

The paper studies the existence and quality of Nash equilibria imposed by oblivious AQM schemes on selfish agents:

1. Existence 2. Efficiency3. Achievability

More Introduction


ContentIntroduction

The ModelExistenceEfficiencyAchievabilitySummary and Future Work

The Model

The Markovian Internet Game



The Internet GamePlayers – The end-to-end selfish traffic agents. Each player has a strategy which is to control the

average rate he tries to push through the network. Users’ Performance Metric – goodput.

Rules – set by the AQM policies (AQM schemes in routers).Nash Equilibrium – No selfish agent has any incentive to unilaterally deviate from it’s current state.Oblivious AQM scheme leads to a protocol equilibrium only if it imposes a Nash equilibrium on the selfish users.

Paper’s focusPaper’s focus – AQM schemes that guarantee bounded average buffer occupancy regardless of the total arrival rate.

The Model


The Markovian Internet Game

The agents generate Poisson traffic. Does not accurately model Internet traffic, yet a

reasonable first step. Each user controls its own offered load = the

average Poisson traffic rate.

The system is modeled as a M/M/1/K queue: Average service time - Without loss of generality

assumed to be unity. Buffers capacity - K=B.

No assumptions are made on the selfish protocol (i.e. TCP, AIMD etc’).

The Model


The M/M/1/K Internet Game

n – Number of users – players.λi – The Poisson average arrival rate of player i.

Ui – Utility function of player i.

μi – Goodput , Ui = μi .p – AQM router drop probability due an average aggregated load (offered load) of λ and an average service time of unity.A symmetric Nash equilibrium - Ensures that every agent has the same goodput at equilibrium. Unless mentioned otherwise, quantities such as the rates, goodput and throughput are averages (Poisson traffic sources).

The Model


Nash Equilibrium Conditions (1)1. No agent can increase their goodput, at Nash

equilibrium, by either increasing or decreasing their throughput:

2. A symmetric Nash equilibrium:

3. Oblivious AQM scheme, hence functions of router state (drop probability, queue length) are independent in i:

nandji jiji

,

The Model

0

i

ii

d

di

i

nd

p

dp

1

Nash Equilibrium Condition :Nash Equilibrium Satisfying Condition –Necessary and sufficient

pU iii 1Utility Function in Nash Equilibrium :

RREEDD

DDTT


Nash Equilibrium Conditions (2)

λNash , μNash , pNash - The aggregate throughput

(offered load), goodput, drop probability respectively.

The Nash equilibrium imposed by an AQM scheme is efficient if the goodput of any selfish agent is bounded below when the throughput (offered load) of the same agent is bounded above:

2

11

c

cpNash

NashNash

pNash is bounded

c1 c2 some constants

The Model

Nash Equilibrium Efficiency Condition :

Existence

Are there oblivious AQM schemes that impose Nash equilibria on selfish users?



Drop-Tail Queuingp - Drop Probability = Probability to find a full system.

From Queuing Theory:

Theorem 1: There is NO Nash Equilibrium for selfish agents and routes implementing Drop-Tail queuing.

Proof: applying the condition for Nash equilibrium:

11

1

B

B

p

Existence

ii

iii ppU 11

00

iii

ii

i

QED.

2

ii

i

11

11

B

B

p

B

21

11


Random Early DetectionApproximated steady state RED model:

From Queuing Theory, at steady state:RED Router, at steady state:

Steady State:

thth

thq

pl

minmaxmin max

p =

0

1

If lq <minth

If minth lq maxth

Otherwisw

“Faster network simulation with scenario pre-filtering” Tech. Rep., USC/ISI Tech Report 550, November 2001.

lq – Queue length

p – Drop Probability

λ – Aggregated Offered Load

p

plq

11

1

lq maxth

thth

th pp

p

pminmax

min11

1max

Existence


RED and Nash EquilibriaTheorem 2: RED Does NOT impose a Nash equilibrium on uncontrolled selfish agents.

Proof: applying the condition for Nash equilibrium:

Summary: RED punishes all flows with the same drop probability. The nature of the drop function is considerably gentle. Misbehaving flows can push more traffic and get less hurt (marginally). There is no incentive for any source to stop pushing packets.

q

qq l

lp

p

pl

1

11

11

1

q

qiii l

lp

11

011

q

q

i

ii

iq

q

i

i

l

l

l

l

QED.

0

i

i 01

1

1 2

d

dl

ll

l q

q

i

q

q

i

i

RED is oblivious:

Nash Equlibria does not exist.

Existence


Virtual Load REDVirtual Load RED model:

Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if

Proof:Throughput at Nash equilibrium is independent of minth:

thth

thvql

minmax

min

p =

0

1

If lvq <minth

If minth lvq maxth

Otherwise

lvq – M/M/1 Queue length when facing the same load :

1vql

1max1min thth

2

1

2

max41,1

1

2

nnnt

t

t thNash

Existence

EfficiencyIf an Oblivious AQM scheme can impose a Nash equilibria, is that equilibria efficient, in terms of achieving high goodput and low drop probability?



Number Of FlowsN

orm

aliz

ed D

ata Total Offered Load

Goodput

minth = 0

maxth =50

n – 1,..,50

Example - Efficiency of VLRED

Efficiency

Proof: Applying Nashequilibrium satisfying andefficiency conditions:

Theorem 4: VLRED is not efficientimposing a Nash Equilibriumon selfish agents

1p α, β - some constants

pn

d

dp

1

11

2

2

lvq2 nμ/α

nμ = θ(lvq2)

th

thvql

max1

max

The goodput falls to zero

asymptotically.QED.


Efficient Nash AQM scheme

Assume – Totaldesirable offered loadat Nash equilibrium:

Oblivious mechanism can ensure an efficient Nash equilibria under selfish behavior of users.

1214

11

2

d

p

dp

n

Efficiency

Goodput

Number Of Flows

Nor

mal

ized

Dat

a

Total Offered Load

Drop Probability

11

11

3

11p

ENAQM drop probability is bounded

AchievabilityHow easy is it for players (users) to reach the equilibrium point? or How can we ensure that agents actually reach the Nash equilibrium state?



Ensuring a Nash equilibrium by an Oblivious AQM

λ*i – Offered load at equilibria when the number of agents is i.

Δi = λ*i - λ*

i-1 = iα α some constant

p = ƒ(λ*i) non decreasing and convex.

Applying Nash equilibrium satisfying and efficiency conditions:

From convexity of p*i :

From efficiency:

The equilibrium imposed by any oblivious AQM strategy is (very) sensitive to the number of agents, thus making it impractical to deploy in the Internet.Agents need the help of the router to reach the equliibria.

i

ii

icf

ic

2'

1

Achievability

c1, c2 - some constants

*1

'1

*2

'2

*1

'1

*1

* ... iii fffpp

ciiii

i

1

1

1

c - constant

2ii The sensitivity coefficient falls faster than the inverse quadric.

Summary and Future Work

OverviewNow what? – Future WorkSome further work



OverviewIntroduction – Today’s Internet. The proposed model – Markovian (M/M/1/K) GameExistence – Drop tail and RED cannot impose a Nash equilibra. VLRED imposes a Nash equilibra.

But the equilibrium points do not have a very high utilization.

Efficiency - ENAQM imposes an efficient Nash equilibra.Achievability - Equilibrium points in oblivious AQM strategies are very sensitive to the change in the number of users. It may be hard to deploy oblivious schemes that do have

Nash equilibria without the explicit help of a protocol.



Now What ? - Future WorkVLRED – Explore why the Nash equilibria do not result in good network utilization. Conjecture – VLRED’s drop function becomes

very harsh as we reach equilibria.

Study gentler versions of VLRED and determine whether such modification can still impose Nash equilibria.Design protocols which lead to efficient network operation, such that no user has any incentive to unilaterally deviate from the protocol – can it be done ? – The Protocol Equilibrium Question.



Some Further work (1)

“Towards Protocol Equilibrium with Oblivious Routers” D. Dutta, A. Goel and J. Heidemann, IEEE INFOCOM 2004. “…In this paper, we show that if routers used

EWMA to measure the aggregate rate, then the best strategy for a selfish agent to minimize its losses is to arrive at a constant rate. Even though the protocol space is arbitrary, our scheme ensures that the best greedy strategy is simple, i.e. send with CBR. Then, we show how we can use the results of an earlier paper to enforce simple and efficient protocol equilibria on selfish traffic agents…”



Some Further work (2)“Pricing Differentiated Services A Game -Theoretic Approach”, E. Altman, D. Barman, R. El Azouzi, D. Ros, B. Tuffin, (accepted for publication in Computer Networks, 2005). “…The goal of this paper is to study pricing of differentiated services

and its impact on the choice of service priority at equilibrium. We consider both TCP connections as well as non controlled (real time) connections…. We first study the performance of the system as a function of the connections’ parameters and their choice of service classes. We then study the decision problem of how to choose the service classes. We model the problem as a noncooperative game. We establish conditions for an equilibrium to exist and to be uniquely defined. We further provide conditions for convergence to equilibrium from non equilibria initial states. We finally study the pricing problem of how to choose prices so that the resulting equilibrium would maximize the network benefit…”

“…The paper (“Oblivious AQM and Nash Equilibria”)(“Oblivious AQM and Nash Equilibria”) restricted itself to symmetric users and symmetric equilibria and the pricing issue was not considered. In this framework, with a common RED buffer, it was shown that an equilibrium does not exist. An equilibrium was obtained and characterized for an alternative buffer management that was proposed, called VLRED. We note that in contrast to (Oblivious AQM and Nash Equilibria”)(Oblivious AQM and Nash Equilibria”), since we also include in the utility of CBR traffic a penalty for losses we do obtain an equilibrium when using RED…”


Thank You !


AppendixesStateful Schemes – Fair QueuingFrom Queuing TheoryNash Equilibrium ConditionsApproximated Steady State RED ModelVLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof Efficient Nash AQM Drop Probability



Stateful Schemes – Fair Queuing - Nagle’s Algorithm

Gateways maintain separate queues for packets from each individual source. The queues are serviced in a round-robin manner. Nagle’s algorithm, by changing the way packets from different sources interact, does not reward, nor leave others vulnerable to, anti-social behavior.

Introduction - Appendix

“Analysis and Simulation of a Fair Queuing Algorithm”,A.Demers, S. Keshav and S.J. Shenker, ACM SIGCOMM,

1989.


From Queuing TheoryM/M/1/K Queuing system - Poisson arrivals, Exponentially distributed service times, one server and finite capacity buffer:

PASTA - Poisson Arrivals See Time Averages – For a queuing system, when the arrival process is Poisson and independent of the service process:The probability that an arriving customer finds i customers in the system Equals The probability that the system is at state i.πi – Probability that the system is in state i, Using birth-death model:

Block Probability -

Kii

i

0

,1

11K

K

KBP

2 KK-1

10

λ

μ

λ

μ

λ

μ

λ

μ

λ

μ


Nash Equilibrium Condition

1.

2.

3.

4.

5.

Substituting 2,3 in 5 we obtain:

nandji jiji

,

d

di

i

pU iii 1

01

ii

i

i pp

nd

p

dp

1Nash Equilibrium Satisfying Condition :

The Model – Appendix

0

i

ii


Approximated Steady State RED Model

Standard RED Transfer Function

Avg. Buffer Occupancy

Dro

p p

rob

.

Approximated RED Transfer Function

Dro

p p

rob

.

Normalized Offered Load

Approximated RED Buffer Occupancy

Normalized Offered Load

Avg

. B

uff

er

Occ

up

an

cy

“Faster network simulation with scenario

pre-filtering”Tech. Rep., USC/ISI Tech Report 550, November

2001.

Existence - Appendix

Example:minth = 10 maxth =20

n – 1,..,50 Pmax = 0.3


VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (1)


Virtual Load RED model:

Proof of Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if Assume the drop probability can be written as a

continues function for all lvq<maxth:

Nash equilibrium condition:

thth

thvql

minmax

min

p =

0

1

If lvq <minth

If minth lvq maxth

Otherwise

lvq – M/M/1 Queue length when facing the same load :

1vql

1max1min thth

2

11minmax

1

minmax

min1

thththth

th

d

dpp

thth

th

thth

nminmax

min11

11minmax

12


VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (2)

Substituting and simplifying we obtain:

This is true if (by Substituting n=1)

1

t


0max)1(2 thnnt

thth t

nnnt min,

2

1

2

max41 2

1max1min thth


Efficient Nash AQM Drop Probability

Total desirable offered load at Nash equilibrium:

Substituting y2=1-λ and then integrating:

k is determined so when there is one user, the drop probability is zero as long as his offered load is less than unity

The offered load at Nash equilibria is boundeddrop probability at equilibria is bounded (proved by

substitution).

y

ykpk

y

yp

y

dy

y

dy

p

dp

1

11log

1

1log1log

112

1

1

Efficiency - Appendix

1214

11

2

d

p

dp

n

11

11

3

11p

k - arbitrary constant to be determined

31k

Oblivious AQM and Nash Equilibria D. Dutta, A. Goel and J. Heidemann USC/ISI USC USC/ISI IEEE...

Documents

Transcript of Oblivious AQM and Nash Equilibria D. Dutta, A. Goel and J. Heidemann USC/ISI USC USC/ISI IEEE...