Network Analysis A brief introduction on queues, delays, and tokens

Lin Gu, lingu@acm.org

Computer Networking: A Top Down Approach 6th edition. Jim Kurose and Keith Ross,. Part of the slides are adapted from course companion materials.

Queueing theory

How long is the queue? How much time will I wait? A branch of applied probability theory Applications in

Telecommunications Traffic control Predicting computer performance Health services (e.g. control of hospital bed

assignments) Airport traffic, airline ticket sales Layout of manufacturing systems. Communication networks

Why do we want to know the characteristics of queues?

Queues and queueing Oct. 16, 2010: 1.03 million

visitors in World Expo., Shanghai If you are one of them, you

wonder How many people ahead of me? How soon can I get into the pavilion?

8 hours? 12 hours? Ultimately, organizer starts to

persuade visitors to leave (drop packets)

Every second, millions of packets flow into a switch/router Would you be able to tell

How long is the queue? How much is the delay?

Would you like to drop packets?

Queuing theory for studying networks

4

View network as collections of queues FIFO data-structures

Queuing theory provides probabilistic analysis of these queues

Examples: Average length Average waiting time Probability queue is at a certain length Probability a packet will be lost

Little’s Law

5

Little’s Law: Mean number tasks in system = mean arrival rate x mean response time Observed before, Little was first to prove

Applies to any system in equilibrium, as long as nothing in black box is creating or destroying tasks

Arrivals Departures

System

Proving Little’s Law

6

J = Shaded area = 9

Same in all cases!

1 2 3 4 5 6 7 8

Packet #

Time

123

1 2 3 4 5 6 7 8

# in System

123

Time

1 2 3

Time inSystem

Packet #

123

Arrivals

Departures

Definitions

7

J: “Area” from previous slide N: Number of jobs (packets) T: Total time : Average arrival rate

N/T W: Average time job is in the system

= J/N L: Average number of jobs in the system

= J/T

The Little’s Law

8

1 2 3 4 5 6 7 8

# in System(L) 1

23

Time (T) 1 2 3

Time inSystem(W)

Packet # (N)

123

=

WL TN )(

NWTLJ

WL )(

Model Queuing System

9

Use Queuing models to Describe the behavior of queuing systems Evaluate system performance

Server System Queuing System

Queue Server

Queuing System

Characteristics of a queueing model

Arrival process: the sequence of requests for service, often specified in terms of inter-arrival time The distribution that determines how the tasks arrive in

the system.

Service mechanism: # of servers and service time The distribution that determines the task processing time

Queue discipline: disposition of blocked customers (customers who find all servers busy)

Kendall notation A/B/m/N –S A – distribution of inter-arrival time B – distribution of service time m – number of servers N – max capacity (∞ if omitted) S – queue discipline (FIFO if omitted) Distributions

M: stands for "Markovian", implying exponential distribution for service times or inter-arrival times.

D: Deterministic (e.g. fixed constant) Ek: Erlang with parameter k Hk: Hyperexponential with parameter k G: General (anything)

Kendall Notation Examples

12

M/M/1: Poisson arrivals and exponential service, 1 server,

infinite capacity (also assumed infinite population), FCFS (FIFO)

The simplest ‘realistic’ queue M/M/m

Same, but m servers G/G/3/20

General arrival and service distributions, 3 servers, 17 queue slots (20-3)

Poisson Process

13

For a Poisson process with average arrival rate , the probability of seeing n arrivals in time interval t

0...)2Pr(

)1Pr()(...]!2

)(1[)1Pr(

1)0Pr()(1...!2

)(1)0Pr(

)(!

)()Pr(

2

2

ttott

ttte

ttott

te

tnEn

ten

t

t

nt

Poisson process & exponential distribution

14

Inter-arrival time t (time between arrivals) in a Poisson process follows exponential distribution with parameter

1)(

)Pr(

tE

et t

Analysis of M/M/1 queue

15

Given: • : Arrival rate of jobs (e.g., packets on input link) • : Service rate of the server (e.g., packets on

output link) Solve:

L: average number of jobs in the queuing system Lq : average number of jobs in the queue W: average waiting time in whole system Wq: average waiting time in the queue

M/M/1 queue model

16

1

Wq

W

L

Lq

Solving queuing systems

17

4 unknowns: L, Lq W, Wq

Relationships: L=W Lq=Wq (steady-state argument) W = Wq + (1/)

If we know L or Lq, we can find the others Finding L can be hard.

In general:

: the probability that the number of items in the system is n

Can we find a closed form expression of and , given only and ?

0

n

nnPL

nP

nP L

Equilibrium conditions

18

0)()(

lim,)(lim, assume westablize, it toFor

)()()()()()(

)()()()(

)]1)()[(()]1)()[((])1)(1)[(()(

)]1)()[((])1)[(()(

11

1000

11

100

t

tPttPPtP

tPtPtPt

tPttP

tPtPt

tPttP

tttPtttPtttttPttP

tttPttttPttP

nn

tnn

t

nnnnn

nnnn

n+1nn-1

)(tPnDefine to be the probability of having n items in the system at time t

Equilibrium conditions

19

11

10

)(

nnn PPP

PP

n+1nn-1

0,0

2

201 , PPPPPPn

n

0

0

000

1,1,1

n

n

n

n

nn PPthenP

1ρρ1

1

ρ1

ρ1ρ,ρ

00

n

n

n

n

then

ρ1ρandρ1ρ

1

0

0

nn

n

n

PP

Solving for L

20

0

n

nnPL )1( )1(00

n

n

n

n nn

0

1)1(n

nnL

1)1()1(L

)1(L

))(1())1()(1(220

n

n

n

n

n

n nnLL

Solving W, Wq and Lq

21

11LW

)(11

WWq

)()(

2

qq WL

Response Time vs. Utilization

22 1W

Waiting vs. Utilization

0

0.05

0.1

0.15

0.2

0.25

0 0.2 0.4 0.6 0.8 1 1.2

W(s

ec)

Summary of M/M/1

23

The simplest ‘realistic’ queue: Poisson arrivals and exponential service, 1 server, infinite capacity (also assumed infinite population), FCFS (FIFO)

Variables: • : Arrival rate of jobs (packets on input link) • : Service rate of the server (output link)

• L: average number of jobs in the queuing system

• Lq : average number of jobs in the queue

• W: average waiting time in whole system

• Wq: average waiting time in the queue

)(

2

qq WL

1LW

)(1

WWq

t

tnEn

ten

nt

1)0Pr(

)(!

)()Pr(

t)1Pr(

ρ

ρ10 P

0)()(

lim,)(lim, assume westablize, toqueue For the

t

tPttPPtP nn

tnn

t

0

n

nnPL

)1( ρ1ρ n

nP

Example

24

On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 ms to forward a packet. Assuming an M/M/1 model, what is the probability of buffer overflow if the gateway had only 13 buffer units? How many buffer units are needed to keep packet loss below one packet per million? mean arrival rate (): 125 packets/s

mean response time (1/): 2 ms Assuming M/M/1:

What is the gateway’s utilization? What is the probability of n packets queued in the gateway? What is the mean number of packets queued in the gateway? The number of buffers so P(overflow) is <10-6?

Example

25

Service rate μ =

Gateway utilization ρ = λ/μ =

Prob. of n packets in system (gateway) =

Mean number of packets in system (gateway) =

1/0.002=500

0.25

nn )25.0(75.0ρ)ρ1(

33.075.0

25.0

ρ1

ρ

Example

26

Probability of buffer overflow with 13 buffer units: = P(more than 13 packets in gateway) = ρ14 = 0.2514 = 3.73x10-9

= 3.73 packets per billion packets

To limit the probability of loss to less than 10-6:61 10ρ n

125.0log/10log 6 n

Policing Mechanisms

Token Bucket: limit input to specified Burst Size and Average Rate.

Bucket can hold b tokens Tokens generated at rate r token/sec unless

bucket full Over interval of length t: number of packets

admitted less than or equal to (r t + b). At most b packets can be transmitted in a burst

(within a short period of time).

Policing Mechanisms (more)

token bucket, WFQ combine to provide guaranteed upper bound on delay, i.e., QoS guarantee!

WFQ

token rate, r

bucket size, b

per-flowrate, R

D = b/Rmax

arrivingtraffic

Appendix

CS352 Fall,200529

Example application of queuing theory

30

Which one is better, multiple-line or single-queue? We can prove using queuing theory that : throughput

improves increases with one queue instead of separate lines