The material here is taken from:

04/22/23 1

A Quick Introduction to Queuing Theory. A Quick Introduction to Queuing Theory.

The material here is taken from:

1) A. M. Law & W. D. Kelton, Simulation, Modeling and Analysis, Third Edition, McGraw-Hill, 2000.

2) H. Kobayashi, Modeling and Analysis, Addison-Wesley, 1978.

3) M. Schwartz, Broadband Integrated Networks, Prentice-Hall, 1996.

4) R. W. Wolff, Stochastic Modeling and the Theory of Queues, Prentice-Hall, 1989.

5) W. Feller, An Introduction to Probability Theory and its Applications, Second Edition, Wiley, 1957.

04/22/23 2


Examples of Queuing Systems.

System Servers Customers

Bank Tellers Customers

Hospital Doctors, nurses, beds Patients

Computer System CPU, I/O devices JobsManufacturing

System Machines, workers Parts

AirportRunways, gates,security check-in

stationsAirplanes,travelers

Communicationsnetwork Nodes, links Messages,

packets

04/22/23 3


Components of a Queuing System.

1) Arrival Process. This specifies how customers arrive to the system. We use Ai to denote the time interval between the arrival of the (i - 1)st and i th customer - the inter-arrival time. We will generally assume that the successive times A1, A2, …, An, … are Independent Identically Distributed random variables. Their mean - or expected - inter-arrival time will be denoted by E(A), and we shall call = 1/E(A) the arrival rate of the customers. Please note the physical units: if Ai is in, say, seconds, then is in reciprocal seconds.

Notice that giving an arrival rate is not sufficient - we will always need to give a probability distribution, for which the arrival rate will provide the mean - or the reciprocal of the mean. Unless specified, though, we usually mean an exponential distribution with given mean...

04/22/23 4


2) Service Mechanism.

This is specified by the number of servers, usually denoted by the variable s, and the probability distribution of the service times. In particular, is S1, S2, …, Sn, … are IID random variables giving the service times of a sequence of customers, the mean service time of a customer will be denoted by E(S), and = 1/E(S) will stand for the service rate of a server.

3) Queue Discipline.

This is the rule by which we choose the next customer to be served. Common ones are

FIFO: First In First Out (a standard queue)

LIFO: Last In First Out (a stack)

Priority: some way is defined to determine the priority of a customer (a priority queue)

04/22/23 5


Notation for Queuing Systems.

A notational system originally introduced by D. G. Kendall has become standard.

1 2 n1. s servers in parallel and one FIFO queue feeding servers.

2. A1, A2, … are IID random variables.

3. S1, S2, … are IID random variables.

4. The Ai's and Si's are independent.

Such a system is denoted by GI/G/s, for a General Independent distribution of the Ai's, a General distribution of the Si's and the presence of s servers.

04/22/23 6

A Quick Introduction to Queuing Theory. A Quick Introduction to Queuing Theory. • A system with one server, exponential arrivals and exponential service is denoted by M/M/1; M is used for Markovian, or memory-less.

• Ek is used for k-Erlang, obtained as a sum of k exponential random variables.

• D is used to denote deterministic (or constant) arrivals or service.

• = /(s is called the utilization factor, and is a measure of how heavily the resources of the queuing systems are used. Note that, under normal circumstances, ≤ 1. is the service rate of a single server, and there are s servers.

04/22/23 7


Measures of Performance for Queuing Systems.

Given a queuing system, what are "relevant" elements to measure? Let:

1) Di = delay in queue of ith customer;

2) Wi = Di + Si = waiting time in system of ith customer;

3) Q(t) = number of customers in queue at time t;

4) L(t) = number of customers in system at time t = number of customers in queue + number of customers being served.

a) Steady-State Average Delay:

d =limn→ ∞

Dii=1

n

∑n

, w.p.1

where w.p. stands for with probability and means that the limit holds for almost all sequences D1, D2, ...

04/22/23 8


b) Steady-State Average Waiting Time:

w =limn→ ∞

Wii=1

n

∑n

, w.p.1

c) Steady-State Time-Average Number in Queue:

Q =limT→ ∞

Q t( )dt0

T

∫T

, w.p.1

c) Steady-State Time-Average Number in System:

L =limT→ ∞

L t( )dt0

T

∫T

, w.p.1

• Note that in all cases < 1 is a necessary condition for the limits to exist (the average number of arrivals must be less than the average number of possible departures).

04/22/23 9


Two conservation equations, valid for any queuing systems for which the limits d and w exist are:

Q = d and

L = w (Little's law, or Little's formula)

A proof can be found in [Wolff, p. 286], and a simpler presentation in [Kobayashi, p. 118].

In words: the time-average number in queue is given by the product of the mean arrival rate and the average delay. The time-average number in system is given by the product of the mean arrival rate and the average waiting time.

Since time in queue and waiting time are related, we can also obtain a formula relating them (their averages): w = d + E(S).

04/22/23 10


The number of arrival and service distributions for which analytical results can be obtained is rather small - exponential distributions (and their Erlang variants) are good candidates - and, usually, at least one of them must be exponential for an analytical attack to be successful.

We now look at the simplest case, the M/M/1 queue.

Another term often used to describe an arrival process with exponentially distributed inter-arrival times is "Poisson Process". This refers to arrivals (for inter-arrival times) or to service times (for servers).

04/22/23 11


Poisson Processes. There are a number of different (but equivalent) definitions of a Poisson process. We will introduce one and derive a number of properties. You should be aware that there are multiple way of deriving the properties.

Definition: a function f(x) is a member of the o(h) class, denoted by if .

f ∈o(h) limh→ 0

f h( )h

=0

Let Pn(t) denote the probability that exactly n arrivals will occur in a time interval of length t ≥ 0. Let be a positive number. We introduce two postulates:

a) The probability that an arrival occurs during (t, t + h) is h + o(h).

b) The probability that more than one arrival occurs during (t, t + h) is o(h).

04/22/23 12


The next step is the derivation of a system of differential equations for Pn(t), n = 0, 1, 2, …. Examine the interval (0, t + h) = (0, t) U [t, t + h). Pn(t + h), n ≥ 1, can be computed as:

a) the probability of n arrivals during (0, t) and no arrivals during [t, t + h);

b) the probability of n - 1 arrivals during (0, t) and one arrival during [t, t + h);

c) the probability of x ≥ 2 arrivals during [t, t + h) and n - x arrivals during (0, t).

These are three mutually exclusive and exhaustive possibilities (they cover all cases). They give:

Pn(t + h)= Pn(t)(1 - h - o(h)) + Pn-1(t) h + o(h)

= Pn(t)(1 - h) + Pn-1(t) h + o(h)

Rearranging and dividing by h gives:

04/22/23 13


Pn t+h( ) −Pn t( )h

=−λPn t( ) +λPn−1 t( )+o h( )h

, n≥1, t≥0

Taking the limit as h -> 0+, gives the (infinite) system of differential equations (actually claim the two-sided limit is justified):

Pn' t( ) =−λPn t( )+λPn−1 t( ), n≥1, t≥0.

Applying similar reasoning to the case n = 0, we have

P0' t( ) =−λP0 t( ), fort≥0.

A moment of thought allows us to conclude that P0(0) = 1 (there have been no arrivals at all), so the last equation comes to us complete with initial condition. Its unique solution is P0(t) =e-t. We can also observe that Pn(0) = 0 for all n ≥ 1. Using the just computed expression for P0(t) , we obtain:

P1'(t) = -P1(t) + e-t, P1(0) = 0.

04/22/23 14


This is a non-homogeneous equation and its solution involves a trick: first fine the general solution of the corresponding homogeneous equation:

P1'(t) = -P1(t) -> P1(t) = C1 e-t.

Replace the constant by a function of t: z(t)e-t, and attempt to determine z(t) by simply inserting into the original equation and seeing what happens:

z'(t) e-t - z(t) e-t = - z(t) e-t + e-t

z'(t) = -> z(t) = C1 + t

P1(t) = (C1 + t) e-t, P1(0) = 0.

Finally:

P1(t) = t e-t.

We can repeat the construction for P2(t), P3(t), … etc., and a final induction proof lets us conclude that:

04/22/23 15


Pn t( )=λt( )

n

n!e−λt

the probability of exactly n arrivals during (0, t). It is easy to verify that

Pn t( )n=0

∞∑ =λt( )

n

n!e−λt

n=0

∞∑ ≡1

From this, we can also compute the probabilities of at least N arrivals, or at most N arrivals, etc. Something we have not quite addressed, yet, is the meaning of . It seems to be an expected arrival rate, but we need to show it.

Let E(n, t) denote the expected number of arrivals during (0, t). From standard probability theory - and some elementary algebra - we have:

E n,t( ) = n⋅Pn t( )n=0

∞∑ = nλt( )

n

n!n=0

∞∑ e−λt =λt

so that is the expected number of arrivals in unit time, or the arrival rate. What is the relationship between and inter-arrival times?

04/22/23 16


Let (t) denote the number of arrivals during (0, t). The probability that the first arrival time, 1, satisfy 1 > t is given by

P(1 > t) = P((t + 0) - (0) = 0)= P0(t) = e-t.

If we assume that the number of arrivals over an interval depends only on the length of the interval, and not on when the interval occurs, we also have ( is the interval between first and second arrival):

P(2 > t| 1 = x) = P((t + x) - (x) = 0) = P0((t +x) - x) = P0(t)= e-t,

andP τi+1 >t | τjj=1

i∑ >x( )=P Λ t+x( )−Λ x( )=0( ) =P0 t( ) =e−λt

We conclude that the probability that any inter-arrival time is greater than t is given by G(t) = P( > t) = P0(t) = e-t. Inter-arrival times are thus IID random variables with cumulative probability distribution function F(t) = 1 - G(t) = 1 - e-t, and probability density function f(t) = e-t:

F t( ) = λe−λτ dτ0

t

∫ =−e−λτ

0

t

=1−e−λt

04/22/23 17


The expected inter-arrival time is thus given by

E τ( )= τ f τ( )0

∞

∫ dτ = τλe−λτ dτ0

∞

∫ =−τe−λτ

0

∞− −e−λτdτ

0

∞

∫=0+ e−λτdτ

0

∞

∫ =−1λ

e−λτ

0

∞=

1λ

Another interesting question - useful to help us choose theoretical distributions matching empirical data sets - is that of the position of the median relative to the mean.

Let be the random variable representing inter-arrival time.

P τ ≤1λ

⎛ ⎝

⎞ ⎠

=F1λ

⎛ ⎝

⎞ ⎠

=1−e−λ

1λ =1−e−1 ≈0.632121;

P τ >1λ

⎛ ⎝

⎞ ⎠ =1−F

1λ

⎛ ⎝

⎞ ⎠ =e−1 ≈0.367879.

A way of interpreting this result is that, for an exponential distribution, the median is less than the mean.

04/22/23 18


We can now ask the question: what is the probability pn(t) that an M/M/1 system with mean arrival rate and mean service rate will have n customers in the system at time t? Can we obtain a steady-state probability, i.e., does exist? We can easily see

limt→ ∞

pn t( )

pn t+Δt( ) =pn t( ) 1−μΔt( )1−λ Δt( ) +μΔtλΔt[ ]+o Δt( )

+pn+1 t( ) μΔt 1−λ Δt( )[ ]+o Δt( )

+pn−1 t( ) λ Δt1−μΔt( )[ ]+o Δt( ), n≥1.

For n = 0, the first term on the right hand must reflect a probability 0 for a departure, and the third term must be absent. Assuming smoothness in t for pn(t), we can use the approximation pn(t + t) = pn(t) + pn'(t)t. Replacing into the equation above, and letting t -> 0, we have

pn' t( ) =−λ +μ( )pn t( )+μpn+1 t( )+λ pn−1 t( )

04/22/23 19


A necessary condition for a steady-state solution is that pn'(t) vanish identically, leaving us with a time-independent system of infinitely many equations:

p0 = p1;

( + ) pn = pn+1 + pn-1, for n ≥ 1.

We recall that is what we called utilization. We can now proceed with, essentially, and induction:

p1 = p0;

p2 = ( + 1)p1 - p0 = 2p0;

….

pn = np0, for all n ≥ 0.

Since , we must have < 1. Summing the geometric series: p0 = (1 - ), pn = (1 - ) n. From = 1 - p0, we can conclude that is the probability the system is not empty.

pnn=0

∞∑ ≡ ρnp0 =0

∞∑ 1

04/22/23 20


The expected number of customers in the system is given by

E n( )= npn0

∞∑ = n 1−ρ( )ρn

0

∞∑ =ρ

1−ρ

An easy derivation of the closed form of the sum follows from observing that

11−ρ( )

2 =ddρ

11−ρ

⎛

⎝ ⎜ ⎞

⎠ ⎟ =

ddρ

ρn

n=0

∞∑( ) = nρn−1

n=0

∞∑ =1ρ

nρn

n=0

∞∑

System Saturation

0

20

40

60

80

100

120

0 0.2 0.4 0.6 0.8 1 1.2

Utilization

Series1

04/22/23 21


Note that with a utilization of = 0.5, the expected number of customers in the system is 1.

One can also show that, for a general service distribution with exponential arrivals, the steady-state average delay in queue is given by the formula

d =λ Var S( )+ E S( )[ ]

2{ }

2 1−λ E S( )[ ]

where S is the service distribution, and Var(S) is the variance of the service distribution.

We can go a little further in this direction by computing variance of the number of customers in the system, and the variance of an exponential distribution with mean 1/. - the service distributions we have considered up to this point are all exponential anyway (although they don't have to be: for ATM 53Byte packets, the service distribution is uniform).

04/22/23 22


Variance of the number of customers in the system.

σ 2 =E n2( )−E n( )

2= n2 pnn=0

∞∑ −E n( )2= 1−ρ( ) n2ρn

n=0

∞∑ −ρ2

1−ρ( )2

We use the same differentiation trick as before, but now differentiate twice:

2

1−ρ( )3 =

d2

dρ2

11−ρ

⎛

⎝ ⎜ ⎞

⎠ ⎟ =

d2

dρ2 ρnn=0

∞

∑( ) = n n−1( )ρn−2n=0

∞

∑

=1ρ2 n2 ρn

n=0

∞∑ −1ρ2 nρn

n=0

∞∑ =1ρ2 n2 ρn

n=0

∞∑ −1

ρ 1−ρ( )2

where we used the result obtained two slides ago. Some simple algebra gives:

n2ρn

n=0

∞∑ =ρ 1+ρ( )1−ρ( )

3

finally leading to:σ 2 =

ρ1−ρ( )

2

04/22/23 23


Variance of an exponential distribution with mean .

Recall that the exponential distribution with mean has probability density function f(t) = e-t. To compute the variance:

Var τ( ) =E τ2( )−E τ( )2 = τ2λe−λτ dτ0

∞

∫ −1λ

⎛ ⎝

⎞ ⎠

2

=−τ2e−λτ0∞| − 2τ −e−λτ

( )0

∞

∫ dτ−1λ

⎛ ⎝

⎞ ⎠

2

= 2τ e−λτ( )

0

∞

∫ dτ −1λ

⎛ ⎝

⎞ ⎠

2

=−2τ1λ

e−λτ0∞| − −2

1λ

e−λτ

0

∞

∫ dτ −1λ

⎛ ⎝

⎞ ⎠

2

=2λ

e−λτ

0

∞

∫ dτ−1λ

⎛ ⎝

⎞ ⎠

2

=−2λ2 e−λτ

0∞| −

1λ

⎛ ⎝

⎞ ⎠

2

=1λ

⎛ ⎝

⎞ ⎠

2

The variance is the square of the mean: another point to help us decide whether an empirical distribution is exponential or not.

04/22/23 24


Expected value and variance of the number of customers in queue.

We observe that when the number of customers in the system is 0, we will have 0 customers in the queue, when the number of customers in the system is 1, the number of customers in queue is still 0, while when the number of customers in the system is n > 1, the number of customers in queue is n - 1. This observation leads to the formula for the expected number of customers in queue:

E nQ( )= n−1( )pnn=1

∞∑where pn = (1 - )n, the probability of having n customers in the system. Making use of the summation formulae just derived, we can compute

E nQ( )=ρ2

1−ρ=ρE n( )

as well as E(n) = E(nQ) + .: the expected number of customers in system is exactly (the utilization) larger than the expected number in queue.

04/22/23 25


We can compute variance:

σQ2 = n−1( )

2pn −E n( )

2

n=1

∞∑ =ρ2 1+ρ−ρ2

( )1−ρ( )

2

making use of the summation formulae previously derived.

Mean Waiting and Mean Delay Time.

In some texts you will find "waiting time" to be defined as "response time" and "delay time" as "waiting time" (the rationale being that the delay time is the time you are waiting in queue - which is actual waiting - while the total time you spent is the time it takes the system to respond to your request, fulfilling it). Using our definitions, we have the relation:

Wi = Di + Si

Let denote the mean waiting time, let denote the mean delay time, let denote the mean number of customers in system, and let denote the mean number of customers in queue.

W D L Q

04/22/23 26


If 1/ is the mean interarrival interval, and 1/ is the mean service time, we have, by definition, . Little's Law (see slide 9) gives:

D =Q λ

=ρ2

λ 1−ρ( )=

ρμ 1−ρ( )

W =L λ

=ρ

λ 1−ρ( )=

1μ 1−ρ( )

What would this be good for? In the various simulations, you computed time-averaged delays and time-averaged waiting times. This provides you with a theoretical check: if your results don't match (within some statistical confidence level) the theoretically predicted ones, you are in some kind of trouble…

They also allow you to avoid - in some simple cases - going through the whole simulation process: a "back-of-the-napkin" computation can give you the results.

The material here is taken from:

Documents

Transcript of The material here is taken from: