s to Ch Workshop 2004

8/13/2019 s to Ch Workshop 2004

1/26

AUSTRALIANRESEARCH COUNCILCentre of Excellence for Mathematicsand Statistics of Complex Systems

Quasi-Birth-and-Death Processes

Peter Taylor

Department of Mathematics and Statistics,

The University of Melbourne


2/26


The M/M/1 queue

0 1 2

Figure 1: Transition structure for an M/M/1 queue


3/26


The M/M/1 queue can be modelled by a continuous-timeMarkov chain with state spaceZ+={0, 1, . . .}and transitionrates

q(n, n + 1) = , n0

q(n, n 1) = , n1.

So the transition matrix is

Q=

0 0

( + ) 0 0 ( + )

0 0 ( + ) .

..

.

..

.

..

.

.. .

. .

.


4/26


A Quasi-Birth-and-Death Process

.. . . M

0

1

1

2

2

3 4

i

i + 1

i 1

phase

l

l

e

ev


5/26


With a suitable ordering of the states, the transition matrix of aQBD has the block partitioned form

Q=

Q1 Q0 0 0

Q2 Q1 Q0 0

0 Q2 Q1 Q0 0 0 Q2 Q1 ...

... ...

... . . .

.

When the QBD is in state(k, i)we say that it is inlevelkandphasei.


6/26


Examples

An M/M/1 queue in a random environment.

A Ph/M/1 queue.

An M/Ph/1 queue.

MAP/Ph/1 queues.

A model for a single cell in a PCS network with dynamicchannel assignment (Lipper and Rumsewicz).

Retrial models.

Etc....


7/26


The transition matrix

Q=

Q1 Q0 0 0 Q2 Q1 Q0 0

0 Q2 Q1 Q0

0 0 Q2 Q1 ... ...

... ...

. . .


8/26


looks like a block version of the transition matrix

Q=

0 0 ( + ) 0

0 ( + )

0 0 ( + ) ... ...

... ...

. . .

for the M/M/1 queue, so it is reasonable to ask whatproperties carry over from the M/M/1 queue in a block sense"to a QBD.


9/26


Stability

The single server queue is stable if and only if < .Let xbe the solution to

x[Q0+ Q1+ Q2] = 0.

Then the QBD is positive recurrent if

xQ0e < xQ2e

,

null recurrent if

xQ0e

= xQ2e

and transient if

xQ0e > xQ2e

.


10/26


The Stationary Distribution

The stationary distribution for a stable M/M/1 queue is derivedby solving the system of second order difference equations

(n)( + ) =(n 1) + (n + 1),

forn1and

(0)=(1).

The characteristic equation is

x2

x( + ) + ,

which has roots1and=/. Thus a summable solution for(n)exists only if


11/26


Now lets think about the QBD. Assume it is positive recurrentand write the stationary distribution as = (0,1, . . .).

Then the equations for the stationary distribution are

n1Q0+ nQ1+ n+1Q2 = 0

forn1and0

Q1+ 1Q2= 0.


12/26


If we happen to be able to find a nonnegative matrix Rand apositive vector x0 such that

Q0+ RQ1+ R2Q2 = 0,

x0

Q1+ RQ2

= 0,

and

x0

k=0

Rke= x0[I R]

1e= 1,

then the stationary distribution would be given by

n = x0Rn.


13/26


We proceed by using the fact that a matrix with a physicalinterpretation has the required properties.

Leti be the mean sojourn time in phaseiof levelk. DefineRto be the matrix whose(i, j)thentry is1/i times the expectedsojourn time in phasej of levelk+ 1before first return to level

k conditional on the process starting in phase iof levelk.

Then, some analysis gives us

Q0+ RQ1+ R2Q2 = 0.


14/26


The physical interpretation also gives us the fact that the

(i, j)thentry ofR is1/i times the expected sojourn time in

phasej of levelk+ before first return to levelk conditionalon the process starting in phaseiof levelk.

Under the assumption that the QBD is positive recurrent, we

know that the total expected time spent in higher levels beforethe process returns to levelkmust be finite, and so

=0

Re

is finite.


15/26


Finally, some more analysis tells us that, under the

assumption of positive recurrence, the matrix Q1+ RQ2 is an

irreducible generator matrix and so there must be a positivesolution x0 to

x0

Q1+ RQ2

= 0.

Thus, the matrixRthat we defined above satisfies all theconditions we need. We have proved that the QBD has thematrix-geometric stationary distribution

n = x0Rn.


16/26


How do we calculate R?

The matrixRis, in fact, the minimal non-negative solutiontothe matrix quadratic equation

Q0+ RQ1+ R2Q2 = 0.

There are a number of good algorithms for deriving this

solution.

I shall discuss these in the final part of this presentation.


17/26


A variation

Sometimes the QBD has the block-partitioned form

Q=

Q1 Q0 0 0 Q2 Q1 Q0 0

0 Q2 Q1 Q0

0 0 Q2 Q1 ... ...

... ...

. . .

.


18/26


In this case the stationary distribution has the form

n =

x0

RR

n1

where

R=Q0[Q1+ RQ2]1

and x0 satisfies

x0

Q1+ RQ2

= 0.


19/26


Hitting Probabilities

Lets assume that the QBD starts in phase iof levelk. We areoften interested in calculating the probability that it hits level

k1in finite time, and does so in phase j. That is we areinterested in thehitting probabilitieson lower levels.

Let us store these probabilities in a matrix G. Thus, the(i, j)th

entry of the matrixGis the probability that the QBD will firstenter levelk1in phasej given that it starts in phase ioflevelk.


20/26


The probabilitiesGij are the same as for the discrete-time

QBD observed only at jump times. This has a transition matrix

of the form

P =

D1 D0 0 0

D2 D1 D0 0

0 D2 D1 D0

0 0 D2 D1 ...

... ...

... . . .

.


21/26


TheDi can be written in terms of theQi via the relations

D0 =

1

Q0D1 =

1Q1+ I

D2 = 1Q2

whereis a diagonal matrix with entries1/(i). Also

D1= 1Q1+ I

where is a diagonal matrix whose entries are the negative

of the diagonal entries of Q1.


22/26


Now if the process starts in state(k, i), then there are threepossibilities for a path that will first hit level k1in state

(k1, j), it can go straight to state(k1, j), with probability[D2]ij ,

it can go to a state of the form state (k, ), and then

eventually hit levelk1in state(k1, j), with probability[D1]iGj , or

it can go to a state of the form state (k+ 1, m),subsequently hit levelk in state(k, )and then eventuallyhit levelk1in state(k1, j), with probability[D0]imGm,Gj .


23/26


Taking these possibilities into account, we derive the equation

G = D2+ D1G + D0G

2

= 1Q2+

1Q1+ I

G + 1Q0G2

and so

Q2+ Q1G + Q0G2 = 0.

As withR, we are looking for the minimal nonnegativesolution to this equation.


24/26


Some extensions

Processes with transition matrices of the form

Q=

Q1 Q0 0 0 Q2 Q1 Q0 0

Q3 Q2 Q1 Q0

Q4

Q3

Q2

Q1

... ...

... ...

. . .

are called Markov chains of GI/M/1 type.


25/26


Processes with transition matrices of the form

Q=

Q1

Q2

Q3

Q4

Q0 Q1 Q2 Q3

0 Q0 Q1 Q2

0 0 Q0 Q1 ...

... ...

... . . .

are called Markov chains of M/G/1 type.


26/26


Markov chains of GI/M/1 type and of M/G/1 type can beanalysed using matrix-analytic methods.

Like QBDs, Markov chains of GI/M/1 type havematrix-geometric stationary distributions.

Analysis of truncated and level-dependent versions of theseprocesses is also possible.

s to Ch Workshop 2004

Documents

Transcript of s to Ch Workshop 2004