Matrix Analytic methods in Markov Modelling
description
Transcript of Matrix Analytic methods in Markov Modelling
Matrix Analytic methods inMarkov Modelling
Continous Time Markov Models
• X: R -> X µ Z (integers)• X(t): state at time t• X: state space (discrete – countable)• R: real numbers (continuous time line)
Continous Time Markov Models
• X is piecewise constant• X is typically cadlag ("continue à droite, limite
à gauche")= RCLL (“right continuous with left limits”)
Transition probabilities
• P(X(t+h)=j|X(t)=i) ¼ h ¸ij for i j
• P(X(t+h)=j|X(t)=j) = 1- i j P(X(t+h)=i|X(t)=j) ¼ 1- hi j ¸ji
• P(X(t+h)=j)= i P(X(t+h)=j and X(h)=i)= i P(X(t+h)=j | X(h)=i) P(X(h)=i)= i j ¸ij h P(X(h)=i) + h(1- i j ¸ji) P(X(h)=j)
Transition probabilities
• P(X(t+h)=j)=i P(X(t+h)=j and X(h)=i)+ (1- k j ¸ik) P(X(h)=j)• P(t)=[P(X(t)=0) P(X(t)=1) P(X(t)=2) ..]• P(t+h) ¼ P(t)H• H=I+hQ• Qij=¸ij• Qjj= 1- i j ¸ji
Taking limits
• P(t+h) ¼ P(t)H• H=I+hQ• P(t+h) ¼ P(t)(I+hQ)=P(t)+hP(t)Q• (P(t+h)-P(t))/h ¼ P(t)Q• d/dt P(t) = P(t)Q• P(t)=P(0)exp(Qt)
Irreducibility
• X is irreducible when states are mutually reachable.
• X is irreducible iff for every i,j 2 X there is a sequence {i(1),i(2),..,I(N) 2 X} such that i(1)=i i(N)=j and ¸i(k),i(k+1) > 0 for every k 2 1,..,N-1
Recurrence• Assume X(t n -)=j and X(tn)=i j then tn is a transition time• Let {tn} be the sequence of consequetive transition times.• {Xn=X(tn)} is called the embedded chain• Let t0=0, X0=i then
k(i)=inf{k>0: X(k)=i}=inf{k>1: X(k)=i}
• Ti=tk(i)
• Ti is the time to next visit at i• X is recurrent if P(Ti<1)=1 for all i
(almost certain return to all states)• X is positive recurrent if E(Ti) · 1 for all i
Stationary probability
• d/dt P(t) = P(t)Q• P(t)=P(0)exp(Qt)• When X is irreducible and positive recurrent
there is a unique probability vector ¦ such that P(t) -> ¦
• ¦ solves ¦ Q=0
Stationary probability
• Ergodicity: ¦i = E(Di)/E(Ti)• ¦i is the fraction of the time in state i• Statistically intuitively appealing
X(t) ==i
Ti
Ditime
ExamplePoisson Counting Process
• Qi,i+1=¸
¸ ¸ ¸ ¸10 2 3
• Counts Poisson events• Birth Chain• Not irreducible• Not recurrent
ExampleBirth/Death(BD)-chain
• Qi,i+1=¸• Qi+1,i=¹
¸ ¸ ¸ ¸10 2 3
• Models a queueing system with Poisson arrival process and independent exponentially distributed service times
• ¸ is arrival rate• ¹ is service rate• Irreducible• Positive recurrent for ¸<¹
¹ ¹ ¹ ¹
ExampleBirth/Death(BD)-chain ¸ ¸ ¸ ¸10 2 3
• ½ = ¸/¹• ¦n=½¦n-1
• ¦n=½n ¦0• P0=1/n=0
1 ½n =1-½• ¦n=½n (1-½)
• E(X) = n=01 n ¦n = ½/(1-½)
¹ ¹ ¹ ¹
ExampleBirth/Death(BD)-chain ¸1 ¸2 ¸3 ¸410 2 3
• ½n = ¸n/¹n
• ¦n=½n ¦n-1
• ¦n=¦i=1n ½i ¦0
• P0=1/n=01 ¦i=1
n ½i
¹1 ¹2 ¹3 ¹4
Markov Modulated Poisson Process
• Has two modes: Modes={ON,OFF}• M(t) 2 Modes is a two state CTMC• Transmits with rate ¸ in ON mode.• Counting proces combines state spaces, i.e.
X = Modes £ {0,1,2,..}• Q(ON,i),(ON,i+1)=¸• Q(ON,i),(OFF,i)=¯• Q(OFF,i),(ON,i)=®• Qi,j=0 otherwise
Markov Modulated Poisson Process
• Q(ON,i),(ON,i+1)=¸• Q(ON,i),(OFF,i)=¯• Q(OFF,i),(ON,i)=®• Qi,j=0 otherwise
1OFF 2 3
¸ ¸ ¸ ¸10 2 3
0
ON
®¯ ® ® ®¯ ¯ ¯
MMPP with exponential service• Q(ON,i),(ON,i+1)=¸• Q(ON,i),(OFF,i)=¯• Q(OFF,i),(ON,i)=®• Q(ON,i),(ON,i-1)=¹• Q(OFF,i),(OFF,i-1)=¹• Qi,j=0 otherwise
1OFF 2 3
¸ ¸ ¸ ¸10 2 3
0
ON
®¯ ® ® ®¯ ¯ ¯
¹ ¹ ¹ ¹
¹¹¹¹
State ordering
• For a state (i,M) we denote i the level of the state
• We order states so that equal levels are gathered
• (i,OFF),(i,ON)(i+1,OFF)(i+1,ON)(i+2,OFF)(i+2,ON)
Generator matrix
)()(
)()(
)()(
)()(
)(
Q
Sub matrices
)(0
A
)()(
A
B
C
Generator matrix by submatrices
ABCAB
CABCAB
CA
Q
0
.....3210 PPPPP
),(),( ONiPOFFiPPi
Balance equations:
P0 A0 + P1 B = 0 eq(0)P0 C + P1 A + P2 B = 0 eq(1)Pi C + Pi+1 A + Pi+2 B = 0 eq(i+1)
We look for a matrix geometric solution, i.e.P1=P0 RP i+1= Pi R
Inserting in eq(0):P0 A0 + P0 R B = 0
and eq(i+1)Pi(C+R A + R2B)=0 for all Pi
Solving for R
• Pi (C+R A + R2 B)=0 for all Pi
• Sufficient that C+R A + R2 B=0 (Ricatti equation)
• Iterative solutionR0=0repeatRn+1=-(C+Rn
2 B) A-1
• Converges for irreducible positive recurrent Q
MM - service• Q(ON,i),(ON,i+1)=¸• Q(OFF,i),(OFF,i+1)=¸• Q(ON,i),(OFF,i)=¯• Q(OFF,i),(ON,i)=®• Q(ON,i),(ON,i-1)=¹• Q(OFF,i),(OFF,i-1)=0• Qi,j=0 otherwise
1OFF 2 3
¸ ¸ ¸ ¸10 2 3
0
ON
®¯ ® ® ®¯ ¯ ¯
¹ ¹ ¹ ¹
¸ ¸ ¸ ¸
Generator matrix
)()(
)()(
)()(
)()(
)()(
Q
Sub matrices
)()(
0 A
)()(
A
B
C
Generally
12344
01233
0122
011
00
AAAABAAAAB
AAABAAB
AB
Q
We still look for a matrix geometric solution:
Now: i=0 1 Ri Ai = 0 A0 + R A1 i=2 1 Ri Ai
Iteration: Rn = -A1
-1 (A0 + i=21 Rn-1
i Ai)
Solving for P0:P0 i=0
1 Ri Bi = 0
Conditions for solution:Irreducibility and pos. recurrence
Miniproject (i)• Let traffic be generated by an on/off Markov process with
on rate: ¸=1, mean rate 0.1 ¸, average on time: T=1• Let service be exponential with rate ¹ = 0.2¸• Construct the generator matrix for the data given above.• Use the iterative algoritme to solve for the load matrix R• Solve for P0 and Pi
• Compute the mean queue length• Compare with M/M/1 results
Miniproject (ii)
• Collect file size or web session duration data• Check for power tails and estimate tail power• Find appropriate parameters for a
hyperexponential approximation of the reliability estimated reliability
• Construct the generator matrix of an equivalent ME/M/1 queue