Markov Process and Excel Spreadsheets
A Computational Adventure
Our bags are packed, we are ready to go…
Three Great Workbooks
Equation Solversolves system of linear equations using matrix inversionUse to solve systems of linear equations, find a matrix inverse, or compute a determinant
MatrixPowerExcel workbook for raising a matrix to higher powersGreat for finding the n-step transition matrix P(n)
AbsorptionExcel workbook for absorption type Markov processesFind the fundamental matrix F and the absorption probabilities FR
Go the course website homepage and select Software then download the filesSoftware CommentsLindo/What's Best/LINGO solves linear, integer, and nonlinear programs
Equation Solver solves system of linear equations using matrix inversion
Frontline Systems Linear, Integer, and Nonlinear Optimization (Excel Solver enhancements)
Excel Solver Excel Add-on solves LP, IP, and NLP problems
Courseware (Text CD ) AMPL -solves linear, integer, and nonlinear programs; TORA - optimization system
Win QSB OR Software from Wiley $84.95Queue Models.xls Excel workbook for solving M/M/s queues
MatrixPower.xls Excel workbook for raising a matrix to higher powers
Absorption.xls Excel workbook for absorption type Markov processes
Queue.xls Alternate queuing spreadsheet for M/M/s and M/G/1 queues
Solving the Steady-State Equations
( ) ( )1 2 3 4 1 2 3 4
.2 .4 0 .4
.1 .1 .5 .30 .3 .3 .4.2 0 .7 .1
.2 .4 0 .4
.1 .1 .5 .30 .3 .3 .4.2 0 .7 .1
P
π π π π π π π π
⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦
A common everyday transition matrixthat is irreducible + ergodic
The algebra
( ) ( )1 2 3 4 1 2 3 4
1 1 2 4
2 1 2 3
3 2 3 4
1 2 3 4
.2 .4 0 .4
.1 .1 .5 .30 .3 .3 .4.2 0 .7 .1
.2 .1 .2.4 .1 .3.5 .3 .7
1
π π π π π π π π
π π π ππ π π ππ π π ππ π π π
⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦
= + += + += + ++ + + = 1 2 4
1 2 3
2 3 4
1 2 3 4
.8 .1 .2 0.4 .9 .3 0
.5 .7 .7 01
π π ππ π π
π π ππ π π π
− − =− + − =
− + − =+ + + =
Carefully enter data into the spreadsheet – Equation_Solver…
Instructions: Enter data including matrix size N then Run Macro "Inverse"
To run Macro enter ALT+F8 or select Tools then Macro from the menu bar.
1 2 4
1 2 3
2 3 4
1 2 3 4
.8 .1 .2 0.4 .9 .3 0
.5 .7 .7 01
π π ππ π π
π π ππ π π π
− − =− + − =
− + − =+ + + =
With great skill, execute the inverse macro… ( )1 2 3 4π π π π
The Absorption State Markov Process
.1 .2 .5 .1 .1
.2 .3 .2 .1 .20 .4 .1 .3 .20 0 0 1 00 0 0 0 1
.1 .1 .1 .2 .5
.1 .2 .2 .3 .2
.3 .2 0 .4 .1
P
R Q
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
I would like to work this problem for you Chuck. However, my
hands are tied.
With assured (untied) hands, enter the R and Q matrices into absorptions.xls
.1 .1 .1 .2 .5
.1 .2 .2 .3 .2
.3 .2 0 .4 .1R Q
⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥= =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦
Absorption Type Markov ProcessEnter Q matrix - Transient States to Transient States
Dim = 3row/col 1 2 3 4 5 6 7 8 9
1 0.1 0.2 0.52 0.2 0.3 0.23 0 0.4 0.14
Entering the R matrix…
Now Run the Macro called Compute
Enter R matrix - Transient to Absorption StatesNbr cols
= 2row/col 1 2 3 4 5 6 7 8 9
1 0.1 0.12 0.1 0.23 0.3 0.24
Look at the results….Fundamental matrix (I-Q)^-1 - Expected Nbr Transitions in Each Transient State
row/col 1 2 3 4 19 20 sum1 1.313 0.907 0.931 3.15042 0.43 1.933 0.668 3.03103 0.191 0.859 1.408 2.45824 0.00005 0.0000
FR matrix - Probability of absorption
row/col1 0.501 0.4992 0.437 0.5633 0.527 0.4734
I still like to help you with this one Chuck
but I am all tied up at the moment.
This sounds great but what if I need the n-step
transition matrix?
The n-step transition matrixusing MatrixPower!
.1 .2 .5 .1 .1
.2 .3 .2 .1 .20 .4 .1 .3 .20 0 0 1 00 0 0 0 1
P
⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦
Let’s do that again…
Matrix Multiplication – the Excel way… MMULT(B3:D5,F3:G5)
Note The formula in the example must be entered as an array formula. Select the range of cells for the resulting matrix starting with the formula cell. Press F2, and then press CTRL+SHIFT+ENTER.
But what if we do not want to use your homegrown
spreadsheets. There must be alternatives?
Students may use one of the matrix calculators found on the internet, purchase WinQSB or another OR software package, program Excel
themselves, or use a calculator that will perform matrix operations.
The Bottom Line
I like to do matrix
operations in my head.
Top Related