modf

Module F - Markov Analysis 1

Module F - Markov Analysis

Module Topics

• The Characteristics of Markov Analysis

• The Transition Matrix

• Steady-State Probabilities

• Additional Examples of Markov Analysis

• Special Types of Transition Matrices

• Excel Solution of the Debt Example


Markov AnalysisOver view

- Markov analysis is a probabilistic technique.

- It provides information about a decision situation.

- It is a descriptive, not an optimizing technique.

- Specifically applicable to systems that exhibit probabilistic movements from one state (or condition) to another.


The Characteristics of Markov Analysis- The brand-switching problem used for discussion.

- The brand-switching problem analyses the probability of customers’ changing brands of a product over time.

- The state of a system is where the system is at a point in time.

- A transition probability is the probability of moving from one state to another during one period of time.

Table F-1 Probabilities of Customer Movement per Month


Properties of Markov Process Analysis

1. The transition probabilities for a given beginning state of the system sum to one.

2. The probabilities apply to all participants in the system.

3. The transition probabilities are constant over time.

4. The states are independent over time.


Markov Analysis Information (1 of 3)

- Information available from Markov analysis is the probability of being in a state at some future time.

- In example, the manager wants to know the probability that a customer would trade with them in month 3 given that the customer trades with them in month 1.

- Probability of a customer’s trading with Petroco in month 3 given that the customer initially traded with Petroco in month 1 is the sum of two branch probabilities associated with Petroco: .36 + .08 = .44.

- For probability of a customer’s purchsasing gasoline from National in month 3, add probabilities associated with National: .24 + .32 = .56.

Figure F-1Probabilities of future states given that a customer trades

with Petroco this month



- For each starting state, the probabilities of ending up in either state in month 3 sum to one.

Probability of Trade in Month 3

Starting State Petroco National Sum

Petroco National

.44

.28 .56 .72

1.00 1.00



- Given that a customer initially purchased gasoline from National, the probability of purchasing gasoline from National in month 3 is .08 + .64 = .72.

- The probability of the customer trading with Petroco in month 3 is .12 + .16 = .28.

Figure F-2Probabilities of future states given that a customer trades

with National this month


The Transition Matrix

- The probabilities of being in a particular state in the future can be determined by using matrix algebra.

- A transition matrix includes the transition probabilities for each state of nature.

First month Next month

Petroco National

T = Petroco

National

80.20.

40.60.


Symbology and Matrix Notation- Symbology

Probability of trading with Petroco in period i, given customer started with Petroco: Pp (i); Probability of trading with National in period i, given customer started with Petroco: Np (i).

- Probability of a customer’s trading at National in month 2, given that customer initially traded at Petroco is

Np(2)

- Probability of a customer’s trading with Petroco and National in a future period i, given that customer traded initially with national is

Pn(i) and Nn (i)

- If a customer is presently trading with Petroco, the following probabilities exist

Pp (1) = 1.0 and Np (1) = 0

- These probabilities in matrix form:

[Pp (1) Np (1)] = [1.0 0.0]


Matrix Multiplication

- Computing probabilities of a customer trading at either station in future months using matrix multiplication. In future time periods, the state probabilities become constant.

Month 2: [Pp(2) Np (2)] = [1.0 0.0]

= [.60 .40]

Month 3: [Pp (3) Np (3)] = [.60 .40]

= [.44 .56]

Month 4: [Pp (4) Np (4)] = [..44 .56]

= [.38 .62]

Month 5: [Pp (5) Np (5)] = [.35 .65]

Month 6: [Pp (6) Np (6)] = [.34 .66]

Month 7: [Pp (7) Np (7)] = [.34 .66]

Month 8: [Pp (8) Np (8)] = [.33 .67]

Month 9: [Pp (9) Np (9)] = [.33 .67]

80.20.

40.60.

80.20.

40.60.

80.20.

40.60.


Future State Probabilities- State probabilities in future time periods:

[Pp(i) Np (i)] = [.33 .67]

Figure F-3The probability Pp(i) for future

values of i


Computing Future State Probabilities- The probability of ending up in a state in the future is independent of the starting state.

- Computing future state probabilities when initial starting state is National:

Month 1: [Pn(1) Nn (1)] = [0.0 1.0]

Month 2: [Pn (2) Nn (2)] = [0.0 1.0] = [.20 .80]

Month 3: [Pn (3) Nn (3)] = [.20 .80] = [.28 .72]

Month 4: [Pn (4) Nn (4)] = [.31 .69]

Month 5: [Pn (5) Nn (5)] = [.32 .68]

Month 6: [Pn (6) Nn (6)] = [.33 .67]

Month 7: [Pn (7) Nn (7)] = [.33 .67]

Month 8: [Pn (8) Nn (8)] = [.33 .67]

Month 9: [Pn (9) Nn (9)] = [.33 .67]

80.20.

40.60.

80.20.

40.60.


Steady-State Probabilities

- Steady-state probabilities are average, constant probabilities that the system will be in a state in the future.

- For service station example:

.33 = probability of a customer’s trading at Petroco after a number of months regrdless of where customer traded in month one.

.67 = probability of a customer’s trading at National after a number of months regardless of where customer traded in month one.


Steady-State Probabilities(continued)

- Combining operations into one matrix:

67.33.

67.33.

Nn(9)Pn(9)

Np(9)Pp(9) : 9Month

.69.31

.62.38

.80.20

.40.60

.72.28

.56.44

Nn(4)Pn(4)

Np(4)Pp(4) :4Month

80.20.

40.60.

.80.20

.40.60

Nn(3)Pn(3)

Np(3)Pp(3) :3Month

.80.20

.40.60

.80.20

.40.60

10

01

Nn(2)Pn(2)

Np(2)Pp(2) :2Month


Direct Algebraic Determination of Steady-State Probabilities

- At some point in the future the state probabilities remain constant from period to period.

- After steady state is reached it is not necessary to designate the time period.

-Steady state probabilities can be computed by developing a set of equations using matrix operations and solving them simultaneously:

[Pp Np] = [Pp Np]

Pp = .6Pp + .2Np

Np = .4Pp + .8Np

Pp + Np = 1.0

Np = 1.0 - Pp

Pp = .6Pp + .2(1.0 - Pp) = .6Pp+ .2 - .2Pp = .2 + .4Pp

..6Pp = .2

Pp = .2/.6 = .33

Np = 1.0 - Pp = 1.0 -.33 = .67

[Pp Np] = [.33 .67]

80.20.

.40.60


Application of the Steady-State Probabilities

(1 of 2)

- Steady-state probabilities can be multiplied by the total system participants to determine the expected number in each state in the future.

- Example: percentage of customers who will trade at a service station during any given month in the long run given that there are 3,000 customers in the community who purchase gasoline:

Petroco: Pp(3,000) = .33(3,000)

= 990 customers

National: Np(3,000) = .67(3,000)

= 2,010 customers


Application of the Steady-State Probabilities (2 of 2)

- Re-evalution with modified transition probabilities resulting from Petroco improving its service.

- Re-evalution indicates Petroco will get 1,200 customers in any given month in the long run, increasing its customer base by 210 customers (1,200 - 990).

- Management must evaluate trade-off of cost of improved service and increase of 210. customers.

- Analysis with new transition probabilities:

[Pp Np] = [Pp Np]

solving, Pp = .2/.5 = .4

and thus, Np = 1- Pp = 1 - .4 = .6

.80.20

.30.70


Additional Examples of Markov AnalysisMachine Breakdowns

- Machine with daily transition matrix:

Day 1 Day 2

Operate Breakdown

T = Operate

Breakdown

.88 = steady-state probability of the machine’s operating

.12 = steady-state probability of the machine’s breaking down

- Management must decide if it should decrease breakdown probability and if cost of doing so is covered by profit achieved by resulting increased production.

.30.70

.10.90


Additional Examples of Markov AnalysisTruck Rental Firm

- Rents in three states; trucks are rented on a daily basis and can be rented and returned in any of the three states.

- Transition matrix: V M NC

Virginia

Maryland

North Carolina

- Steady-state probabilities: Virginia Maryland North Carolina

[.471 .244 .285]

- Given total fleet of 200 trucks, long run expectation of number of trucks in each state:

[ 94 49 57 ]

50.10.40.

20.50.30.

20.20.60.


Special Types of Transition Matrices- Once the system leaves a transient state, it will never return (state 3 in example):

1 2 3

1

T = 2

3

- A transition matrix is cyclic when the system moves back and forth between states:

1 2

T = 1

2

- Once the system moves into an absorbing state, it is trapped and cannot leave (state 3 in example): 1 2 3

1

T = 2

3

001.0

0.70.30

0.60.40

00.1

0.10

1.000

.20.40.40

.10.60.30


The Debt Example(1 of 6)

- Example of absorbing state matrix.

- States are the months during which customer carries a debt.

- States: p = debt paid, 1 = debt held 1 month, 2 = debt held 2 months, b = “bad” debt.

- p and b are absorbing states.

- Transition matrix for A to Z Office Supply Company accounts receivable:

p 1 2 b

p

T = 1

2

b

1000

.5000.50

0.300.70

0001



- Rearrange the transition matrix:

p b 1 2

p

T = b

1

2

- Label submatrices:

T =

0050.50.

30.0070.

0010

0001

QR

0I



- Form identity matrix, I =

- Matrix of zeros, 0 =

- Transition probabilities of the debt’s being absorbed in next period,

R =

- Transition probabilities for movement between both nonabsorbing states,

Q =

10

01

00

00

50.50.

070.

00

.300



- The fundamental matrix, F, indicates the expected number of times the system will be in any of the nonabsorbing states before absorbtion occurs.

- Determine fundamental matrix, F:

F = (I - Q)-1

- Determine F by taking inverse of the difference between identity matrix, I, and Q:

F =

10

30.1



- The F R matrix reflects the probability that the debt will eventually be absorbed given any starting state.

- Calculating the the F R matrix:

F R =

=

10

30.1

.50.50

0.70

.50.50

.15.85



- Determine what portions of these funds will be collected and what will result in bad debts given that company has accounts receivable of $4,000 in month 1 and $6,000 in month 2.

Determination of accounts receivable = [4,000 6,000]

= [6,400 3,600]

- Of $10,000 owed, bank can expect to receive $6,400 and $3,600 will become bad debt.

.50.50

.15.85


Assignment • Problems 6, 12,18,24,25,27, and

29

modf

Documents

Transcript of modf