MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.
-
Upload
ethan-jordan -
Category
Documents
-
view
221 -
download
0
description
Transcript of MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.
![Page 1: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/1.jpg)
MA3264 Mathematical ModellingLecture 5
Discrete Probabilistic Modelling
![Page 2: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/2.jpg)
Markov Chains
vertices represent states and are labeled 1,..,K
Can be illustrated using labeled directed graphs
edges represent transitions between states and are labeled by numbers in the interval [0,1]
Page 217, Figure 6.1
1 2
p1
q1
qp
Transition Matrix T=
qpqp
11
NOTE Our TM = transpose book’s TM
![Page 3: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/3.jpg)
Transition Probabilities
The label on a directed edge from vertex j to vertex i is the probability that if the system is in state j at time n then it will be in state i at time n+1 FOR EVERY n (in other words these so called transition probabilities (TP) do not change over time), for convenience we place these TP into a transition matrix T
Imagine a dynamic* system that can be in one of K states during each time interval [n,n+1)
1 2
p1
q1
qp
qpqp
11
* marked by usually continuous and productive activity or change
![Page 4: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/4.jpg)
Transition ProbabilitiesFor the situation displayed in the graph below, if a system is in state 1 at time n = 1, then the probability that the system will change to be is state 2 at time n = 2 equals 1-p and the probability that the system will stay the same to be in state 1 at time n = 2 equals p, similar considerations apply if the system is in state 2 at time n = 1
1 2
p1
q1
qp
qpqp
11
The sum of the elements in each column of T equals 1
![Page 5: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/5.jpg)
State VectorsIf at time n, the probability of a system being in state 1 equals a(n) and the probability of being in state 2 equals b(n) then this may be represented by a state vector
)()(
)(nbna
nv
Clearly the entries in this matrix are in [0,1] and their sum equals 1
![Page 6: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/6.jpg)
Dynamics of State Vectors
The dynamics of this vector are derived from probability
)()()()(
)1(2221
1211
nbTnaTnbTnaT
nv
)()()(
2221
1211 nTvnbna
TTTT
![Page 7: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/7.jpg)
Law of Large NumbersIf we have two (or more) large populations of individuals, each of whom can be in state 1 or state 2 at each time n, then a(n), b(n) can be interpreted as the frequency (or fractions) of individuals that are in state 1, 2 at time n
Likewise, if a state vector has entries that represent frequencies, then these frequencies can be interpreted as probabilities of an individual who is chosen randomly to be in state 1, 2 at time n
Likewise, the entries of a transition matrix can be either interpreted as probabilities or as frequencies
This dual interpretation aspect can be initially confusing but becomes much more obvious through applications
![Page 8: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/8.jpg)
Rental Car ApplicationExample 1. Rental Car Company (pages 217-218)
1 24.
3.
7.6.
7.4.3.6.
T
If we let nn qp , the fraction of cars in Orlando, Tampa
at time n, then
nnn qpp 3.6.1
nnn qpq 7.4.1
n
n
n
n
qp
Tqp
1
1
11
OrlandoTampa
![Page 9: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/9.jpg)
Long Term Behavior
Can we find the long-term behavior ?
7.4.3.6.
T
0
0
2
22
1
1
qp
Tqp
Tqp
Tqp n
n
n
n
n
n
n
This matrix notation gives n-step transitions
61.52.39.48.2T
583.556.417.444.3T
5749.5668.4251.4332.4T
5717.5710.4283.4290.6T
![Page 10: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/10.jpg)
Long Term Behavior
If 1,lim
ba
bbaa
T k
kthen
bbaa
bbaa
T
baba
ba
T7.4.3.6.
ba
ba
T
5714.4286.
ba
![Page 11: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/11.jpg)
MATLAB Experiment>> T = [.6 .3;.4 .7]
T =
0.6000 0.3000 0.4000 0.7000
>> for k = 1:20Tk = T^k;a1(k) = Tk(1,1);b1(k) = Tk(2,1);a2(k) = Tk(1,2);b2(k) = Tk(2,2);End
>> plot([a1' a2' b1' b2'])
![Page 12: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/12.jpg)
Radioactive DecayModel decay of an atom of Polonium 209 to Lead 205after its half life (=102 years), its state vector evolves as
1 2
21
0
121
Transition Matrix T=
10
2121
1. Po209, 2. Pb205
1615161
8781
4341
2121
01
10
10
![Page 13: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/13.jpg)
Radioactive DecayState vector dynamics of # Po209 atoms remaining(after 102 years)
1 2
41
1
1000
21
41
21
2141
T
1. two Po209 2. one Po209 3. zero Po209
3214
1
21
1
21
![Page 14: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/14.jpg)
Traffic Light
With fixed transition times of one minute
1 2 1
010001100
T
1. red 2. green 3. yellow
31 1
1
001
100
010
001
100
010
001
![Page 15: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/15.jpg)
Convergence CriteriaTheorem (Perron-Frobenius) If T is an n x n transition matrix then the following three statements are true
]111[v
ST kk
lim(ii)
(i) 1 is an eigenvalue of T since vT = v where
exists (converges) if and only if all other
(iii) If there exists an integer p > 0 such that every entry of the matrix is positive then the if condition in (ii) is satisfied and every entry of w is positive.
PT
eigenvalues have modulus < 1,then the right eigenvector w (Tw = w) for eigenvalue 1 (normalized so the sum of the entries of w equals 1), is the steady state of T and each column of S equals w
![Page 16: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/16.jpg)
Numerical Examples
7.4.3.6.
Matrix Eigenvalues Steady State Vector
15.05.
4.2.2.2.6.05.4.2.75.
1,3.
5714.4286.
1,5.
10
1,55.,2.
2500.1944.5556.
![Page 17: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/17.jpg)
Suggested Reading&Problems in Textbook
6.1 Probabilistic Modelling with Discrete Systems, pages 217-222
http://aix1.uottawa.ca/~jkhoury/markov.htm
Recommended Websites
http://en.wikipedia.org/wiki/Markov_chain
http://www.eng.buffalo.edu/~kofke/applets/MarkovApplet1.html
![Page 18: MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.](https://reader036.fdocuments.in/reader036/viewer/2022082501/5a4d1afc7f8b9ab059984195/html5/thumbnails/18.jpg)
Tutorial 5 Due Week 29 Sept – 3 Oct
Page 222. Problem 1.
Page 222. Problems 2
Page 222-223. Project Problem 1