Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week....
Transcript of Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week....
![Page 1: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/1.jpg)
Biostatistics-Lecture 4
Ruibin Xi
Peking University
School of Mathematical Sciences
![Page 2: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/2.jpg)
2
Markov Models • A discrete (finite) system:
– N distinct states.
– Begins (at time t=1) in some initial state(s).
– At each time step (t=1,2,…) the system moves
from current to next state (possibly the same as
the current state) according to transition
probabilities associated with current state.
• This kind of system is called a finite, or discrete Markov model
• After Andrei Andreyevich Markov (1856 -1922)
![Page 3: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/3.jpg)
3
Outline
• Markov Chains (Markov Models)
• Hidden Markov Chains (HMMs)
• Algorithmic Questions
• Biological Relevance
![Page 4: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/4.jpg)
4
Discrete Markov Model: Example • Discrete Markov
Model with 5 states.
• Each aij represents the probability of moving from state i to state j
• The aij are given in a matrix A = {aij}
• The probability to start in a given state i is pi , The vector p repre-sents these start probabilities.
![Page 5: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/5.jpg)
5
Markov Property • Markov Property: The state of the system at time t+1
depends only on the state of the system at time t
Xt=1 Xt=2 Xt=3 Xt=4 Xt=5
] x X | x P[X
] x X , x X , . . . , x X , x X | x P[X
tt11t
00111-t1-ttt11t
t
t
![Page 6: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/6.jpg)
6
Markov Chains
Stationarity Assumption
Probabilities independent of t when process is “stationary”
So,
This means that if system is in state i, the probability that
the system will next move to state j is pij , no matter what
the value of t is
t 1 j t i ijfor all t, P[X x |X x ] p
![Page 7: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/7.jpg)
7
• raining today rain tomorrow prr = 0.4
• raining today no rain tomorrow prn = 0.6
• no raining today rain tomorrow pnr = 0.2
• no raining today no rain tomorrow prr = 0.8
Simple Minded Weather Example
![Page 8: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/8.jpg)
8
Simple Minded Weather Example Transition matrix for our example
• Note that rows sum to 1
• Such a matrix is called a Stochastic Matrix
8.02.0
6.04.0P
![Page 9: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/9.jpg)
9
Coke vs. Pepsi
Given that a person’s last cola purchase was Coke ™,
there is a 90% chance that her next cola purchase will
also be Coke ™.
If that person’s last cola purchase was Pepsi™, there
is an 80% chance that her next cola purchase will also
be Pepsi™.
coke pepsi
0.1 0.9 0.8
0.2
![Page 10: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/10.jpg)
10
Coke vs. Pepsi Given that a person is currently a Pepsi purchaser,
what is the probability that she will purchase Coke
two purchases from now?
66.034.0
17.083.0
8.02.0
1.09.0
8.02.0
1.09.02P
8.02.0
1.09.0P
The transition matrices are:
(corresponding to
one purchase ahead)
![Page 11: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/11.jpg)
11
Coke vs. Pepsi
Given that a person is currently a Coke
drinker, what is the probability that she will
purchase Pepsi three purchases from now?
562.0438.0
219.0781.0
66.034.0
17.083.0
8.02.0
1.09.03P
![Page 12: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/12.jpg)
12
Coke vs. Pepsi Assume each person makes one cola purchase per
week. Suppose 60% of all people now drink Coke, and
40% drink Pepsi.
What fraction of people will be drinking Coke three
weeks from now?
6438.0438.04.0781.06.0)0( )3(
101
)3(
000
1
0
)3(
03
pQpQpQXPi
ii
Let (Q0,Q1)=(0.6,0.4) be the initial probabilities.
We will regard Coke as 0 and Pepsi as 1
We want to find P(X3=0)
8.02.0
1.09.0P
P00
![Page 13: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/13.jpg)
13
Equilibrium (Stationary) Distribution
coke pepsi
0.1 0.9 0.8
0.2
• Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction will be drinking Coke 10,100,1000,10000 … weeks from now?
• For each week, probability is well defined. But does it converge to some equilibrium distribution [p0,p1]?
• If it does, then eqs. : .9p0+.2p1 =p0, .8p1+.1p0 =p1
must hold, yielding p0= 2/3, p1=1/3 .
![Page 14: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/14.jpg)
Equilibrium (Stationary) Distribution
Whether or not there is a stationary distribution, and
whether or not it is unique if it does exist, are determined
by certain properties of the process.
– Irreducible means that every state is accessible from every other state.
– Aperiodic means that any state return to itself can occur at irregular times.
– Positive recurrent means that the expected return time is finite for every state.
coke pepsi
0.1 0.9 0.8
0.2
![Page 15: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/15.jpg)
15
Equilibrium (Stationary) Distribution
• If the Markov chain is positive recurrent, there exists a stationary distribution. If it is positive recurrent and irreducible, there exists a unique stationary distribution, and furthermore the process constructed by taking the stationary distribution as the initial distribution is ergodic. Then the average of a function f over samples of the Markov chain is equal to the average with respect to the stationary distribution,
![Page 16: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/16.jpg)
16
Equilibrium (Stationary) Distribution
• Writing P for the transition matrix, a stationary distribution is a vector π which satisfies the equation
– Pπ = π .
• In this case, the stationary distribution π is an eigenvector of the transition matrix, associated with the eigenvalue 1.
![Page 17: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/17.jpg)
17
Discrete Markov Model - Example
• States – Rainy:1, Cloudy:2, Sunny:3
• Matrix A –
• Problem – given that the weather on day 1 (t=1) is sunny(3), what is the probability for the observation O:
![Page 18: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/18.jpg)
18
Discrete Markov Model – Example (cont.)
• The answer is -
![Page 19: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/19.jpg)
19
Third Example: A Friendly Gambler
Game starts with 10$ in gambler’s pocket
– At each round we have the following:
• Gambler wins 1$ with probability p
• Gambler loses 1$ with probability 1-p
– Game ends when gambler goes broke (no sister in bank),
or accumulates a capital of 100$ (including initial capital)
– Both 0$ and 100$ are absorbing states
0 1 2 N-1 N
p p p p
1-p 1-p 1-p 1-p
Start
(10$)
or
![Page 20: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/20.jpg)
20
Fourth Example: A Friendly Gambler
0 1 2 N-1 N
p p p p
1-p 1-p 1-p 1-p
Start
(10$)
Irreducible means that every state is accessible from every other state. Aperiodic means that any state return to itself can occur at irregular times. Positive recurrent means that the expected return time is finite for every state. If the Markov chain is positive recurrent, there exists a stationary distribution. Is the gambler’s chain positive recurrent? Does it have a stationary distribution (independent upon initial distribution)?
![Page 21: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/21.jpg)
Markov Chains
See http://www.statslab.cam.ac.uk/~james/Markov/ Or http://probability.ca/MT/ for more detail
![Page 22: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/22.jpg)
22
Markov Chain A Simple Example
• States: fair coin F, unfair (biased) coin B
• Discrete times: flip 1, 2, 3, …
• Initial probability: pF = 0.6, pB = 0.4
• Transition probability
• Prob(FFBBFFFB)
P(O = {FFBBFFFB} | A,p )
= pF ×aFF ×aFB ×aBB ×aBF ×aFF ×aFF ×aFB
= 0.6 ´ 0.9 ´ 0.1´ 0.7´ 0.3´ 0.9 ´ 0.9 ´ 0.1= 9.19 ´10-4
F B 0.1
0.3 0.9 0.7
![Page 23: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/23.jpg)
23
Hidden Markov Model
• Coin toss example
• Coin transition is a Markov chain
• Probability of H/T depends on the coin used
• Observation of H/T is a hidden Markov chain (coin state is hidden)
![Page 24: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/24.jpg)
24
Hidden Markov Model
• Elements of an HMM (coin toss) – N, the number of states (F / B)
– M, the number of distinct observation (H / T)
– A = {aij} state transition probability
– B = {bj(k)} emission probability
– p ={pi} initial state distribution • pF = 0.4, pB = 0.6
7.03.0
1.09.0A
2.0)(,8.0)(
5.0)(,5.0)(
TbHb
TbHb
BB
FF
![Page 25: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/25.jpg)
25
HMM Applications
• Stock market: bull/bear market hidden Markov chain, stock daily up/down observed, depends on big market trend
• Speech recognition: sentences & words hidden Markov chain, spoken sound observed (heard), depends on the words
• Digital signal processing: source signal (0/1) hidden Markov chain, arrival signal fluctuation observed, depends on source
• Bioinformatics: sequence motif finding, gene prediction, genome copy number change, protein structure prediction, protein-DNA interaction prediction
![Page 26: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/26.jpg)
26
Basic Problems for HMM
1. Given , how to compute P(O|) observing sequence O = O1O2…OT
• Probability of observing HTTHHHT … • Forward procedure, backward procedure
2. Given observation sequence O = O1O2…OT and , how to choose state sequence Q = q1q2…qt
• What is the hidden coin behind each flip • Forward-backward, Viterbi
3. How to estimate =(A,B,p) so as to maximize P(O| )
• How to estimate coin parameters • Baum-Welch (Expectation maximization)
![Page 27: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/27.jpg)
27
Problem 1: P(O|)
• Suppose we know the state sequence Q
– O = H T T H H H T – Q = F F B F F B B
– Q = B F B F B B B
• Each given path Q has a probability for O
2.08.05.05.02.05.05.0
)()()()()()()(),|(
TbHbHbHbTbTbHbQOP BBFFBFF
2.08.08.05.02.05.08.0
)()()()()()()(),|(
TbHbHbHbTbTbHbQOP BBBFBFB
)()...()(),|( 2211 TqTqq ObObObQOP
![Page 28: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/28.jpg)
28
Problem 1: P(O|)
• What is the prob of this path Q?
– Q = F F B F F B B
– Q = B F B F B B B
• Each given path Q has its own probability
7.01.09.03.01.09.06.0
)|(
BBFBFFBFFBFFF aaaaaaQP p
)|( QP
7.07.01.03.01.03.04.0
)|(
BBBBFBBFFBBFB aaaaaaQP p
![Page 29: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/29.jpg)
29
Problem 1: P(O|)
• Therefore, total pb of O = HTTHHHT
• Sum over all possible paths Q: each Q with its own pb multiplied by the pb of O given Q
• For path of T long and N hidden states, there are NT paths, unfeasible calculation
QallQall
QOPQPQOPOP ),|()|()|,()|(
![Page 30: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/30.jpg)
30
Solution to Prob1: Forward Procedure
• Use dynamic programming
• Summing at every time point
• Keep previous subproblem solution to speed up current calculation
![Page 31: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/31.jpg)
31
Forward Procedure
• Coin toss, O = HTTHHHT
• Initialization
– Pb of seeing H1 from F1 or B1
H T T H …
a1(F) = pFbF (H) = 0.4 ´ 0.5 = 0.2
a1(B) = pBbB (H) = 0.6 ´ 0.8 = 0.48
)()( 11 Obi iip
B
F
![Page 32: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/32.jpg)
32
Forward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
– Pb of seeing T2 from F2 or B2
F2 could come from F1 or B1
Each has its pb, add them up
H T T H …
)()( 11 Obi iip
)(])([)( 1
1
1
tj
N
i
ijtt Obaij
+ B B
F
+
F
![Page 33: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/33.jpg)
33
Forward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
H T T H …
0712.02.0)7.048.01.02.0()())()(()(
162.05.0)3.048.09.02.0()())()(()(
112
112
TbaBaFB
TbaBaFF
BBBFB
FBFFF
)()( 11 Obi iip
)(])([)( 1
1
1
tj
N
i
ijtt Obaij
+ B B
F
+
F
![Page 34: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/34.jpg)
34
Forward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
H T T H …
013208.02.0]7.00712.01.0162.0[)(
08358.05.0]3.00712.09.0162.0[)(
3
3
B
F
)()( 11 Obi iip
)(])([)( 1
1
1
tj
N
i
ijtt Obaij
+ B
+ B
+
+ B B
F F
+
F
+
F
![Page 35: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/35.jpg)
35
Forward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
• Termination
H T T H …
)()( 11 Obi iip
)(])([)( 1
1
1
tj
N
i
ijtt Obaij
+ B
+ B
+
+ B B
F F
+
F
+
F
)()()()|( 44
1
BFiOPN
i
t
![Page 36: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/36.jpg)
36
Solution to Prob1: Backward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Pb of coin to see certain flip after it
...H H H T
1)()(
1)(
**
*
BF
i
TT
T
B
F
![Page 37: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/37.jpg)
37
Backward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
• Pb of coin to see certain flip after it
1)(* iT
)()()(1
11 jObaiN
j
ttjijt
![Page 38: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/38.jpg)
38
Backward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
...H H H T ?
1)(* iT
)()()(1
11 jObaiN
j
ttjijt
29.02.07.05.03.01)(1)()(
47.02.01.05.09.01)(1)()(
1
1
TbaTbaB
TbaTbaF
BBBFBFT
BFBFFFT
+
+
B
F
![Page 39: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/39.jpg)
39
Backward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
...H H H T
1)(* iT
)()()(1
11 jObaiN
j
ttjijt
2329.029.08.07.047.05.03.0)()()()()(
2347.029.08.01.047.05.09.0)()()()()(
112
112
BHbaFHbaB
BHbaFHbaF
TBBBTFBFT
TBFBTFFFT
+ +
+ +
B B
+
+
B
F F F
![Page 40: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/40.jpg)
40
Backward Procedure
• Coin toss, O = HTTHHHT
• Initialization
• Induction
• Termination
• Both forward and backward could be used to solve problem 1, which should give identical results
1)(* iT
)()()(1
11 jObaiN
j
ttjijt
)()()()((*) 110 BHbFHb BBFF pp
![Page 41: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/41.jpg)
41
Solution to Problem 2 Forward-Backward Procedure
• First run forward and backward separately
• Keep track of the scores at every point
• Coin toss
– α: pb of this coin for seeing all the flips now and before
– β: pb of this coin for seeing all the flips after
H T T H H H T
α1(F) α2(F) α3(F) α4(F) α5(F) α6(F) α7(F)
α1(B) α2(B) α3(B) α4(B) α5(B) α6(B) α7(B)
β1(F) β2(F) β3(F) β4(F) β5(F) β6(F) β7(F)
β1(B) β2(B) β3(B) β4(B) β5(B) β6(B) β7(B)
![Page 42: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/42.jpg)
42
Solution to Problem 2 Forward-Backward Procedure
• Coin toss
• Gives probabilistic prediction at every time point • Forward-backward maximizes the expected number
of correctly predicted states (coins)
N
j
tt
ttt
jj
iii
1
)()(
)()()(
659.00038.00016.00047.00025.0
0047.00025.0..
)()()()(
)()()(
3333
333
ge
BBFF
FFF
![Page 43: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/43.jpg)
43
Solution to Problem 2 Viterbi Algorithm
• Report the path that is most likely to give the observations
• Initiation
• Recursion
• Termination
• Path (state sequence) backtracking
0)(
)()(
1
11
i
Obi ii
p
])([maxarg)(
)(])([max)(
11
11
ijtNi
t
tjijtNi
t
aij
Obaij
)]([maxarg
)]([max*
1
*
1
iq
iP
TNi
T
TNi
1,...,2,1),( *
11
* TTtqq ttt
![Page 44: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/44.jpg)
44
Viterbi Algorithm
• Observe: HTTHHHT
• Initiation 0)(
)()(
1
11
i
Obi ii
p
48.08.06.0)(
2.05.04.0)(
1
1
B
F
![Page 45: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/45.jpg)
45
Viterbi Algorithm
H T T H
F
B
![Page 46: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/46.jpg)
46
Viterbi Algorithm
• Observe: HTTHHHT
• Initiation
• Recursion
0)(
)()(
1
11
i
Obi ii
p
])([maxarg)(
)(])([max)(
11
11
ijtNi
t
tjijtNi
t
aij
Obaij
BBFF
TbaBaFB
TbaBaFF
BBBFBi
FBFFFi
)(,)(
0672.02.0)336.0,02.0max()()])(,)(([max)(
09.05.0)144.0,18.0max()()])(,)(([max)(
22
112
112
Max instead of +, keep track path
![Page 47: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/47.jpg)
47
Viterbi Algorithm
• Max instead of +, keep track of path
• Best path (instead of all path) up to here
H T T H
F F
B B
![Page 48: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/48.jpg)
48
Viterbi Algorithm
• Observe: HTTHHHT
• Initiation
• Recursion
0)(
)()(
1
11
i
Obi ii
p
])([maxarg)(
)(])([max)(
11
11
ijtNi
t
tjijtNi
t
aij
Obaij
BBFF
TbaBaFB
TbaBaFF
BBBFBi
FBFFFi
)(,)(
0094.02.0)7.00672.0,1.009.0max(
)()])(,)(([max)(
0405.05.0)3.00672.0,9.009.0max(
)()])(,)(([max)(
33
223
223
Max instead of +, keep track path
![Page 49: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/49.jpg)
49
Viterbi Algorithm
• Max instead of +, keep track of path
• Best path (instead of all path) up to here
H T T H
F F
B B
F
B
F
B
![Page 50: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/50.jpg)
50
Viterbi Algorithm • Terminate, pick state that gives final best δ
score, and backtrack to get path
H T T H
• BFBB most likely to give HTTH
F F
B B
F
B
F
B B B
F
B
![Page 51: Biostatistics-Lecture 4 - PKU...Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of](https://reader033.fdocuments.in/reader033/viewer/2022041704/5e4371733a4cb425bd7f44d1/html5/thumbnails/51.jpg)
51
Solution to Problem 3
• No optimal way to do this, so find local maximum
• Baum-Welch algorithm (equivalent to expectation-maximization)
– Random initialize =(A,B,p)
– Update =(A,B,p)
• p: % of F vs B on Viterbi path
• A: frequency of F/B transition on Viterbi path
• B: frequency of H/T emitted by F/B