1
Russell and Norvig, AIMA : Chapter 15Part A: 15.1 , 15.2
Probabilistic Reasoning over Time
Presented to: Prof. Dr. S. M. Aqil Burney
Presented by: Zain Abbas (MSCS-UBIT)
2
Agenda
Temporal probabilistic agents
Inference: Filtering, prediction,
smoothing and most likely
explanation
Hidden Markov models
Kalman filters
3
Agenda
Temporal probabilistic agents
Inference: Filtering, prediction,
smoothing and most likely
explanation
Hidden Markov models
Kalman filters
4
environmentagent
?
sensors
actuators
t1, t2, t3, …
Temporal Probabilistic Agents
5
Time and Uncertainty
The world changes, we need to track and predict it
Examples: diabetes management, traffic monitoring
Basic idea: copy state and evidence variables for each
time step
Xt – set of unobservable state variables at time t
e.g., BloodSugart, StomachContentst
Et – set of evidence variables at time t
e.g., MeasuredBloodSugart, PulseRatet, FoodEatent
Assumes discrete time step
6
States and Observations
Process of change is viewed as series of snapshots, each
describing the state of the world at a particular time
Each time slice involves a set or random variables indexed
by t:
1. the set of unobservable state variables Xt
2. the set of observable evidence variable Et
The observation at time t is Et = et for some set of values et
The notation Xa:b denotes the set of variables from Xa to Xb
7
Markov Assumption Current state depends on only a finite history of
previous states
i.e. Xt depends on some bounded set of X0:t-1
First-order Markov process
P(Xt|X0:t-1) = P(Xt|Xt-1)
Second-order Markov process
P(Xt|X0:t-1) = P(Xt|Xt-2, Xt-1)
Markov processes
8
Sensor Markov assumption
P(Et|X0:t, E0:t-1) = P(Et|Xt)
Assume stationary process
transition model P(Xt|Xt-1) and sensor model
P(Et|Xt) are the same for all t
In a stationary process, the changes in
the world state are governed by laws that
do not themselves change over time
Markov processes
9
Example
10
Complete Joint Distribution
Given: Transition model: P(Xt|Xt-1)
Sensor model: P(Et|Xt)
Prior probability: P(X0) Then we can specify complete joint
distribution of over all variables :
t
iiiiitt XEPXXPXPEEXXXP
110110 )|()|()(),...,,,...,,(
11
Agenda
Temporal probabilistic agents
Inference: Filtering, prediction,
smoothing and most likely
explanation
Hidden Markov models
Kalman filters
12
Inference in Temporal Models
Having set up the structure of a generic temporal model, we can formulate the basic inference tasks that must be solved.
They are as follows: Filtering or Monitoring Prediction Smoothing or Hindsight Most likely explanation
13
What is the probability that it is raining today, given all the umbrella observations up through today?
P(Xt|e1:t) - computing current belief state, given all evidence to date
Filtering or Monitoring
14
Filtering or Monitoring
Transition modelPosterior
distribution at time t
Prediction
Sensor model
Filtering
15
Proof
Forward Algorithm
Bayes Rule
Chain Rule
Conditional Independence
Marginal Probability
Chain Rule
Conditional Independence
Dividing the evidence
t
t
t
xtttttt
ttx
ttttt
xttttt
tttt
ttttt
ttt
ttttt
exPxXPXeP
exPexXPXeP
exXPXeP
eXPXeP
eXPeXeP
eeXP
eeXPeXP
)|()|()|(
)|(),|()|(
)|,()|(
)|()|(
)|(),|(
)|,(
),|()|(
:1111
:1:1111
:1111
:1111
:11:111
:111
1:111:11
16
What is the probability that it will rain the day after tomorrow, given all the umbrella observations up through today?
P(Xt+k|e1:t) computing probability of some future state
Prediction
17
Example
18
Example
Let us illustrate the filtering process for two steps in the basic umbrella example
Assume that security guard has some prior belief as to whether it rained on day 0, just before the observation sequence begins. Let's suppose this is P(R0)=<0.5,0.5>
On day 1, the umbrella appears, so U1= true. The prediction from t=0 to t=1 is
19
Example … continued
Updating the evidence for t=1 gives
Prediction from t=1 to t=2 is
Updating the evidence for t=2 gives
or
o rPrRPRPo
11
5.0,5.0
182.0,818.01.0,45.0
373.0,627.0182.07.0,3.0818.03.0,7.0
117.0,883.0075.0,565.0
20
Example… continued
21
What is the probability that it rained yesterday, given all the umbrella observations through today?
P(Xk|e1:t) 0 ≤ k < tcomputing probability of past state (hindsight)
Smoothing or Hindsight
22
Smoothing or Hindsight
Smoothing computes P(Xk|e1:t), the posterior distribution of the state at some past time k given a complete sequence of observations from 1 to t
23
Smoothing or Hindsight
bk+1:t
Bayes Rule
Chain Rule
Conditional Independence
Dividing the evidence
)|()|(
)|(),|(
)|,(
),|()|(
:1:1
:1:1:1
:1:1
:1:1:1
kkktk
kkkktk
ktkk
tkkktk
eXPXeP
eXPeXeP
eeXP
eeXPeXP
f1:k
24
Smoothing or Hindsight
Sensor modelBackward
message at time
k+1
Sensor model
Backward Message at time k
25
Smoothing or Hindsight
Backward Algorithm
11111:2
111:21
111:1
111:1
11:1:1
)|()|()|(
)|()|,(
)|()|(
)|(),|(
)|,()|(
kkkkkktk
kkkktkk
kkkktk
kkkkktk
kkktkktk
XxPxePxeP
XxPxeeP
XxPxeP
XxPxXeP
XxePXeP
Bayes Rule
Chain Rule
Conditional Independence
Marginal Probability
Conditional Independence
26
Example
27
If the umbrella appeared the first three days but not on the fourth, what is the most likely weather sequence to produce these umbrella sightings?
arg maxx1:xtP(x1:t|e1:t)given sequence of observation, find sequence of states that is most likely to have generated those observations.
Most likely explanation
28
Most likely explanation
Enumeration Enumerate all possible state sequence Compute the joint distribution and find the
sequence with the maximum joint distribution
Smooth Calculate the posterior distribution
for each time step k In each step k, find the state with maximum
posterior distribution Combine these states to form a sequence
)|( :1 tk eXP
29
Most likely explanation
Most likely path to each xt+1 = most likely path to some xt plus one more step
Identical to filtering except f1:t replaced by
)|,(max)|(max)|(
)|,(max
:11:1111
1:11:1
1:1
:1
tttx
ttx
tt
tttx
eXxPxXPXeP
eXxP
tt
t
)|,(max :11:1:11:1
tttx
t eXxPmt
30
Proof… Viterbi Algorithm
)|,(max)|(max)|(
)|,()|(max)|(
)|,()|(max)|(
)|,(),,|(max
)|,,(max
),|,(max
)|,(max
:11:1111
:11:1111
:11:1111
:11:11:1:11
:111:1
1:11:1
1:11:1
1:1
:1
:1
:1
:1
:1
:1
tttx
ttx
tt
tttttx
tt
tttttx
tt
tttttttx
ttttx
ttttx
tttx
exxPxXPXeP
exxPxXPXeP
exxPxXPXeP
eXxPXxeeP
eeXxP
eeXxP
eXxP
tt
t
t
t
t
t
t
Bayes Rule
Chain Rule
Conditional Independence
Divide Evidence
Chain Rule
31
Example
Top Related