Bayes Filter

Probabilistic RoboticsBayesian filtering

Martin MagnussonApril 3, 2012

Intro and recap Robot environment interaction Bayes filters Outro

Agenda

1 Intro and recap

2 Robot environment interaction

3 Bayes filters

4 Outro


Recap of last lecture

Important relations

Conditional probability(the prob that X = x if we know, or assume, that Y = y)

p(x | y) = p(x, y) joint probp(y) scaled to fit y

note that p(x, y) p(y)

Joint probability of x and y(the prob that both X = x and Y = y)

p(x, y) = p(x | y) cond prob

p(y)scaled to fit

= p(y |x)p(x)

Conditional independence: x and y are independent, given z,iff

p(x, y | z) = p(x | z)p(y | z)



Causal vs. diagnostic reasoning

Environment state X: open or closed.

Robot sensor reading Y: open or closed.

Assume we know p(Y = y |X = open)(i.e., quality of the sensor causal knowledge)

and need to know p(X = open |Y = y)(i.e., prob that the door is open diagnostic knowledge)

Bayes rule lets us use causal knowledge to infer diagnosticknowledge:

p(open | y) = p(y | open)p(open)p(y)

(How to compute p(y)? Well see that later.)



Bayes formula, explained

Prior: p(x) (probability before sensor input)

Posterior: p(x | y) (probability after input = diagnosis)

Bayes rule: probability that x is true given y (the posterior)increases with

the prior of x (i.e., prob of x before the test),and the prob of finding y in a world where x is true

decreases withthe prior prob of finding y (i.e., prob of getting test result ywithout knowing the state of x)

The denominator doesnt depend on x, so its the same forboth p(cancer |pos) and p(cancer |pos) and is used tomake the posterior p(x | y) integrate to 1.



Bayes formula, robotics exampleX: world state, Z: robot measurements.

Noisy sensors:

p(Z = open |X = open) = 0.6p(Z = closed |X = open) = 0.4

}hard to sense open door

p(Z = open |X = closed) = 0.2p(Z = closed |X = closed) = 0.8

}easy to sense closed door

Prior probabilities

p(X = open) = 0.5

p(X = closed) = 0.5



State estimation example

Suppose the robot senses Z = open.

What is the probability that the door is actually open; thatis, p(X = open |Z = open)?Apply Bayes formula:

p(X = open |Z = open) =

=p(Z = open |X = open)p(X = open)

p(Z = open |X = open)p(X = open) + p(Z = open |X = closed)p(X = closed)

=0.6 0.5

0.6 0.5 + 0.2 0.5 = 0.75



Law of total probability

Where does the denominator come from? If all y are pairwisedisjoint and fill up all of , then

Theorem (Discrete case)

p(x) =

y

p(x | y)p(y) =

y

p(x, y)

Follows from the definition of conditional probability andKolmogorovs axioms.

Robot state variables fulfil the requirements: Can only be inone state at a time, and all outcomes = .



Law of total probability, illustration

p(x) =n

i=1

p(x, yi)=n

i=1

p(x | yi)p(yi)

x

y2y1

y3

y4

y5

y6

y7



Law of total probability, proof

If x occurs, then one of yi must also occur (since yi aredisjoint and fill ).

So x occurs and both x and one yi occurs areequivalent.

Equivalent to ni=1(x yi) occurs.

y

p(y) axiom 1= 1

p(x) = ni=1(x yi)axiom 3

=

ni=1

p(x, yi)

p(x)def. of joint prob.

=n

i=1

p(x | yi)p(yi)


State

State

Description of what the robot needsto know.

State at time t is denoted xt.

State transitions over time:x0 x1 . . .The set of all states from time t1 totime t2:xt1:t2 = xt1 ,xt1+1,xt1+1, . . . ,xt2

Internal state Typically the pose [x, y, ].

External state Map, other agents, etc.

x, y


State

Markov state

The Markov property

The conditional probability distribution offuture states depends only upon the presentstate, not on the sequence of events thatpreceded it.

In other words, past (x0:t1) and future(xt+1:) states are conditionally independent,given the present state xt.


State

Markov state, example

Positions of chess pieces is Markovstate (complete state), in idealised

chess. . .

. . . but not in real-world chess!

In reality, complete state descriptions are infeasible.


Interaction

Measurements

Sensor input from environment.

Measurement at time t is denoted zt.

Measurements decrease uncertainty.


Interaction

Actions

Action at time t is denoted ut.Typical actions:

the robot turns its wheels to move,the robot uses its manipulator to grasp an object,do nothing (and let time pass by).

Note thatactions are never carried out with absolute certainty,actions generally increase uncertainty.


Interaction

Modelling actions

The outcome of an action u is modelled by the conditionalprobability distribution

p(x |u,x)

That is, the probability that, when in state x, executing action u,changes the state to x.

1 state x = [10 m,5 m,0]

2 action u = move 1 m forward

3 what is, for example, p(x = [11 m,5 m,0])?(p < 1 because of wheel slip, etc.)


Belief

Belief

We never know the true state of the robot.

All we have is the belief.

Represent belief through conditional probabilitydistribution:

bel(xt) = p(xt | z1:t,u1:t)

A belief distribution assigns a probability density (or mass)to each possible outcome, (given a sequence of actions andmeasurements).

Belief distributions are posterior probabilities over statevariables, conditioned on the available data.


Belief

Prediction vs. belief

Represent belief through conditional probability distribution:


Prediction: the belief distribution before incorporating ameasurement

bel(xt) = p(xt | z1:t1,u1:t)

Belief: the belief distribution after a measurement



The algorithm

Bayes filters: framework

Given:1 stream of observations z and action data u

{z1:t,u1:t} = {u1, z1, . . . ,ut, zt}

2 sensor model p(z |x) (how accurate the sensors are)3 action model p(x |u,x) (how reliable the actuators are)4 prior probability of the system state p(x).

Wanted:estimate of the state x (the belief)


Update the belief recursively: bel(xt) is computed frombel(xt1).


The algorithm

Bayes filters: assumptions

xt2 xt1 xt

ut2

zt2

ut1 ut

zt1 zt

x0 ... ...

Markov assumption impliesstatic worldindependent controlsperfect model no approximation errors

p(xt |x0:t1, z1:t,u1:t) = p(xt |xt1,ut) state transition probabilityp(zt |x0:t, z1:t,u1:t) = p(zt |xt) measurement probability


Example

State estimation, example

Robot observing a door

Given a sensor reading open fromthe camera, what is theprobability that the door isactually open?

p(X = open |Z = open)


Example

State estimation example, sensor model

Xt = {open, closed} : world state

Zt = {open, closed}: robot measurements.

Noisy sensors:

p(Zt = sense_open |Xt = open) = 0.6p(Zt = sense_closed |Xt = open) = 0.4

}hard to sense open door

p(Zt = sense_open |Xt = closed) = 0.2p(Zt = sense_closed |Xt = closed) = 0.8

}easy to sense closed door


Example

State estimation example, actions

Actions Ut = {push,null}

p(Xt = open |Ut = push,Xt1 = open) = 1p(Xt = closed |Ut = push,Xt1 = open) = 0

}door stays open

p(Xt = open |Ut = push,Xt1 = closed) = 0.8p(Xt = closed |Ut = push,Xt1 = closed) = 0.2

}cant always open door

p(Xt = open |Ut = null,Xt1 = open) = 1p(Xt = closed |Ut = null,Xt1 = open) = 0p(Xt = open |Ut = null,Xt1 = closed) = 0p(Xt = closed |Ut = null,Xt1 = closed) = 1

no other agents


Example

State estimation example, t = 1Suppose at time t = 1, the robot takes action U1 = null andsenses Z1 = open.We want to compute an updated belief distribution bel(X1).With Bayes filter, we can do that using the prior beliefbel(X0).

bel(X1 = open)

= p(X = open |Z = open) =

=p(Z = open |X = open)p(X = open)

p(Z = open |X = open)p(X = open) + p(Z = open |X = closed)p(X = closed)

=0.6 0.5

0.6 0.5 + 0.2 0.5= 0.75

bel(X1 = closed) =0.2 0.5

0.6 0.5 + 0.2 0.5 = 0.25 = 1 bel(X1 = open)


Example

State transisions

p(x |u,x) for u = push

X = open X = closed

0

0.8

1 0.2

This is a simple two-state Markov chain.

If the door is closed, the action push succeeds in 80% of thecases.


Example

Integrating the outcome of actions

We know p(x |u,x) (thats our action model).How to compute the posterior p(x |u)? I.e., the resultingbelief after the action.

Integrate over all prior states x.

The law of total probability gives us

p(x |u) =x

p(x |u,x)p(x) discrete case

p(x |u) =

p(x |u,x)p(x) dx continuous case


Example

State estimation example, executing an actionSuppose at time t = 2, the robot takes action u2 = push.

p(X = open | u2)

=x

p(X = open | u2, x)p(x)

= p(X = open | u2,X = open)p(X = open)+ p(X = open | u2,X = closed)p(X = closed)

= 1 0.75 + 0.8 0.25 = 0.95

p(X = closed | u2)

=x

p(X = closed | u2, x)p(x)

= p(X = closed | u2,X = open)p(X = open)+ p(X = closed | u2,X = closed)p(X = closed)

= 0 0.75 + 0.2 0.25 = 0.05


Example

Combining evidence

How can we integrate the next observation Z2?

More generally, how can we estimate p(X |Z1, . . . ,Zn)?


Example

Bayes rule, with background knowledge

p(x | y) = p(y |x)p(x)p(y)

We can also condition Bayes rule on additional RVs(background knowledge):

p(x | y, z) = p(y |x, z)p(x | z)p(y | z)


Example

State estimation example, t = 2

After taking action u2 = push, it senses z2 = open.

bel(X2 = open)

= p(X2 = open | z1, z2) =

=p(z2 |X1 = open)p(X1 = open | z1)

p(z2 |X1 = open)p(X1 = open | z1) + p(z2 |X1 = closed)p(X1 = closed | z1)

=0.6 0.75

0.6 0.75 + 0.2 0.25= 0.90

bel(X2 = closed) =0.2 0.25

0.6 0.75 + 0.2 0.25= 0.10 = 1 bel(X2 = open)


Example

The Bayes filter algorithm

Giventhe previous belief distribution,the latest action,and the latest sensor measurement,

compute an updated belief distribution for time t.

1: function BayesFilter(bel(Xt1),ut, zt)2: for all xt do3: bel(xt) =

p(xt |ut,xt1) bel(xt1) dxt1 . control update

4: bel(xt) = p(zt |xt) bel(xt) p(zt)1 . measurement update5: end for6: return bel(Xt)7: end function


Example

The Bayes filter algorithm explained

The control update comes from the law of total probability:For all prior states xt1, sum up (integrate)

the product of the prior for xt1and the prob that u makes the transition from xt1 to xt.

The measurement update comes from Bayes ruleThe prob of getting zt in xttimes the prior for xt (after the control update),divided by the prior of zt, in order to make the total mass ofbel(xt) = 1.


Example

Why cant we use the Bayes filter in reality?

Because we cant compute the update rule for continuous statespaces!

Because of the integral in the denominator (normaliser) ofBayes rule

Because of the integral in the control update


SummaryMarkov assumptions: we dont need history of all previousstates.Sensor measurements Z decrease uncertainty, robot actionsU increase uncertainty.Belief is represented as posterior PDF over possible stateoutcomes, conditioned on sensor data and actions.

Bayes rule allows us to compute probabilities that are hardto assess otherwise.Under the Markov assumption, recursive Bayesian updatingcan be used to efficiently combine evidence.Bayes filters are a probabilistic tool for estimating the stateof dynamic systems.The Bayes filter cannot be implemented for realistic,continuous, state spaces. (The remainder of the course willdiscuss approximations.)


Next lecture

Time and space

10.1512.00, Wednesday April 11T-111

Reading material

Thrun et al., Chapters 5 and 6
Intro and recap

Bayes Filter

Documents

Transcript of Bayes Filter