14: Conditional DistributionsDavid VarodayanFebruary 7, 2020Adapted from slides by Lisa Yan

1

Sum of independent random variables

2

Review

𝑋𝑋~Bin 𝑛𝑛1,𝑝𝑝 ,𝑌𝑌~Bin(𝑛𝑛2,𝑝𝑝) 𝑋𝑋 + 𝑌𝑌 ~Bin(𝑛𝑛1 + 𝑛𝑛2,𝑝𝑝)𝑋𝑋,𝑌𝑌 independent

𝑋𝑋~Poi 𝜆𝜆1 ,𝑌𝑌~Poi 𝜆𝜆2𝑋𝑋,𝑌𝑌 independent

𝑋𝑋 + 𝑌𝑌 ~Poi(𝜆𝜆1 + 𝜆𝜆2)

𝑋𝑋~𝒩𝒩 𝜇𝜇1,𝜎𝜎12𝑌𝑌~𝒩𝒩 𝜇𝜇2,𝜎𝜎22𝑋𝑋,𝑌𝑌 independent

𝑋𝑋 + 𝑌𝑌 ~𝒩𝒩(𝜇𝜇1 + 𝜇𝜇2,𝜎𝜎12 + 𝜎𝜎22)

Note: these also hold in the general case (≥ 2 variables)

Quick questionLet 𝐴𝐴~Bin 30, 0.01 and 𝐵𝐵~Bin 50, 0.02 be independent RVs. Can we use sum of independent Poisson RVs to approximate 𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 ?

3

Review

Quick questionLet 𝐴𝐴~Bin 30, 0.01 and 𝐵𝐵~Bin 50, 0.02 be independent RVs. Can we use sum of independent Poisson RVs to approximate 𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 ?

4

Review

𝐴𝐴 ≈ 𝑋𝑋~Poi 𝜆𝜆1 = 30 ⋅ 0.01 = 0.3𝐵𝐵 ≈ 𝑌𝑌~Poi 𝜆𝜆2 = 50 ⋅ 0.02 = 1 𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 ≈ 𝑃𝑃 𝑋𝑋 + 𝑌𝑌 = 2

Sol 1: Approximate as sum of Poissons

𝑋𝑋 + 𝑌𝑌~Poi 𝜆𝜆 = 𝜆𝜆1 + 𝜆𝜆2 = 1.3

=𝜆𝜆2

2!𝑒𝑒−𝜆𝜆

Sol 2: No approximation

𝑃𝑃 𝐴𝐴 + 𝐵𝐵 = 2 = �𝑘𝑘=0

2

𝑃𝑃 𝐴𝐴 = 𝑘𝑘 𝑃𝑃 𝐵𝐵 = 2 − 𝑘𝑘

≈ 0.2302

= �𝑘𝑘=0

230𝑘𝑘 0.01𝑘𝑘 0.99 30−𝑘𝑘 50

2 − 𝑘𝑘 0.022−𝑘𝑘0.9848+𝑘𝑘 ≈ 0.2327

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

5

Ratio of probabilities interlude

Convolution

6

the convolutionof 𝑓𝑓𝑋𝑋 and 𝑓𝑓𝑌𝑌

For independent continuous random variables 𝑋𝑋 and 𝑌𝑌:

𝑓𝑓𝑋𝑋+𝑌𝑌 𝛼𝛼 = �−∞

∞𝑓𝑓𝑋𝑋 𝑘𝑘 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑘𝑘 𝑑𝑑𝑘𝑘

the convolutionof 𝑝𝑝𝑋𝑋 and 𝑝𝑝𝑌𝑌

Recall for independent discrete random variables 𝑋𝑋 and 𝑌𝑌:

𝑃𝑃 𝑋𝑋 + 𝑌𝑌 = 𝑛𝑛 = �𝑘𝑘

𝑃𝑃 𝑋𝑋 = 𝑘𝑘 𝑃𝑃 𝑌𝑌 = 𝑛𝑛 − 𝑘𝑘

Sum of independent UniformsLet 𝑋𝑋~Uni 0,1 and 𝑌𝑌~Uni 0,1 be independent random variables.What is the distribution of 𝑋𝑋 + 𝑌𝑌, 𝑓𝑓𝑋𝑋+𝑌𝑌?

7

𝑓𝑓𝑋𝑋+𝑌𝑌 𝛼𝛼 = �−∞

∞𝑓𝑓𝑋𝑋 𝑥𝑥 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑥𝑥 𝑑𝑑𝑥𝑥

𝑋𝑋 and 𝑌𝑌independent+ continuous

𝑓𝑓𝑋𝑋+𝑌𝑌 𝛼𝛼 = �−∞

∞𝑓𝑓𝑋𝑋 𝑘𝑘 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑘𝑘 𝑑𝑑𝑘𝑘

𝑓𝑓𝑋𝑋 𝑘𝑘 𝑓𝑓𝑌𝑌 𝛼𝛼 − 𝑘𝑘 = 1 when:

𝑓𝑓𝑋𝑋+𝑌𝑌 𝛼𝛼 = �𝑎𝑎 0 ≤ 𝑎𝑎 ≤ 1

2 − 𝑎𝑎 1 ≤ 𝑎𝑎 ≤ 20 otherwise

0

1/2

𝛼𝛼

𝑓𝑓 𝑋𝑋+𝑌𝑌𝛼𝛼

1/2 1 3/2 2

1

00 ≤ 𝑘𝑘 ≤ 1 and0 ≤ 𝛼𝛼 − 𝑘𝑘 ≤ 1 ⇒ 𝛼𝛼 − 1 ≤ 𝑘𝑘 ≤ 𝛼𝛼

The precise integrationbounds on 𝑘𝑘 depend on 𝛼𝛼.

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

8

Properties of Expectation, extended to two RVs1. Linearity:

𝐸𝐸 𝑎𝑎𝑋𝑋 + 𝑏𝑏𝑌𝑌 + 𝑐𝑐 = 𝑎𝑎𝐸𝐸 𝑋𝑋 + 𝑏𝑏𝐸𝐸 𝑌𝑌 + 𝑐𝑐

2. Expectation of a sum = sum of expectation:

𝐸𝐸 𝑋𝑋 + 𝑌𝑌 = 𝐸𝐸 𝑋𝑋 + 𝐸𝐸 𝑌𝑌

3. Unconscious statistician:

9

(we’ve seen this; we’ll prove this next)

𝐸𝐸 𝑔𝑔 𝑋𝑋,𝑌𝑌 = �𝑥𝑥

�𝑦𝑦

𝑔𝑔 𝑥𝑥,𝑦𝑦 𝑝𝑝𝑋𝑋,𝑌𝑌(𝑥𝑥,𝑦𝑦)

𝐸𝐸 𝑔𝑔 𝑋𝑋,𝑌𝑌 = �−∞

∞�−∞

∞𝑔𝑔 𝑥𝑥,𝑦𝑦 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 𝑑𝑑𝑥𝑥 𝑑𝑑𝑦𝑦

Proof of expectation of a sum of RVs

10

Even if the joint distribution is unknown, you can calculate the expectation of sum as sum of expectations.

Example: 𝐸𝐸 ∑𝑖𝑖=1𝑛𝑛 𝑋𝑋𝑖𝑖 = ∑𝑖𝑖=1𝑛𝑛 𝐸𝐸 𝑋𝑋𝑖𝑖 despite dependent trials 𝑋𝑋𝑖𝑖

𝐸𝐸 𝑋𝑋 + 𝑌𝑌 = 𝐸𝐸 𝑋𝑋 + 𝐸𝐸 𝑌𝑌

�𝑥𝑥

�𝑦𝑦

𝑥𝑥 + 𝑦𝑦 𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 LOTUS,𝑔𝑔 𝑋𝑋,𝑌𝑌 = 𝑋𝑋 + 𝑌𝑌

= �𝑥𝑥

�𝑦𝑦

𝑥𝑥𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 + �𝑥𝑥

�𝑦𝑦

𝑦𝑦𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

Linearity of summations(cont. case: linearity of integrals)

= �𝑥𝑥

𝑥𝑥�𝑦𝑦

𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 + �𝑦𝑦

𝑦𝑦�𝑥𝑥

𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

Marginal PMFs for 𝑋𝑋 and 𝑌𝑌= �𝑥𝑥

𝑥𝑥𝑝𝑝𝑋𝑋 𝑥𝑥 + �𝑦𝑦

𝑦𝑦𝑝𝑝𝑌𝑌 𝑦𝑦

= 𝐸𝐸 𝑋𝑋 + 𝐸𝐸[𝑌𝑌]

𝐸𝐸 𝑋𝑋 + 𝑌𝑌 =

Expectations of common RVs

11

𝑋𝑋~Bin(𝑛𝑛,𝑝𝑝) 𝐸𝐸 𝑋𝑋 = 𝑛𝑛𝑝𝑝

𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐸𝐸 𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

Review

Expectations of common RVs

12

𝑋𝑋~Bin(𝑛𝑛,𝑝𝑝) 𝐸𝐸 𝑋𝑋 = 𝑛𝑛𝑝𝑝

𝑌𝑌~NegBin(𝑟𝑟, 𝑝𝑝) 𝐸𝐸 𝑌𝑌 = 𝑟𝑟𝑝𝑝

𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐸𝐸 𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

How should we define 𝑌𝑌𝑖𝑖? A. 𝑌𝑌𝑖𝑖 = 𝑖𝑖th trial is heads. 𝑌𝑌𝑖𝑖~Ber 𝑝𝑝 , 𝑖𝑖 = 1, … ,𝑛𝑛B. 𝑌𝑌𝑖𝑖 = # trials to get 𝑖𝑖th success (after 𝑖𝑖 − 1 th success)

𝑌𝑌𝑖𝑖~Geo 𝑝𝑝 , 𝑖𝑖 = 1, … , 𝑟𝑟C. 𝑌𝑌𝑖𝑖 = # successes in 𝑛𝑛 trials

𝑌𝑌𝑖𝑖~Bin 𝑛𝑛,𝑝𝑝 , 𝑖𝑖 = 1, … , 𝑟𝑟, we look for 𝑃𝑃 𝑌𝑌𝑖𝑖 = 1

𝑌𝑌 = �𝑖𝑖=1

?

𝑌𝑌𝑖𝑖

Suppose:

Expectations of common RVs

13

𝑋𝑋~Bin(𝑛𝑛,𝑝𝑝) 𝐸𝐸 𝑋𝑋 = 𝑛𝑛𝑝𝑝

𝑌𝑌~NegBin(𝑟𝑟, 𝑝𝑝) 𝐸𝐸 𝑌𝑌 = 𝑟𝑟𝑝𝑝

𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐸𝐸 𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

𝑌𝑌 = �𝑖𝑖=1

𝑟𝑟

𝑌𝑌𝑖𝑖

Let 𝑌𝑌𝑖𝑖 = # trials to get 𝑖𝑖thsuccess (after𝑖𝑖 − 1 th success)

𝑌𝑌𝑖𝑖~Geo 𝑝𝑝 ,𝐸𝐸 𝑌𝑌𝑖𝑖 = 1𝑝𝑝

𝐸𝐸 𝑌𝑌 = 𝐸𝐸 �𝑖𝑖=1

𝑟𝑟

𝑌𝑌𝑖𝑖 = �𝑖𝑖=1

𝑟𝑟

𝐸𝐸 𝑌𝑌𝑖𝑖 = �𝑖𝑖=1

𝑟𝑟1𝑝𝑝

=𝑟𝑟𝑝𝑝

Midterm exam

When: Monday, February 10, 7:00pm-9:00pmWhere: Cubberley Auditorium

Not permitted: book/computer/calculatorPermitted: Three 8.5”x11” double-sided sheets of notes

Covers: Up to (and including) week 4 + Lecture Notes 11Practice: http://web.stanford.edu/class/cs109/exams/midterm.htmlReview session: Tomorrow Saturday, 3-5pm, STLC 111

Announcements

14

not recorded; materials will be posted though

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

15

CS109 roadmap

16

Multiple events:

𝑃𝑃 𝐸𝐸 𝐹𝐹 =𝑃𝑃 𝐸𝐸𝐹𝐹𝑃𝑃 𝐹𝐹

conditional probability

𝑃𝑃 𝐸𝐸𝐹𝐹 = 𝑃𝑃 𝐸𝐸 𝑃𝑃(𝐹𝐹)

independence

Joint (Multivariate) distributions

joint PMF/PDF

𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

intersection

𝑃𝑃 𝐸𝐸 ∩ 𝐹𝐹= 𝑃𝑃 𝐸𝐸𝐹𝐹

conditional distributions?

today

independentRVs

sum of independent RVs

Discrete conditional distributionsRecall the definition of the conditional probability of event 𝐸𝐸 given event 𝐹𝐹:

𝑃𝑃 𝐸𝐸 𝐹𝐹 =𝑃𝑃 𝐸𝐸𝐹𝐹𝑃𝑃 𝐹𝐹

For discrete random variables 𝑋𝑋 and 𝑌𝑌, the conditional PMF of 𝑋𝑋 given 𝑌𝑌 is

𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦

𝑃𝑃 𝑌𝑌 = 𝑦𝑦

𝑝𝑝𝑋𝑋|𝑌𝑌 𝑥𝑥|𝑦𝑦 =𝑝𝑝𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑝𝑝𝑌𝑌 𝑦𝑦

17

Different notation,same idea:

Quick checkNumber or function?

1. 𝑃𝑃 𝑋𝑋 = 2 𝑌𝑌 = 5

2. 𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 5

3. 𝑃𝑃 𝑋𝑋 = 2 𝑌𝑌 = 𝑦𝑦

4. 𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 𝑦𝑦

18

True or false?

5.

6.

7. �𝑥𝑥

�𝑦𝑦

𝑃𝑃 𝑋𝑋 = 𝑥𝑥|𝑌𝑌 = 𝑦𝑦 𝑃𝑃 𝑌𝑌 = 𝑦𝑦 = 1

�𝑦𝑦

𝑃𝑃 𝑋𝑋 = 2|𝑌𝑌 = 𝑦𝑦 = 1

�𝑥𝑥

𝑃𝑃 𝑋𝑋 = 𝑥𝑥|𝑌𝑌 = 5 = 1

𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦

𝑃𝑃 𝑌𝑌 = 𝑦𝑦

Discrete probabilities of CS109Each student responds with

(major 𝑋𝑋, year 𝑌𝑌, bool excited about midterm 𝑀𝑀):

19

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑌𝑌 = 1 .006 .000 .000 .000 .000 .000𝑌𝑌 = 2 .155 .069 .034 .006 .023 .029𝑌𝑌 = 3 .092 .063 .023 .006 .006 .000𝑌𝑌 = 4 .017 .029 .011 .006 .000 .000𝑌𝑌 ≥ 5 .029 .006 .011 .006 .000 .000𝑌𝑌 = 1 .000 .000 .000 .000 .000 .000𝑌𝑌 = 2 .126 .040 .017 .017 .000 .017𝑌𝑌 = 3 .046 .040 .006 .011 .000 .006𝑌𝑌 = 4 .006 .006 .000 .000 .000 .000𝑌𝑌 ≥ 5 .006 .000 .017 .011 .000 .000

𝑀𝑀=

0𝑀𝑀

=1

𝑃𝑃 𝑌𝑌 ≥ 5,𝑋𝑋 = 1 ,𝑀𝑀 = 1

Joint PMF of 𝑋𝑋,𝑌𝑌,𝑀𝑀

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .299 .167 .080 .023 .029 .029𝑀𝑀 = 1 .184 .086 .040 .040 .000 .023

𝑃𝑃 𝑋𝑋 = 1 ,𝑀𝑀 = 1 = 0.18

Joint PMF of 𝑋𝑋,𝑀𝑀

Discrete probabilities of CS109The below tables are probability tables for the conditional PMFs𝑃𝑃 𝑀𝑀 = 𝑚𝑚|𝑋𝑋 = 𝑥𝑥 and 𝑃𝑃 𝑋𝑋 = 𝑥𝑥|𝑀𝑀 = 𝑚𝑚 .

1. Which table is which?2. Fill in the missing probability.

20

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .299 .167 .080 .023 .029 .029𝑀𝑀 = 1 .184 .086 .040 .040 .000 .023

𝑃𝑃 𝑋𝑋 = 1 ,𝑀𝑀 = 1 = 0.18

Joint PMF of 𝑋𝑋,𝑀𝑀

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .619 .659 .667 .364 1.000 .556𝑀𝑀 = 1 .381 .341 .333 .636 .000 .444

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .477 .266 .128 .037 .046 .046𝑀𝑀 = 1 .231 .108 .108 .000 .062

𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦

𝑃𝑃 𝑌𝑌 = 𝑦𝑦

Be clear about which probabilities should sum to one.

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

21

Ratio of probabilities interlude

Relative probabilities of continuous random variablesLet 𝑋𝑋 = time to finish problem set 2.Suppose 𝑋𝑋~𝒩𝒩 10,2 .How much more likely are you to complete in 10 hours than 5 hours?

22

𝑃𝑃 𝑋𝑋 = 10𝑃𝑃 𝑋𝑋 = 5

=

A. 00

= undefined

B. 𝑓𝑓 10𝑓𝑓 5

Relative probabilities of continuous random variables

23

Ratios of PDFs are meaningful

𝑃𝑃 𝑋𝑋 = 𝑎𝑎 = 𝑃𝑃 𝑎𝑎 −𝜀𝜀2≤ 𝑋𝑋 ≤ 𝑎𝑎 +

𝜀𝜀2

𝑃𝑃 𝑋𝑋 = 10𝑃𝑃 𝑋𝑋 = 5

=𝑓𝑓 10𝑓𝑓 5

𝑃𝑃 𝑋𝑋 = 𝑎𝑎𝑃𝑃 𝑋𝑋 = 𝑏𝑏

=𝜀𝜀𝑓𝑓 𝑎𝑎𝜀𝜀𝑓𝑓 𝑏𝑏

=𝑓𝑓 𝑎𝑎𝑓𝑓 𝑏𝑏Therefore

= �𝑎𝑎−𝜀𝜀2

𝑎𝑎+𝜀𝜀2𝑓𝑓 𝑥𝑥 𝑑𝑑𝑥𝑥 ≈ 𝜀𝜀𝑓𝑓(𝑎𝑎)

=

1𝜎𝜎 2𝜋𝜋

𝑒𝑒−10 − 𝜇𝜇 2

2𝜎𝜎2

1𝜎𝜎 2𝜋𝜋

𝑒𝑒−5 − 𝜇𝜇 2

2𝜎𝜎2

=𝑒𝑒−

10 −10 2

2⋅2

𝑒𝑒−5 − 10 2

2⋅2

=𝑒𝑒0

𝑒𝑒−254

= 518

Let 𝑋𝑋 = time to finish problem set 2.Suppose 𝑋𝑋~𝒩𝒩 10,2 .How much more likely are you to complete in 10 hours than 5 hours?

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

24

Ratio of probabilities interlude

For continuous RVs 𝑋𝑋 and 𝑌𝑌, the conditional PDF of 𝑋𝑋 given 𝑌𝑌 is

𝑓𝑓𝑋𝑋|𝑌𝑌 𝑥𝑥|𝑦𝑦 =𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦

Intuition:

Note that conditional PDF 𝑓𝑓𝑋𝑋|𝑌𝑌 is a true density:

Continuous conditional distributions

25

𝑃𝑃 𝑋𝑋 = 𝑥𝑥 𝑌𝑌 = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦

𝑃𝑃 𝑌𝑌 = 𝑦𝑦𝑓𝑓𝑋𝑋|𝑌𝑌 𝑥𝑥 𝑦𝑦 𝜀𝜀𝑋𝑋 =

𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 𝜀𝜀𝑥𝑥𝜀𝜀𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦 𝜀𝜀𝑦𝑦

�−∞

∞𝑓𝑓𝑥𝑥 𝑥𝑥|𝑦𝑦 𝑑𝑑𝑥𝑥 = �

−∞

∞ 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦

𝑑𝑑𝑥𝑥 =∫−∞∞ 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 𝑑𝑑𝑥𝑥

𝑓𝑓𝑌𝑌 𝑦𝑦=𝑓𝑓𝑌𝑌 𝑦𝑦𝑓𝑓𝑌𝑌 𝑦𝑦

= 1

Bayes’ Theorem with Continuous RVsFor continuous RVs 𝑋𝑋 and 𝑌𝑌,

𝑓𝑓𝑌𝑌|𝑋𝑋 𝑦𝑦|𝑥𝑥 =𝑓𝑓𝑋𝑋|𝑌𝑌 𝑥𝑥|𝑦𝑦 𝑓𝑓𝑌𝑌 𝑦𝑦

𝑓𝑓𝑋𝑋 𝑥𝑥

26

𝑃𝑃 𝑌𝑌 = 𝑦𝑦 𝑋𝑋 = 𝑥𝑥 =𝑃𝑃 𝑋𝑋 = 𝑥𝑥|𝑌𝑌 = 𝑦𝑦 𝑃𝑃 𝑌𝑌 = 𝑦𝑦

𝑃𝑃 𝑋𝑋 = 𝑥𝑥

𝑓𝑓𝑌𝑌|𝑋𝑋 𝑦𝑦 𝑥𝑥 𝜀𝜀𝑌𝑌 =𝑓𝑓𝑋𝑋|𝑌𝑌 𝑥𝑥|𝑦𝑦 𝜀𝜀𝑋𝑋 𝑓𝑓𝑌𝑌 𝑦𝑦 𝜀𝜀𝑦𝑦

𝑓𝑓𝑋𝑋 𝑥𝑥 𝜀𝜀𝑋𝑋

Intuition:

Tracking in 2-D space?

27

You want to knowthe 2-D location ofan object.

Your satellite pinggives you a noisy 1-Dmeasurement of thedistance of the objectfrom the satellite (0,0).

Tracking in 2-D space• You have a prior belief about the 2-D location of an object, 𝑋𝑋,𝑌𝑌 .• You observe a noisy distance measurement, 𝐷𝐷 = 4.• What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

28

posteriorbelief

likelihood(of evidence)

priorbelief

normalization constant

Recall Bayes terminology:

𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑥𝑥, 𝑦𝑦|𝑑𝑑 =𝑓𝑓𝐷𝐷|𝑋𝑋,𝑌𝑌 𝑑𝑑|𝑥𝑥, 𝑦𝑦 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥, 𝑦𝑦

𝑓𝑓𝐷𝐷 𝑑𝑑

Top-down view

Tracking in 2-D space• You have a prior belief about the 2-D location of an object, 𝑋𝑋,𝑌𝑌 .• You observe a noisy distance measurement, 𝐷𝐷 = 4.• What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

29

Let 𝑋𝑋,𝑌𝑌 = object’s 2-D location.(your satellite is at (0,0)

Suppose the prior distribution is asymmetric bivariate normal distribution:

𝑥𝑥

𝑦𝑦

𝑓𝑓 𝑋𝑋,𝑌𝑌𝑥𝑥,𝑦𝑦

3-D view

𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 =1

2𝜋𝜋22𝑒𝑒−

𝑥𝑥−3 2+ 𝑦𝑦−3 2

2 22

normalizing constant

= 𝐾𝐾1 ⋅ 𝑒𝑒− 𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

Tracking in 2-D space• You have a prior belief about the 2-D location of an object, 𝑋𝑋,𝑌𝑌 .• You observe a noisy distance measurement, 𝐷𝐷 = 4.• What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

30

Let 𝐷𝐷 = distance from the satellite (radially).Suppose you knew your actual position: 𝑥𝑥,𝑦𝑦 .• 𝐷𝐷 is still noisy! Suppose noise is unit variance: 𝜎𝜎2 = 1• On average, 𝐷𝐷 is your actual position: 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2

If you knew your actual location 𝑥𝑥,𝑦𝑦 , you could say how likely

a measurement 𝐷𝐷 = 4 is!!

Tracking in 2-D space

31

prob

abili

ty d

ensi

ty 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2

𝜎𝜎2 = 1

Distance measurement of a ping is normal with respect to the true location.

Tracking in 2-D space• You have a prior belief about the 2-D location of an object, 𝑋𝑋,𝑌𝑌 .• You observe a noisy distance measurement, 𝐷𝐷 = 4.• What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

32

If noise is normal: 𝐷𝐷|𝑋𝑋,𝑌𝑌~𝑁𝑁 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2 ,𝜎𝜎2 = 1

Distance measurement of a ping is normal with respect to the true location.

If you knew your actual location 𝑥𝑥,𝑦𝑦 , you could say how likely

a measurement 𝐷𝐷 = 4 is!!

prob

abili

ty d

ensi

ty 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2

𝜎𝜎2 = 1

Tracking in 2-D space• You have a prior belief about the 2-D location of an object, 𝑋𝑋,𝑌𝑌 .• You observe a noisy distance measurement, 𝐷𝐷 = 4.• What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

33

If you knew your actual location 𝑥𝑥,𝑦𝑦 , you could say how likely

a measurement 𝐷𝐷 = 4 is!!𝐷𝐷|𝑋𝑋,𝑌𝑌~𝒩𝒩 𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2 ,𝜎𝜎2 = 1

𝑓𝑓𝐷𝐷|𝑋𝑋,𝑌𝑌 𝐷𝐷 = 𝑑𝑑|𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦 =1

𝜎𝜎 2𝜋𝜋𝑒𝑒− 𝑑𝑑−𝜇𝜇 2

2𝜎𝜎2

=12𝜋𝜋

𝑒𝑒− 𝑑𝑑− 𝑥𝑥2+𝑦𝑦2

2

2 = 𝑒𝑒− 𝑑𝑑− 𝑥𝑥2+𝑦𝑦2

2

2

normalizing constant

𝐾𝐾2 ⋅

Tracking in 2-D space• You have a prior belief about the 2-D location of an object, 𝑋𝑋,𝑌𝑌 .• You observe a noisy distance measurement, 𝐷𝐷 = 4.• What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

34

Observation likelihood

𝑓𝑓𝐷𝐷|𝑋𝑋,𝑌𝑌 𝑑𝑑|𝑥𝑥,𝑦𝑦 = 𝐾𝐾2 ⋅ 𝑒𝑒− 𝑑𝑑− 𝑥𝑥2+𝑦𝑦2

2

2

𝜇𝜇 = 𝑥𝑥2 + 𝑦𝑦2

𝜎𝜎2 = 1Top-down view

Prior belief

𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 = 𝐾𝐾1 ⋅ 𝑒𝑒− 𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑥𝑥,𝑦𝑦|4 = 𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦|𝐷𝐷 = 4Posteriorbelief

Tracking in 2-D spaceWhat is your updated (posterior) belief of the 2-D location of the object after observing the measurement?

35

𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦|𝐷𝐷 = 4 =𝑓𝑓𝐷𝐷|𝑋𝑋,𝑌𝑌 𝐷𝐷 = 4|𝑋𝑋 = 𝑥𝑥,𝑌𝑌 = 𝑦𝑦 𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦

𝑓𝑓(𝐷𝐷 = 4)Bayes’Theorem

=𝐾𝐾2 ⋅ 𝑒𝑒

−4− 𝑥𝑥2+𝑦𝑦2

2

2 ⋅ 𝐾𝐾1 ⋅ 𝑒𝑒−

𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

𝑓𝑓(𝐷𝐷 = 4)

likelihood of 𝐷𝐷 = 4 prior belief

=𝐾𝐾3 ⋅ 𝑒𝑒

−4− 𝑥𝑥2+𝑦𝑦2

2

2 +𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

𝑓𝑓(𝐷𝐷 = 4)

= 𝐾𝐾4 ⋅ 𝑒𝑒−

4− 𝑥𝑥2+𝑦𝑦22

2 + 𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

Tracking in 2-D space: Posterior belief

36

𝑓𝑓𝑋𝑋,𝑌𝑌 𝑥𝑥,𝑦𝑦 = 𝐾𝐾1 ⋅ 𝑒𝑒− 𝑥𝑥−3 2+ 𝑦𝑦−3 2

8

Prior belief Posterior beliefTop-down view

𝑦𝑦

3-D view

𝑥𝑥

𝑦𝑦

𝑥𝑥

Top-down view 3-D view

𝑓𝑓𝑋𝑋,𝑌𝑌|𝐷𝐷 𝑥𝑥,𝑦𝑦|4 =

𝐾𝐾4⋅ 𝑒𝑒−

4− 𝑥𝑥2+𝑦𝑦22

2 +𝑥𝑥−3 2+ 𝑦𝑦−3 2

8