14: Conditional Distributions

Post on 05-Apr-2022

5 views 0 download

Preview:

Click to see full reader
Report this document
SHARE

Transcript of 14: Conditional Distributions

14: Conditional DistributionsDavid VarodayanFebruary 7, 2020Adapted from slides by Lisa Yan

1

Sum of independent random variables

2

Review

𝑋𝑋~Bin 𝑛𝑛1,𝑝𝑝 ,π‘Œπ‘Œ~Bin(𝑛𝑛2,𝑝𝑝) 𝑋𝑋 + π‘Œπ‘Œ ~Bin(𝑛𝑛1 + 𝑛𝑛2,𝑝𝑝)𝑋𝑋,π‘Œπ‘Œ independent

𝑋𝑋~Poi πœ†πœ†1 ,π‘Œπ‘Œ~Poi πœ†πœ†2𝑋𝑋,π‘Œπ‘Œ independent

𝑋𝑋 + π‘Œπ‘Œ ~Poi(πœ†πœ†1 + πœ†πœ†2)

𝑋𝑋~𝒩𝒩 πœ‡πœ‡1,𝜎𝜎12π‘Œπ‘Œ~𝒩𝒩 πœ‡πœ‡2,𝜎𝜎22𝑋𝑋,π‘Œπ‘Œ independent

𝑋𝑋 + π‘Œπ‘Œ ~𝒩𝒩(πœ‡πœ‡1 + πœ‡πœ‡2,𝜎𝜎12 + 𝜎𝜎22)

Note: these also hold in the general case (β‰₯ 2 variables)

Quick questionLet 𝐴𝐴~Bin 30, 0.01 and 𝐡𝐡~Bin 50, 0.02 be independent RVs. Can we use sum of independent Poisson RVs to approximate 𝑃𝑃 𝐴𝐴 + 𝐡𝐡 = 2 ?

3

Review

Quick questionLet 𝐴𝐴~Bin 30, 0.01 and 𝐡𝐡~Bin 50, 0.02 be independent RVs. Can we use sum of independent Poisson RVs to approximate 𝑃𝑃 𝐴𝐴 + 𝐡𝐡 = 2 ?

4

Review

𝐴𝐴 β‰ˆ 𝑋𝑋~Poi πœ†πœ†1 = 30 β‹… 0.01 = 0.3𝐡𝐡 β‰ˆ π‘Œπ‘Œ~Poi πœ†πœ†2 = 50 β‹… 0.02 = 1 𝑃𝑃 𝐴𝐴 + 𝐡𝐡 = 2 β‰ˆ 𝑃𝑃 𝑋𝑋 + π‘Œπ‘Œ = 2

Sol 1: Approximate as sum of Poissons

𝑋𝑋 + π‘Œπ‘Œ~Poi πœ†πœ† = πœ†πœ†1 + πœ†πœ†2 = 1.3

=πœ†πœ†2

2!π‘’π‘’βˆ’πœ†πœ†

Sol 2: No approximation

𝑃𝑃 𝐴𝐴 + 𝐡𝐡 = 2 = οΏ½π‘˜π‘˜=0

2

𝑃𝑃 𝐴𝐴 = π‘˜π‘˜ 𝑃𝑃 𝐡𝐡 = 2 βˆ’ π‘˜π‘˜

β‰ˆ 0.2302

= οΏ½π‘˜π‘˜=0

230π‘˜π‘˜ 0.01π‘˜π‘˜ 0.99 30βˆ’π‘˜π‘˜ 50

2 βˆ’ π‘˜π‘˜ 0.022βˆ’π‘˜π‘˜0.9848+π‘˜π‘˜ β‰ˆ 0.2327

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

5

Ratio of probabilities interlude

Convolution

6

the convolutionof 𝑓𝑓𝑋𝑋 and π‘“π‘“π‘Œπ‘Œ

For independent continuous random variables 𝑋𝑋 and π‘Œπ‘Œ:

𝑓𝑓𝑋𝑋+π‘Œπ‘Œ 𝛼𝛼 = οΏ½βˆ’βˆž

βˆžπ‘“π‘“π‘‹π‘‹ π‘˜π‘˜ π‘“π‘“π‘Œπ‘Œ 𝛼𝛼 βˆ’ π‘˜π‘˜ π‘‘π‘‘π‘˜π‘˜

the convolutionof 𝑝𝑝𝑋𝑋 and π‘π‘π‘Œπ‘Œ

Recall for independent discrete random variables 𝑋𝑋 and π‘Œπ‘Œ:

𝑃𝑃 𝑋𝑋 + π‘Œπ‘Œ = 𝑛𝑛 = οΏ½π‘˜π‘˜

𝑃𝑃 𝑋𝑋 = π‘˜π‘˜ 𝑃𝑃 π‘Œπ‘Œ = 𝑛𝑛 βˆ’ π‘˜π‘˜

Sum of independent UniformsLet 𝑋𝑋~Uni 0,1 and π‘Œπ‘Œ~Uni 0,1 be independent random variables.What is the distribution of 𝑋𝑋 + π‘Œπ‘Œ, 𝑓𝑓𝑋𝑋+π‘Œπ‘Œ?

7

𝑓𝑓𝑋𝑋+π‘Œπ‘Œ 𝛼𝛼 = οΏ½βˆ’βˆž

βˆžπ‘“π‘“π‘‹π‘‹ π‘₯π‘₯ π‘“π‘“π‘Œπ‘Œ 𝛼𝛼 βˆ’ π‘₯π‘₯ 𝑑𝑑π‘₯π‘₯

𝑋𝑋 and π‘Œπ‘Œindependent+ continuous

𝑓𝑓𝑋𝑋+π‘Œπ‘Œ 𝛼𝛼 = οΏ½βˆ’βˆž

βˆžπ‘“π‘“π‘‹π‘‹ π‘˜π‘˜ π‘“π‘“π‘Œπ‘Œ 𝛼𝛼 βˆ’ π‘˜π‘˜ π‘‘π‘‘π‘˜π‘˜

𝑓𝑓𝑋𝑋 π‘˜π‘˜ π‘“π‘“π‘Œπ‘Œ 𝛼𝛼 βˆ’ π‘˜π‘˜ = 1 when:

𝑓𝑓𝑋𝑋+π‘Œπ‘Œ 𝛼𝛼 = οΏ½π‘Žπ‘Ž 0 ≀ π‘Žπ‘Ž ≀ 1

2 βˆ’ π‘Žπ‘Ž 1 ≀ π‘Žπ‘Ž ≀ 20 otherwise

0

1/2

𝛼𝛼

𝑓𝑓 𝑋𝑋+π‘Œπ‘Œπ›Όπ›Ό

1/2 1 3/2 2

1

00 ≀ π‘˜π‘˜ ≀ 1 and0 ≀ 𝛼𝛼 βˆ’ π‘˜π‘˜ ≀ 1 β‡’ 𝛼𝛼 βˆ’ 1 ≀ π‘˜π‘˜ ≀ 𝛼𝛼

The precise integrationbounds on π‘˜π‘˜ depend on 𝛼𝛼.

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

8

Properties of Expectation, extended to two RVs1. Linearity:

𝐸𝐸 π‘Žπ‘Žπ‘‹π‘‹ + π‘π‘π‘Œπ‘Œ + 𝑐𝑐 = π‘Žπ‘ŽπΈπΈ 𝑋𝑋 + 𝑏𝑏𝐸𝐸 π‘Œπ‘Œ + 𝑐𝑐

2. Expectation of a sum = sum of expectation:

𝐸𝐸 𝑋𝑋 + π‘Œπ‘Œ = 𝐸𝐸 𝑋𝑋 + 𝐸𝐸 π‘Œπ‘Œ

3. Unconscious statistician:

9

(we’ve seen this; we’ll prove this next)

𝐸𝐸 𝑔𝑔 𝑋𝑋,π‘Œπ‘Œ = οΏ½π‘₯π‘₯

�𝑦𝑦

𝑔𝑔 π‘₯π‘₯,𝑦𝑦 𝑝𝑝𝑋𝑋,π‘Œπ‘Œ(π‘₯π‘₯,𝑦𝑦)

𝐸𝐸 𝑔𝑔 𝑋𝑋,π‘Œπ‘Œ = οΏ½βˆ’βˆž

βˆžοΏ½βˆ’βˆž

βˆžπ‘”π‘” π‘₯π‘₯,𝑦𝑦 𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 𝑑𝑑π‘₯π‘₯ 𝑑𝑑𝑦𝑦

Proof of expectation of a sum of RVs

10

Even if the joint distribution is unknown, you can calculate the expectation of sum as sum of expectations.

Example: 𝐸𝐸 βˆ‘π‘–π‘–=1𝑛𝑛 𝑋𝑋𝑖𝑖 = βˆ‘π‘–π‘–=1𝑛𝑛 𝐸𝐸 𝑋𝑋𝑖𝑖 despite dependent trials 𝑋𝑋𝑖𝑖

𝐸𝐸 𝑋𝑋 + π‘Œπ‘Œ = 𝐸𝐸 𝑋𝑋 + 𝐸𝐸 π‘Œπ‘Œ

οΏ½π‘₯π‘₯

�𝑦𝑦

π‘₯π‘₯ + 𝑦𝑦 𝑝𝑝𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 LOTUS,𝑔𝑔 𝑋𝑋,π‘Œπ‘Œ = 𝑋𝑋 + π‘Œπ‘Œ

= οΏ½π‘₯π‘₯

�𝑦𝑦

π‘₯π‘₯𝑝𝑝𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 + οΏ½π‘₯π‘₯

�𝑦𝑦

𝑦𝑦𝑝𝑝𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦

Linearity of summations(cont. case: linearity of integrals)

= οΏ½π‘₯π‘₯

π‘₯π‘₯�𝑦𝑦

𝑝𝑝𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 + �𝑦𝑦

𝑦𝑦�π‘₯π‘₯

𝑝𝑝𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦

Marginal PMFs for 𝑋𝑋 and π‘Œπ‘Œ= οΏ½π‘₯π‘₯

π‘₯π‘₯𝑝𝑝𝑋𝑋 π‘₯π‘₯ + �𝑦𝑦

π‘¦π‘¦π‘π‘π‘Œπ‘Œ 𝑦𝑦

= 𝐸𝐸 𝑋𝑋 + 𝐸𝐸[π‘Œπ‘Œ]

𝐸𝐸 𝑋𝑋 + π‘Œπ‘Œ =

Expectations of common RVs

11

𝑋𝑋~Bin(𝑛𝑛,𝑝𝑝) 𝐸𝐸 𝑋𝑋 = 𝑛𝑛𝑝𝑝

𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐸𝐸 𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

Review

Expectations of common RVs

12

𝑋𝑋~Bin(𝑛𝑛,𝑝𝑝) 𝐸𝐸 𝑋𝑋 = 𝑛𝑛𝑝𝑝

π‘Œπ‘Œ~NegBin(π‘Ÿπ‘Ÿ, 𝑝𝑝) 𝐸𝐸 π‘Œπ‘Œ = π‘Ÿπ‘Ÿπ‘π‘

𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐸𝐸 𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

How should we define π‘Œπ‘Œπ‘–π‘–? A. π‘Œπ‘Œπ‘–π‘– = 𝑖𝑖th trial is heads. π‘Œπ‘Œπ‘–π‘–~Ber 𝑝𝑝 , 𝑖𝑖 = 1, … ,𝑛𝑛B. π‘Œπ‘Œπ‘–π‘– = # trials to get 𝑖𝑖th success (after 𝑖𝑖 βˆ’ 1 th success)

π‘Œπ‘Œπ‘–π‘–~Geo 𝑝𝑝 , 𝑖𝑖 = 1, … , π‘Ÿπ‘ŸC. π‘Œπ‘Œπ‘–π‘– = # successes in 𝑛𝑛 trials

π‘Œπ‘Œπ‘–π‘–~Bin 𝑛𝑛,𝑝𝑝 , 𝑖𝑖 = 1, … , π‘Ÿπ‘Ÿ, we look for 𝑃𝑃 π‘Œπ‘Œπ‘–π‘– = 1

π‘Œπ‘Œ = �𝑖𝑖=1

?

π‘Œπ‘Œπ‘–π‘–

Suppose:

Expectations of common RVs

13

𝑋𝑋~Bin(𝑛𝑛,𝑝𝑝) 𝐸𝐸 𝑋𝑋 = 𝑛𝑛𝑝𝑝

π‘Œπ‘Œ~NegBin(π‘Ÿπ‘Ÿ, 𝑝𝑝) 𝐸𝐸 π‘Œπ‘Œ = π‘Ÿπ‘Ÿπ‘π‘

𝑋𝑋 = �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 𝐸𝐸 𝑋𝑋 = 𝐸𝐸 �𝑖𝑖=1

𝑛𝑛

𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐸𝐸 𝑋𝑋𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝑝𝑝 = 𝑛𝑛𝑝𝑝Let 𝑋𝑋𝑖𝑖 = 𝑖𝑖th trial is heads𝑋𝑋𝑖𝑖~Ber 𝑝𝑝 ,𝐸𝐸 𝑋𝑋𝑖𝑖 = 𝑝𝑝

π‘Œπ‘Œ = �𝑖𝑖=1

π‘Ÿπ‘Ÿ

π‘Œπ‘Œπ‘–π‘–

Let π‘Œπ‘Œπ‘–π‘– = # trials to get 𝑖𝑖thsuccess (after𝑖𝑖 βˆ’ 1 th success)

π‘Œπ‘Œπ‘–π‘–~Geo 𝑝𝑝 ,𝐸𝐸 π‘Œπ‘Œπ‘–π‘– = 1𝑝𝑝

𝐸𝐸 π‘Œπ‘Œ = 𝐸𝐸 �𝑖𝑖=1

π‘Ÿπ‘Ÿ

π‘Œπ‘Œπ‘–π‘– = �𝑖𝑖=1

π‘Ÿπ‘Ÿ

𝐸𝐸 π‘Œπ‘Œπ‘–π‘– = �𝑖𝑖=1

π‘Ÿπ‘Ÿ1𝑝𝑝

=π‘Ÿπ‘Ÿπ‘π‘

Midterm exam

When: Monday, February 10, 7:00pm-9:00pmWhere: Cubberley Auditorium

Not permitted: book/computer/calculatorPermitted: Three 8.5”x11” double-sided sheets of notes

Covers: Up to (and including) week 4 + Lecture Notes 11Practice: http://web.stanford.edu/class/cs109/exams/midterm.htmlReview session: Tomorrow Saturday, 3-5pm, STLC 111

Announcements

14

not recorded; materials will be posted though

http://web.stanford.edu/class/cs109/exams/midterm.html

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

15

CS109 roadmap

16

Multiple events:

𝑃𝑃 𝐸𝐸 𝐹𝐹 =𝑃𝑃 𝐸𝐸𝐹𝐹𝑃𝑃 𝐹𝐹

conditional probability

𝑃𝑃 𝐸𝐸𝐹𝐹 = 𝑃𝑃 𝐸𝐸 𝑃𝑃(𝐹𝐹)

independence

Joint (Multivariate) distributions

joint PMF/PDF

𝑝𝑝𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦

intersection

𝑃𝑃 𝐸𝐸 ∩ 𝐹𝐹= 𝑃𝑃 𝐸𝐸𝐹𝐹

conditional distributions?

today

independentRVs

sum of independent RVs

Discrete conditional distributionsRecall the definition of the conditional probability of event 𝐸𝐸 given event 𝐹𝐹:

𝑃𝑃 𝐸𝐸 𝐹𝐹 =𝑃𝑃 𝐸𝐸𝐹𝐹𝑃𝑃 𝐹𝐹

For discrete random variables 𝑋𝑋 and π‘Œπ‘Œ, the conditional PMF of 𝑋𝑋 given π‘Œπ‘Œ is

𝑃𝑃 𝑋𝑋 = π‘₯π‘₯ π‘Œπ‘Œ = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦

𝑃𝑃 π‘Œπ‘Œ = 𝑦𝑦

𝑝𝑝𝑋𝑋|π‘Œπ‘Œ π‘₯π‘₯|𝑦𝑦 =𝑝𝑝𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,π‘¦π‘¦π‘π‘π‘Œπ‘Œ 𝑦𝑦

17

Different notation,same idea:

Quick checkNumber or function?

1. 𝑃𝑃 𝑋𝑋 = 2 π‘Œπ‘Œ = 5

2. 𝑃𝑃 𝑋𝑋 = π‘₯π‘₯ π‘Œπ‘Œ = 5

3. 𝑃𝑃 𝑋𝑋 = 2 π‘Œπ‘Œ = 𝑦𝑦

4. 𝑃𝑃 𝑋𝑋 = π‘₯π‘₯ π‘Œπ‘Œ = 𝑦𝑦

18

True or false?

5.

6.

7. οΏ½π‘₯π‘₯

�𝑦𝑦

𝑃𝑃 𝑋𝑋 = π‘₯π‘₯|π‘Œπ‘Œ = 𝑦𝑦 𝑃𝑃 π‘Œπ‘Œ = 𝑦𝑦 = 1

�𝑦𝑦

𝑃𝑃 𝑋𝑋 = 2|π‘Œπ‘Œ = 𝑦𝑦 = 1

οΏ½π‘₯π‘₯

𝑃𝑃 𝑋𝑋 = π‘₯π‘₯|π‘Œπ‘Œ = 5 = 1

𝑃𝑃 𝑋𝑋 = π‘₯π‘₯ π‘Œπ‘Œ = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦

𝑃𝑃 π‘Œπ‘Œ = 𝑦𝑦

Discrete probabilities of CS109Each student responds with

(major 𝑋𝑋, year π‘Œπ‘Œ, bool excited about midterm 𝑀𝑀):

19

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

π‘Œπ‘Œ = 1 .006 .000 .000 .000 .000 .000π‘Œπ‘Œ = 2 .155 .069 .034 .006 .023 .029π‘Œπ‘Œ = 3 .092 .063 .023 .006 .006 .000π‘Œπ‘Œ = 4 .017 .029 .011 .006 .000 .000π‘Œπ‘Œ β‰₯ 5 .029 .006 .011 .006 .000 .000π‘Œπ‘Œ = 1 .000 .000 .000 .000 .000 .000π‘Œπ‘Œ = 2 .126 .040 .017 .017 .000 .017π‘Œπ‘Œ = 3 .046 .040 .006 .011 .000 .006π‘Œπ‘Œ = 4 .006 .006 .000 .000 .000 .000π‘Œπ‘Œ β‰₯ 5 .006 .000 .017 .011 .000 .000

𝑀𝑀=

0𝑀𝑀

=1

𝑃𝑃 π‘Œπ‘Œ β‰₯ 5,𝑋𝑋 = 1 ,𝑀𝑀 = 1

Joint PMF of 𝑋𝑋,π‘Œπ‘Œ,𝑀𝑀

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .299 .167 .080 .023 .029 .029𝑀𝑀 = 1 .184 .086 .040 .040 .000 .023

𝑃𝑃 𝑋𝑋 = 1 ,𝑀𝑀 = 1 = 0.18

Joint PMF of 𝑋𝑋,𝑀𝑀

Discrete probabilities of CS109The below tables are probability tables for the conditional PMFs𝑃𝑃 𝑀𝑀 = π‘šπ‘š|𝑋𝑋 = π‘₯π‘₯ and 𝑃𝑃 𝑋𝑋 = π‘₯π‘₯|𝑀𝑀 = π‘šπ‘š .

1. Which table is which?2. Fill in the missing probability.

20

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .299 .167 .080 .023 .029 .029𝑀𝑀 = 1 .184 .086 .040 .040 .000 .023

𝑃𝑃 𝑋𝑋 = 1 ,𝑀𝑀 = 1 = 0.18

Joint PMF of 𝑋𝑋,𝑀𝑀

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .619 .659 .667 .364 1.000 .556𝑀𝑀 = 1 .381 .341 .333 .636 .000 .444

CS𝑋𝑋 = 1

SymSys/MCS/EE𝑋𝑋 = 2

Other Eng/Sci/

Math𝑋𝑋 = 3

Hum/SocSci/

Ling𝑋𝑋 = 4

Double major𝑋𝑋 = 5

Undec.𝑋𝑋 = 6

𝑀𝑀 = 0 .477 .266 .128 .037 .046 .046𝑀𝑀 = 1 .231 .108 .108 .000 .062

𝑃𝑃 𝑋𝑋 = π‘₯π‘₯ π‘Œπ‘Œ = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦

𝑃𝑃 π‘Œπ‘Œ = 𝑦𝑦

Be clear about which probabilities should sum to one.

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

21

Ratio of probabilities interlude

Relative probabilities of continuous random variablesLet 𝑋𝑋 = time to finish problem set 2.Suppose 𝑋𝑋~𝒩𝒩 10,2 .How much more likely are you to complete in 10 hours than 5 hours?

22

𝑃𝑃 𝑋𝑋 = 10𝑃𝑃 𝑋𝑋 = 5

=

A. 00

= undefined

B. 𝑓𝑓 10𝑓𝑓 5

Relative probabilities of continuous random variables

23

Ratios of PDFs are meaningful

𝑃𝑃 𝑋𝑋 = π‘Žπ‘Ž = 𝑃𝑃 π‘Žπ‘Ž βˆ’πœ€πœ€2≀ 𝑋𝑋 ≀ π‘Žπ‘Ž +

πœ€πœ€2

𝑃𝑃 𝑋𝑋 = 10𝑃𝑃 𝑋𝑋 = 5

=𝑓𝑓 10𝑓𝑓 5

𝑃𝑃 𝑋𝑋 = π‘Žπ‘Žπ‘ƒπ‘ƒ 𝑋𝑋 = 𝑏𝑏

=πœ€πœ€π‘“π‘“ π‘Žπ‘Žπœ€πœ€π‘“π‘“ 𝑏𝑏

=𝑓𝑓 π‘Žπ‘Žπ‘“π‘“ 𝑏𝑏Therefore

= οΏ½π‘Žπ‘Žβˆ’πœ€πœ€2

π‘Žπ‘Ž+πœ€πœ€2𝑓𝑓 π‘₯π‘₯ 𝑑𝑑π‘₯π‘₯ β‰ˆ πœ€πœ€π‘“π‘“(π‘Žπ‘Ž)

=

1𝜎𝜎 2πœ‹πœ‹

π‘’π‘’βˆ’10 βˆ’ πœ‡πœ‡ 2

2𝜎𝜎2

1𝜎𝜎 2πœ‹πœ‹

π‘’π‘’βˆ’5 βˆ’ πœ‡πœ‡ 2

2𝜎𝜎2

=π‘’π‘’βˆ’

10 βˆ’10 2

2β‹…2

π‘’π‘’βˆ’5 βˆ’ 10 2

2β‹…2

=𝑒𝑒0

π‘’π‘’βˆ’254

= 518

Let 𝑋𝑋 = time to finish problem set 2.Suppose 𝑋𝑋~𝒩𝒩 10,2 .How much more likely are you to complete in 10 hours than 5 hours?

Today’s plan

Sum of two Uniform independent RVs

Expectation of sum of two RVs

Discrete conditional distributions

Continuous conditional distributions

24

Ratio of probabilities interlude

For continuous RVs 𝑋𝑋 and π‘Œπ‘Œ, the conditional PDF of 𝑋𝑋 given π‘Œπ‘Œ is

𝑓𝑓𝑋𝑋|π‘Œπ‘Œ π‘₯π‘₯|𝑦𝑦 =𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,π‘¦π‘¦π‘“π‘“π‘Œπ‘Œ 𝑦𝑦

Intuition:

Note that conditional PDF 𝑓𝑓𝑋𝑋|π‘Œπ‘Œ is a true density:

Continuous conditional distributions

25

𝑃𝑃 𝑋𝑋 = π‘₯π‘₯ π‘Œπ‘Œ = 𝑦𝑦 =𝑃𝑃 𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦

𝑃𝑃 π‘Œπ‘Œ = 𝑦𝑦𝑓𝑓𝑋𝑋|π‘Œπ‘Œ π‘₯π‘₯ 𝑦𝑦 πœ€πœ€π‘‹π‘‹ =

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 πœ€πœ€π‘₯π‘₯πœ€πœ€π‘¦π‘¦π‘“π‘“π‘Œπ‘Œ 𝑦𝑦 πœ€πœ€π‘¦π‘¦

οΏ½βˆ’βˆž

βˆžπ‘“π‘“π‘₯π‘₯ π‘₯π‘₯|𝑦𝑦 𝑑𝑑π‘₯π‘₯ = οΏ½

βˆ’βˆž

∞ 𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,π‘¦π‘¦π‘“π‘“π‘Œπ‘Œ 𝑦𝑦

𝑑𝑑π‘₯π‘₯ =βˆ«βˆ’βˆžβˆž 𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 𝑑𝑑π‘₯π‘₯

π‘“π‘“π‘Œπ‘Œ 𝑦𝑦=π‘“π‘“π‘Œπ‘Œ π‘¦π‘¦π‘“π‘“π‘Œπ‘Œ 𝑦𝑦

= 1

Bayes’ Theorem with Continuous RVsFor continuous RVs 𝑋𝑋 and π‘Œπ‘Œ,

π‘“π‘“π‘Œπ‘Œ|𝑋𝑋 𝑦𝑦|π‘₯π‘₯ =𝑓𝑓𝑋𝑋|π‘Œπ‘Œ π‘₯π‘₯|𝑦𝑦 π‘“π‘“π‘Œπ‘Œ 𝑦𝑦

𝑓𝑓𝑋𝑋 π‘₯π‘₯

26

𝑃𝑃 π‘Œπ‘Œ = 𝑦𝑦 𝑋𝑋 = π‘₯π‘₯ =𝑃𝑃 𝑋𝑋 = π‘₯π‘₯|π‘Œπ‘Œ = 𝑦𝑦 𝑃𝑃 π‘Œπ‘Œ = 𝑦𝑦

𝑃𝑃 𝑋𝑋 = π‘₯π‘₯

π‘“π‘“π‘Œπ‘Œ|𝑋𝑋 𝑦𝑦 π‘₯π‘₯ πœ€πœ€π‘Œπ‘Œ =𝑓𝑓𝑋𝑋|π‘Œπ‘Œ π‘₯π‘₯|𝑦𝑦 πœ€πœ€π‘‹π‘‹ π‘“π‘“π‘Œπ‘Œ 𝑦𝑦 πœ€πœ€π‘¦π‘¦

𝑓𝑓𝑋𝑋 π‘₯π‘₯ πœ€πœ€π‘‹π‘‹

Intuition:

Tracking in 2-D space?

27

You want to knowthe 2-D location ofan object.

Your satellite pinggives you a noisy 1-Dmeasurement of thedistance of the objectfrom the satellite (0,0).

Tracking in 2-D spaceβ€’ You have a prior belief about the 2-D location of an object, 𝑋𝑋,π‘Œπ‘Œ .β€’ You observe a noisy distance measurement, 𝐷𝐷 = 4.β€’ What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

28

posteriorbelief

likelihood(of evidence)

priorbelief

normalization constant

Recall Bayes terminology:

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ|𝐷𝐷 π‘₯π‘₯, 𝑦𝑦|𝑑𝑑 =𝑓𝑓𝐷𝐷|𝑋𝑋,π‘Œπ‘Œ 𝑑𝑑|π‘₯π‘₯, 𝑦𝑦 𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯, 𝑦𝑦

𝑓𝑓𝐷𝐷 𝑑𝑑

Top-down view

Tracking in 2-D spaceβ€’ You have a prior belief about the 2-D location of an object, 𝑋𝑋,π‘Œπ‘Œ .β€’ You observe a noisy distance measurement, 𝐷𝐷 = 4.β€’ What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

29

Let 𝑋𝑋,π‘Œπ‘Œ = object’s 2-D location.(your satellite is at (0,0)

Suppose the prior distribution is asymmetric bivariate normal distribution:

π‘₯π‘₯

𝑦𝑦

𝑓𝑓 𝑋𝑋,π‘Œπ‘Œπ‘₯π‘₯,𝑦𝑦

3-D view

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 =1

2πœ‹πœ‹22π‘’π‘’βˆ’

π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

2 22

normalizing constant

= 𝐾𝐾1 β‹… π‘’π‘’βˆ’ π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

8

Tracking in 2-D spaceβ€’ You have a prior belief about the 2-D location of an object, 𝑋𝑋,π‘Œπ‘Œ .β€’ You observe a noisy distance measurement, 𝐷𝐷 = 4.β€’ What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

30

Let 𝐷𝐷 = distance from the satellite (radially).Suppose you knew your actual position: π‘₯π‘₯,𝑦𝑦 .β€’ 𝐷𝐷 is still noisy! Suppose noise is unit variance: 𝜎𝜎2 = 1β€’ On average, 𝐷𝐷 is your actual position: πœ‡πœ‡ = π‘₯π‘₯2 + 𝑦𝑦2

If you knew your actual location π‘₯π‘₯,𝑦𝑦 , you could say how likely

a measurement 𝐷𝐷 = 4 is!!

Tracking in 2-D space

31

prob

abili

ty d

ensi

ty πœ‡πœ‡ = π‘₯π‘₯2 + 𝑦𝑦2

𝜎𝜎2 = 1

Distance measurement of a ping is normal with respect to the true location.

Tracking in 2-D spaceβ€’ You have a prior belief about the 2-D location of an object, 𝑋𝑋,π‘Œπ‘Œ .β€’ You observe a noisy distance measurement, 𝐷𝐷 = 4.β€’ What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

32

If noise is normal: 𝐷𝐷|𝑋𝑋,π‘Œπ‘Œ~𝑁𝑁 πœ‡πœ‡ = π‘₯π‘₯2 + 𝑦𝑦2 ,𝜎𝜎2 = 1

Distance measurement of a ping is normal with respect to the true location.

If you knew your actual location π‘₯π‘₯,𝑦𝑦 , you could say how likely

a measurement 𝐷𝐷 = 4 is!!

prob

abili

ty d

ensi

ty πœ‡πœ‡ = π‘₯π‘₯2 + 𝑦𝑦2

𝜎𝜎2 = 1

Tracking in 2-D spaceβ€’ You have a prior belief about the 2-D location of an object, 𝑋𝑋,π‘Œπ‘Œ .β€’ You observe a noisy distance measurement, 𝐷𝐷 = 4.β€’ What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

33

If you knew your actual location π‘₯π‘₯,𝑦𝑦 , you could say how likely

a measurement 𝐷𝐷 = 4 is!!𝐷𝐷|𝑋𝑋,π‘Œπ‘Œ~𝒩𝒩 πœ‡πœ‡ = π‘₯π‘₯2 + 𝑦𝑦2 ,𝜎𝜎2 = 1

𝑓𝑓𝐷𝐷|𝑋𝑋,π‘Œπ‘Œ 𝐷𝐷 = 𝑑𝑑|𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦 =1

𝜎𝜎 2πœ‹πœ‹π‘’π‘’βˆ’ π‘‘π‘‘βˆ’πœ‡πœ‡ 2

2𝜎𝜎2

=12πœ‹πœ‹

π‘’π‘’βˆ’ π‘‘π‘‘βˆ’ π‘₯π‘₯2+𝑦𝑦2

2

2 = π‘’π‘’βˆ’ π‘‘π‘‘βˆ’ π‘₯π‘₯2+𝑦𝑦2

2

2

normalizing constant

𝐾𝐾2 β‹…

Tracking in 2-D spaceβ€’ You have a prior belief about the 2-D location of an object, 𝑋𝑋,π‘Œπ‘Œ .β€’ You observe a noisy distance measurement, 𝐷𝐷 = 4.β€’ What is your updated (posterior) belief of the 2-D location of the object after

observing the measurement?

34

Observation likelihood

𝑓𝑓𝐷𝐷|𝑋𝑋,π‘Œπ‘Œ 𝑑𝑑|π‘₯π‘₯,𝑦𝑦 = 𝐾𝐾2 β‹… π‘’π‘’βˆ’ π‘‘π‘‘βˆ’ π‘₯π‘₯2+𝑦𝑦2

2

2

πœ‡πœ‡ = π‘₯π‘₯2 + 𝑦𝑦2

𝜎𝜎2 = 1Top-down view

Prior belief

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 = 𝐾𝐾1 β‹… π‘’π‘’βˆ’ π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

8

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ|𝐷𝐷 π‘₯π‘₯,𝑦𝑦|4 = 𝑓𝑓𝑋𝑋,π‘Œπ‘Œ|𝐷𝐷 𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦|𝐷𝐷 = 4Posteriorbelief

Tracking in 2-D spaceWhat is your updated (posterior) belief of the 2-D location of the object after observing the measurement?

35

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ|𝐷𝐷 𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦|𝐷𝐷 = 4 =𝑓𝑓𝐷𝐷|𝑋𝑋,π‘Œπ‘Œ 𝐷𝐷 = 4|𝑋𝑋 = π‘₯π‘₯,π‘Œπ‘Œ = 𝑦𝑦 𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦

𝑓𝑓(𝐷𝐷 = 4)Bayes’Theorem

=𝐾𝐾2 β‹… 𝑒𝑒

βˆ’4βˆ’ π‘₯π‘₯2+𝑦𝑦2

2

2 β‹… 𝐾𝐾1 β‹… π‘’π‘’βˆ’

π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

8

𝑓𝑓(𝐷𝐷 = 4)

likelihood of 𝐷𝐷 = 4 prior belief

=𝐾𝐾3 β‹… 𝑒𝑒

βˆ’4βˆ’ π‘₯π‘₯2+𝑦𝑦2

2

2 +π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

8

𝑓𝑓(𝐷𝐷 = 4)

= 𝐾𝐾4 β‹… π‘’π‘’βˆ’

4βˆ’ π‘₯π‘₯2+𝑦𝑦22

2 + π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

8

Tracking in 2-D space: Posterior belief

36

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ π‘₯π‘₯,𝑦𝑦 = 𝐾𝐾1 β‹… π‘’π‘’βˆ’ π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

8

Prior belief Posterior beliefTop-down view

𝑦𝑦

3-D view

π‘₯π‘₯

𝑦𝑦

π‘₯π‘₯

Top-down view 3-D view

𝑓𝑓𝑋𝑋,π‘Œπ‘Œ|𝐷𝐷 π‘₯π‘₯,𝑦𝑦|4 =

𝐾𝐾4β‹… π‘’π‘’βˆ’

4βˆ’ π‘₯π‘₯2+𝑦𝑦22

2 +π‘₯π‘₯βˆ’3 2+ π‘¦π‘¦βˆ’3 2

8

Top related

Kernel Embeddings of Conditional Distributionslsong/papers/SonFukGre13.pdfΒ Β· Kernel embeddings of conditional distributions lead to simple algorithms to solve these problems, which
 Documents
Prbability Theory and Mathematical Statistics Lecture 05: Probability Distributions ...Β Β· 2019. 3. 27.Β Β· Introduction Multivariate Distributions Marginal Distributions Conditional
 Documents
Conditional density estimation with spatially … smagrini@unive.it Abstract Estimating conditional distributions for the analysis of convergence is usually done using a standard kernel
 Documents
Kernel Embeddings of Conditional Distributionslsong/papers/SonFukGre13.pdfKernel Embeddings of Conditional Distributions Le Songy, Kenji Fukumizuzand Arthur Gretton yGeorgia Institute
 Documents
Limit theorems for conditional distributions with regard ...alzaig/limit.pdfΒ Β· LIMIT THEOREMS FOR CONDITIONAL DISTRIBUTIONS WITH REGARD FOR LARGE DEVIATIONS 1189 In Sec. 2, we introduce
 Documents
Conditional Distributions and the Bivariate Normal Distribution James H. Steiger.
 Documents
Probability - University of Texas at Austinsniekum/classes/343-S20/...Today Probability Random Variables Joint and Marginal Distributions Conditional Distributions Product Rule, Chain
 Documents
1 Chapter 14 Probabilistic Reasoning. 2 Outline Syntax of Bayesian networks Semantics of Bayesian networks Efficient representation of conditional distributions.
 Documents
Testing probability distributions using conditional …algorithm to obtain a draw from DS, the conditional distribution of Drestricted to a subset Sof the domain (where Sis specified
 Documents
Bayes’ Net Semantics Probabilities in BNscs188/sp10/slides...Bayes’ nets implicitly encode joint distributions As a product of local conditional distributions To see what probability
 Documents
Conditional Value-at-Risk, Methodology and Applications ...Β Β· distributions CVaR may differ from conditional expected loss exceeding VaR. β€’ Acerbi et al.1,2 recently redefined
 Documents
Conditional distributions - Stanford UniversityΒ Β· Conditional distributions Will Monroe July 26, 2017 with materials by Mehran Sahami and Chris Piech
 Documents
JointMeasurabilityandtheOne-wayFubiniProperty ...hammond/fubini.pdfΒ Β· Fubini property, exact law of large numbers, conditional distributions, characterizations of independence. βˆ—Part
 Documents
Chapter 11: Inference for Distributions of Section 11.2 ...mshache.weebly.com/uploads/2/2/9/4/22949878/11.2_powerpoint.pdfΒ Β· Example: Comparing Conditional Distributions ips (a)
 Documents
Testing probability distributions using conditional samplesccanonne/files/talks/cond-2013-03-08.pdfΒ Β· Testing probability distributions using conditional samples (when testers get
 Documents
Marginal and Conditional distributions. Theorem: (Marginal distributions for the Multivariate Normal distribution) have p-variate Normal distribution.
 Documents
Ch 14 Probability Distributions
 Documents
Conditional Distributions and Functions of Jointly Distributed … · 2020-01-03Β Β· (0, (Y (| f xy 0 X (,f X fx Y X. page 5: Lecture 5: Conditional Distributions and Functions of
 Documents
Lecture 14: Multivariate Distributions
 Documents
Determining conditional Gaussian distributions for ...projekter.aau.dk/projekter/files/225732843/main.pdfDetermining conditional Gaussian distributions for decomposable graphical models
 Documents

Copyright Β© 2022 FDOCUMENTS