Lecture 04: Conditional Probability - Stanford University · 2018. 7. 3. · Conditional...

Lecture04:ConditionalProbabilityLisaYanJuly2,2018

Announcements

ProblemSet#1dueonFriday◦Gradescope submissionportalup

UsePiazza

NoclassorOHonWednesdayJuly4th

2

Summaryfromlasttime

Samplespace,S:Thesetofallpossibleoutcomesofanexperiment.Eventspace,E:AsubsetofS.Ifwehaveequallylikelyoutcomes,thenP(E)=|E|/|S|.

Twokeytacticstocountingprobabilities:• DeMorgan’s lawandcalculatingP(Ec)helpsusavoidtrickycounting

situations.• Wecantreatindistinctobjectsasdistinctifithelpscreateequallylikely

outcomes.

3

Countsofballsandurns

Supposewehaven=2 balls,labeledAandB,areplacedinm=2 urns,Urns1and2,whereeachballisequallylikelytobeplacedinanyurn.

4

Urn 1 Urn 2

A,B -A BB A- A,B

Possibilities

Equallylikely

Urn 1 Urn 2 Prob

2 0 1/41 1 2/40 2 1/4

Counts

Notequallylikely

Goalsfortoday

Conditionalprobability◦Definitionofconditionalprobability◦ChainRule◦ LawofTotalProbability◦Bayes’Theorem◦ TheMontyHallproblem

5

Dice,ourmisunderstoodfriends

Rolltwo6-sideddice,yieldingvaluesD1 andD2.LetEbeevent:D1 + D2=4.

WhatisP(E)?

6

|S| =36

E ={(1,3),(2,2),(3,1)}

P(E) =3/36

Problem:

Solution:

=1/12

🤔

Dice,ourmisunderstoodfriendsRolltwo6-sideddice,yieldingvaluesD1 andD2.LetEbeevent:D1 + D2=4.LetFbeevent:D1 =2.WhatisP(E,givenFalreadyobserved)?

7

S ={(2,1),(2,2),(2,3),(2,4),(2,5),(2,6)}

E ={(2,2)}

P(E,givenFalreadyobserved)

Solution:

=1/6

ConditionalProbability

ConditionalprobabilityistheprobabilitythatEoccursgiventhatFhasalreadyoccurred.Thisisknownas“ConditioningonF.”

Writtenas: P(E|F)

Means: “P(E,givenFalreadyobserved)”

Samplespace,Sà allpossibleoutcomesconsistentwithF(i.e.SÇ F)

Eventspace,Eà alloutcomesinEconsistentwithF(i.e.EÇ F)

8

ConditionalProbability

ConditionalprobabilityistheprobabilitythatEoccursgiventhatFhasalreadyoccurred.Thisisknownas“ConditioningonF.”

Withequallylikelyoutcomes:

P(E|F)=

9

#ofoutcomesinEconsistentwithF#ofoutcomesinSconsistentwithF

|EF||SF|

|EF||F|

P (E|F ) =3

14⇡ 0.21

P (E) =8

50⇡ 0.16

= =

🙆

ConditionalProbability,ingeneralGeneraldefinition:

P(E|F)=(evenwhenoutcomesarenotequallylikely)

Implies: P(EF)=P(E|F)P(F)

WhatifP(F)=0?

à P(E|F)undefined,ofcourse!

Youhaveobservedtheimpossible!10

P(EF)P(F)

TheChainRule

P(EF)=P(FE)=P(F|E)P(E)Bycommutativityofsetintersection,

🤔

Slicingupthespam24emailsaresent,6eachto4users.• 10ofthe24emailsarespam.• Allpossibleoutcomesareequallylikely.E=user1receives3spamemails.WhatisP(E)?F=user2receives6spamemails.WhatisP(E|F)?G=user3receives5spamemails.WhatisP(G|F)?

11

Solution:103

143

246

≈ 0.3245

P(E)=

43

143

186

≈0.0784

P(E|F)=

141

186

= 0

Nowaytochoose5spamfrom4remainingspamemails!

45P(G|F)=

P(EF)P(F)P(E|F)=

|EF||F|

Forequallylikelyoutcomes: =

BitstringsAbitstringwithm0’sandn1’ssentonanetwork.Alldistinctarrangementsofbitsareequallylikely.

E=firstbitreceivedisa1F=k offirstr bitsreceivedare1’s

WhatisP(E|F)?

12

Solution: P(EF)P(F)

P(E|F) = = P(F|E)P(E)P(F)

P(F|E)= ,P(E)= nm+n ,P(F)=

𝑛 − 1𝑘 − 1

𝑚𝑟 − 𝑘

𝑚 + 𝑛 − 1𝑟 − 1

𝑛𝑘

𝑚𝑟 − 𝑘

𝑚 + 𝑛𝑟

Bydefinitionofconditionalprobability

Bitstrings

13

Solution:

P(E|F)=

𝑛 − 1𝑘 − 1

𝑚𝑟 − 𝑘

𝑚 + 𝑛 − 1𝑟 − 1

𝑛𝑘

𝑚𝑟 − 𝑘

𝑚 + 𝑛𝑟

Abitstringwithm0’sandn1’ssentonanetwork.Alldistinctarrangementsofbitsareequallylikely.


WhatisP(E|F)?P(EF)P(F)

P(E|F) = = P(F|E)P(E)P(F)

Bydefinitionofconditionalprobability

nm+n =

kr

Bitstrings

14

Solution2:

P(E|F)=P(pickingoneofk 1’soutofr bits)

Abitstringwithm0’sandn1’ssentonanetwork.Alldistinctarrangementsofbitsareequallylikely.


WhatisP(E|F)? Byinsight

=kr Rockon!!!

🤔

CardpilesAdeckof52cardsisrandomlydividedinto4piles(13cardsperpile).

WhatisP(eachpilecontainsexactlyoneace)?

16

E1 ={AceSpades(AS)inanyonepile}

E2 ={ASandAceHearts(AH)indifferentpiles}

E3 ={AS,AH,AceDiamonds(AD)indifferentpiles}

E4 ={All4acesindifferentpiles}

ComputeP(E1 E2 E3 E4)

=P(E1)P(E2|E1)P(E3|E1 E2)P(E4 |E1 E2 E3)

Solution:

Cardpiles

17

E1 ={AceSpades(AS)inanyonepile}

E2 ={ASandAceHearts(AH)indifferentpiles}

E3 ={AS,AH,AceDiamonds(AD)indifferentpiles}

E4 ={All4acesindifferentpiles}

P(E1)

P(E2|E1)

P(E3|E1 E2)

P(E4 |E1 E2 E3)

P(E1 E2 E3 E4)

≈ 0.105

39x26x13

51x50x49=

=1

=39/51 (39cardsnotinASpile)

=26/50 (26notinASorAHpiles)=13/49 (13notinAS,AH,ADpiles)

CardpilesAdeckof52cardsisrandomlydividedinto4piles(13cardsperpile).

WhatisP(eachpilecontainsexactlyoneace)?

18

E:multipartexperiment:

1. Deal4acesintodifferentpiles.

2. Distributerestofcardssuchthateachpilehas13cards.

S:Distributeallcardssuchthateachpilehas13cards.

Solution2:

4812,12,12,12

4!

5213,13,13,13

P(E)= |E||S| ≈ 0.105

🙆

LawofTotalProbability

P(E) =P(EF)+P(EFc)=P(F)P(E|F)+P(Fc)P(E|Fc)

19

IfyoucannotcalculateP(E)directly,ithelpstointroduceaneweventFandremoveit.

E=EFÈ EFc

Note:EFÇ EFc =ÆEF +

S

EFc

S

E

S

F E=

GeneralLawofTotalProbability

IfF1,F2,…,Fn areasetofmutuallyexclusiveandexhaustiveevents,then

20

𝑃 𝐸 ==𝑃>

?

𝐹? 𝑃 𝐸 𝐹?

BreakAttendance: t inyurl.com/cs109summer2018

21

🤔

MajoringinCSWhatistheprobabilitythatarandomlychosenstudentoncampusisaCSmajorif:

• 25%ofstudentsoncampusarejuniors

• Ifastudentisajunior,thentheyhave30%likelihoodofbeingaCSmajor

• Ifastudentisnotajunior,thentheyhave20%likelihoodofbeingaCSmajor

22

M=astudentisaCSmajor, J=astudentisajuniorWeknow: P(J)=0.25, P(M|J)=0.30, P(M|Jc)=0.20

P(M)= P(J)P(M|J) + P(Jc)P(M|Jc)

0.25x0.3 + 0.75x0.2

=0.225

LawofTotalprobability

Solution:

𝑃 𝐸 ==𝑃>

?

𝐹? 𝑃 𝐸 𝐹?

GeneralLawofTotalProbability

ThomasBayes

23

Rev.ThomasBayes(~1701-1761):BritishmathematicianandPresbyterianminister

Amomentofsilence…

24

🙆

Bayes’Theorem

25

𝑃 𝐹 𝐸 =𝑃 𝐸 𝐹 𝑃 𝐹

𝑃 𝐸


𝑃 𝐸|𝐹 𝑃 𝐹 + 𝑃 𝐸 𝐹B 𝑃(𝐹B) Expandedform:

Mostcommonform:

BayesianInterpretation#1

26


𝑃 𝐸

Wanttofind(WTF):P(F|E)

Know:P(E|F)

Bayes’isawayof“flipping”thecondition.

🤔

HIVtestingAtestis98%effectiveatdetectingHIV.• However,thetesthasa“falsepositive”rateof1%.• 0.5%oftheUSpopulationhasHIV.E=youtestpositiveforHIVwiththistestF=youactuallyhaveHIVWhatisP(F|E)?

27

Solution:

P(E|F)=0.98 “truepositive”P(E|Fc)=0.01 “falsepositive”P(F)=0.005 prior


𝑃 𝐸|𝐹 𝑃 𝐹 + 𝑃 𝐸 𝐹B 𝑃(𝐹B) (0.98)(0.005)

(0.98)(0.005)+(0.01)(1–0.005)=

≈ 0.330


𝑃 𝐸|𝐹 𝑃 𝐹 + 𝑃 𝐸 𝐹B 𝑃(𝐹B)

Bayes’Theorem

AllPeople,S

Bayes’TheoremIntuition

28

Peoplewhotestpositive,E

PeoplewithHIV,F

Say we have 1000 people:

5 have HIVand test positive

985 do not have HIVand test negative.

10 do not have HIVand test positive.

» 0.333

BayesianInterpretation#2

31

Beforethetest:prior: youmayhaveHIV,butyou

are95.5%confident.(0.5%populationhasHIV)

Thetest:likelihood: thetestreturnsusefulinformationbasedon

whetheryouhaveHIV(98%yesifHIV,1%yesifnotHIV)

Afterthetest:posterior: younowknowhavemoreinformationabout

whetheryouhaveHIV,basedontestresults.


𝑃 𝐸 posterior

likelihood prior

Normalizingconstant

BayesianInterpretations

32


𝑃 𝐸 posterior

likelihood prior

Normalizingconstant

“Youprobablythinkthatyouarebetternow,betternow,Youonlysaythat'cause I'mnotaround,notaround”– PostMalone,2018Posterior

MontyHallProblem

33

WayneBrady+

🤔

MontyHallProblemakaLet’sMakeaDealBehindonedoorisaprize(equallylikelytobeanydoor).

Behindtheothertwodoorsisnothing

1. Wechooseadoor

2. Hostopens1ofother2doors,revealingnothing

3. Wearegivenanoptiontochangetotheotherdoor.

Shouldweswitch?

34

DoorsA,B,C

Note:Ifwedon’tswitch,P(win)=1/3 (random)

WearecomparingP(win)andP(win|switch).

🤔

Ifweswitch

P(Aprize)=1/3

1.HostopensBorC

2.Weswitch

3.Wealwayslose

P(win|Aprize,pickedA,switched)=0

35

Solution:Withoutlossofgenerality,saywepickA(outofDoorsA,B,C).

P(Bprize)=1/3

1.HostmustopenC

2.WeswitchtoB

3.Wealwayswin

P(win|Bprize,pickedA,switched)=1

P(Cprize)=1/3

1.HostmustopenB

2.WeswitchtoC

3.Wealwayswin

P(win|Cprize,pickedA,switched)=1

P(win|pickedA,switched)=1/3*0+1/3*1+1/3*1=2/3

MontyHall,1000envelopeversion

Startwith1000envelopes(ofwhich1istheprize).

1. Youchoose1envelope.

2. Iopen998ofremaining999(showingtheyareempty).

3. Shouldyouswitch?

36

P(envelopeisprize)=1/1000

P(youwinwithoutswitching)=

P(youwinwithswitching)=

999/1000=P(998emptyenvelopeshadprize)+P(lastotherenvelopehasprize)

=P(remainingotherenvelopehasprize)

P(other999envelopeshaveprize)=999/1000

1original#envelopes

original#envelopes- 1original#envelopes

Lecture 04: Conditional Probability - Stanford University · 2018. 7. 3. · Conditional...

Documents

Transcript of Lecture 04: Conditional Probability - Stanford University · 2018. 7. 3. · Conditional...