03: Intro to Probabilityweb.stanford.edu/class/archive/cs/cs109/cs109.1208/lectures/03...ย ยท Proof...
Transcript of 03: Intro to Probabilityweb.stanford.edu/class/archive/cs/cs109/cs109.1208/lectures/03...ย ยท Proof...
03: Intro to ProbabilityLisa Yan
April 10, 2020
1
Lisa Yan, CS109, 2020
Quick slide reference
2
3 Defining Probability 03a_definitions
13 Axioms of Probability 03b_axioms
20 Equally likely outcomes 03c_elo
30 Corollaries 03d_corollaries
37 Exercises LIVE
Todayโs discussion thread: https://us.edstem.org/courses/109/discussion/24492
Defining Probability
3
Gradescope quiz, blank slide deck, etc.
http://cs109.stanford.edu/
03a_definitions
Lisa Yan, CS109, 2020
Key definitions
An experiment in probability:
Sample Space, ๐: The set of all possible outcomes of an experiment
Event, ๐ธ: Some subset of ๐ (๐ธ โ ๐).
Outcome
4
Experiment
๐
๐ธ
Lisa Yan, CS109, 2020 5
Key definitions
Event, ๐ธ
โข Flip lands heads๐ธ = Heads
โข โฅ 1 head on 2 coin flips๐ธ = (H,H), (H,T), (T,H)
โข Roll is 3 or less:๐ธ = 1, 2, 3
โข Low email day (โค 20 emails)๐ธ = ๐ฅ | ๐ฅ โ โค, 0 โค ๐ฅ โค 20
โข Wasted day (โฅ 5 TT hours):๐ธ = ๐ฅ | ๐ฅ โ โ, 5 โค ๐ฅ โค 24
Sample Space, ๐
โข Coin flip๐ = Heads, Tails
โข Flipping two coins๐ = (H,H), (H,T), (T,H), (T,T)
โข Roll of 6-sided die๐ = {1, 2, 3, 4, 5, 6}
โข # emails in a day๐ = ๐ฅ | ๐ฅ โ โค, ๐ฅ โฅ 0
โข TikTok hours in a day๐ = ๐ฅ | ๐ฅ โ โ, 0 โค ๐ฅ โค 24
Lisa Yan, CS109, 2020
What is a probability?
A number between 0 and 1
to which we ascribe meaning.*
6
*our belief that an event ๐ธ occurs.
Lisa Yan, CS109, 2020
What is a probability?
๐ = # of total trials
๐(๐ธ) = # trials where ๐ธ occurs
7
Let ๐ธ = the set of outcomes
where you hit the target.
Lisa Yan, CS109, 2020
What is a probability?
๐ = # of total trials
๐(๐ธ) = # trials where ๐ธ occurs
8
Let ๐ธ = the set of outcomes
where you hit the target.
Lisa Yan, CS109, 2020
What is a probability?
๐ = # of total trials
๐(๐ธ) = # trials where ๐ธ occurs
9
Let ๐ธ = the set of outcomes
where you hit the target.
๐ ๐ธ โ 0.500
Lisa Yan, CS109, 2020
What is a probability?
๐ = # of total trials
๐(๐ธ) = # trials where ๐ธ occurs
10
Let ๐ธ = the set of outcomes
where you hit the target.
๐ ๐ธ โ 0.667
Lisa Yan, CS109, 2020
What is a probability?
๐ = # of total trials
๐(๐ธ) = # trials where ๐ธ occurs
11
Let ๐ธ = the set of outcomes
where you hit the target.
๐ ๐ธ โ 0.458
Lisa Yan, CS109, 2020 12Not just yetโฆ
Axioms of Probability
13
03b_axioms
Lisa Yan, CS109, 2020
Quick review of sets
14
Review of Sets
E F
S ๐ธ and ๐น are events in ๐.
Experiment:
Die roll
๐ = 1, 2, 3, 4, 5, 6Let ๐ธ = 1, 2 , and ๐น = 2,3
Lisa Yan, CS109, 2020
Quick review of sets
15
Review of Sets
๐ธ and ๐น are events in ๐.
Experiment:
Die roll
๐ = 1, 2, 3, 4, 5, 6Let ๐ธ = 1, 2 , and ๐น = 2,3
E
S
F
def Union of events, ๐ธ โช ๐น
The event containing all outcomes in ๐ธ or ๐น.
๐ธ โช ๐น = {1,2,3}
Lisa Yan, CS109, 2020
Quick review of sets
16
Review of Sets
๐ธ and ๐น are events in ๐.
Experiment:
Die roll
๐ = 1, 2, 3, 4, 5, 6Let ๐ธ = 1, 2 , and ๐น = 2,3
E
S
F
def Intersection of events, ๐ธ โฉ ๐น
The event containing all outcomes in ๐ธ and ๐น.
๐ธ โฉ ๐น = ๐ธ๐น = {2}
def Mutually exclusive events ๐นand ๐บ means that ๐น โฉ ๐บ = โ
G
Lisa Yan, CS109, 2020
Quick review of sets
17
Review of Sets
๐ธ and ๐น are events in ๐.
Experiment:
Die roll
๐ = 1, 2, 3, 4, 5, 6Let ๐ธ = 1, 2 , and ๐น = 2,3
E
S
F
def Complement of event ๐ธ, ๐ธ๐ถ
The event containing all outcomes in that are not in ๐ธ.
๐ธ๐ถ = {3, 4, 5, 6}
Lisa Yan, CS109, 2020
3 Axioms of Probability
Definition of probability: ๐ ๐ธ = lim๐โโ
๐(๐ธ)
๐
Axiom 1: 0 โค ๐ ๐ธ โค 1
Axiom 2: ๐ ๐ = 1
Axiom 3: If ๐ธ and ๐น are mutually exclusive (๐ธ โฉ ๐น = โ ),then ๐ ๐ธ โช ๐น = ๐ ๐ธ + ๐ ๐น
18
Lisa Yan, CS109, 2020
Axiom 3 is the (analytically) useful Axiom
Axiom 3: If ๐ธ and ๐น are mutually exclusive (๐ธ โฉ ๐น = โ ),then ๐ ๐ธ โช ๐น = ๐ ๐ธ + ๐ ๐น
More generally, for any sequence ofmutually exclusive events ๐ธ1 , ๐ธ2 , โฆ :
19
(like the Sum Rule
of Counting, but for
probabilities)
๐
๐ธ1 ๐ธ2
๐ธ3
Equally Likely Outcomes
20
03c_elo
Lisa Yan, CS109, 2020
Equally Likely Outcomes
Some sample spaces have equally likely outcomes.
โข Coin flip: S = {Head, Tails}
โข Flipping two coins: S = {(H, H), (H, T), (T, H), (T, T)}
โข Roll of 6-sided die: S = {1, 2, 3, 4, 5, 6}
If we have equally likely outcomes, then P(Each outcome)
Therefore
21
(by Axiom 3)
Lisa Yan, CS109, 2020
Roll two dice
Roll two 6-sided fair dice. What is P(sum = 7)?
๐ = { (1, 1) , (1, 2), (1, 3), (1, 4), (1, 5), (1, 6),
(2, 1) , (2, 2), (2, 3), (2, 4), (2, 5), (2, 6),
(3, 1) , (3, 2), (3, 3), (3, 4), (3, 5), (3, 6),
(4, 1) , (4, 2), (4, 3), (4, 4), (4, 5), (4, 6),
(5, 1) , (5, 2), (5, 3), (5, 4), (5, 5), (5, 6),
(6, 1) , (6, 2), (6, 3), (6, 4), (6, 5), (6, 6)}
๐ธ =
22
๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
Lisa Yan, CS109, 2020
Target revisited
23
Lisa Yan, CS109, 2020
Target revisited
Screen size = 800 ร 800
Radius of target: 200
The dart is equally likely to land anywhere on the screen. What is ๐ ๐ธ , the probability of hitting the target?
24
Let ๐ธ = the set of outcomeswhere you hit the target.
๐ = 8002 ๐ธ โ ๐ โ 2002
๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
Lisa Yan, CS109, 2020
Target revisited
Screen size = 800 ร 800
Radius of target: 200
The dart is equally likely to land anywhere on the screen. What is ๐ ๐ธ , the probability of hitting the target?
25
Let ๐ธ = the set of outcomeswhere you hit the target.
๐ = 8002 ๐ธ โ ๐ โ 2002
๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
Lisa Yan, CS109, 2020
Play the lottery.
What is ๐ win ?
๐ = {Lose,Win}
๐ธ = {Win}
26
Not equally likely outcomes ๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
The hard part: defining outcomes consistently
across sample space and events
Lisa Yan, CS109, 2020
Cats and sharks
4 cats and 3 sharks in a bag. 3 drawn.
What is P(1 cat and 2 sharks drawn)?
27
Note: Do indistinct objects give you an equally likely sample space?
A.
B.
C.
D.
E. Zero/other
๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
๐ค
(No)
Make indistinct items distinct
to get equally likely outcomes.
Lisa Yan, CS109, 2020
Cats and sharks (ordered solution)
4 cats and 3 sharks in a bag. 3 drawn.
What is P(1 cat and 2 sharks drawn)?
28
Define
โข ๐ = Pick 3 distinct items
โข๐ธ = 1 distinct cat,2 distinct sharks
๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
Make indistinct items distinct
to get equally likely outcomes.
Lisa Yan, CS109, 2020
Cats and sharks (unordered solution)
4 cats and 3 sharks in a bag. 3 drawn.
What is P(1 cat and 2 sharks drawn)?
29
๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
Define
โข ๐ = Pick 3 distinct items
โข๐ธ = 1 distinct cat,2 distinct sharks
Make indistinct items distinct
to get equally likely outcomes.
Corollaries of Probability
30
03d_corollaries
Lisa Yan, CS109, 2020
Axioms of Probability
Definition of probability: ๐ ๐ธ = lim๐โโ
๐(๐ธ)
๐
Axiom 1: 0 โค ๐ ๐ธ โค 1
Axiom 2: ๐ ๐ = 1
Axiom 3: If ๐ธ and ๐น are mutually exclusive (๐ธ โฉ ๐น = โ ),then ๐ ๐ธ โช ๐น = ๐ ๐ธ + ๐ ๐น
31
Review
Lisa Yan, CS109, 2020
3 Corollaries of Axioms of Probability
Corollary 1: ๐ ๐ธ๐ถ = 1 โ ๐(๐ธ)
32
Lisa Yan, CS109, 2020
Proof of Corollary 1
Corollary 1: ๐ ๐ธ๐ถ = 1 โ ๐(๐ธ)
Proof:
๐ธ, ๐ธ๐ถ are mutually exclusive Definition of ๐ธ๐ถ
๐ ๐ธ โช ๐ธ๐ถ = ๐ ๐ธ + ๐ ๐ธ๐ถ Axiom 3
๐ = ๐ธ โช ๐ธ๐ถ Everything must either bein ๐ธ or ๐ธ๐ถ, by definition
1 = ๐ ๐ = ๐ ๐ธ +๐ ๐ธ๐ถ Axiom 2
๐ ๐ธ๐ถ = 1 โ ๐(๐ธ) Rearrange
33
Lisa Yan, CS109, 2020
3 Corollaries of Axioms of Probability
Corollary 1: ๐ ๐ธ๐ถ = 1 โ ๐(๐ธ)
Corollary 2: If ๐ธ โ ๐น, then ๐ ๐ธ โค ๐(๐น)
Corollary 3: ๐ ๐ธ โช ๐น = ๐ ๐ธ + ๐ ๐น โ ๐ ๐ธ๐น(Inclusion-Exclusion Principle for Probability)
34
Lisa Yan, CS109, 2020
Selecting Programmers
โข P(student programs in Java) = 0.28
โข P(student programs in Python) = 0.07
โข P(student programs in Java and Python) = 0.05.
What is P(student does not program in (Java or Python))?
35
1. Define events& state goal
2. Identify knownprobabilities
3. Solve
Lisa Yan, CS109, 2020
Corollary 3: ๐ ๐ธ โช ๐น = ๐ ๐ธ + ๐ ๐น โ ๐ ๐ธ๐น(Inclusion-Exclusion Principle for Probability)
General form:
Inclusion-Exclusion Principle (Corollary 3)
๐ ๐ธ โช ๐น โช ๐บ =
๐ ๐ธ + ๐ ๐น + ๐(๐บ)
โ ๐ ๐ธ โฉ ๐น โ ๐ ๐ธ โฉ ๐บ โ ๐ ๐น โฉ ๐บ
+ ๐ ๐ธ โฉ ๐น โฉ ๐บ
36
E
FG
r = 1:
r = 2:
r = 3:
(live)03: Intro to ProbabilityOishi Banerjee and Cooper RaterinkAdapted from Lisa YanJune 26, 2020
37
Lisa Yan, CS109, 2020
Reminders: Lecture with
โข Turn on your camera if you are able, mute your mic in the big room
โข Virtual backgrounds are encouraged (classroom-appropriate)
Breakout Rooms for meeting your classmatesโฆ Just like sitting next to someone new Our best approximation to sitting next to someone new
We will use Ed instead of Zoom chatโข Lots of activity and questions, thank you all!
38
Todayโs discussion thread: https://us.edstem.org/courses/667/discussion/82037
Lisa Yan, CS109, 2020
Summary so far
39
Sort objects
(permutations)
Distinct
(distinguishable)
Some
distinct Distinct Indistinct
Distinct
1 group ๐ groups
Counting tasks on ๐ objects
Choose ๐ objects
(combinations)
Put objects in ๐buckets
๐
๐๐๐๐!
Ordered Unordered
๐ ๐ธ =|๐ธ|
|๐|
Equally likely
outcomes
Review
Lisa Yan, CS109, 2020
Indistinguishable? Distinguishable? Probability?
We choose 3 books from a set of 4 distinct (distinguishable) and 2 indistinct (indistinguishable) books.
Let event ๐ธ = our choice does not include both indistinct books.
1. What is |๐ธ|?
2. What is ๐ ๐ธ ?
40
Review
distinguishable,
equally likely
outcomes
report
countmake indistinct
compute
probability
Think, then Breakout Rooms
Then check out the question on the next slide (Slide 44). Post any clarifications here!
https://us.edstem.org/courses/667/discussion/82037
Think by yourself: 2 min
Breakout rooms: 5 min. Introduce yourself!
41
๐ค
Lisa Yan, CS109, 2020
Poker Straights and Computer Chips
1. Consider 5-card poker hands.โข โstraightโ is 5 consecutive rank
cards of any suit
What is P(Poker straight)?
2. Consider the โofficialโ definition of a Poker Straight:โข โstraightโ is 5 consecutive rank cards of any suit
โข straight flushโ is 5 consecutive rank cards of same suit
What is P(Poker straight, but not straight flush)?
3. Computer chips: ๐ chips are manufactured, 1 of which is defective.๐ chips are randomly selected from ๐ for testing.
What is P(defective chip is in ๐ selected chips?)
42
โข What is an example of an outcome?
โข Is each outcome equally likely?
โข Should objects be
ordered or unordered?
๐ค
Lisa Yan, CS109, 2020
Any Poker Straight
1. Consider 5-card poker hands.
โข โstraightโ is 5 consecutive rankcards of any suit
What is P(Poker straight)?
43
Define
โข ๐ (unordered)
โข ๐ธ (unordered,consistent with S)
Compute ๐ Poker straight =
Lisa Yan, CS109, 2020
Consider 5-card poker hands.
โข โstraightโ is 5 consecutive rank cards of any suit
โข โstraight flushโ is 5 consecutive rank cards of same suit
What is P(Poker straight, but not straight flush)?
44
โOfficialโ Poker Straight
Define
โข ๐ (unordered)
โข ๐ธ (unordered,consistent with S)
Compute ๐ Official Poker straight =
Lisa Yan, CS109, 2020
Define
โข ๐ (unordered)
โข ๐ธ (unordered,consistent with S)
Compute
๐ chips are manufactured, 1 of which is defective.๐ chips are randomly selected from ๐ for testing.
What is ๐(defective chip is in ๐ selected chips?)
45
Chip defect detection
๐ ๐ธ =
Lisa Yan, CS109, 2020
๐ chips are manufactured, 1 of which is defective.๐ chips are randomly selected from ๐ for testing.
What is P(defective chip is in ๐ selected chips?)
46
Chip defect detection, solution #2
Redefine experiment
1. Choose ๐ indistinct chips (1 way)
2. Throw a dart and make one defective
Define
โข ๐ (unordered)
โข ๐ธ (unordered,consistent with S)
Interlude for announcements
47
Lisa Yan, CS109, 2020
Section
48
Week 1โs section: pre-recorded Python review session
Week 2+: 12:30-1:30PT Thursdays, live on Zoom (will be recorded)
Lisa Yan, CS109, 2020
Interesting probability news
49
โThe study finds that very few chords govern most of
the music, a phenomenon that is also known in
linguistics, where very few words dominate language
corporaโฆ. It characterizes Beethoven's specific
composition style for the String Quartets, through a
distribution of all the chords he used, how often they
occur, and how they commonly transition from one to
the other.โ
https://actu.epfl.ch/news/de
coding-beethoven-s-music-
style-using-data-scienc/
Lisa Yan, CS109, 2020
3 Corollaries of Axioms of Probability
Corollary 1: ๐ ๐ธ๐ถ = 1 โ ๐(๐ธ)
Corollary 2: If ๐ธ โ ๐น, then ๐ ๐ธ โค ๐(๐น)
Corollary 3: ๐ ๐ธ โช ๐น = ๐ ๐ธ + ๐ ๐น โ ๐ ๐ธ๐น(Inclusion-Exclusion Principle for Probability)
50
Review
Lisa Yan, CS109, 2020
Takeaway: Mutually exclusive events
51
๐ แซ๐=1
โ
๐ธ๐ =
๐
๐ธ1 ๐ธ2
๐ธ3
Axiom 3,
Mutually exclusive
events
E
F G
Inclusion-
Exclusion
Principle
The challenge of probability is in defining events.
Some event probabilities are easier to compute than others.
๐ แซ๐=1
โ
๐ธ๐ =
Review
Lisa Yan, CS109, 2020
Serendipity
52
Lisa Yan, CS109, 2020
Serendipity
โข The population of Stanford is ๐ = 17,000 people.
โข You are friends with ๐ = people.
โข Walk into a room, see ๐ = 360 random people.
โข Assume you are equally likely to see each person at Stanford.
What is the probability that you see someone you know in the room?
53
Breakout Rooms
Check out the question on the next slide (Slide 57). Post any clarifications here!
https://us.edstem.org/courses/667/discussion/82037
Breakout rooms: 5 min. Introduce yourself if you havenโt yet!
54
๐ค
Lisa Yan, CS109, 2020
Serendipity
โข The population of Stanford is ๐ = 17,000 people.
โข You are friends with ๐ = 100 people.
โข Walk into a room, see ๐ = 360 random people.
โข Assume you are equally likely to see each person at Stanford.
What is the probability that you see someone you know in the room?
Define
โข ๐ (unordered)
โข ๐ธ: โฅ 1 friend in the room
55
What strategy should you use?
A. ๐ exactly 1 + ๐ exactly 2๐ exactly 3 + โฏ
B. 1 โ ๐ see no friends
๐ค
Lisa Yan, CS109, 2020
Serendipity
โข The population of Stanford is ๐ = 17,000 people.
โข You are friends with ๐ = 100 people.
โข Walk into a room, see ๐ = 360 random people.
โข Assume you are equally likely to see each person at Stanford.
What is the probability that you see someone you know in the room?
Define
โข ๐ (unordered)
โข ๐ธ: โฅ 1 friend in the room
56
It is often much easier to compute ๐ ๐ธ๐ .
Lisa Yan, CS109, 2020
The Birthday Paradox Problem
What is the probability that in a set of n people, at least one pair of them will share the same birthday?
For you to think about (and discuss in section!)
57
Lisa Yan, CS109, 2020
In a 52 card deck, cards are flipped one at a time.After the first ace (of any suit) appears, consider the next card.
Is P(next card = Ace Spades) < P(next card = 2 Clubs)?
Card Flipping
๐ค(by yourself)
Lisa Yan, CS109, 2020
In a 52 card deck, cards are flipped one at a time.After the first ace (of any suit) appears, consider the next card.
Is P(next card = Ace Spades) < P(next card = 2 Clubs)?
Card Flipping
Sample space | S | = 52!
Event ๐ธ๐ด๐ , next card
is Ace Spades
๐ธ2๐ถ, next card
is 2 Clubs
1. Take out Ace of Spades.
2. Shuffle leftover 51 cards.
3. Add Ace Spades after first ace.
|๐ธ๐ด๐| = 51! โ 1
1. Take out 2 Clubs.
2. Shuffle leftover 51 cards.
3. Add 2 Clubs after first ace.
|๐ธ2๐ถ| = 51! โ 1
๐ ๐ธ๐ด๐ = ๐ ๐ธ2๐ถ