HUDM4122 Probability and Statistical Inference January 26, 2015.
HUDM4122 Probability and Statistical Inference - upenn.edu · Flipping a fair coin Flipping same...
Transcript of HUDM4122 Probability and Statistical Inference - upenn.edu · Flipping a fair coin Flipping same...
HUDM4122Probability and Statistical Inference
February 16, 2015
In the last class
• We started Ch. 4.4 in Mendenhall, Beaver, &Beaver
Today
• Ch. 4.4-4.6 in Mendenhall, Beaver, & Beaver
Today
• Sampling without Replacement• Permutations• Combinations• Independence• Conditional Probability
We ended last class with this problem
• I call a radio station, where they make me pick anumber between 1 and 10
• If I get today’s winning number, I get free ticketsto hear Justin Bieber (oh lucky me)
• Let’s say I call 5 days in a row
• What is the probability I get tickets on exactly oneday? (my daughter will still be excited)
Solution A (professor)110 + 110 910 + 110 910 910+ 110 910 910 910+ 110 910 910 910 910= 0.40951
Solution B (class)110 ( 910) ∗ 5= 0.32805
These represent two differentproblems
• One (my solution) is sampling withoutreplacement
• The other is the sum of five independentevents
• So which one is right?
Let’s first compute the sample space
• I call a radio station, where they make me picka number between 1 and 10
• I call 5 days in a row
• 10*10*10*10*10 = 100,000
Let’s first compute the sample space
• I call a radio station, where they make me picka number between 1 and 10
• I call 5 days in a row
• 10*10*10*10*10 = 100,000• A.K.A. too many to list out
Let’s first compute the sample space
• I call a radio station, where they make me picka number between 1 and 10
• I call 5 days in a row
• 10*10*10*10*10 = 100,000• A.K.A. too many to list out
– Or is it?
bieber-tix-example.xlsx
bieber-tix-example.xlsx
• 0.32805
So, the professor was wrong
So, the professor was wrong
• It turns out sleep deprivation is bad forcognition
So, the professor was wrong
• It turns out sleep deprivation is bad forcognition
• Don’t try this on your midterm
So why was this correct?110 ( 910) ∗ 5
5 days to win 1 ticket110 ( 910) ∗ 5• CXXXX• XCXXX• XXCXX• XXXCX• XXXXC
5 days
In each of 5 days,1 answer where right110 ( 910) ∗ 5
5 daysCorrect answer
And 4 more days where wrong;and there are 9 wrong answers110 ( 910) ∗ 5
5 casesCorrect answer
Wrong answers
Wrong days
Each day is independent from other days –that’s why this is the correct math110 ( 910) ∗ 5
5 casesCorrect answer
Wrong answers
Wrong days
Questions? Comments?
The sample space for multi-stage datacollection can be calculated using
• The Extendedmn rule
The Extendedmn rule
• Let’s say you have k stages of your datacollection
The Extendedmn rule
• Let’s say you have k stages of data collection– Unlike the book, I don’t call it an experiment,
because that term is usually given a more specificmeaning by researchers
The Extendedmn rule
• Let’s say you have k stages of data collection• And there are n1 ways to accomplish the first
stage• And n2 ways to accomplish the 2nd stage• And n3 ways to accomplish the 3rd stage• And nk ways to accomplish the kth stage
• Then the sample space = n1 * n2 * n3 … * nk
Any questions about theExtendedmn rule?
• Let’s say you have k stages of data collection• And there are n1 ways to accomplish the first
stage• And n2 ways to accomplish the 2nd stage• And n3 ways to accomplish the 3rd stage• And nk ways to accomplish the kth stage
• Then the sample space = n1 * n2 * n3 … * nk
Note that there doesn’t have to be thesame probability in each stage!
This can come in useful in cases thatare not truly independent
• Unlike the Justin Bieber example
Independence
• Two events A and B are independent if A doesnot affect B and B does not affect A
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin again
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again
Bob parties late night beforemidterm
Bob falls asleep duringmidterm
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again
Bob parties late night beforemidterm
Bob falls asleep duringmidterm
Bob’s grade on midterm Bob’s grade on final
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again
Bob parties late night beforemidterm
Bob falls asleep duringmidterm
Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelled
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again
Bob parties late night beforemidterm
Bob falls asleep duringmidterm
Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelledBob gets expelled Bob takes job at McDonald’s
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again
Bob parties late night beforemidterm
Bob falls asleep duringmidterm
Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelledBob gets expelled Bob takes job at McDonald’s
Bob takes job at McDonald’s Bob wins lottery
Which of these are independent?A B
Flipping a fair coin Flipping same fair coin againFlipping a biased coin Flipping same biased coin again
Bob parties late night beforemidterm
Bob falls asleep duringmidterm
Bob’s grade on midterm Bob’s grade on finalBob disrupts class Bob gets expelledBob gets expelled Bob takes job at McDonald’s
Bob takes job at McDonald’s Bob wins lotteryBob wins lottery Bob becomes ill due to
congenital heart problem
Example of probability calculation withnon-independent events
• Let’s say that I invite 6 friends over to playBeer Hunter
This can come in useful in cases thatare not truly independent
• Let’s say that 6 friends decide to play BeerHunter
• Rules are– 6 cans of beer– 1 violently shaken before playing and then
shuffled– Each person chooses a can and opens it
Initial Sample Space = 6
• 1 Bad outcome• 5 Perfectly fine outcomes
– If you don’t like beer, imagine it’s root beer
Probability of bad outcome
• The probability of a bad outcome for friend 1is 1/6
Probability of bad outcome
• The probability of a bad outcome for friend 1is 1/6
• But if friend 1 comes out OK• The probability of a bad outcome for friend 2
is 1/5, not 1/6!
Probability of bad outcome
• The probability of a bad outcome for friend 1is 1/6
• But if friend 1 comes out OK• The probability of a bad outcome for friend 2
is 1/5, not 1/6!
• Does everyone see why?
Probability of bad outcome
• The probability of a bad outcome for friend 1is 1/6
• But if friend 1 comes out OK• The probability of a bad outcome for friend 2
is 1/5, not 1/6!
• Does everyone see why?– Friend 1 already opened a beer can, and it went
ok– This only leaves 5 closed beer cans
Probability of bad outcome
• If friend 2 comes out OK• The probability of a bad outcome for friend 3
is 1/4
Probability of bad outcome
• If friend 2 comes out OK• The probability of a bad outcome for friend 3
is 1/4
• If friend 3 comes out OK• The probability of a bad outcome for friend 4
is 1/3
Probability of bad outcome
• If friend 4 comes out OK• The probability of a bad outcome for friend 5
is 1/2
Probability of bad outcome
• If friend 4 comes out OK• The probability of a bad outcome for friend 5
is 1/2
• If friend 5 comes out OK
Probability of bad outcome
• If friend 4 comes out OK• The probability of a bad outcome for friend 5
is 1/2
• If friend 5 comes out OK• Friend 6 will need to get a clean shirt
True sample space
• 6 *5 * 4 * 3 * 2 * 1
• This is called sampling without replacement
Any questions?
Now you do an example
• In pairs
Now you do an example
• Let’s say I’m a roadie for the band Van Halen
Now you do an example
• Let’s say I’m a roadie for the band Van Halen– Professors need to moonlight to make ends meet
in this city
Now you do an example
• Let’s say I’m a roadie for the band Van Halen• They have a “no brown M&M’s” clause in their
contract, and if any of the four members get abrown M&M, I’m fired
Now you do an example
• Let’s say I’m a roadie for the band Van Halen• They have a “no brown M&M’s” clause in their
contract, and if any of the four members get abrown M&M, I’m fired
• I’ve just handed them a bowl with 20 M&Ms,including one brown M&M
• Each band member takes 1 M&M withoutlooking
Now you do an example• Let’s say I’m a roadie for the band Van Halen• They have a “no brown M&M’s” clause in their
contract, and if any of the four members get abrown M&M, I’m fired
• I’ve just handed them a bowl with 20 M&Ms,including one brown M&M
• Each band member takes 1 M&M without looking
• What is the sample space?• What is the probability I get fired?
Questions? Comments?
Another example: permutations
• An application of sampling withoutreplacement
• How many orderings can you have between acertain number of objects?
Example• Let’s say that I’m redecorating my office in
preparation for a visit from a funder from the USarmy (“Bob”), a funder from the National ScienceFoundation (“Janet”), and a funder from the USDepartment of Education (“Ed”)
• I want to place Bob’s book, Janet’s book, and Ed’sbook in a place of honor next to my desk
• How many different orders can I put their booksin?
Example
• The first book could be Bob’s, Janet’s, or Ed’s• If the first book is Bob’s, the second book can
only be Janet’s or Ed’s• If the first book is Bob’s, and the second book
is Janet’s, then the third book can only be Ed’s
• We’re back to the same math of 3*2*1
Any questions?
Formal equations
• The sample space for n stages, samplingwithout replacement, is
• n * (n-1) * (n-2) * (n-3) * (n-4) until (n-k)=1
• This is written n!– Pronounced “n factorial”
Formal equations
• The number of permutations for n objects,taking all of them together, is
• Still n!
• n * (n-1) * (n-2) * (n-3) * (n-4) until (n-k)=1
Do it yourself
• What is the number of permutations for 4objects?
• What is the number of permutations for 6objects?
• What is the number of permutations for 10objects?
Do it yourself• What is the number of permutations for 4
objects?– 4*3*2*1=24
• What is the number of permutations for 6objects?– 6*5*4*3*2*1=720
• What is the number of permutations for 10objects?– 10*9*8*7*6*5*4*3*2*1= 3,628,800
Ryan’s daughter suggests
• “Daddy, why don’t we try organizing all thebooks on your bookshelves in every possibleway?”
• I own approximately 600 books
• Is this a good idea?
Questions? Comments?
More general case
• If we only want to pick r objects out of the ntotal objects, the equation becomes
• ! !
Using general case equation
• If I want to find out how many orderings of 2books I can get from 6 total books– n!/(n-r)!– 6!/(6-2)!– 6!/4!– 720/24– 30 possible orderings
Using general case equation
• If I want to find out how many orderings of 4books I can get from 6 total books– n!/(n-r)!– 6!/(6-4)!– 6!/2!– 720/2– 360 possible orderings
Any questions?
Related problem: Combinations
• If we don’t care about order, but only want toknow how many combinations of items wecan get
Combination formula
• The number of combinations of r objects outof n total objects is
• !! !
Example
• I have five friends, and three tickets to seeFerrari Truck
• How many combinations of friends could Ipotentially bring?
Example
• I have five friends, and three tickets to seeFerrari Truck
• How many combinations of friends could Ipotentially bring?
• !! ! = !! ! = !!( !) = ( ) = = 10
Example
• I have 600 books, and want to take 3 books ona ridiculously long flight to the FirstUzbekistani Conference on Educational DataMining
• How many combinations of books could Ipotentially bring?
Example
• I have 600 books, and want to take 3 books on aridiculously long flight to the First UzbekistaniConference on Educational Data Mining
• How many combinations of books could Ipotentially bring?
• !! ! = !! ! = !! ! = ∗ ∗! =
• 35.8 million
Your turn
• Peter’s Pizzeria has 6 toppings, and a 2-topping special
• How many combinations of toppings couldyou get, and have the special?
Questions?
An application
• In my son’s play group, he has 6 playmates– 5 friends and 1 frenemy
• What is the probability that on a specificplaydate with 2 friends, it will involve thefrenemy?
Can be written as
• Number of playdates that involve frenemy = 5– Frenemy plus each of 5 friends
• Total number of combinations (2 of 6)
• !! ! = !! ! = !!( !) = ( ) = = 15
Can be written as
• Number of playdates that involve frenemy = 5– Frenemy plus each of 5 friends
• Total number of combinations (2 of 6) = 15
• 5/15 = 1/3
Any questions?
Conditional Probability
• Let’s take two non-independent events, A andB
• P(A | B) =• Probability of A,• Given that we know that B occurred
Conditional Probability
• Let’s take two non-independent events, A and B
• P(A | B) =• Probability of A,• Given that we know that B occurred
• Note that this tells us nothing aboutP(A | ~B)P(A) overall
General Multiplication Rule
• Probability of A and B equals• Probability of A• Multiplied by• Probability of B, given A
General Multiplication Rule
• Probability of A and B equals• Probability of A• Multiplied by• Probability of B, given A
• Formally• ∩ = ( | )
Example
• P(Q) = 0.2• P(R|Q) = 0.7
• P(Q ∩ R) = ?
Example
• P(Q) = 0.2• P(R|Q) = 0.7
• P(Q ∩ R) = 0.14
You try it
• P(F) = 0.7• P(G|F) = 0.5
• P(F ∩ G) = ?
You try it
• P(F) = 0.7• P(G|F) = 0.5
• P(F ∩ G) = 0.35
You try it
• P(X) = 0.3• P(X|Y) = 0.9
• P(X ∩ Y) = ?
You try it
• P(X) = 0.3• P(X|Y) = 0.9
• P(X ∩ Y) = ?
• Impossible to calculate from information given
A Concrete Example
• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam | parties late) = 0.7• P(Bob does badly on exam | ~parties late) = 0.2
• What is the probability that Bob parties late anddoes badly?
A Concrete Example
• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam | parties late) = 0.7• P(Bob does badly on exam | ~parties late) = 0.2
• What is the probability that Bob parties late anddoes badly?
• 0.35
Conditional Probability Formula
• A mathematical transformation of the GeneralMultiplication Rule
• That rule was
• ∩ = ( | )
Conditional Probability Formula
• If you divide both sides by P(A)
• ( ∩ )( ) = ( | )( )• Which resolves to
• ( ∩ )( ) = ( | )
Conditional Probability Formula
• P(B|A) = ( ∩ )( )
Conditional Probability Formula
• P(B|A) = ( ∩ )( )• Note that P(A) can’t equal 0, or you’re dividing
by 0…
Example
• P(B|A) = ( ∩ )( )• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam AND parties late) =
0.35
• What is the probability that does badly, giventhat he parties late?
Example
• P(B|A) = ( ∩ )( )• P(Bob parties late, night before exam) = 0.5• P(Bob does badly on exam AND parties late) =
0.35
• What is the probability that does badly, given thathe parties late?– 0.7
You try it
• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from
wealthy family) = 0.21
• What is the probability that a student goes tocollege, given that he/she comes from a wealthyfamily?
You try it
• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from
wealthy family) = 0.21
• What is the probability that a student goes tocollege, given that he/she comes from a wealthyfamily?– 0.84
What if we want to determine
• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from
wealthy family) = 0.21
• What is the probability that a student comesfrom a wealthy family, given that he/she went tocollege?
What if we want to determine
• P(B|A) = ( ∩ )( )• P(Student is from a wealthy family) = 0.25• P(Student goes to college AND comes from
wealthy family) = 0.21
• What is the probability that a student comesfrom a wealthy family, given that he/she went tocollege?– We can’t tell, using this formula…
Upcoming Classes
• 2/18 Bayes Theorem– Ch. 4-7– HW3 due
• 2/23 Discrete Random Variables and TheirProbability Distributions– Ch. 4-8
Homework 3
• Due in 2 days• In the ASSISTments system
Questions? Comments?