07: Variance, Bernoulli, Binomial - Stanford...
Transcript of 07: Variance, Bernoulli, Binomial - Stanford...
07: Variance, Bernoulli, BinomialJerry CainApril 12, 2021
1
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Quick slide reference
2
3 Variance 07a_variance_i
10 Properties of variance 07b_variance_ii
17 Bernoulli RV 07c_bernoulli
22 Binomial RV 07d_binomial
34 Exercises LIVE
Variance
3
07a_variance_i
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Stanford, CAπΈ high = 68Β°FπΈ low = 52Β°F
Washington, DCπΈ high = 67Β°FπΈ low = 51Β°F
4
Average annual weather
Is πΈ π enough?
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Stanford, CAπΈ high = 68Β°FπΈ low = 52Β°F
Washington, DCπΈ high = 67Β°FπΈ low = 51Β°F
5
Average annual weather
0
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.4
ππ=π₯
ππ=π₯
Stanford high temps Washington high temps
35 50 65 80 90 35 50 65 80 90
68Β°F 67Β°F
Normalized histograms are approximations of PMFs.
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Variance = βspreadβConsider the following three distributions (PMFs):
β’ Expectation: πΈ π = 3 for all distributionsβ’ But the βspreadβ in the distributions is different!β’ Variance, Var π : a formal quantification of βspreadβ
6
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Variance
The variance of a random variable π with mean πΈ π = π is
Var π = πΈ π β π (
β’ Also written as: πΈ π β πΈ π !
β’ Note: Var(X) β₯ 0β’ Other names: 2nd central moment, or square of the standard deviation
Var π
def standard deviation SD π = Var π7
Units of π!
Units of π
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Variance of Stanford weather
8
Varianceof π
Var π = πΈ π β πΈ π !
Stanford, CAπΈ high = 68Β°FπΈ low = 52Β°F
0
0.1
0.2
0.3
0.4
ππ=π₯
Stanford high temps
35 50 65 80 90
π π β π !
57Β°F 121 (Β°F)2
πΈ π = π = 68Β°F
71Β°F 9 (Β°F)2
75Β°F 49 (Β°F)2
69Β°F 1 (Β°F)2
β¦ β¦
Variance πΈ π β π ! = 39 (Β°F)2
Standard deviation = 6.2Β°F
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Stanford, CAπΈ high = 68Β°F
Washington, DCπΈ high = 67Β°F
9
Comparing variance
0
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.4
ππ=π₯
ππ=π₯
Stanford high temps Washington high temps
35 50 65 80 90 35 50 65 80 90
Var π = 39 Β°F ! Var π = 248 Β°F !
Varianceof π
Var π = πΈ π β πΈ π !
68Β°F 67Β°F
Properties of Variance
10
07b_variance_ii
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Properties of varianceDefinition Var π = πΈ π β πΈ π !
def standard deviation SD π = Var π
Property 1 Var π = πΈ π! β πΈ π !
Property 2 Var ππ + π = π!Var π
11
Units of π!
Units of π
β’ Property 1 is often easier to compute than the definitionβ’ Unlike expectation, variance is not linear
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Properties of varianceDefinition Var π = πΈ π β πΈ π !
def standard deviation SD π = Var π
Property 1 Var π = πΈ π! β πΈ π !
Property 2 Var ππ + π = π!Var π
12
Units of π!
Units of π
β’ Property 1 is often easier to compute than the definitionβ’ Unlike expectation, variance is not linear
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Computing variance, a proof
13
Var π = πΈ π β πΈ π !
= πΈ π! β πΈ π !
= πΈ π! β π!= πΈ π! β 2π! + π!= πΈ π! β 2ππΈ π + π!
= πΈ π β π ! Let πΈ π = π
Everyone, please
welcome the second
moment!
Varianceof π
Var π = πΈ π β πΈ π !
= πΈ π! β πΈ π !
=,"
π₯ β π !π π₯
=,"
π₯! β 2ππ₯ + π! π π₯
=,"
π₯!π π₯ β 2π,"
π₯π π₯ + π!,"
π π₯
β 1
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Let Y = outcome of a single die roll. Recall πΈ π = 7/2 .Calculate the variance of Y.
14
Variance of a 6-sided dieVarianceof π
Var π = πΈ π β πΈ π !
= πΈ π! β πΈ π !
1. Approach #1: Definition
Var π =161 β
72
!+162 β
72
!
+163 β
72
!+164 β
72
!
+165 β
72
!+166 β
72
!
2. Approach #2: A property
πΈ π! =161! + 2! + 3! + 4! + 5! + 6!
= 91/6
Var π = 91/6 β 7/2 !
= 35/12 = 35/12
2nd moment
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Properties of varianceDefinition Var π = πΈ π β πΈ π !
def standard deviation SD π = Var π
Property 1 Var π = πΈ π! β πΈ π !
Property 2 Var ππ + π = π!Var π
15
Units of π!
Units of π
β’ Property 1 is often easier to compute than the definitionβ’ Unlike expectation, variance is not linear
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Property 2: A proofProperty 2 Var ππ + π = π!Var π
16
Var ππ + π= πΈ ππ + π ! β πΈ ππ + π ! Property 1
= πΈ π!π! + 2πππ + π! β ππΈ π + π !
= π!πΈ π! + 2πππΈ π + π! β π! πΈ[π] ! + 2πππΈ[π] + π!
= π!πΈ π! β π! πΈ[π] !
= π! πΈ π! β πΈ[π] !
= π!Var π Property 1
Proof:
Factoring/Linearity of Expectation
Bernoulli RV
17
07c_bernoulli
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment with two outcomes: βsuccessβ and βfailure.βdef A Bernoulli random variable π maps βsuccessβ to 1 and βfailureβ to 0.
Other names: indicator random variable, boolean random variable
Examples:β’ Coin flipβ’ Random binary digitβ’ Whether a disk drive crashed
Bernoulli Random Variable
18
π π = 1 = π 1 = ππ π = 0 = π 0 = 1 β ππ~Ber(π)
Support: {0,1} VarianceExpectation
PMF
πΈ π = πVar π = π(1 β π)
Remember this nice property of expectation. It will come back!
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Jacob Bernoulli
19
Jacob Bernoulli (1654-1705), also known as βJamesβ, was a Swiss mathematician
One of many mathematicians in Bernoulli familyThe Bernoulli Random Variable is named for him
My academic great14 grandfather
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Defining Bernoulli RVs
20
Serve an ad.β’ User clicks w.p. 0.2β’ Ignores otherwise
Let π: 1 if clicked
π~Ber(___)π π = 1 = ___π π = 0 = ___
Roll two dice.β’ Success: roll two 6βsβ’ Failure: anything else
Let π : 1 if success
π~Ber(___)
πΈ π = ___
π~Ber(π) π" 1 = ππ" 0 = 1 β ππΈ π = π
Run a programβ’ Crashes w.p. πβ’ Works w.p. 1 β π
Let π: 1 if crash
π~Ber(π)π π = 1 = ππ π = 0 = 1 β π π€
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Defining Bernoulli RVs
21
Serve an ad.β’ User clicks w.p. 0.2β’ Ignores otherwise
Let π: 1 if clicked
π~Ber(___)π π = 1 = ___π π = 0 = ___
Roll two dice.β’ Success: roll two 6βsβ’ Failure: anything else
Let π : 1 if success
π~Ber(___)
πΈ π = ___
π~Ber(π) π" 1 = ππ" 0 = 1 β ππΈ π = π
Run a programβ’ Crashes w.p. πβ’ Works w.p. 1 β π
Let π: 1 if crash
π~Ber(π)π π = 1 = ππ π = 0 = 1 β π
Binomial RV
22
07d_binomial
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: π independent trials of Ber(π) random variables.def A Binomial random variable π is the number of successes in π trials.
Examples:β’ # heads in n coin flipsβ’ # of 1βs in randomly generated length n bit stringβ’ # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
23
π = 0, 1, β¦ , π:
π π = π = π π = ππ π! 1 β π "#!π~Bin(π, π)
Support: {0,1, β¦ , π}
PMF
πΈ π = ππVar π = ππ(1 β π)Variance
Expectation
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021 24
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Reiterating notation
The parameters of a Binomial random variable:β’ π: number of independent trialsβ’ π: probability of success on each trial
25
1. The random variable
2. is distributed as a
3. Binomial 4. with parameters
π ~ Bin(π, π)
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Reiterating notation
If π is a binomial with parameters π and π, the PMF of π is
26
π ~ Bin(π, π)
π π = π = ππ π( 1 β π )*(
Probability Mass Function for a BinomialProbability that πtakes on the value π
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Three coin flipsThree fair (βheadsβ with π = 0.5) coins are flipped.β’ π is number of headsβ’ π~Bin 3, 0.5
Compute the following event probabilities:
27
π~Bin(π, π) π π = ππ π# 1 β π $%#
π π = 0
π π = 1
π π = 2
π π = 3
π π = 7P(event)
π€
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Three coin flipsThree fair (βheadsβ with π = 0.5) coins are flipped.β’ π is number of headsβ’ π~Bin 3, 0.5
Compute the following event probabilities:
28
π~Bin(π, π) π π = ππ π# 1 β π $%#
π π = 0 = π 0 = 30 π$ 1 β π % = &
'
π π = 1
π π = 2
π π = 3
π π = 7
= π 1 = 31 π& 1 β π ! = %
'
= π 2 = 32 π! 1 β π & = %
'
= π 3 = 33 π% 1 β π $ = &
'
= π 7 = 0P(event) PMF
Extra math note:By Binomial Theorem,we can proveβ()$* π π = π = 1
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: π independent trials of Ber(π) random variables.def A Binomial random variable π is the number of successes in π trials.
Examples:β’ # heads in n coin flipsβ’ # of 1βs in randomly generated length n bit stringβ’ # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
29
π~Bin(π, π)Range: {0,1, β¦ , π} Variance
Expectation
PMF
πΈ π = ππVar π = ππ(1 β π)
π = 0, 1, β¦ , π:
π π = π = π π = ππ π! 1 β π "#!
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Ber π = Bin(1, π)
Binomial RV is sum of Bernoulli RVs
Bernoulliβ’ π~Ber(π)
Binomialβ’ π~Bin π, πβ’ The sum of π independent
Bernoulli RVs
30
π =,+)&
*
π+ , π+ ~Ber(π)
+
+
+
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: π independent trials of Ber(π) random variables.def A Binomial random variable π is the number of successes in π trials.
Examples:β’ # heads in n coin flipsβ’ # of 1βs in randomly generated length n bit stringβ’ # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
31
π~Bin(π, π)Range: {0,1, β¦ , π}
πΈ π = ππVar π = ππ(1 β π)Variance
Expectation
PMF π = 0, 1, β¦ , π:
π π = π = π π = ππ π! 1 β π "#!
Proof:
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: π independent trials of Ber(π) random variables.def A Binomial random variable π is the number of successes in π trials.
Examples:β’ # heads in n coin flipsβ’ # of 1βs in randomly generated length n bit stringβ’ # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
32
π~Bin(π, π)Range: {0,1, β¦ , π}
PMF
πΈ π = ππVar π = ππ(1 β π)
Weβll prove this later in the course
VarianceExpectation
π = 0, 1, β¦ , π:
π π = π = π π = ππ π! 1 β π "#!
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
No, give me the variance proof right now
33
proofwiki.org