The Gaussian (Normal) Distribution: More Details & Some Applications.
10: The Normal (Gaussian) Distribution
Transcript of 10: The Normal (Gaussian) Distribution
Lisa Yan, CS109, 2020
Quick slide reference
2
3 Normal RV 10a_normal
15 Normal RV: Properties 10b_normal_props
21 Normal RV: Computing probability 10c_normal_prob
30 Exercises LIVE
Lisa Yan, CS109, 2020
def An Normal random variable π is defined as follows:
Other names: Gaussian random variable
Normal Random Variable
5
π π₯ =1
π 2ππβ π₯βπ 2/2π2
π~π©(π, π2)Support: ββ,β
Variance
Expectation
πΈ π = π
Var π = π2
π~π©(π, π2)mean
variance
π₯
ππ₯
Lisa Yan, CS109, 2020
Carl Friedrich Gauss
Carl Friedrich Gauss (1777-1855) was a remarkably influentialGerman mathematician.
Did not invent Normal distribution but rather popularized it6
Lisa Yan, CS109, 2020
Why the Normal?
β’ Common for natural phenomena: height, weight, etc.
β’ Most noise in the world is Normal
β’ Often results from the sum of many random variables
β’ Sample means are distributed normally
7
Thatβs what they
want you to believeβ¦
Lisa Yan, CS109, 2020
Why the Normal?
β’ Common for natural phenomena: height, weight, etc.
β’ Most noise in the world is Normal
β’ Often results from the sum of many random variables
β’ Sample means are distributed normally
8
Actually log-normal
Just an assumption
Only if equally weighted
(okay this one is true, weβll see
this in 3 weeks)
Lisa Yan, CS109, 2020
0
0.05
0.1
0.15
0.2
0.25
0 β¦ 44 48 52 56 60 64 β¦ 900 β¦ 44 48 52 56 60 64 β¦ 90
Okay, so why the Normal?
Part of CS109 learning goals:
β’ Translate a problem statement into a random variable
In other words: model real life situations with probability distributions
9
value
How do you model student heights?
β’ Suppose you have data from one classroom.
Fits perfectly!
But what about in
another classroom?
Lisa Yan, CS109, 2020
A Gaussian maximizes entropy for a
given mean and variance.
Part of CS109 learning goals:
β’ Translate a problem statement into a random variable
In other words: model real life situations with probability distributions
0
0.05
0.1
0.15
0.2
0.25
0 β¦ 44 48 52 56 60 64 β¦ 900 β¦ 44 48 52 56 60 64 β¦ 90
Okay, so why the Normal?
10
Occamβs Razor:
βNon sunt multiplicanda
entia sine necessitate.β
Entities should not be multiplied
without necessity.
value
How do you model student heights?
β’ Suppose you have data from one classroom.
β’ Same mean/var
β’ Generalizes well
Lisa Yan, CS109, 2020
I encourage you to stay critical of how
to model real-world phenomena.
Why the Normal?
β’ Common for natural phenomena: height, weight, etc.
β’ Most noise in the world is Normal
β’ Often results from the sum of many random variables
β’ Sample means are distributed normally
11
Actually log-normal
Just an assumption
Only if equally weighted
(okay this one is true, weβll see
this in 3 weeks)
Lisa Yan, CS109, 2020
Anatomy of a beautiful equation
Let π~π© π, π2 .
The PDF of π is defined as:
12
π π₯ =1
π 2ππβ
π₯ β π 2
2π2
normalizing constantexponential
tail
symmetric
around π
variance π2
manages spread
π₯
ππ₯
Lisa Yan, CS109, 2020
Campus bikes
You spend some minutes, π, travelingbetween classes.
β’ Average time spent: π = 4 minutes
β’ Variance of time spent: π2 = 2 minutes2
Suppose π is normally distributed. What is the probability you spend β₯ 6 minutes traveling?
13
π~π©(π = 4, π2 = 2)
π π β₯ 6 = ΰΆ±6
β
π(π₯)ππ₯ = ΰΆ±6
β 1
π 2ππβ
π₯ β π 2
2π2 ππ₯
(call me if you analytically solve this)Loving, not scaryβ¦except this time
Lisa Yan, CS109, 2020
Computing probabilities with Normal RVs
For a Normal RV π~π© π, π2 , its CDF has no closed form.
π π β€ π₯ = πΉ π₯ = ΰΆ±ββ
π₯ 1
π 2ππβ
π¦ β π 2
2π2 ππ¦
However, we can solve for probabilities numerically using a function Ξ¦:
πΉ π₯ = Ξ¦π₯ β π
π
14
Cannot be
solved
analytically
β οΈ
CDF of
π~π© π, π2A function that has been
solved for numerically
To get here, weβll first
need to know some
properties of Normal RVs.
Lisa Yan, CS109, 2020
Properties of Normal RVs
Let π~π© π, π2 with CDF π π β€ π₯ = πΉ π₯ .
1. Linear transformations of Normal RVs are also Normal RVs.
If π = ππ + π, then π~π©(ππ + π, π2π2).
2. The PDF of a Normal RV is symmetric about the mean π.
πΉ π β π₯ = 1 β πΉ π + π₯
16
Lisa Yan, CS109, 2020
1. Linear transformations of Normal RVs
Let π~π© π, π2 with CDF π π β€ π₯ = πΉ π₯ .
Linear transformations of X are also Normal.
If π = ππ + π, then π~π© ππ + π, π2π2
Proof:
β’ πΈ π = πΈ ππ + π = ππΈ π + π = ππ + π
β’ Var π = Var ππ + π = π2Var π = π2π2
β’ π is also Normal
17
Proof in Ross,
10th ed (Section 5.4)
Linearity of Expectation
Var ππ + π = π2Var π
Lisa Yan, CS109, 2020
2. Symmetry of Normal RVs
Let π~π© π, π2 with CDF π π β€ π₯ = πΉ π₯ .
The PDF of a Normal RV is symmetric about the mean π.
πΉ π β π₯ = 1 β πΉ π + π₯
18
π(π₯)
π₯π
Lisa Yan, CS109, 2020
Using symmetry of the Normal RV
19
1. π π β€ π§
2. π π < π§
3. π π β₯ π§
4. π π β€ βπ§
5. π π β₯ βπ§
6. π(π¦ < π < π§) π€
A. πΉ π§
B. 1 β πΉ(π§)
C. πΉ π§ β πΉ(π¦)
= πΉ π§
π§
π(π§)
πΉ π β π₯ = 1 β πΉ π + π₯
π = 0
Let π~π© 0,1 with CDF π π β€ π§ = πΉ π§ .
Suppose we only knew numeric valuesfor πΉ π§ and πΉ π¦ , for some π§, π¦ β₯ 0.
How do we compute the following probabilities?
Lisa Yan, CS109, 2020
Using symmetry of the Normal RV
20
1. π π β€ π§
2. π π < π§
3. π π β₯ π§
4. π π β€ βπ§
5. π π β₯ βπ§
6. π(π¦ < π < π§)
A. πΉ π§
B. 1 β πΉ(π§)
C. πΉ π§ β πΉ(π¦)
= πΉ π§
= πΉ π§
= 1 β πΉ(π§)
= 1 β πΉ(π§)
= πΉ π§
= πΉ π§ β πΉ(π¦)
Symmetry is particularly useful when
computing probabilities of zero-mean
Normal RVs.
π§
π(π§)
πΉ π β π₯ = 1 β πΉ π + π₯
π = 0
Let π~π© 0,1 with CDF π π β€ π§ = πΉ π§ .
Suppose we only knew numeric valuesfor πΉ π§ and πΉ π¦ , for some π§, π¦ β₯ 0.
How do we compute the following probabilities?
Lisa Yan, CS109, 2020
Computing probabilities with Normal RVs
Let π~π© π, π2 .
To compute the CDF, π π β€ π₯ = πΉ π₯ :
β’ We cannot analytically solve the integral (it has no closed form)
β’ β¦but we can solve numerically using a function Ξ¦:
πΉ π₯ = Ξ¦π₯ β π
π
22
CDF of the
Standard Normal, π
Lisa Yan, CS109, 2020
The Standard Normal random variable π is defined as follows:
Other names: Unit Normal
CDF of π defined as:
Standard Normal RV, π
23
π~π©(0, 1) Variance
Expectation πΈ π = π = 0
Var π = π2 = 1
π π β€ π§ = Ξ¦(π§)
Note: not a new distribution; just
a special case of the Normal
Lisa Yan, CS109, 2020
Ξ¦ has been numerically computed
24
π π β€ 1.31 = Ξ¦(1.31)
ππ§
π§
Ξ¦(π§)
Standard Normal Table only has
probabilities Ξ¦(π§) for π§ β₯ 0.
Lisa Yan, CS109, 2020
History fact: Standard Normal Table
25
The first Standard Normal Table was computed by Christian Kramp, French astronomer (1760β1826), in Analysedes RΓ©fractions Astronomiques et Terrestres, 1799
Used a Taylor series expansion to the third power
Lisa Yan, CS109, 2020
Probabilities for a general Normal RV
Let π~π© π, π2 . To compute the CDF π π β€ π₯ = πΉ π₯ ,we use Ξ¦, the CDF for the Standard Normal π~π©(0, 1):
πΉ π₯ = Ξ¦π₯ β π
πProof:
26
πΉ π₯ = π π β€ π₯
= π π β π β€ π₯ β π = ππ β π
πβ€π₯ β π
π
= π π β€π₯ β π
π
Algebra + π > 0
Definition of CDF
β’πβπ
π=
1
ππ β
π
πis a linear transform of π.
β’ This is distributed as π©1
ππ β
π
π,1
π2π2 =π© 0,1
β’ In other words, πβπ
π= π~π© 0,1 with CDF Ξ¦.= Ξ¦
π₯ β π
π
Lisa Yan, CS109, 2020
Probabilities for a general Normal RV
Let π~π© π, π2 . To compute the CDF π π β€ π₯ = πΉ π₯ ,we use Ξ¦, the CDF for the Standard Normal π~π©(0, 1):
πΉ π₯ = Ξ¦π₯ β π
πProof:
27
πΉ π₯ = π π β€ π₯
= π π β π β€ π₯ β π = ππ β π
πβ€π₯ β π
π
= π π β€π₯ β π
π
Algebra + π > 0
Definition of CDF
β’πβπ
π=
1
ππ β
π
πis a linear transform of π.
β’ This is distributed as π©1
ππ β
π
π,1
π2π2 =π© 0,1
β’ In other words, πβπ
π= π~π© 0,1 with CDF Ξ¦.= Ξ¦
π₯ β π
π
1. Compute π§ = π₯ β π /π.
2. Look up Ξ¦ π§ in Standard Normal table.
Lisa Yan, CS109, 2020
Campus bikes
You spend some minutes, π, traveling between classes.β’ Average time spent: π = 4 minutesβ’ Variance of time spent: π2 = 2 minutes2
Suppose π is normally distributed. What is the probability you spend β₯ 6 minutes traveling?
28
π~π©(π = 4, π2 = 2) π π β₯ 6 = ΰΆ±6
β
π(π₯)ππ₯ (no analytic solution)
1. Compute π§ =π₯βπ
π2. Look up Ξ¦(π§) in table
π π β₯ 6 = 1 β πΉπ₯(6)
= 1 β Ξ¦6 β 4
2
Γ
β 1 β Ξ¦ 1.41
1 β Ξ¦ 1.41β 1 β 0.9207= 0.0793
Lisa Yan, CS109, 2020
Is there an easier way? (yes)
Let π~π© π, π2 . What is π π β€ π₯ = πΉ π₯ ?
β’ Use Python
β’ Use website tool
29
from scipy import statsX = stats.norm(mu, std)X.cdf(x)
SciPy reference:https://docs.scipy.org/doc/scipy/refere
nce/generated/scipy.stats.norm.html
Website tool: https://web.stanford.edu/class/cs109
/handouts/normalCDF.html
Lisa Yan, CS109, 2020
The Normal (Gaussian) Random Variable
Let π~π© π, π2 .
The PDF of π is defined as:
31
π π₯ =1
π 2ππβ
π₯ β π 2
2π2
normalizing constantexponential
tail
symmetric
around π
variance π2
manages spread
π₯
ππ₯
Review
ThinkSlide 34 has a question to go over by yourself.
Post any clarifications here!
https://us.edstem.org/courses/667/discussion/89934
Think by yourself: 2 min
32
π€(by yourself)
Lisa Yan, CS109, 2020
Normal Random Variable
Match PDF to distribution:
π© 0, 1
π©(β2, 0.5)
π© 0, 5
π©(0, 0.2)
33
A.
B.
C.
D.
π~π©(π, π2)mean variance
π€(by yourself)
π₯
ππ₯
Lisa Yan, CS109, 2020
Knowing how to use a Standard Normal Table will
still be useful in our understanding of Normal RVs.
Computing probabilities with Normal RVs: Old school
34
*particularly useful if we had closed book exams with no calculator**
**we have open book exams with calculators this quarter
Ξ¦ π§ for non-negative π§
*
Lisa Yan, CS109, 2020
Computing probabilities with Normal RVs
Let π~π© π, π2 . What is π π β€ π₯ = πΉ π₯ ?
1. Rewrite in terms of standard normal CDF Ξ¦ by computing π§ =π₯βπ
π.
Linear transforms of Normals are Normal:
πΉ π₯ = Ξ¦π₯ β π
π
2. Then, look up in a Standard Normal Table, where π§ β₯ 0.
Normal PDFs are symmetric about their mean:
Ξ¦ βπ§ = 1 β Ξ¦ π§
35
Review
π =πβπ
π, where π~ π© 0,1
Lisa Yan, CS109, 2020
Get your Gaussian On
Let π~π© π = 3, π2 = 16 . Std deviation π = 4.
1. π π > 0
36
β’ If π~π© π, π2 , then
πΉ π₯ = Ξ¦π₯βπ
π
β’ Symmetry of the PDF of
Normal RV implies
Ξ¦ βπ§ = 1 β Ξ¦ π§
Breakout Rooms
Slide 39 has two questions to go over in groups.
Post any clarifications here!
https://us.edstem.org/courses/667/discussion/89934
Breakout rooms: 5 mins
37
π€
Lisa Yan, CS109, 2020
Get your Gaussian On
Let π~π© π = 3, π2 = 16 .Note standard deviation π = 4.
How would you write each of the belowprobabilities as a function of thestandard normal CDF, Ξ¦?
1. π π > 0 (we just did this)
2. π 2 < π < 5
3. π π β 3 > 6
38
β’ If π~π© π, π2 , then
πΉ π₯ = Ξ¦π₯βπ
π
β’ Symmetry of the PDF of
Normal RV implies
Ξ¦ βπ§ = 1 β Ξ¦ π§
π€
Lisa Yan, CS109, 2020
Get your Gaussian On
Let π~π© π = 3, π2 = 16 . Std deviation π = 4.
1. π π > 0
2. π 2 < π < 5
39
β’ If π~π© π, π2 , then
πΉ π₯ = Ξ¦π₯βπ
π
β’ Symmetry of the PDF of
Normal RV implies
Ξ¦ βπ§ = 1 β Ξ¦ π§
Lisa Yan, CS109, 2020
Get your Gaussian On
Let π~π© π = 3, π2 = 16 . Std deviation π = 4.
1. π π > 0
2. π 2 < π < 5
3. π π β 3 > 6
40
Compute π§ =π₯βπ
π
β’ If π~π© π, π2 , then
πΉ π₯ = Ξ¦π₯βπ
π
β’ Symmetry of the PDF of
Normal RV implies
Ξ¦ βπ₯ = 1 β Ξ¦ π₯
π π < β3 + π π > 9
= πΉ β3 + 1 β πΉ 9
= Ξ¦β3 β 3
4+ 1 βΞ¦
9 β 3
4
Look up Ξ¦(z) in table
Lisa Yan, CS109, 2020
Get your Gaussian On
Let π~π© π = 3, π2 = 16 . Std deviation π = 4.
1. π π > 0
2. π 2 < π < 5
3. π π β 3 > 6
41
Compute z =π₯βπ
πLook up Ξ¦(z) in table
π π < β3 + π π > 9
= πΉ β3 + 1 β πΉ 9
= Ξ¦β3 β 3
4+ 1 βΞ¦
9 β 3
4
= Ξ¦ β3
2+ 1 βΞ¦
3
2
= 2 1 β Ξ¦3
2
β 0.1337
β’ If π~π© π, π2 , then
πΉ π₯ = Ξ¦π₯βπ
π
β’ Symmetry of the PDF of
Normal RV implies
Ξ¦ βπ₯ = 1 β Ξ¦ π₯
Lisa Yan, CS109, 2020
Announcements
43
Problem Set 3
Due: Friday 7/13 1pm PT
Timβs OH permanently moved to 8-10pm PT, Wednesday
Lisa Yan, CS109, 2020
Interesting probability news
44
https://www.forbes.com/sites/lanceeliot/2020/04/12/on-
the-probabilities-of-social-distancing-as-gleaned-from-ai-self-
driving-cars/#218da4489472
Breakout Rooms
Slide 47 has two questions to go over in groups.
Post any clarifications here!
https://us.edstem.org/courses/667/discussion/89934
Breakout rooms: 5 mins
45
π€
Lisa Yan, CS109, 2020
Noisy Wires
Send a voltage of 2 V or β2 V onwire (to denote 1 and 0, respectively).
β’ π = voltage sent (2 or β2)β’ π = noise, π~π© 0, 1β’ π = π + π voltage received.
Decode: 1 if π β₯ 0.50 otherwise.
1. What is P(decoding error | original bit is 1)?i.e., we sent 1, but we decoded as 0?
2. What is P(decoding error | original bit is 0)?
These probabilities are unequal. Why might this be useful?46
π€πΉπ (π)
π = π
Lisa Yan, CS109, 2020
Noisy Wires
Send a voltage of 2 V or β2 V onwire (to denote 1 and 0, respectively).
β’ π = voltage sent (2 or β2)β’ π = noise, π~π© 0, 1β’ π = π + π voltage received.
Decode: 1 if π β₯ 0.50 otherwise.
1. What is P(decoding error | original bit is 1)?i.e., we sent 1, but we decoded as 0?
47πΉπ (π)
π = π
π π < 0.5| π = 2 = π 2 + π < 0.5 = π π < β1.5 Y is Standard Normal
= Ξ¦ β1.5 = 1 β Ξ¦ 1.5 β 0.0668
Lisa Yan, CS109, 2020
Noisy Wires
Send a voltage of 2 V or β2 V onwire (to denote 1 and 0, respectively).
β’ π = voltage sent (2 or β2)β’ π = noise, π~π© 0, 1β’ π = π + π voltage received.
Decode: 1 if π β₯ 0.50 otherwise.
1. What is P(decoding error | original bit is 1)?i.e., we sent 1, but we decoded as 0?
2. What is P(decoding error | original bit is 0)?
48πΉπ (π)
π = π
0.0668
β 0.0062π π β₯ 0.5| π = β2 = π β2 + π β₯ 0.5 = π π β₯ 2.5Asymmetric decoding probability: We would like to avoid
mistaking a 0 for 1. Errors the other way are more tolerable.
Lisa Yan, CS109, 2020
ELO ratings
50
What is the probability that the Warriors win?
How do you model zero-sum games?
Lisa Yan, CS109, 2020
ELO ratings
Each team has an ELO score π, calculated based on theirpast performance.
β’ Each game, a team hasability π΄~π© π, 2002 .
β’ The team with the highersampled ability wins.
What is the probabilitythat Warriors winthis game?
Want: π Warriors win = π π΄π > π΄π΅
51
0
0.0005
0.001
0.0015
0.002
0.0025
1000 1500 2000 2500
π=1470
0
0.0005
0.001
0.0015
0.002
0.0025
1000 1500 2000 2500
π=1657
Arpad Elo
Warriors π΄π~π© π = 1657, 2002
Opponents π΄π΅~π© π = 1470, 2002
Lisa Yan, CS109, 2020
ELO ratings
52
Want: π Warriors win = π π΄π > π΄π΅
β 0.7488, calculated by sampling
from scipy import statsWARRIORS_ELO = 1657OPPONENT_ELO = 1470STDEV = 200NTRIALS = 10000
nSuccess = 0for i in range(NTRIALS):w = stats.norm.rvs(WARRIORS_ELO, STDEV)b = stats.norm.rvs(OPPONENT_ELO, STDEV)if w > b:nSuccess += 1
print("Warriors sampled win fraction", float(nSuccess) /NTRIALS)