Central Limit Theorem - Dartmouth College
Transcript of Central Limit Theorem - Dartmouth College
Central Limit Theorem
Β§ number of measurements needed to estimate the mean
Β§ convergence of the estimated mean Β§ a better approximation than the
Chebyshev Inequality (LLN).
Xingru [email protected]
XC 2020
Central Limit Theorem
π! π"
π# π$
π% β―
π& π'normal
density function
+
+
+
+
+
XC 2020
Central Limit Theorem
π! π"
π# π$
π% β―
π& π'normal
density function
++
+
+
+
If π' is the sum of π mutually independent random variables, then the distribution function of π' is well-approximated by a normal density function.
XC 2020
CENTRAL LIMIT THEOREM FOR BERNOULLI TRIALS
binomial distribution
XC 2020
Bernoulli Trials
A Bernoulli trials process is a sequence of π chance experiments such that
Β§ Each experiment has two possible outcomes, which we may call success and failure.
Β§ The probability π of success on each experiment is the same for each experiment, and this probability is not affected by any knowledge of previous outcomes. The probability π of failure is given by π = 1 β π.
50%Toss a Coinhead & tail
60%Weather Forecast
not rain & rain
12.5%Wheel of Fortune50 points & others
πBernoulli Trialsoutcome 1 & 2
XC 2020
Binomial Distribution
Β§ Let π be a positive integer and let π be a real number between 0 and 1.
Β§ Let π΅ be the random variable which counts the number of successes in a Bernoulli trials process with parameters π and π.
Β§ Then the distribution π(π, π, π) of π΅ is called the binomial distribution.
Binomial Distribution
π π, π, π =πππ(π')(
1 2 3 4 5
β β β β β
XC 2020
Binomial Distribution
Β§ Consider a Bernoulli trials process with probability π for success on each trial.
Β§ Let π* = 1 or 0 according as the πth outcome is a success or failure.
Β§ Let π' = π& + π% +β―+ π'. Then π' is the number of successes in π trials.
Β§ We know that π' has as its distribution the binomial probabilities π(π, π, π).
XC 2020
π increases
drifts off to the right
flatten out
XC 2020
The Standardized Sum of π4
Β§ Consider a Bernoulli trials process with probability π for success on each trial.
Β§ Let π* = 1 or 0 according as the πth outcome is a success or failure.
Β§ Let π' = π& + π% +β―+ π'. Then π' is the number of successes in π trials.
Β§ We know that π' has as its distribution the binomial probabilities π(π, π, π).
Β§ The standardized sum of π' is given by
π'β =,!)'-'-.
.
π = β― π% = β―
XC 2020
The Standardized Sum of π4
Β§ Consider a Bernoulli trials process with probability π for success on each trial.
Β§ Let π* = 1 or 0 according as the πth outcome is a success or failure.
Β§ Let π' = π& + π% +β―+ π'. Then π' is the number of successes in π trials.
Β§ We know that π' has as its distribution the binomial probabilities π(π, π, π).
Β§ The standardized sum of π' is given by
π'β =,!)'-'-.
.
Β§ π'β always has expected value 0 and variance 1.π = 0
π% = 1
XC 2020
The Standardized Sum of π4
=π'β =π' β πππππ
π(π, π, 1)π(π, π, 2)
π(π, π, 3)π(π, π, 4)
π(π, π, 5)
π(π, π, 0)
β―π(π, π, π)
π(π, π, π)
0 1 2 3 4 5 β¦ π
0 β πππππ
1 β πππππ
2 β πππππ
3 β πππππ
4 β πππππ
5 β πππππ
β¦ π β πππππ
π
π₯XC 2020
π'β =π' β πππππ
π = 0
π% = 1
XC 2020
standardized sum
standard normal distribution
shapes: sameheights: different
why?
XC 2020
standardized sum standard normal distribution
shapes: sameheights: different why?
standardized sum: β(/0' β* = 1
standard normal distribution: β«)1
21π π₯ ππ₯ = 1
XC 2020
standardized sum: β(/0' β* = 1
standard normal distribution: β«)1
21π π₯ ππ₯ = 1 β β(/0' π π₯* ππ₯
C(/0
'
β* = 1 C(/0
'
π π₯* ππ₯ β 1
=β( = π π₯( ππ₯
=π π₯( =β(ππ₯
XC 2020
=β) = π(π, π, π)=π'β =π' β πππππ =ππ₯ = π₯)*& β π₯) =
1πππ
π(π, π, 1)π(π, π, 2)
π(π, π, 3)π(π, π, 4)
π(π, π, 5)
π(π, π, 0)
β―π(π, π, π)
0 1 2 3 4 5 β¦ π
0 β πππππ
1 β πππππ
2 β πππππ
3 β πππππ
4 β πππππ
5 β πππππ
β¦ π β πππππ
=π = ππππ₯) + ππ π : the integer nearest to π
XC 2020
β( = π(π, π, π)
ππ₯ = π₯(2& β π₯( =1πππ
π = ππππ₯( + ππ
=π π₯( = 3"45= πππ π(π, π, ππππ₯( + ππ )
XC 2020
Central Limit Theorem for Binomial Distributions
Β§ For the binomial distribution π(π, π, π) we have
lim'β1
ππππ π, π, ππ + π₯ πππ = π(π₯),
where π(π₯) is the standard normal density.
Β§ The proof of this theorem can be carried out using Stirlingβs approximation.
Stirlingβs FormulaThe sequence π! is asymptotically equal
to π'π)' 2ππ.
XC 2020
Approximating Binomial Distributions
=lim'β1
ππππ π, π, ππ + π₯ πππ = π(π₯)
?π π, π, π β β―
normalbinomial
binomial normal
XC 2020
Approximating Binomial Distributions
=lim'β,
ππππ π, π, ππ + π₯ πππ = π(π₯) ?π π, π, π β β―
π = ππ + π₯ πππ
π₯ =π β πππππ
π π, π, π βπ(π₯)πππ
π(π, π, π) β1πππ
π(π β πππππ
)
XC 2020
Example
Toss a coin
Find the probability of exactly 55 heads in 100 tosses of a coin.
=lim'β,
ππππ π, π, ππ + π₯ πππ = π(π₯)
π(π, π, π) β1πππ
π(π β πππππ
)
XC 2020
Example
Toss a coin
Find the probability of exactly 55 heads in 100 tosses of a coin.
π(π, π, π) β1πππ
π(π β πππππ
)π = 100 π =12
ππ = 50 πππ = 5
π = 55
π₯ =π β πππππ = 1 π(100,
12, 55) β
15π(1)
XC 2020
Example
Toss a coin
Find the probability of exactly 55 heads in 100 tosses of a coin.
π(π, π, π) β1πππ π(
π β πππππ )
π 100,12, 55 β
15π 1 =
15(12π
π)&/%)
Standard normal distribution π
π = 0 and π = 1
π8 π₯ = &%9:
π) 5); #/%:# = &%9π)5#/%.
XC 2020
π(π, π, 1)π(π, π, 2)
π(π, π, 3)π(π, π, 4)
π(π, π, 5)
π(π, π, 0)
β―π(π, π, π)
0 1 2 3 4 5 β¦ π
0 β πππππ
1 β πππππ
2 β πππππ
3 β πππππ
4 β πππππ
5 β πππππ
β¦ π β πππππ
A specific outcome
XC 2020
π(π, π, 1)π(π, π, 2)
π(π, π, 3)π(π, π, 4)
π(π, π, 5)
π(π, π, 0)
β―π(π, π, π)
0 1 2 3 4 5 β¦ π
0 β πππππ
1 β πππππ
2 β πππππ
3 β πππππ
4 β πππππ
5 β πππππ
β¦ π β πππππ
Outcomes in an Interval
XC 2020
Central Limit Theorem for Binomial Distributions
Β§ Let π' be the number of successes in n Bernoulli trials with probability π for success, and let π and π be two fixed real numbers.
Β§ Then
lim'β21
π π β€ ,!)'-'-.
β€ π = β«<=π π₯ ππ₯ = NA(π, π).
Β§ We denote this area by NA(π, π).
π β€π' β πππππ
= π'β β€ π π πππ + ππ β€ π' β€ π πππ + ππ
XC 2020
lim'β21
π π β€ ,!)'-'-.
β€ π = NA(π, π).
π β€π' β πππππ
= π'β β€ π π πππ + ππ β€ π' β€ π πππ + ππ
π β€ π' β€ π π =π β πππππ
β€ π'β β€π β πππππ
= π
lim'β21
π π β€ π' β€ π = lim'β21
π *)'-'-.
β€ π'β β€>)'-'-.
= NA(*)'-'-.
, >)'-'-.
).
Approximating Binomial Distributions
XC 2020
π β€ π' β€ π π =π β πππππ
β€ π'β β€π β πππππ
= π
lim'β21
π π β€ π' β€ π = lim'β21
π *)'-'-.
β€ π'β β€>)'-'-.
= NA(*)'-'-.
, >)'-'-.
).
Approximating Binomial Distributions (more accurate)
lim@βBC
π π β€ π@ β€ π = NA(DE!"E@F
@FG,HB!"E@F
@FG).
π π
XC 2020
Example
Toss a coin
A coin is tossed 100 times. Estimate the probability that the number of heads lies between 40 and 60 (including the end points).
=lim'β21
π π β€ π' β€ π = NA(π β 12 β ππ
πππ ,π + 12 β ππ
πππ )
XC 2020
Toss a coin
A coin is tossed 100 times. Estimate the probability that the number of heads lies between 40 and 60 (including the end points).
π = 100 π =12
ππ = 50 πππ = 5
π = 40
π β 12 β πππππ
= β2.1
π = 60
π + 12 β πππππ
= 2.1
the outcome will not deviate by more than two standard deviations from the expected value
XC 2020
Toss a coin
A coin is tossed 100 times. Estimate the probability that the number of heads lies between 40 and 60 (including the end points).
π = 40
π β 12 β πππππ = β2.1
π = 60
π + 12 β πππππ = 2.1
=lim$β&'
π π β€ π$ β€ π = NA(π β 12 β ππ
πππ,π + 12 β ππ
πππ)
=lim$β&'
π 40 β€ π$ β€ 60 = NA β2.1,2.1= 2NA 0, 2.1
2π 2.1π
XC 2020
Approximating Binomial Distribution
πΊπβ = πlim$β'
ππππ π, π, ππ + π₯ πππ = π(π₯)
πΊπ = π
π(π, π, π) β1πππ π(
π β πππππ )
π β€ πΊπβ β€ π
lim'β*,
π π β€ -!.'/'/0
β€ π = NA(π, π).
π β€ πΊπ β€ π
lim$β&'
π π β€ π$ β€ π = NA(()!")$*
$*+,,&!")$*
$*+).
CLT
XC 2020
CENTRAL LIMIT THEOREM FOR DISCRETE INDEPENDENT TRIALS
any independent trials process such that the individual trials have finite variance
XC 2020
The Standardized Sum of π4
Β§ Consider an independent trials process with common distribution function π π₯defined on the integers, with expected value π and variance π%.
Β§ Let π' = π& + π% +β―+ π' be the sum of π independent discrete random variables of the process.
Β§ The standardized sum of π' is given by
π'β =,!)';':#
.
Β§ π'β always has expected value 0 and variance 1.
πΈ(π') = β―
π(π') = β―
XC 2020
independent trials process
π(π' = π)
π π₯)=π β ππππ%
=π'β =π' β ππππ%
normal
ππ-π(π$ = π)
XC 2020
Approximation Theorem
Β§ Let π&, π%, β―, π' be an independent trials process and let π' = π& + π% +β―+ π'.
Β§ Assume that the greatest common divisor of the differences of all the values that the π( can take on is 1.
Β§ Let πΈ π( = π and π π( = π%.
Β§ Then for π large,
π(π' = π) β &':#
π(()';':#
).
Β§ Here π(π₯) is the standard normal density.
XC 2020
Central Limit Theorem for a Discrete Independent Trials Process
Β§ Let π' = π& + π% +β―+ π' be the sum of n discrete independent random variables with common distribution having expected value π and variance π%.
Β§ Then, for π < π,
lim'β21
π π β€ ,!)';':#
β€ π = &%9β«<
= π)5#/%ππ₯ = NA(π, π).
XC 2020
Approximating a Discrete Independent Trials Process
π(π' = π) β1ππ%
π(π β ππππ%
)
lim'β*,
π π β€ π' β€ π =
NA(1.'2'3"
, 4.'2'3"
).
πΊπ = π
π β€ πΊπ β€ π
CLT
XC 2020
Approximating a Discrete Independent Trials Process
lim'β*,
π π β€ π' β€ π =
NA(5.#".'/
'/0,6*#".'/
'/0).
π β€ πΊπ β€ π
π(π' = π) β1ππ%
π(π β ππππ%
)
πΊπ = π
π(π, π, π) β1πππ
π(π β πππππ
)
πΊπ = π
lim'β*,
π π β€ π' β€ π =
NA(1.'2'3"
, 4.'2'3"
).
π β€ πΊπ β€ π
Bernoulli
XC 2020
ExampleA surveying instrument makes an error of -2, -1, 0, 1, or 2 feet with equal probabilities when measuring the height of a 200-foot tower.
XC 2020
Find the expected value and the variance for the height obtained using this instrument once.
Example
height = error + 200
error -2 -1 0 1 2
probability15
15
15
15
15
πΈ error =15β2 β 1 + 0 + 1 + 2 = 0
π error =15(β2)%+(β1)%+0% + 1% + 2% = 2
πΈ height= 0 + 200 = 200
π height = 2
XC 2020
Estimate the probability that in 18 independent measurements of this tower, the average of the measurements is between 199 and 201, inclusive.
Example
π = 200 π% = 2
=lim'β21
π π β€ π' β€ π = NA(π β ππππ%
,π β ππππ%
).
199 β€π'π=π'18
β€ 201π = 18
XC 2020
Example
π = 200 π% = 2
=lim'β*,
π π β€ π' β€ π = NA(π β ππππ%
,π β ππππ%
).
199 β€π'π=π'18
β€ 201π = 18
π 199 β€π'18
β€ 201 = π 3582 β€ π' β€ 3618
β NA3582 β 3600
36,3618 β 3600
36= NA(β3, 3).
XC 2020
03
01
02
Β§ independent
Discrete Random Variables
§ independent§ identical
Bernoulli Trials
§ independent§ identical
Discrete Trials
Central Limit Theorem
XC 2020
Central Limit Theorem
Β§ Let π&, π%, β―, π' be a sequence of independent discrete random variables. There exist a constant π΄, such that π* β€ π΄ for all π.
Β§ Let π' = π& + π% +β―+ π' be the sum. Assume that π' β β.
Β§ For each π, denote the expected value and variance of π* by π* and π*%, respectively.
Β§ Define the expected value and variance of π' to be π' and π '%, respectively.
Β§ For π < π,
lim'β21
π π β€ ,!)@!A!
β€ π = &%9β«<
= π)5#/%ππ₯ = NA(π, π).
XC 2020
CENTRAL LIMIT THEOREM FOR CONTINUOUS INDEPENDENT TRIALScontinuous random variables with a common density function
XC 2020
Central Limit Theorem
Β§ Let π' = π& + π% +β―+ π' be the sum of π independent continuous random variables with common density function π having expected value π and variance π%.
Β§ Let
π'β =,!)';':#
.
Β§ Then we have, for all π < π,
lim'β21
π π β€ π'β β€ π = &%9β«<
= π)5#/%ππ₯ = NA(π, π).
XC 2020
Example
Β§ Suppose a surveyor wants to measure a known distance, say of 1 mile, using a transit and some method of triangulation.
Β§ He knows that because of possible motion of the transit, atmospheric distortions, and human error, any one measurement is apt to slightly in error.
Β§ He plans to make several measurements and take an average.
Β§ He assumes that his measurements are independent random variables with common distribution of mean π = 1 and standard deviation π =0.0002.
XC 2020
Example
Β§ He can say that if π is large, the average ,!
'has a density function that
is approximately normal, with mean 1mile, and standard deviation 0.000%
'miles.
Β§ How many measurements should he make to be reasonably sure that his average lies within 0.0001 of the true value?
XC 2020
Example
π = 1
Expected Value
π = 0.0002
Standard Deviationπ' = π& + π% +β―+ π'
Sum of π independent measurements
Β§ He can say that if π is large, the average ,!' has a density function that is approximately normal, with mean 1 mile, and standard deviation 0.000%
'miles.
πU = (0.0002)UVariance π΄' =
π& + π% +β―+ π'π
πΈ π΄' = ππ· π΄' =
ππ
π π΄' =π%
π
Average of π independent measurements
XC 2020
Example
Chebyshev inequality
LLN
π'β =π' β ππππ%
CLT
Β§ How many measurements should he make to be reasonably sure that his average lies within 0.0001 of the true value?
XC 2020
ππ
πΊπ
π·(|πΏ β π|β₯ πΊ)
inequalityChebyshevπ·(|πΏ β π| β₯ πΊ) β€
ππ
πΊπ
This is a sample text. Insert your desired text
here.
This is a sample text. Insert your desired text here.
Let π be a discrete random variable with expected value π = πΈ(π) and variance π% = π(π), and let π > 0 be any positive number. Then
not necessarily positive
XC 2020
Example
Chebyshev inequality
LLN
Β§ How many measurements should he make to be reasonably sure that his average lies within 0.0001 of the true value?
π = 1 π% = (0.0002)%
=π(|π β π| β₯ π) β€π%
π%
π π΄' β π β₯ 0.0001 β€π%
ππ%=
0.0002 %
π 0.0001 % =4π
XC 2020
Example
Chebyshev inequality
LLN
Β§ How many measurements should he make to be reasonably sure that his average lies within 0.0001 of the true value?
=π(|π β π| β₯ π) β€π%
π%
π π΄' β π β₯ 0.0001 β€π%
ππ%=4π= 0.05 π = 80
XC 2020
Example
π'β =π' β ππππ%
CLT
=lim'β21
π π β€ π'β β€ π = NA(π, π)
π π΄' β π β€ 0.0001 = π π' β ππ β€ 0.5ππ
= ππ' β ππππ%
β€ 0.5 π = π β0.5 π β€ π'β β€ 0.5 π
β NA β0.5 π, 0.5 π
π = 1 π% = (0.0002)%
XC 2020
Example
π'β =π' β ππππ%
CLT
=lim'β21
π π β€ π'β β€ π = NA(π, π)
π π΄' β π β€ 0.0001 β NA β0.5 π, 0.5 π = 0.95
0.5 π = 2 π = 16
XC 2020
Example
Chebyshev inequality
LLN
π'β =π' β ππππ%
CLT
Β§ How many measurements should he make to be reasonably sure that his average lies within 0.0001 of the true value?
π = 80 π = 16
XC 2020