8. Limit theorems
description
Transcript of 8. Limit theorems
![Page 1: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/1.jpg)
ENGG 2040C: Probability Models and Applications
Andrej Bogdanov
Spring 2014
8. Limit theorems
![Page 2: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/2.jpg)
Many times we do not need to calculate probabilities exactly
An approximate or qualitative estimate often sufficesP(magnitude 7+ earthquake within 10 years) = ?
This is often a much easier task
![Page 3: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/3.jpg)
What do you think?
I toss a coin 1000 times. The probability that I get a streak of 14 consecutive heads is
< 10% ≈ 50% > 90%A B C
![Page 4: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/4.jpg)
Consecutive heads
where Ii is an indicator r.v. for the event “14 consecutive heads starting at position i”
Let N be the number of occurrences of 14 consecutive heads in 1000 coin flips.
N = I1 + … + I987
E[Ii ] = P(Ii = 1) = 1/214E[N ] = 987 ⋅ 1/214
= 987/16384
≈ 0.0602
![Page 5: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/5.jpg)
Markov’s inequality
For every non-negative random variable X and every value a:
P(X ≥ a) ≤ E[X] / a.
E[N ] ≈ 0.0602
P[N ≥ 1] ≤ E[N ] / 1 ≤ 6%.
![Page 6: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/6.jpg)
Proof of Markov’s inequality
For every non-negative random variable X: and every value a:
P(X ≥ a) ≤ E[X] / a.
E[X ] = E[X | X ≥ a ] P(X ≥ a) + E[X | X < a ] P(X < a)
≥ 0≥ a ≥ 0
E[X ] ≥ a P(X ≥ a) + 0.
![Page 7: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/7.jpg)
Hats
1000 people throw their hats in the air. What is the probability at least 100 people get their hat back?
N = I1 + … + I1000where Ii is the indicator for the event that person i
gets their hat. Then E[Ii ] = P(Ii = 1) = 1/n
Solution
E[N ] = n 1/n
= 1 P[N ≥ 100] ≤ E[N ] / 100 = 1%.
![Page 8: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/8.jpg)
Patterns
A coin is tossed 1000 times. Give an upper bound on the probability that the pattern HH occurs:
(b) at most 100 times
(a) at least 500 times
![Page 9: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/9.jpg)
Patterns
Let N be the number of occurrences of HH.
P[N ≥ 500] ≤ E[N ] / 500
= 249.75/500
≈ 49.88%so 500+ HHs occur with probability ≤
49.88%.P[N ≤ 100] ≤ ? P[999 – N ≥
899]
(b)P[N ≤ 100] =
≤ E[999 – N ] / 899= (999 – 249.75)/
899≤ 83.34%
Last time we calculated E[N ] = 999/4 = 249.75.
(a)
![Page 10: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/10.jpg)
Computer simulation of patterns# toss n coins and count number of consecutive head pairsdef consheads(n): count = 0 lastone = randint(0, 1) thisone = randint(0, 1) for i in range(n - 1): if lastone == 1 and thisone == 1: count = count + 1 lastone = thisone thisone = randint(0, 1) return count
>>> for i in range(100): print(consheads(1000), end = “ ”)
264 260 256 263 272 224 256 254 275 231 242 232 247 268 229 270 231 272 241 238 257 239 251 252 255 249 267 223 272 254 219 266 271 265 212 262 239 253 265 254 262 231 271 242 258 255 219 281 238 246 242 263 245 239 270 199 251 229 240 253 282 258 237 276 247 221 242 226 232 244 222 258 255 294 239 267 253 259 236 239 236 243 254 240 232 248 270 252 232 282 248 244 251 223 226 222 288 266 268 236
![Page 11: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/11.jpg)
Chebyshev’s inequality
For every random variable X and every t:
P(|X – m| ≥ ts) ≤ 1 / t2.
where m = E[X], s = √Var[X].
![Page 12: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/12.jpg)
Patterns
E[N ] = 999/4 = 249.75Var[N] = (5⋅999 – 7)/16 = 311.75
m = 249.75s ≈ 17.66
(a)
P(X ≥ 500)≤ P(|X – m| ≥ 14.17s) ≤ 1/14.172 ≈ 0.50%
(b)
P(X ≤ 100)≤ P(|X – m| ≥ 8.47s) ≤ 1/8.472 ≈ 1.39%
![Page 13: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/13.jpg)
Proof of Chebyshev’s inequality
For every random variable X and every a:
P(|X – m| ≥ ts) ≤ 1 / t2.
where m = E[X], s = √Var[X].
P(|X – m| ≥ ts) = P((X – m)2 ≥ t2s2) ≤ E[(X – m)2] / t2s2 = 1 / t2.
![Page 14: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/14.jpg)
An illustration
mm – ts m + ts
sP(|X – m| ≥ ts ) ≤ 1 / t2.
m a
P( X ≥ a ) ≤ m / a.
0
Markov’s inequality:
Chebyshev’s inequality:
![Page 15: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/15.jpg)
Polling
1
2
3
45 6
7 8
9
![Page 16: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/16.jpg)
Polling
Xi =1 if i
0 if i
X1,…, Xn are independent Bernoulli(m)
where m is the fraction of blue voters
X = X1 + … + Xn
X/n is the pollster’s estimate of m
![Page 17: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/17.jpg)
Polling
How accurate is the pollster’s estimate X/n?
E[X] =
= mn E[X1] + … + E[Xn]
Var[X]
= Var [X1] + … + Var [Xn] = s2n
m = E[Xi], s = √Var[Xi]
X = X1 + … + Xn
![Page 18: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/18.jpg)
Polling
E[X] = mnVar[X] = s2n
P( |X – mn| ≥ ts √n ) ≤ 1 / t2.
P( |X/n – m| ≥ e) ≤ d.
confidenceerror
samplingerror
X = X1 + … + Xn
den
![Page 19: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/19.jpg)
The weak law of large numbers
For every e, d > 0 and n ≥ s2/(e2d): P(|X/n – m| ≥ e) ≤ d
X1,…, Xn are independent with same p.m.f. (p.d.f.)m = E[Xi], s = √Var[Xi],
X = X1 + … + Xn
![Page 20: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/20.jpg)
Polling
Say we want confidence error d = 10% and sampling error e = 5% . How many people should we poll?
For e, d > 0 and n ≥ s2/(e2d):
P(|X/n – m| ≥ e) ≤ d
n ≥ s2/(e2d) ≥ 4000s2
For Bernoulli(m) samples, s2 = m (1 – m) ≤ 1/4
This suggests we should poll about 1000 people.
![Page 21: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/21.jpg)
A polling simulation
number of people polled n
X 1 +
… +
Xn
nX1, …, Xn independent Bernoulli(1/2)
polls
ter’s
est
imat
e
![Page 22: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/22.jpg)
A polling simulation
number of people polled n
X 1 +
… +
Xn
n
20 simulations
polls
ter’s
est
imat
e
![Page 23: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/23.jpg)
A more precise estimate
Let’s assume n is large.
Weak law of large numbers:X1 + … + Xn ≈ mn with high
probability
X1,…, Xn are independent with same p.m.f. (p.d.f.)
P( |X – mn| ≥ ts √n ) ≤ 1 / t2.
this suggests X1 + … + Xn ≈ mn + Ts √n
![Page 24: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/24.jpg)
Some experiments
X = X1 + … + Xn Xi independent Bernoulli(1/2)
n = 6
n = 40
![Page 25: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/25.jpg)
Some experiments
X = X1 + … + Xn Xi independent Poisson(1)
n = 3
n = 20
![Page 26: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/26.jpg)
Some experiments
X = X1 + … + Xn Xi independent Uniform(0, 1)
n = 2
n = 10
![Page 27: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/27.jpg)
The normal random variable
f(t) = (2p)-½ e-t /22
tp.d.f. of a normal random variable
![Page 28: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/28.jpg)
The central limit theorem
X1,…, Xn are independent with same p.m.f. (p.d.f.)
where T is a normal random variable.
m = E[Xi], s = √Var[Xi], X = X1 + … + Xn
For every t (positive or negative):
lim P(X ≤ mn + ts √n ) = P(T ≤ t)n → ∞
![Page 29: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/29.jpg)
Polling again
Probability model
X = X1 + … + Xn Xi independent Bernoulli(m)
m = fraction that will vote blue
E[Xi] = m, s = √Var[Xi] = √m(1 - m) ≤ ½.
Say we want confidence error d = 10% and sampling error e = 5% . How many people should we poll?
![Page 30: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/30.jpg)
Polling again
lim P(X ≤ mn – ts √n ) = P(T ≤ -t)n → ∞
5% n
lim P(X ≥ mn + ts √n ) = P(T ≥ t)n → ∞
5% n
lim P(X/n is not within 5% of m) = P(T ≤ -t) + P(T ≥ t)n → ∞
= 2 P(T ≤ -t)
ts √n = 5% n t = 5%√n/s
![Page 31: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/31.jpg)
The c.d.f. of a normal random variable
t
F(t)
P(T ≤ -t)t-t
P(T ≥ t)
![Page 32: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/32.jpg)
Polling again
confidence error = 2 P(T ≤ -t)
We want a confidence error of ≤ 10%:
= 2 P(T ≤ -5%√n/s)≤ 2 P(T ≤ -√n/10)
We need to choose n so that P(T ≤ -√n/10) ≤ 5%.
![Page 33: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/33.jpg)
Polling again
t
F(t)
P(T ≤ -√n/10) ≤ 5%
-√n/10 ≈ -1.645
n ≈ 16.452 ≈ 271
http://stattrek.com/online-calculator/normal.aspx
![Page 34: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/34.jpg)
Party
Give an estimate of the probability that the average arrival time of a guest is past 8:40pm.
Ten guests arrive independently at a party between 8pm and 9pm.
![Page 35: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/35.jpg)
Acute triangles
Drop three points at random on a square. What is the probability that they form an acute triangle?
![Page 36: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/36.jpg)
Simulation
# indicate whether the triangle with the given vertices is acutedef is_acute(x1, y1, x2, y2, x3, y3):
def dot(x1, y1, x2, y2, x0, y0): return (x1 - x0) * (x2 - x0) + (y1 - y0) * (y2 - y0)
a1 = dot(x2, y2, x3, y3, x1, y1) a2 = dot(x3, y3, x1, y1, x2, y2) a3 = dot(x1, y1, x2, y2, x3, y3) return a1 > 0 and a2 > 0 and a3 > 0
# count the fraction of acute triangles among n random samplesdef simulate_triangles(n): count = 0 for i in range(n): if is_acute(uniform(0.0, 1.0), uniform(0.0, 1.0), uniform(
count = count + 1 return 1.0 * count / n
Idea: Conduct a poll among random triangles!
![Page 37: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/37.jpg)
Simulation
Want sampling error e = .01, confidence error d = .051. Rigorous estimate: By weak law of large numbers, we can choose n = s2/(e2d) ≤ 50,000 > simulate_triangles(50000) 0.27326> simulate_triangles(50000)0.27392> simulate_triangles(50000)0.27612
![Page 38: 8. Limit theorems](https://reader036.fdocuments.in/reader036/viewer/2022062410/568161bd550346895dd19c00/html5/thumbnails/38.jpg)
Simulation
Want sampling error e = .01, confidence error d = .052. Non-rigorous (but better) estimate: Central limit theorem suggests choosing n such that ts √n ≤ en, P(Normal < -t) = d
> simulate_triangles(5366) 0.28158777487886694> simulate_triangles(5366)0.27003354453969436> simulate_triangles(5366)0.2849422288483041
t = 1.465
n = (t/2e)2 ≈ 5366