Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani...
Transcript of Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani...
![Page 1: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/1.jpg)
Advanced Probability Theory (Math541)
Instructor: Kani Chen
(Classic)/Modern Probability Theory (1900-1960)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 1 / 17
![Page 2: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/2.jpg)
Primitive/Classic Probability:
(16th-19th century)
Gerolamo Cardano (1501-1576)“Liber de Ludo Aleae” (games of chance)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
![Page 3: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/3.jpg)
Primitive/Classic Probability:
(16th-19th century)
Gerolamo Cardano (1501-1576)“Liber de Ludo Aleae” (games of chance)
Galilei Galileo (1564-1642)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
![Page 4: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/4.jpg)
Primitive/Classic Probability:
(16th-19th century)
Gerolamo Cardano (1501-1576)“Liber de Ludo Aleae” (games of chance)
Galilei Galileo (1564-1642)
Pierre de Fermat (1601-1665) and Blaise Pascal (1623-1662)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
![Page 5: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/5.jpg)
Primitive/Classic Probability:
(16th-19th century)
Gerolamo Cardano (1501-1576)“Liber de Ludo Aleae” (games of chance)
Galilei Galileo (1564-1642)
Pierre de Fermat (1601-1665) and Blaise Pascal (1623-1662)
Jakob Bernoulli (1654 -1705) Bernoullitrial/distribution/r.v/numbersDaniel (1700-1782) (utility function)Johann (1667-1748) (L’Hopital rule)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 2 / 17
![Page 6: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/6.jpg)
Abraham de Moivre (1667-1754)“Doctrine of Chances”
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
![Page 7: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/7.jpg)
Abraham de Moivre (1667-1754)“Doctrine of Chances”
Pierre-Simon Laplace (1749-1827)Laplace transformDe Moivre-Laplace Theorem (the central limit theorem)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
![Page 8: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/8.jpg)
Abraham de Moivre (1667-1754)“Doctrine of Chances”
Pierre-Simon Laplace (1749-1827)Laplace transformDe Moivre-Laplace Theorem (the central limit theorem)
Simeon Denis Poisson (1781-1840).Poisson distribution/process, the law of rare events
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
![Page 9: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/9.jpg)
Abraham de Moivre (1667-1754)“Doctrine of Chances”
Pierre-Simon Laplace (1749-1827)Laplace transformDe Moivre-Laplace Theorem (the central limit theorem)
Simeon Denis Poisson (1781-1840).Poisson distribution/process, the law of rare events
Emile Borel ( 1871-1956).Borel sets/measurable, Borel-Cantelli lemma, Borel strong law.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 3 / 17
![Page 10: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/10.jpg)
Foundation of modern probability:
Keynes, J. M. (1921): A Treatise on Probability.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 4 / 17
![Page 11: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/11.jpg)
Foundation of modern probability:
Keynes, J. M. (1921): A Treatise on Probability.
von Mises, R. (1928): Probability, Statistics and Truth.(1931): Mathematical Theory of Probability and Statistics.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 4 / 17
![Page 12: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/12.jpg)
Foundation of modern probability:
Keynes, J. M. (1921): A Treatise on Probability.
von Mises, R. (1928): Probability, Statistics and Truth.(1931): Mathematical Theory of Probability and Statistics.
Kolmogorov, A. (1933): Foundations of the Theory of Probability.Kolmogorov’s axioms: Probability space trio: (Ω,F , P).
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 4 / 17
![Page 13: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/13.jpg)
Measure-theoretic Probabilities
(Chapter 1 begins)
Sets, set operations (∩, ∪ and complement), set of sets/subsets,algebra, σ-algebra, measurable space (Ω,F), product space ...
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
![Page 14: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/14.jpg)
Measure-theoretic Probabilities
(Chapter 1 begins)
Sets, set operations (∩, ∪ and complement), set of sets/subsets,algebra, σ-algebra, measurable space (Ω,F), product space ...
measures, probabilities, product measure, independence ofevents, conditional probabilities, conditional expectation withrespect to σ-algebra.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
![Page 15: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/15.jpg)
Measure-theoretic Probabilities
(Chapter 1 begins)
Sets, set operations (∩, ∪ and complement), set of sets/subsets,algebra, σ-algebra, measurable space (Ω,F), product space ...
measures, probabilities, product measure, independence ofevents, conditional probabilities, conditional expectation withrespect to σ-algebra.
Transformation/map, measurable transformation/map, randomvariables/elements defined through mapping X = X (ω).
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
![Page 16: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/16.jpg)
Measure-theoretic Probabilities
(Chapter 1 begins)
Sets, set operations (∩, ∪ and complement), set of sets/subsets,algebra, σ-algebra, measurable space (Ω,F), product space ...
measures, probabilities, product measure, independence ofevents, conditional probabilities, conditional expectation withrespect to σ-algebra.
Transformation/map, measurable transformation/map, randomvariables/elements defined through mapping X = X (ω).
Expectation/integral.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
![Page 17: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/17.jpg)
Measure-theoretic Probabilities
(Chapter 1 begins)
Sets, set operations (∩, ∪ and complement), set of sets/subsets,algebra, σ-algebra, measurable space (Ω,F), product space ...
measures, probabilities, product measure, independence ofevents, conditional probabilities, conditional expectation withrespect to σ-algebra.
Transformation/map, measurable transformation/map, randomvariables/elements defined through mapping X = X (ω).
Expectation/integral.
Caratheodory’s extension theorem, Kolmogorov’s extensiontheorem, Dynkin’s π − λ theorem, The Radon-Nikodym Theorem.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17
![Page 18: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/18.jpg)
Convergence.
Convergence modes:convergence almost sure (strong),in probability,in Lp (most commonly, L1 or L2),in distribution/law (weak).The relations of the convergence modes.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 6 / 17
![Page 19: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/19.jpg)
Convergence.
Convergence modes:convergence almost sure (strong),in probability,in Lp (most commonly, L1 or L2),in distribution/law (weak).The relations of the convergence modes.
Dominated convergence theorem,(extension to uniformly integrable r.v.s.)monotone convergence theorem.Fatou’s lemma.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 6 / 17
![Page 20: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/20.jpg)
Law of Large numbers.
X1, ..., Xn, ... are iid random variables with mean µ. Let Sn =∑n
i=1 Xi .
Weak law of Large numbers:
Sn/n → µ, in probability.
i.e., for any ǫ > 0,
P(|Sn/n − µ| > ǫ) → 0.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 7 / 17
![Page 21: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/21.jpg)
Law of Large numbers.
X1, ..., Xn, ... are iid random variables with mean µ. Let Sn =∑n
i=1 Xi .
Weak law of Large numbers:
Sn/n → µ, in probability.
i.e., for any ǫ > 0,
P(|Sn/n − µ| > ǫ) → 0.
Kolmogorov’s Strong law of Large numbers:
Sn/n → µ, almost surely.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 7 / 17
![Page 22: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/22.jpg)
Law of Large numbers.
X1, ..., Xn, ... are iid random variables with mean µ. Let Sn =∑n
i=1 Xi .
Weak law of Large numbers:
Sn/n → µ, in probability.
i.e., for any ǫ > 0,
P(|Sn/n − µ| > ǫ) → 0.
Kolmogorov’s Strong law of Large numbers:
Sn/n → µ, almost surely.
Consistency in statistical estimation.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 7 / 17
![Page 23: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/23.jpg)
Large deviation (strengthening weak law)
For fixed t > 0, how fast is P(Sn/n − µ > t) → 0?
Under regularity conditions,
1n
log P(Sn/n − µ > t) ≈ γ(t),
where γ is determined by the commmon distribution of Xi .
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 8 / 17
![Page 24: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/24.jpg)
Large deviation (strengthening weak law)
For fixed t > 0, how fast is P(Sn/n − µ > t) → 0?
Under regularity conditions,
1n
log P(Sn/n − µ > t) ≈ γ(t),
where γ is determined by the commmon distribution of Xi .
Special case, Xi are iid N(0, 1), then γ(t) = −t2/2.Note that 1 − Φ(x) ≈ φ(x)/x .
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 8 / 17
![Page 25: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/25.jpg)
Law of iterated logarithm (strengthening strong law)
Sn/n − µ → 0 a.s.Is there a proper an such that the “limit” of Sn/an is nonzero finite?
Kolmogorov’s law of iterated logarithm.
lim supn→∞
Sn/n − µ√
2σ2n log log(n)→ 1, a.s.
where σ2 = var(Xi).
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 9 / 17
![Page 26: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/26.jpg)
Law of iterated logarithm (strengthening strong law)
Sn/n − µ → 0 a.s.Is there a proper an such that the “limit” of Sn/an is nonzero finite?
Kolmogorov’s law of iterated logarithm.
lim supn→∞
Sn/n − µ√
2σ2n log log(n)→ 1, a.s.
where σ2 = var(Xi).
Marcinkiewicz-Zygmund strong law: for 0 < p < 2,
Sn − ncn1/p
→ 0, a.s.,
if and only if E(|Xi |p) < ∞, where c = µ for 1 ≤ p < 2.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 9 / 17
![Page 27: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/27.jpg)
Convergence of Series.
The convergence of Sn for independent X1, ..., Xn.
Khintchine’s convergence theorem: if E(Xi) = 0 and∑
n var(Xn) < ∞, then Sn converges a.s. as well as in L2.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 10 / 17
![Page 28: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/28.jpg)
Convergence of Series.
The convergence of Sn for independent X1, ..., Xn.
Khintchine’s convergence theorem: if E(Xi) = 0 and∑
n var(Xn) < ∞, then Sn converges a.s. as well as in L2.
Kolmogorov’s three series theorem:Sn converges a.s. if and only if
∑
n P(|Xn| > 1) < ∞,∑
n E(Xn1|Xn|≤1) < ∞, and∑
n var(Xn1|Xn|≤1) < ∞.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 10 / 17
![Page 29: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/29.jpg)
Convergence of Series.
The convergence of Sn for independent X1, ..., Xn.
Khintchine’s convergence theorem: if E(Xi) = 0 and∑
n var(Xn) < ∞, then Sn converges a.s. as well as in L2.
Kolmogorov’s three series theorem:Sn converges a.s. if and only if
∑
n P(|Xn| > 1) < ∞,∑
n E(Xn1|Xn|≤1) < ∞, and∑
n var(Xn1|Xn|≤1) < ∞.
Kronecker Lemma:If 0 < bn ↑ ∞ and
∑
n(an/bn) < ∞ then∑n
j=1 aj/bn → 0.A technique to show law of large numbers via convergence ofseries.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 10 / 17
![Page 30: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/30.jpg)
The central limit theorem (Chapter 2).
De Moivre-Lapalace TheoremFor X1, X2... independent, under proper conditions,
Sn − E(Sn)√
var(Sn)→ N(0, 1) in distribution.
i .e., P(Sn − E(Sn)
√
var(Sn)< x
)
→ P(N(0, 1) < x)
for all x .
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 11 / 17
![Page 31: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/31.jpg)
The conditions and extensions
Lyapunov.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
![Page 32: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/32.jpg)
The conditions and extensions
Lyapunov.
Lindeberg (1922) condition: Xj is mean 0 with variance σ2j , such
thatn
∑
j=1
E(X 2j 1|Xj |>ǫsn) = o(s2
n)
for all ǫ > 0, where s2n =
∑nj=1 σ2
j .
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
![Page 33: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/33.jpg)
The conditions and extensions
Lyapunov.
Lindeberg (1922) condition: Xj is mean 0 with variance σ2j , such
thatn
∑
j=1
E(X 2j 1|Xj |>ǫsn) = o(s2
n)
for all ǫ > 0, where s2n =
∑nj=1 σ2
j .
Feller (1935): If CLT holds, σn/sn → 0 and sn → ∞, then theabove Lindeberg condition holds.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
![Page 34: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/34.jpg)
The conditions and extensions
Lyapunov.
Lindeberg (1922) condition: Xj is mean 0 with variance σ2j , such
thatn
∑
j=1
E(X 2j 1|Xj |>ǫsn) = o(s2
n)
for all ǫ > 0, where s2n =
∑nj=1 σ2
j .
Feller (1935): If CLT holds, σn/sn → 0 and sn → ∞, then theabove Lindeberg condition holds.
Extensions to martingales, Markov process ...
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
![Page 35: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/35.jpg)
The conditions and extensions
Lyapunov.
Lindeberg (1922) condition: Xj is mean 0 with variance σ2j , such
thatn
∑
j=1
E(X 2j 1|Xj |>ǫsn) = o(s2
n)
for all ǫ > 0, where s2n =
∑nj=1 σ2
j .
Feller (1935): If CLT holds, σn/sn → 0 and sn → ∞, then theabove Lindeberg condition holds.
Extensions to martingales, Markov process ...
Inference in statistics (accuracy justification of estimation:confidence interval, test of hypothesis.)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 12 / 17
![Page 36: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/36.jpg)
Rate of convergence to normality
Suppose X1, ..., Xn are iid.
Berry-Esseen Bound:
∣
∣
∣P
(Sn − nµ√nσ2
< x)
− P(N(0, 1) < x)∣
∣
∣≤ cγ/σ3
n1/2
for all x , where γ = E(|Xi |3) and c is a universal constant.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 13 / 17
![Page 37: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/37.jpg)
Rate of convergence to normality
Suppose X1, ..., Xn are iid.
Berry-Esseen Bound:
∣
∣
∣P
(Sn − nµ√nσ2
< x)
− P(N(0, 1) < x)∣
∣
∣≤ cγ/σ3
n1/2
for all x , where γ = E(|Xi |3) and c is a universal constant.
Extensions, e.g., to non iid cases, etc..
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 13 / 17
![Page 38: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/38.jpg)
Rate of convergence to normality
Suppose X1, ..., Xn are iid.
Berry-Esseen Bound:
∣
∣
∣P
(Sn − nµ√nσ2
< x)
− P(N(0, 1) < x)∣
∣
∣≤ cγ/σ3
n1/2
for all x , where γ = E(|Xi |3) and c is a universal constant.
Extensions, e.g., to non iid cases, etc..
Edgeworth expansion.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 13 / 17
![Page 39: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/39.jpg)
Random Walk (Chapter 3)
Given X1, X2, ... iid, we study the behavior S1, S2, ... as a sequenceof random variables.
Stopping times.T is an integer-valued r.v. such that T = n only depends on thevalues of X1, ..., Xn, on, in other words, the values S1, ..., Sn.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 14 / 17
![Page 40: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/40.jpg)
Random Walk (Chapter 3)
Given X1, X2, ... iid, we study the behavior S1, S2, ... as a sequenceof random variables.
Stopping times.T is an integer-valued r.v. such that T = n only depends on thevalues of X1, ..., Xn, on, in other words, the values S1, ..., Sn.
Wald’s identity/equation:Under proper conditions, E(ST ) = µE(T ).
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 14 / 17
![Page 41: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/41.jpg)
Random Walk (Chapter 3)
Given X1, X2, ... iid, we study the behavior S1, S2, ... as a sequenceof random variables.
Stopping times.T is an integer-valued r.v. such that T = n only depends on thevalues of X1, ..., Xn, on, in other words, the values S1, ..., Sn.
Wald’s identity/equation:Under proper conditions, E(ST ) = µE(T ).
Interpretation: if Xi is the gain/loss of the i-th game, which is fair inthe sense that E(Xi) = 0. Then Sn is the cumulative gain/loss inthe first n games. Under proper conditions, any exit strategy(stopping time) shall still break even.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 14 / 17
![Page 42: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/42.jpg)
Tool box:
Indicator functions.(e.g.,Borel-Cantelli Lemma → Borel’s strong law of large numbers→ Kolmogorov’s 0-1 law.)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 15 / 17
![Page 43: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/43.jpg)
Tool box:
Indicator functions.(e.g.,Borel-Cantelli Lemma → Borel’s strong law of large numbers→ Kolmogorov’s 0-1 law.)
Truncation.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 15 / 17
![Page 44: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/44.jpg)
Tool box:
Indicator functions.(e.g.,Borel-Cantelli Lemma → Borel’s strong law of large numbers→ Kolmogorov’s 0-1 law.)
Truncation.
Characteristic functions/moment generating functions. (in provingthe CLT and its convergence rates)
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 15 / 17
![Page 45: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/45.jpg)
Inequalities
Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
![Page 46: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/46.jpg)
Inequalities
Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski.
Markov, Chebyshev.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
![Page 47: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/47.jpg)
Inequalities
Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski.
Markov, Chebyshev.
Kolmogorov.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
![Page 48: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/48.jpg)
Inequalities
Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski.
Markov, Chebyshev.
Kolmogorov.
Exponential: Berstein, Bennett, Hoeffding,
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
![Page 49: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/49.jpg)
Inequalities
Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski.
Markov, Chebyshev.
Kolmogorov.
Exponential: Berstein, Bennett, Hoeffding,
Doob (for martingale).
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
![Page 50: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/50.jpg)
Inequalities
Jensen, Holder, Cauchy-Schwartz, Lyapunov, Minkowski.
Markov, Chebyshev.
Kolmogorov.
Exponential: Berstein, Bennett, Hoeffding,
Doob (for martingale).
Khintchine, Marcinkiewicz-Zygmund, Burkholder-Gundy,
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 16 / 17
![Page 51: Advanced Probability Theory (Math541) - …makchen/MATH5411/overview541.pdf · Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 5 / 17. Convergence. Convergence](https://reader030.fdocuments.in/reader030/viewer/2022012322/5b7492ae7f8b9a26208ca667/html5/thumbnails/51.jpg)
Martingales (Chapter 4).
1. Conditional expectation with respect to σ-algebra.2. Definition of martingales,3. Inequalities.4. Optional sampling theorem.5. Martingale convergence theorem.
Instructor: Kani Chen (HKUST) Advanced Probability Theory (Math541) 17 / 17