Probability theory 2011 Convergence concepts in probability theory Definitions and relations...
-
date post
21-Dec-2015 -
Category
Documents
-
view
220 -
download
0
Transcript of Probability theory 2011 Convergence concepts in probability theory Definitions and relations...
Probability theory 2011
Convergence concepts in probability theory
Definitions and relations between convergence concepts Sufficient conditions for almost sure convergence Convergence via transforms The law of large numbers and the central limit theorem
Probability theory 2011
Coin-tossing: relative frequency of heads
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 50 100 150 200 250 300 350 400
Re
lati
ve
fre
qu
en
cy
of
he
ad
s
Series1
Series2
Series3
Convergence of each trajectory?
Convergence in probability?
Probability theory 2011
Convergence to a constant
The sequence {Xn} of random variables converges almost
surely to the constant c if and only if
P({ ; Xn() c as n }) = 1
The sequence {Xn} of random variables converges in
probability to the constant c if and only if, for all > 0,
P({ ; | Xn() – c| > }) 0 as n
Probability theory 2011
An (artificial) example
Let X1, X2,… be a sequence of independent binary random variables such
that
P(Xn = 1) = 1/n and P(Xn = 0) = 1 – 1/n
Does Xn converge to 0 in probability?
Does Xn converge to 0 almost surely?
Common exception set?
Probability theory 2011
The law of large numbers for random variableswith finite variance
Let {Xn} be a sequence of independent and identically distributed random
variables with mean and variance 2, and set
Sn = X1 + … + Xn
Then
Proof: Assume that = 0. Then
.
0 ,0)|(| all foras nn
SP n
2
)()|(|
n
SVar
n
SP
n
n
Probability theory 2011
Convergence to a random variable: definitions
The sequence {Xn} of random variables converges almost surely to the
random variable X if and only if
P({ ; Xn() X() as n }) = 1
Notation:
The sequence {Xn} of random variables converges in probability to the
random variable X if and only if, for all > 0,
P({ ; | Xn() – X()| > }) 0 as n
Notation:
nXX san .. as
nXX pn as
Probability theory 2011
Convergence to a random variable: an example
Assume that the concentration of NO in air is continuously
recorded and let Xt, be the concentration at time t.
Consider the random variables:
Does Yn converge to Y in probability?
Does Yn converge to Y almost surely?
tt XY 10max
1/2/10 ,...,,,max XXXXY nnn
Probability theory 2011
Convergence in distribution: an example
Let Xn Bin(n, c/n). Then the distribution of Xn converges to
a Po(c) distribution as n
.
Binomial and Poisson distributions (n = 20, c = 0.1)
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0 1 2 3 4 5 6 7 8 9 10
Pro
bab
ilit
y
BinomialPoisson
p = 0.1)
Probability theory 2011
Convergence in distribution and in norm
The sequence Xn converges in distribution to the random variable X as
n iff
for all x where FX(x) is continuous.
Notation:
The sequence Xn converges in quadratic mean to the random variable X
as n iff
Notation:
nxFxF XX n )()( as
nXXE n 0 || 2 as
nXX dn as
nXX n as 2
Probability theory 2011
Relations between the convergence concepts
Almost sure convergence
Convergence in r-mean
Convergence in probability
Convergence in distribution
Probability theory 2011
Convergence in probability impliesconvergence in distribution
Note that, for all > 0,
||
||
XXxXP
XXxXP
xXP
nn
nn
n
Probability theory 2011
Convergence almost surely -convergence in r-mean
Consider a branching process in which the offspring distribution has mean 1.
Does it converge to zero almost surely?Does it converge to zero in quadratic mean?
Let X1, X2,… be a sequence of independent random variables such that
P(Xn = n2) = 1/n2 and P(Xn = 0) = 1 – 1/n2
Does Xn converge to 0 in probability?
Does Xn converge to 0 almost surely?
Does Xn converge to 0 in quadratic mean?
Probability theory 2011
Relations between different types ofconvergence to a constant
Almost sure convergence
Convergence in r-mean
Convergence in probability
Convergence in distribution
Probability theory 2011
Convergence via generating functions
Let X, X1, X2, … be a sequence of nonnegative, integer-
valued random variables, and suppose that
Then
ntgtg XX n )()( as
nXX dn as
Is the limit function of a sequence of generating functions a generating function?
Probability theory 20101
Convergence via moment generating functions
Let X, X1, X2, … be a sequence of random variables, and
suppose that
Then
htntt XX n ||, )()( foras
nXX dn as
Is the limit function of a sequence of moment generating functions a moment generating function?
Probability theory 2011
Convergence via characteristic functions
Let X, X1, X2, … be a sequence of random variables, and
suppose that
Then
tntt XX nforas , )()(
nXX dn as
Is the limit function of a sequence of characteristic functions a characteristic function?
Probability theory 2011
Convergence to a constantvia characteristic functions
Let X1, X2, … be a sequence of random variables, and
suppose that
Then
net itcX n
)( as
ncX pn as
Probability theory 2011
The law of large numbers(for variables with finite expectation)
Let {Xn} be a sequence of independent and identically
distributed random variables with expectation , and set
Sn = X1 + … + Xn
Then
.
nn
SX pn
n as
Probability theory 2011
The strong law of large numbers(for variables with finite expectation)
Let {Xn} be a sequence of independent and identically
distributed random variables with expectation , and set
Sn = X1 + … + Xn
Then
.
nn
SX san
n as..
Probability theory 2011
The central limit theorem
Let {Xn} be a sequence of independent and identically distributed random
variables with mean and variance 2, and set
Sn = X1 + … + Xn
Then
Proof: If = 0, we get
.
nNn
nS dn as)1,0(
nn
XS
n
S n
to
n
t
n
t
n
tt
nn
)(2
1)()()(22
Probability theory 2011
Rate of convergence in the central limit theorem
Example: XU(0,1)
.
n
XExxF
n
nSx n 3
3||7975.0|)()(|sup
nnn
XE 3.8
)1212/(1
4/17975.0
||7975.0
3
3
Probability theory 2011
Sums of exponentially distributed random variables
0.00
0.20
0.40
0.60
0.80
1.00
1.20
0 5 10 15 20 25
Cu
mu
lati
ve d
istr
ibu
tio
n f
un
ctio
n
gamma(10;1) N(10;sqr(10))
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0 5 10 15 20 25
Pro
bab
ility
den
sity
gamma(10;1) N(10;sqr(10))
Probability theory 2011
Convergence of empirical distribution functions
Proof: Write Fn(x) as a sum of indicator functions
Bootstrap techniques: The original distribution is replaced with the empirical distribution
))(1)((,0())()(( xFxFNxFxFn dn
n
xnsobservatioxFn
#)(
Probability theory 2011
Resampling techniques- the bootstrap method
**2
*1 ,...,, Nxxx
3467
798839
41
8570
62
90 58 4460
73
22
587988
41
88
8570
90
22 34 4460
41
60Sampling with replacement
Resampled dataObserved data
x **2
*1 ...,,, Nxxx
Probability theory 2011
Characteristics of infinite sequences of events
Let {An, n = 1, 2, …} be a sequence of events, and define
Example: Consider a queueing system and let
An = {the queueing system is empty at time n}
1* inflim
n nmmnn AAA
1
* suplimn nm
mnn AAA
Probability theory 2011
The probability that an event occurs infinitely often - Borel-Cantelli’s first lemma
Let {An, n = 1, 2, …} be an arbitrary sequence of events. Then
Example: Consider a queueing system and let
An = {the queueing system is empty at time n}
0.).()(1
oiAPAP nn
n
Is the converse true?
Probability theory 2011
The probability that an event occurs infinitely often- Borel-Cantelli’s second lemma
Let {An, n = 1, 2, …} be a sequence of independent events. Then
1.).()(1
oiAPAP nn
n
Probability theory 2011
Necessary and sufficient conditions for almost sure convergence of independent random variables
Let X1, X2, … be a sequence of independent random variables. Then
)|(|01
..
nn
san XPnasX
Probability theory 2011
Exercises: Chapter VI
6.1, 6.6, 6.9, 6.10, 6.17, 6.21, 6.25, 6.49