Basics of probability in statistical simulation and stochastic programming
description
Transcript of Basics of probability in statistical simulation and stochastic programming
![Page 1: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/1.jpg)
Basics of probability in statistical simulation and stochastic programming
Leonidas SakalauskasInstitute of Mathematics and InformaticsVilnius, LithuaniaEURO Working Group on Continuous Optimization
Lecture 2
![Page 2: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/2.jpg)
Content
Random variables and random functions
Law of Large numbers Central Limit Theorem Computer simulation of random
numbers Estimation of multivariate integrals
by the Monte-Carlo method
![Page 3: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/3.jpg)
Simple remark
Probability theory displays the library of mathematical probabilistic models
Statistics gives us the manual how to choose the probabilistic model coherent with collected data
Statistical simulation (Monte-Carlo method) gives us knowledge how to simulate random environment by computer
![Page 4: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/4.jpg)
Random variable
Random variable is described by
Set of support Probability measure
Probability measure is described by distribution function:
)(Pr)( xXobxF
)(XSUPP
![Page 5: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/5.jpg)
Probabilistic measure
Probabilistic measure has three components:
Continuous;Discrete (integer);Singular.
![Page 6: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/6.jpg)
Continuous r.v.
Continuous r.v. is described by probability density function )(xf
Thus:
x
dyyfxF )()(
![Page 7: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/7.jpg)
Continuous variable
If probability measure is absolutely continuous, the expected value of random function:
dxxpxfXEf )()()(
![Page 8: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/8.jpg)
Discrete variable
Discrete r.v. is described by mass probabilities:
n
n
ppp
xxx
,...,,
,...,,
21
21
![Page 9: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/9.jpg)
Discrete variable
If probability measure is discrete, the expected value of random function is sum or series:
n
iii pxfXEf
1
)()(
![Page 10: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/10.jpg)
Singular variable
Singular r.v. probabilistic measure is concentrated on the set having zero Borel measure (say, Kantor set).
![Page 11: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/11.jpg)
Law of Large Numbers (Chebyshev, Kolmogorov)
,lim 1 zN
zN
i i
N
here are independent copies of r. v. ,
Ez
Nzzz ,...,, 21
![Page 12: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/12.jpg)
What did we learn ?
N
N
i i 1
is the sample of copies of r.v. , distributed with the density .
Nzzz ,...,, 21
dzzpzxf )(),(The integral
is approximated by the sampling average
,,...,1),,( Njzxf jj if the sample size N is large, here
)(zp
![Page 13: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/13.jpg)
Central limit theorem (Gauss, Lindeberg, ...)
),(/
lim xxN
xP N
N
,2
1)( 2
2
x y
dyex
here
,EX 222 )( XEXD,1
N
x
x
N
ii
N
![Page 14: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/14.jpg)
Beri-Essen theorem
N
XExxFN
x
3
3
41.0)()(sup
xxobxF NN Pr)(where
![Page 15: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/15.jpg)
What did we learn ?
According to the LLN:
,1
1
N
iixN
x,
)(1
2
2
N
xxN
iNi
N
xx
EXXE
N
iNi
1
3
3
Thus, apply CLT to evaluate the statistical error of approximation and its validity.
![Page 16: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/16.jpg)
Example
Let some event occurred n times repeating N independent experiments.
Then confidence interval of probability of event :
N
ppp
N
ppp
)1(96.1,
)1(96.1
here ,N
np (1,96 – 0,975 quantile of normal distribution,
confidence interval – 5% )
6)1( ppNIf the Beri-Esseen condition is valid: !!!
![Page 17: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/17.jpg)
Statistical integrating …
b
a
dxxfI )( ???
Main idea – to use the gaming of a large number of random events
![Page 18: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/18.jpg)
Statistical integration
dxxpxfXEf )()()(
,)(
1
N
xfNi i )(pxi
![Page 19: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/19.jpg)
Statistical simulation and Monte-Carlo method
x
dzzpzxfxF min)(),()(
,min),(
1
x
N
i i
N
zxf )(pzi
(Shapiro, (1985), etc)
![Page 20: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/20.jpg)
Simulation of random variables
There is a lot of techniques and methods to simulate r.v. Let r.v. be uniformly distributed in the interval (0,1]
Then, the random variable , where ,
)(UFU
is distributed with the cumulative distribution function )(F
![Page 21: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/21.jpg)
))sin(cos()( xaxxxf
0
))sin(cos()( dxexaxxaF x
N=100, 1000
![Page 22: Basics of probability in statistical simulation and stochastic programming](https://reader035.fdocuments.in/reader035/viewer/2022081512/5681474a550346895db48ac0/html5/thumbnails/22.jpg)
Wrap-Up and conclusions
o the expectations of random functions, defined by the multivariate integrals, can be approximated by sampling averages according to the LLN, if the sample size is sufficiently large;
o the CLT can be applied to evaluate the reliability and statistical error of this approximation