Introduction to stochastic process Dr. Nur Aini Masruroh.
-
date post
19-Dec-2015 -
Category
Documents
-
view
237 -
download
4
Transcript of Introduction to stochastic process Dr. Nur Aini Masruroh.
![Page 1: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/1.jpg)
Introduction to stochastic processIntroduction to stochastic process
Dr. Nur Aini Masruroh
![Page 2: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/2.jpg)
Outlines Outlines
Concept of probability
Random variables
Expected value
Conditional probability
Limit theorem
Stochastic processes
![Page 3: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/3.jpg)
Probability in Industrial EngineeringProbability in Industrial Engineering
Random arrivals of jobsRandom service timeRandom requests for resourcesProbability of queue overflowService delaysScheduling problemsFlow control and routingRevenue managementRisk assessment for decision analysisEtc
![Page 4: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/4.jpg)
Basic probabilityBasic probability
Random experiment: an experiment whose outcome cannot be determined in advanced
Sample space (S): set of all possible outcomes Event (E): a subset of sample space, it occurs if the
outcome of the experiment is an element of that subset A number P(E) is defined and satisfies:
0 ≤ P(E) ≤ 1 P(S) =1 For any sequence of events E1, E2, … that are mutually
exclusive
11
)(i
ii
i EPEp
![Page 5: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/5.jpg)
Random variablesRandom variables
Consider a random experiment having sample space S. A random variable X is a function that assigns a real value to each
outcome in S The distribution function F of the random variable X is defined for
any real number x by
A random variable X is said to be discrete if its set of possible values is countable. For discrete random variables,
A random variable is called continuous if there exists a function f(x), called the probability density function, such that
xXPxF ,)(
xy
yXPxF )(
B
dxxfBXP )(inis For every set B
![Page 6: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/6.jpg)
Random variablesRandom variables
Since it follows that
The join distribution function F of two random variables X and Y is defined by
F(x, y) = P{X≤ x, Y ≤ y} The distribution functions of X and Y,
Fx(x) = P{X≤ x} and FY(y) = P{Y≤y}
It can be obtained from F(x, y) by making use of the continuity property of the probability operator.
Continuity property:
Similarly,
F(x, y) = Fx(x)FY(y) if X and Y are independent
x
dxxfxF )()( )()( xFdx
dxf
xXPyYxXP nn
,lim
),(lim)(or),(lim)( yxFyFyxFxFx
Yy
X
![Page 7: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/7.jpg)
Cumulative Distribution Function (CDF)Cumulative Distribution Function (CDF)
CDF, F, of the random variable X is defined by all real numbers b, -∞ < b < ∞, by F(b) = P(X ≤ b)
F(b) denotes the cumulative probability that the random variable X takes on a value that is less or equal to b (the total probability mass)
CDF is a non-decreasing function: P(X≤ a) ≤ P(X ≤ b) if a < b and thus F(a) ≤ F(b) for a < b
F(∞) = 1 and F(-∞)=0: bounded CDF is a right continuous function: for any b, F(b+), value
of F(b) just to the right of b, equals to F(b) as b+ get closer to b. The CDF is right continuous at all values, but may be left discontinuous at some values
![Page 8: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/8.jpg)
Probabilities from CDFProbabilities from CDF
P(X ≤ b) = F(b) P(X<b) = F(b – ε) = F(b-)
ε is a small number
P(X=b) = P(X ≤ b) – P(X < b) = F(b) – F(b-) P(a < X ≤ b) = F(b) – F(a) P(a ≤ X ≤ b) = F(b) – F(a-) P(a ≤ X < b) = F(b-) – F(a-) P(a < X < b) = F(b-) – F(a)
![Page 9: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/9.jpg)
Probability Density Function (pdf)Probability Density Function (pdf)
The pdf of a continuous random variable X is defined by the derivative of the continuous CDF at differentiable intervals : dF(b)/db = f(b)
The pdf of a continuous random variable tells us the density of the mass at each point on the sample space
The value of pdf is not probability, for example, if f(x) = 2x, 0 ≤ x ≤ 1, and 0 otherwise, f(1) = 2 is obviously not a probability value
Graphically, pdf is the area under f(x) from ain interval a,b.
Note: if pmf has meaning itself, the value itself for pdf has no meaning
b
a
dxxfbXaP )()(
![Page 10: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/10.jpg)
Discrete random variable: summaryDiscrete random variable: summary
A random variable X associates a number with each outcome of an experiment
A discrete random variable takes on a finite number The probabilistic behavior of a discrete random variable
X is described by its probability mass function (pmf), p(u) P(uj) = p(X=uj)
All the probabilistic information about the discrete random variable X is summarized in its pmf
A discrete random variable X has a CDF F(b) which is: Right-continuous Staircase CDF
![Page 11: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/11.jpg)
Continuous random variable: summaryContinuous random variable: summary
No mass to define pmf: every event has zero probability Example: p(height of people= 173.7897654) 0 However P(147< height of people < 187) 1
The set of possible values are uncountable while the set of possible values were finite or countably infinite for a discrete random variable
Sample space is not a discrete set, but a continuous space (or interval)
A continuous random variable X has a CDF F(b) which is: Continuous at all b, -∞ < b < ∞ Differentiable at all b (except possibly at finite set of points)
![Page 12: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/12.jpg)
Expected valueExpected value
Expectation of the random variable X,
The variance of the random variable X is defined by
Var X = E[(X – E[X])2]
= E[X2] – E2[X]
Two jointly distributed random variables X and Y are said to be uncorrelated if their covariance defined by
Cov (X, Y) = E[(X – EX)(Y – EY)]
= E[XY] – E[X]E[Y]
x
XxXxP
Xdxxxf
xxdFXE
discreteisif}{
continueisif)(
)(][
![Page 13: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/13.jpg)
Expected valueExpected value
Expectation of a sum of random variables is equal to the sum of the expectations
Variances:
n
ii
n
ii XEXE
11
[
),(2)(11
ji
n
i jii
n
ii XXCovXVarXVar
![Page 14: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/14.jpg)
Conditional probabilitiesConditional probabilities
In general, given information about the outcome of some events, we may revise our probabilities of other events
We do this through the use of conditional probabilities The probability of an event X given specific outcomes of another
event Y is called the conditional probability X given Y The conditional probability of event X given event Y and other
background information ξ, is denoted by p(X|Y, ξ) and is given by
0)|(for)|(
)|(),|(
Yp
Yp
YXpYXp
![Page 15: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/15.jpg)
Bayes’ TheoremBayes’ Theorem
Given two uncertain events X and Y. Suppose the probabilities p(X|ξ) and p(Y|X, ξ) are known, then
X
XYpXpYp
where
Yp
XYpXpYXp
)||()|()|(
)|(
),|()|(),|(
![Page 16: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/16.jpg)
Factorization rule for joint probabilityFactorization rule for joint probability
![Page 17: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/17.jpg)
Limit theorem Limit theorem Strong Law of Large Numbers
If X1, X2, … are independent and identically distributed with mean μ, then
Central Limit Theorem If X1, X2, … are independent and identically distributed with
mean μ and variance σ2, then
Thus if we let where X1, X2, … are independent and identically distributed, then the Strong of LLN states that, with probability1, Sn/n will converge to E[Xi]; whereas the CLT states that Sn will have an asymptotic normal distribution as n → ∞
1...
lim 1
n
XXP n
n
dxean
nXXP x
an
n
2/12
2
1...lim
n
i in XS1
![Page 18: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/18.jpg)
Stochastic processStochastic process
X(t) is the state of the process (measurable characteristic of interest) at time t
• the state space of the a stochastic process is defined as the set of all possible values that the random variables X(t) can assume
• when the set T is countable, the stochastic process is a discrete time process; denote by {Xn, n=0, 1, 2, …}
• when T is an interval of the real line, the stochastic process is a continuous time process; denote by {X(t), t≥0}
![Page 19: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/19.jpg)
Stochastic processStochastic process
Hence,• a stochastic process is a family of random
variables that describes the evolution through time of some (physical) process.
• usually, the random variables X(t) are dependent and hence the analysis of stochastic processes is very difficult.
• Discrete Time Markov Chains (DTMC) is a special type
of stochastic process that has a very simple dependence among X(t) and renders nice results in the analysis of {X(t), t∈T} under very mild assumptions.
![Page 20: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/20.jpg)
Example of stochastic processesExample of stochastic processes
Refer to X(t) as the state of the process at time t A stochastic process {X(t), t T} is a time indexed ∈
collection of random variables
X(t) might equal the total number of customers that have entered a supermarket by time t
X(t) might equal the number of customers in the supermarket at time t
X(t) might equal the stock price of a company at time t
![Page 21: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/21.jpg)
Counting processCounting process
Definition: A stochastic process {N(t), t≥0} is a counting process if
N(t) represents the total number of “events” that have occurred up to time t
![Page 22: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/22.jpg)
Counting process Counting process
Examples: If N(t) equal the number of persons who have entered a
particular store at or prior to time t, then {N(t), t≥0} is a counting process in which an event corresponds to a person entering the store
• If N(t) equal the number of persons in the store at time t, then {N(t), t≥0} would not be a counting process. Why?
If N(t) equals the total number of people born by time t, then {N(t), t≥0} is a counting process in which an event corresponds to a child is born
If N(t) equals the number of goals that Ronaldo has scored by time t, then {N(t), t≥0} is a counting process in which an event occurs whenever he scores a goal
![Page 23: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/23.jpg)
Counting processCounting process
A counting process N(t) must satisfy N(t)≥0 N(t) is integer valued If s ≤t, then N(s) ≤ N(t) For s<t, N(t)-N(s) equals the number of events that have
occurred in the interval (s,t), or the increments of the counting process in (s,t)
A counting process has Independent increments if the number of events which occur in
disjoint time intervals are independent Stationary increments if the distribution of the number of
events which occur in any interval of time depends only on the length of the time interval
![Page 24: Introduction to stochastic process Dr. Nur Aini Masruroh.](https://reader030.fdocuments.in/reader030/viewer/2022012903/56649d2d5503460f94a04b0f/html5/thumbnails/24.jpg)
Independent incrementIndependent increment
This property says that numbers of events in disjoint intervals are independent random variables.
Suppose that t1< t2≤ t3< t4. Then N(t2)-N(t1), the number of events occurring in (t1,t2], is independent of N(t4)-N(t3), the number of events occurring in (t3, t4].