Post on 09-Jul-2020
CPSC 531Systems Modeling and Simulation
Review
2
Independent Events
• Independent events are those that don’t have any effect on each other. That is, knowing one of them occurs does not provide any information about the occurrence of the other event. Mathematically, A and B are independent if P(A|B) = P(A) and P(B|A) = P(B)
• From conditional probability definition, we have P(AB) = P(A|B)P(B). Therefore, if A and B are independent events, P(AB) = P(A)P(B)
3
Law of Total Probability• Let B1, B2, B3, …, Bk be mutually disjoint and collectively
exhaustive events from the sample space S. Then, for any event A in S, we have ∑=
=
k
jjj BAPBPAP
1)|()()(
ExplanationA = (B1A)U (B2A)U… (BkA).
The (BjA)’s are disjoint events. Therefore, using laws of conditional probability we get:
kjBP
BAPBPABPAP
j
k
jjj
k
jj
,...,1for 0)( if
,)|()()()(11
=>
∑=∑===
B1 B2
B3B4
A
4
Bayes’ Theorem• Partition: The events B1, B2, B3, …, Bk form a
partition of a set S if they are mutually disjoint and SBk
i i ==U 1
• Bayes’ Theorem: Suppose that B1, B2, B3, …, Bkform a partition of sample space S such that P(Bj)> 0 for j = 1, …, k. Let A be an event in S such that P(A)>0. Then, for i = 1, …, k,
∑=
=kj jj
iii
BAPBP
BAPBPABP
1 )|()(
)|()()|(
5
Random Variables
• A random variable is a real-valued mapping that assigns a numerical value to each possible outcome of an experiment.
• Consider arrival of jobs at a CPU. Let X be the number of jobs that arrive per unit time. X is a random variable that can take the values {0,1,2,…}.
6
Discrete Random Variables and PMF• A random variable X is said to be discrete if the number of
possible values of X is finite, or at most, an infinite sequence of different values.
• Discrete random variables are characterized by the probabilities of the values attained by the variable. These probabilities are referred to as the Probability Mass Function (PMF) of X. Mathematically, we define PMF as:
∑=
=
==
==
xsX
X
sP
xsXsP
xXPxp
)(
)(
}))(|({
)()(
7
Properties of PMF and CDF
1)(0
)(
)(
)()(
1)(1)(
,1)(0
≤≤
=≤=
≤<−∞=
==ℜ∈∀≤≤
∑
∑∑
≤
ℜ∈
xF
xp
tXP
tXPtF
xporxp
xxp
X
txX
X
i iXx X
X
:CDF
:PMF
8
Expectation
• Definition: weighted average of possible values of X.
• ci ‘s are constants• works even if Xi’s are not independent
∑ ==x
X xpxXE µ)(][
∑∑==
=
=
n
iii
n
iii XEcXcE
XcEcXE
11
][][
][][
9
Binomial Random Variable
• Consider n Bernoulli trials, where each trial can result in a success with probability p. The number of successes X in such a n-trial sequence is a binomial random variable.
• The PMF for this random variable is given by:
where p is the probability of success of a Bernoulli trial.E[X] = np
=−
===−
otherwise
nkppk
n
kXPkpknk
X
,0
,...,2,1,0,)1()}{()(
10
Binomial PMFBinomial Distribution ( n = 10)
0
0.05
0.1
0.15
0.2
0.25
0.3
0 1 2 3 4 5 6 7 8 9 10
Number of successes (k )
P (
{X =
k}) p = 1/2
p = 1/4
p = 3/4
p = 1/2
11
Geometric Random Variable
• The number of Bernoulli trials, X, until first success is a Geometric random variable.
• PMF is given as:
• CDF is given as:
• Mean and variance:
2
1
1
1
1)(
1][
0,)1(1)1()(
,0
,...2,1,)1()(
p
pXVar
pXE
tppptF
otherwise
kppkp
t
i
tiX
k
X
−==
≥−−=−=
=−=
∑=
−
−
12
Geometric PMFGeometric Distribution (p = 0.5)
0
0.1
0.2
0.3
0.4
0.5
0.6
0 1 2 3 4 5 6 7 8 9 10
Number of trials until first success (k )
P (
{X =
k})
13
Example: Modeling Packet Loss
• Geometric r.v. gives number of trials required to get first success
• It is easy to see pX(k) = (1-p)k-1p, k = 1,2,…where p is the probability of success of a trial
• Modeling packet losses seen at a router• We can model using a Bernoulli process
{ Y0, Y1, Y2,…} where Yi represents a Bernoulli trial for packet number i
• We can say:P{Yi = 1} = p (i.e., a packet loss) P{Yi = 0} = 1 -p (i.e., no loss)
• So number of successful packet transmissions before first loss, X, is geometrically distributedP{( X= n)} = p (1-p)n-1 , n = 1,2,…(good length distribution)
14
Poisson Random Variable
• A discrete random variable, X, that takes only non-negative integer values is said to be Poisson with parameter λ > 0, if X has the following PMF:
• Poisson PMF with parameter λ is a good approximation of Binomial PMF with parameters n and p, providedλ = np, n is very large, and p is very small.
==
−
otherwise
kk
ekp
k
X
,0
,...2,1,0,!)(λλ
15
Poisson PMFPoisson Distribution
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 1 2 3 4 5 6 7 8 9 10
Number of events (k)
P [X
= k
]
λ = 0.5λ = 1λ = 5
16
Poisson Approximation to Binomial
Binomial Distribution ( n = 100, p =0.02)
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5 10
Number of successes (k )
P (
{X =
k})
Poisson Distribution (λ = 2)
0
0.05
0.1
0.15
0.2
0.25
0.3
0 5 10
Number of events (k )
P ({
X =
k})
Binomial distribution with large n and small p can be approximated by Poisson distribution with λ = np
17
Poisson Random Variable (cont.)
• CDF of Poisson Random Variable:
• Mean and variance:
• Consider N independent Poisson random variables Xi, i=1,2,3,…,N, with parameters Xi. Then X=X1+X2+…+XN is also a Poisson r.v. with parameter λ=λ1+λ2+...+λΝ
λλ
λλ
==
≥=∑=
−
)(][
0,!
)(0
XVarXE
tk
etFt
k
k
X
18
Example: Job arrivals
• Consider modeling number of job arrivals at a shop in an interval (0,t]
• Let λ be the rate of arrival of jobs• In an interval ∆t → 0
P{one arrival in ∆t} = λ ∆tP{two or more arrivals in ∆t} is negligible
• Divide the interval (0,t] into n subintervals of equal lengths
• Assume arrival of jobs in each interval to be independent of arrivals in another interval
19
Example: Job arrivals (…)
• If n → ∞, the interval can be viewed as a sequence of Bernoulli trials with
• The number of successes k in n trials can be given by the Binomial distribution’s PMF
n
ttp λλ =∆=
knk ppk
n−−
= )1(
20
Example: Job arrivals (…)
ondistributi Poisson the is which
e
is interval time in events ofy probabilit the , Letting
e
to reduces above the Setting
get to for Substitute
-
t-
!
]1,0(1
,...2,1,0,!
)(
,...,1,0,1
k
kk
kk
t
n
nkn
t
n
t
k
n
t/np
k
k
knk
λ
λ
λλ
λ
λ
λ
→=
∞→=
−
=
−
21
Continuous Random Variable
• A random variable X is said to be continuous if there exists a non-negative function f(x),∀x∈(−∞,∞), with the property that for any set Aof real numbers:
• f(x) is called the “probability density function”(PDF) of X
∫=∈A
dxxfAXP )(})({
22
Properties of PDF
}.{],[.,.
)(})({
1)(
,0)(
BXPbaBei
dxxfbXaP
dxxf
xxf
b
a
∈=
=≤≤
=
∀≥
∫∫∞∞−
find to want weand
one. equals curve the under area i.e.,
23
Properties of PDF (continued)
})({
})({
})({
})({
0)(})({
bXaP
bXaP
bXaP
bXaP
dxxfaXPa
a
≤<=<≤=<<=≤≤
=== ∫:property above the of eConsequenc
:values individual to value 0 assign onsdistributi Continuous
24
Cumulative Distribution Function
• The CDF FX(⋅) of a continuous random variable X with PDF fX(⋅) can be obtained as follows:
∫=−∞∈=
∞−
x
X
X
dttf
xXPxF
)(
]}),(({)(
25
CDF - PDF Relationship
• The PDF can be obtained from the CDF and vice versa:
• Distribution of a continuous random variable can be represented using either the PDF or the CDF.
)()(
)(' xfdx
xdFxF X
XX ==
26
PDF and CDF of Uniform R.V.
• The PDF of a uniform random variable X in the interval [a, b] is:
• The CDF of X is:
( )1
,
0, otherwise
a x bf x b a
< <= −( )
0,
,
1,
x a
x aF x a x b
b ax b
≤ −= < < − ≥How did we get F(x)?
∫∫ −
−=
−=
−=<<
∞−
x
a
x
ab
ax
ab
dt
ab
dtbxaF )(
27
Uniform R.V. PDF and CDFPDF of Uniform R.V. (a=1, b=3)
0
0.5
1
0 1 2 3 4
x
f(x)
CDF of Uniform R.V. (a=1, b=3)
0
0.5
1
0 1 2 3 4
x
F(x)
28
Exponential Distribution
β
β
β
β
1][
0,1
0,0)(
,0
0,)(
=
≥−<=
≥=
−
−
XE
xe
xxF
otherwise
xexf
xX
x
X
29
Exponential Models
• This distribution has been used to model:• Inter-arrival times between IP packets• Inter-arrival times between calls at a call
centre• Inter-arrival times between web sessions from
a web client• Service time distributions• Lifetime of products
• Widely used in queuing theory
30
Exponential PDF and CDFPDF of Exponential Distribution
0
1
2
3
4
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
x
f(x)
β=0.5β=1.0β=2.0β=4.0
CDF of Exponential Distribution
0
0.2
0.4
0.6
0.8
1
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5
x
F(x)
β=2.0
31
Memory-less Property of Exponential Distribution• Suppose inter-arrival times of IP packets are
modelled using Exponential distribution. The memory-less property states that the distribution of the expected time to a packet arrival is independent of the duration there have been no packet arrivals
• Suppose X is an exponentially distributed r.v. and X ≥ t (i.e., no arrivals for time t or less). Then,
P({ X ≥ t+h | X ≥ t }) = P({ X ≥ h })
32
Pareto Distribution
• If X is a random variable with a Pareto distribution, then its PDF is given by:
where xm, is the minimum possible value of X, also called a location parameter, and k is positive, also called a shape parameter.
• CDF of Pareto distribution is given by:
0,0,,)(1
>>≥=+
kxxxx
xkxf mmk
km
km
x
xxF
−=1)(
33
Pareto PDF and CDF
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
10 100 1000
prob
abili
ty d
ensi
ty, f
(x)
x
Pareto PDF with x_min = 3, k = 1.2
0.00
0.20
0.40
0.60
0.80
1.00
10 100 1000
cum
mul
ativ
e di
strib
utio
n, F
(x)
x
Pareto CDF with x_min = 3, k = 1.2
Example: Distribution of file sizes on a web server• A PDF shows a high probability that a file will be under 10 KB in
size, and a very small probability of being larger than 100 KB• A CDF curve shows proportion of files within certain size threshold,
e.g. nearly all files are under 100 KB in size
34
Pareto Models
• This highly left-skewed distribution is heavy-tailed meaning that a random variable can have extreme values.
• Common models:• Distribution of income• Distribution of files in a P2P system• The values of oil reserves in oil fields (a few large
fields, many small fields) • The length distribution in jobs assigned to
supercomputers (a few large ones, many small ones) • The standardized price returns on individual stocks
35
Normal Distribution
• X is a normal random variable with mean µ and variance σ2 if X has the following PDF:
• The CDF of a normal distribution is:
• There is no closed form for FX(x).
∫∞−
−−
−−
=≤=
∞<<∞−=
x t
X
x
X
dtxXPxF
xxf
e
e
2
2
2
2
2
)(
2
)(
2
1})({)(
,2
1)(
σµ
σµ
σπ
σπ
36
PDF of Normal Distribution (µ = 0)
0
0.2
0.4
0.6
0.8
1
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6
x
f(x)
σ=1σ=0.5σ=2
PDF of Normal Distribution
This PDF has a “bell” shape with “peak” at x=0.
37
Standard Normal Distribution
• If X is normally distributed with parameters µand σ2, then
is normally distributed with parameters 0 and 1. Z is called the “standard normal distribution”.
σµ)( −
=X
Z
38
Computing CDF of Normal Distribution
• Transform X to standard normal distribution Zand use tables
•
• Area under standard normal curve in (-∞,z) is equal to area under normal curve in (-∞,x)
• Same method is used to obtain P(a < X < b), by calculating P(X < b) − P(X < a)
• Alternative formula:
)1,0()(
),(~ 2 NX
ZNX is then Ifσµσµ −
=
−+=2
12
1)(
σµx
erfxFX
39
Normal CDF - Example
),0(
)2.0(
2.02
54.5)()4.5()4.5(
)4.5()4,5(
)()()(
z
ZP
zforzZPXPF
FfindNFor
xzforzZPxXPxF
X
X
X
interval the over e.g. given, is area which
check always but curve, the under area gives table The
table" ondistributi Normal Standard"
called table the from read is ≤
=−
=≤=≤=
−=≤=≤=
σµ
40
Normal CDF - Example
• Rows mark z-value up to 1 decimal digit• Columns mark z-value’s 2nd decimal digit• For z = 0.2read value at (0.2, 0.0) → 0.0793• Then account for the rest of the area on the left
of y-axis to get the FX(5.4) = 0.5 + 0.0793 = 0.5793
0.28520.28230.27940.27640.27340.27040.26730.26420.26110.25800.7
0.25490.25170.24860.24540.24220.23890.23570.23240.22910.22570.6
0.22240.21900.21570.21230.20880.20540.20190.19850.19500.19150.5
0.18790.18440.18080.17720.17360.17000.16640.16280.15910.15540.4
0.15170.14800.14430.14060.13680.13310.12930.12550.12170.11790.3
0.11410.11030.10640.10260.09870.09480.09100.08710.08320.07930.2
0.07530.07140.06750.06360.05960.05570.05170.04780.04380.03980.1
0.03590.03190.02790.02390.01990.01600.01200.00800.00400.00000.0
0.09 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0.00
41
Chi-Square Test
• Prepare a histogram of the empirical data with kcells
• Let Oi and Ei be the observed and expected frequency of the ith cell, respectively. Compute the following:
• has a Chi-Square distribution with (k-1) degrees of freedom
20χ
∑ −==
k
i i
ii
E
EO
1
220
)(χ
42
Chi-Square Test (continued …)
• Define a null hypothesis, H0, that observations come from a specified distribution
• The null hypothesis cannot be rejected at a significance level of α if
true) is H|H P(reject
level cesignifican of meaning
00=
<−−−
α
χχ α2
]1,1[20 sk Obtained from a table
43
More on Chi-Square Test
• Errors in cells with small Ei’s affect the test statistics more than cells with large Ei’s.
• Minimum size of Ei debated: [BCNN05] recommends a value of 3 or more; if not combine adjacent cells.
• Test designed for discrete distributions and large sample sizes only. For continuous distributions, Chi-Square test is only an approximation (i.e., level of significance holds only for n→∞).
44
Chi-Square Test Example• Example: 500 random numbers generated using a
random number generator; observations categorized into cells at intervals of 0.1, between 0 and 1. At level of significance of 0.1, are these numbers IID U(0,1)?
Interval Oi Ei [(Oi-Ei)^2]/Ei1 50 50 02 48 50 0.083 49 50 0.024 42 50 1.285 52 50 0.086 45 50 0.57 63 50 3.388 54 50 0.329 50 50 0
10 47 50 0.18500 5.84
0.10. of level cesignifican at accepted Hypothesis
table the from ;68.14;85.5 2]9,9.0[
20 == χχ
45
Fundamental a.k.a. Operational Laws
• Utilization Law• Forced Flow Law• Service Demand Law• Little’s Law• Interactive Response Time Law
46
Utilization Law
iiiii
i SXC
B
T
C
T
B U =×==
Utilization of a resource (system) is equal to the product of the throughput of the resource (system) and average service time of the resource (system)
• Utilization of a resource is the fraction of time that resource is busy.• Ui is always between 0 and 1.
47
Forced Flow Law
• Each “system-level” request may require multiple visits to a system “resource”. • E.g., A database transaction may require several disk
accesses;
• This law relates system throughput to the resource throughput
Xk = Vk × X0
2
1
3
• A system consists of many resources
• Vi := average # of visits per request to resource i
• Xi := throughput at resource i
48
Service Demand Law
Di := mean time spent by a typical request obtaining service from resource i
• Contrast Di with Si
Si := mean service time per visit for resource iDi = Si Vi
Di = Ui/Xi × Xi/X0 = Ui /X0
• Typically X0 and Ui are easier to obtain than Siand Vi.
49
Little’s Law• The most famous Operational Law• Average number in system equals product of the
departure rate of customers (i.e., throughput of the system) and the average time each customer spends in the system.
Ni = Xi × Ri
Arrivals CompletionsNumber in System = N
Black Box == pub
Ri
50
Interactive Response Time Law
X0 = System throughput
N = # of clients (terminals)
Z = Client’s avg. think timeR = Avg. System Response time
Let Nt = avg. # of clients in think mode
Let Nw = avg. # of clients waiting for response
Nt + Nw = N
Nt = X0 Z [Box 1]Nw = X0 R [Box 2],
⇒ N = X0(R+Z)
⇒ R = (N/X0) - Z
Terminals
Subsystem
Box 1
Box 2
R
Z
X