Random Number Generators
2
Why do we need random variables?
• random components in simulation
→ need for a method which generates numbers that are random
• examples
– interarrival times
– service times
–demand sizes
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
3
terminology
• Random Numbers
– stochastic variable that meets certain conditions
–numbers randomly drawn from some known distribution
– It is impossible to “generate” them with the help of a digital computer.
• Pseudo-random Numbers
–numbers which seem to be randomly drawn from some known
distribution
–may be generated with the help of a digital computer
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
4
techniques for generating random variables
– ten-sided die (each throw generates a decimal)
– throwing a coin n times • get a binary number between 0 and 2^n-1
–other physical devices
–observe substances undergoing atomic decay• e.g., Caesium-137, Krypton-85,. . .
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
5
techniques for generating pseudo-random variables
• numbers are generated using a recursive formula
– ri is a function of ri-1, ri-2, …
– relation is deterministic – no random numbers
– if the mathematical relation not known and well chosen it is practically
impossible to predict the next number• In most cases we want the generated random numbers to simulate a
uniform distribution over (0, 1), that is a U(0, 1)-distribution
– there’re many simple techniques to transform uniform (U(a, b)) samples
into samples from other well-known distributions
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
1
0 1 0 1
6
terminology
• seed
– random number generators are usually initialized with a starting
number which is called the “seed”
–different seeds generates different streams of pseudorandom numbers
– the same seed results in the same stream
• cycle length
– length of the stream of pseudorandom numbers without repetition
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
7
methods for generating pseudo-random variables
• all methods usually generate uniformly distributed pseudorandom numbers in the interval [0,1]
–midsquare method (outdated)
– congruential method (popular)
– shift register (Tausworthe generator)
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
8
midsquare method
• arithmetic generator
–proposed by von Neumann and Metropolis in 1940s
• algorithm
– start with an initial number Z0 with m digits (seed) [m even]
–square it and get a number with 2m digits (add a zero at the beginning
if the number has only 2m-1 digits)
–obtain Z1 by taking the middle m digits
– to be in the interval [0,1) divide the Zi by 10m
–etc….
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
9
midsquare method: example (Z0 = 7182, m = 4)
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
i01234567…
Zi
7182 5811 7677 9363 6657 3156 9603
2176 …
Ui
0.71820.58110.76770.93630.66570.31560.96030.2176
…
Zi2
51581124 33767721 58936329 87665769 44315649
9960336 92217609
4734976 …
10
midsquare method (cont.)
• advantages
– fairly simple• disadvantage
– tends to degenerate fairly rapidly to zero• example: try Z0 = 1009
–not random as predictable
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
11
Linear Congruential Generator (LCG)
• LCGs
– first proposed by Lehmer (1951).
–produce pseudorandom numbers such that each single number
determines its successor by means of a linear function followed by a
modular reduction
– to be in the interval [0,1) divide the Zi by m
–Z0 seed a multiplier (integer)
– c increment (integer) m modulus (integer)• Variations are possible
– combinations of previous numbers instead of using only the last value040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
12
LCG: example
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
i01234567…
Zi
7 6
1 8
11 10
5 12 …
Ui
0.43750.3750.06250.5 0.6875 0.625 0.31250.75
…
a = 5c = 3m = 16Z0 = 7
Z1 = (5*Z0 + 3) mod 16 = 38 mod 16 = 6Z2 = (5*Z1 + 3) mod 16 = 33 mod 16 = 1Z3 = (5*Z2 + 3) mod 16 = 8 mod 16 = 8…
13
properties of LCGs
• parameters need to be chosen carefully
–nonnegative
–0 < m a < m c < m Z0 < m
• disadvantages
–not random
– if a and m are properly chosen, the Uis will “look like” they are randomly
and uniformly distributed between 0 and 1.
– can only take rational values 0, 1/m, 2/m, … (m-1)/m
– looping
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
14
LCG (cont.)
• looping
–whenever Zi takes on a value it has had previously, exactly the same
sequence of values is generated, and this cycle repeats itself endlessly.
– length of sequence = period of generator
–period can be at most m • if so: LCG has full period
–LCG has full period iff• the only positive integer that divides both m and c is 1• if q is a prime number that divides m then q divides a-1• if 4 divides m, then 4 divides a-1
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
15
Linear Feedback Shift Register Generators (LFSR)
• developed by Tausworthe (1965)
– related to cryptographic methods
–operate directly on bits to form random numbers• linear combination of the last q bits
– ci constants (equal to 0 or 1, cq = 1)
– in most applications only two of the cj coefficients are nonzero
–execution can be sped up • modulo 2 equivalent to exclusive-or
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
16
LFSR (cont.)
• form a sequence of binary integers W1, W2, …
– string together l consecutive bits
– consider them as numbers in base 2
• transform them into Uis
• maximum period 2q -1
– if l is relatively prime to 2q-1 LFSR has full period
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
17
LFSR : example
• r = 3 q = 5 b1 = b2 = b3 = b4 = b5 = 1 l = 4
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
I 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 17 18
bi 1 1 1 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1
i 1 2 3
Wi (base 2) 1111 1000 1101
Wi (base 10) 15 8 13
Ui 0.9375 0.5 0.8125
Testing RNGsEmpirical Tests
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 18
19
random number generators
• methods presented so far…
– completely deterministic
–Uis appear as if they were U(0,1)
• test their actual quality
–how well do the generated Uis resemble values of true IID U(0,1)
• different kinds of tests
–empirical tests• based on actual Uis produced by RNG
– theoretical tests040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
20
Chi-Square test (with all parameters known)
• checks whether the Uis appear to be IID U(0,1)
–divide [0,1] into k subintervals of equal length (k > 100)
–generate n random variables: U1, U2, .. Un
– calculate fj (number of Uis that fall into jth subinterval)
– calculate test statistic
– for large n χ2 will have an approximate chi-square distribution with k-1
df under the null hypothesis that the Uis are IID U(0,1)
– reject null hypothesis at level ® if
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
Generating Random Variates
22
Generating Random Variates
• General Approach
– Inverse Transformation
–Composition
–Convolution• Generating Continuous Random Variates
–Uniform U(a,b)
–Exponential
–m-Erlang
–Gamma, Weibull
–Normal
–etc…040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
23
Inverse Transformation
• generate continuous random variate X
–distribution function F (continuous, strictly increasing)• 0 < F(x) < 1• if x1 < x2 then 0 < F(x1) · F(x2) < 1
– inverse of F: F-1
• algorithm
–generate U ~ U(0,1)
– return X = F-1(U)
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
24
Inverse Transformation (cont.)
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
x
F(x)
0
1
U1
X1X2
U2
• returned value X has desired distribution F
25
Inverse Transformation (cont.)
• create X according to exponential distribution with mean ¯
• in order to find F-1 set
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
take natural logarithm (base e)
26
Inverse Transformation for discrete variates
• distribution function
• probability mass function
• algorithm
–generate U ~ U(0,1)
–determine smallest possible integer i such that U · F(xi)
– return X = xi
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
27
Composition
• applies then the distribution function F can be expressed as a convex combination of other distribution functions F1, F2, ..
–we hope to be able to sample from the Fj’s more easily than from the
original F
– if X has a density it can be written as
• algorithm
–generate positive random integer J such that P(J = j) = pj
–Return X with distribution function FJ
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
28
Composition (example)
• double-exponential distribution (laplace distribution)
– can be rewritten as
–where IA(x) is the indicator function of set A
• f(x) is a convex combination of
– f1(x) = ex I(-1, 0) and f2(x) = e-x I[0, 1)
–p1 = p2 = 0.5
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
29
Convolution
• desired random variable X can be expressed as sum of other random variables that are IID and can be generated more readily then X directly
• algorithm
–generate Y1, Y2, … Ym IID each with distribution function G
– return X = Y1 + Y2 + + Ym
• example: m-Erlang random variable X with mean ¯
– sum of m IID exponential random variables with common mean ¯/m
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
30
Generate Continuous Variates
• Uniform distribution X(a,b)
X = a + (b-a)U• Exponential (mean ¯)
X = - ¯ ln (1-U) or X = -¯ ln U• m-Erlang
040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I
Top Related