Random Number Generators

2

Why do we need random variables?

• random components in simulation → need for a method which generates numbers that are random

• examples– interarrival times–service times–demand sizes

040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

3

terminology

• Random Numbers –stochastic variable that meets certain conditions–numbers randomly drawn from some known distribution– It is impossible to “generate” them with the help of a digital computer.

• Pseudo-random Numbers –numbers which seem to be randomly drawn from some known

distribution–may be generated with the help of a digital computer


4

techniques for generating random variables

– ten-sided die (each throw generates a decimal)– throwing a coin n times • get a binary number between 0 and 2^n-1–other physical devices–observe substances undergoing atomic decay• e.g., Caesium-137, Krypton-85,. . .


5

techniques for generating pseudo-random variables

• numbers are generated using a recursive formula– ri is a function of ri-1, ri-2, …

– relation is deterministic – no random numbers– if the mathematical relation not known and well chosen it is practically

impossible to predict the next number• In most cases we want the generated random numbers to simulate a

uniform distribution over (0, 1), that is a U(0, 1)-distribution– there’re many simple techniques to transform uniform (U(a, b)) samples

into samples from other well-known distributions


1

0 1 0 1

6

terminology

• seed– random number generators are usually initialized with a starting

number which is called the “seed”–different seeds generates different streams of pseudorandom numbers– the same seed results in the same stream

• cycle length– length of the stream of pseudorandom numbers without repetition


7

methods for generating pseudo-random variables

• all methods usually generate uniformly distributed pseudorandom numbers in the interval [0,1]

–midsquare method (outdated)–congruential method (popular)–shift register (Tausworthe generator)


8

midsquare method

• arithmetic generator–proposed by von Neumann and Metropolis in 1940s

• algorithm–start with an initial number Z0 with m digits (seed) [m even]

–square it and get a number with 2m digits (add a zero at the beginning if the number has only 2m-1 digits)

–obtain Z1 by taking the middle m digits

– to be in the interval [0,1) divide the Zi by 10m

–etc….


9

midsquare method: example (Z0 = 7182, m = 4)


i01234567…

Zi

7182 5811 7677 9363 6657 3156 9603

2176 …

Ui

0.71820.58110.76770.93630.66570.31560.96030.2176

…

Zi2

51581124 33767721 58936329 87665769 44315649 9960336

92217609 4734976

…

10

midsquare method (cont.)

• advantages– fairly simple• disadvantage

– tends to degenerate fairly rapidly to zero• example: try Z0 = 1009

–not random as predictable


11

Linear Congruential Generator (LCG)

• LCGs– first proposed by Lehmer (1951).–produce pseudorandom numbers such that each single number

determines its successor by means of a linear function followed by a modular reduction

– to be in the interval [0,1) divide the Zi by m

–Z0 seed a multiplier (integer)

–c increment (integer) m modulus (integer)• Variations are possible

–combinations of previous numbers instead of using only the last value040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

12

LCG: example


i01234567…

Zi

7 6

1 8

11 10

5 12 …

Ui

0.43750.3750.06250.5 0.6875 0.625 0.31250.75

…

a = 5c = 3m = 16Z0 = 7

Z1 = (5*Z0 + 3) mod 16 = 38 mod 16 = 6Z2 = (5*Z1 + 3) mod 16 = 33 mod 16 = 1Z3 = (5*Z2 + 3) mod 16 = 8 mod 16 = 8…

13

properties of LCGs

• parameters need to be chosen carefully–nonnegative–0 < m a < m c < m Z0 < m• disadvantages

–not random– if a and m are properly chosen, the Uis will “look like” they are randomly

and uniformly distributed between 0 and 1.–can only take rational values 0, 1/m, 2/m, … (m-1)/m– looping


14

LCG (cont.)

• looping–whenever Zi takes on a value it has had previously, exactly the same

sequence of values is generated, and this cycle repeats itself endlessly.– length of sequence = period of generator

–period can be at most m • if so: LCG has full period–LCG has full period iff• the only positive integer that divides both m and c is 1• if q is a prime number that divides m then q divides a-1• if 4 divides m, then 4 divides a-1


15

Linear Feedback Shift Register Generators (LFSR)

• developed by Tausworthe (1965)– related to cryptographic methods–operate directly on bits to form random numbers• linear combination of the last q bits

–ci constants (equal to 0 or 1, cq = 1)

– in most applications only two of the cj coefficients are nonzero

–execution can be sped up • modulo 2 equivalent to exclusive-or


16

LFSR (cont.)

• form a sequence of binary integers W1, W2, … –string together l consecutive bits–consider them as numbers in base 2

• transform them into Uis

• maximum period 2q -1– if l is relatively prime to 2q-1 LFSR has full period


17

LFSR : example

• r = 3 q = 5 b1 = b2 = b3 = b4 = b5 = 1 l = 4


I 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 17 18

bi 1 1 1 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1

i 1 2 3Wi (base 2) 1111 1000 1101Wi (base 10) 15 8 13Ui 0.9375 0.5 0.8125

Testing RNGsEmpirical Tests

040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 18

19

random number generators

• methods presented so far…–completely deterministic–Uis appear as if they were U(0,1)

• test their actual quality–how well do the generated Uis resemble values of true IID U(0,1)

• different kinds of tests–empirical tests• based on actual Uis produced by RNG

– theoretical tests040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I

20

Chi-Square test (with all parameters known)

• checks whether the Uis appear to be IID U(0,1)–divide [0,1] into k subintervals of equal length (k > 100)–generate n random variables: U1, U2, .. Un

–calculate fj (number of Uis that fall into jth subinterval)

–calculate test statistic

– for large n χ2 will have an approximate chi-square distribution with k-1 df under the null hypothesis that the Uis are IID U(0,1)

– reject null hypothesis at level ® if


Generating Random Variates

22

Generating Random Variates

• General Approach– Inverse Transformation–Composition–Convolution• Generating Continuous Random Variates

–Uniform U(a,b)–Exponential–m-Erlang–Gamma, Weibull–Normal–etc…


23

Inverse Transformation

• generate continuous random variate X –distribution function F (continuous, strictly increasing)• 0 < F(x) < 1• if x1 < x2 then 0 < F(x1) · F(x2) < 1

– inverse of F: F-1

• algorithm–generate U ~ U(0,1)– return X = F-1(U)


24

Inverse Transformation (cont.)


x

F(x)

0

1U1

X1X2

U2

• returned value X has desired distribution F

25

Inverse Transformation (cont.)

• create X according to exponential distribution with mean ¯

• in order to find F-1 set


take natural logarithm (base e)

26

Inverse Transformation for discrete variates

• distribution function

• probability mass function

• algorithm–generate U ~ U(0,1)–determine smallest possible integer i such that U · F(xi)

– return X = xi


27

Composition

• applies then the distribution function F can be expressed as a convex combination of other distribution functions F1, F2, ..

–we hope to be able to sample from the Fj’s more easily than from the

original F

– if X has a density it can be written as

• algorithm–generate positive random integer J such that P(J = j) = pj

–Return X with distribution function FJ


28

Composition (example)

• double-exponential distribution (laplace distribution)

–can be rewritten as

–where IA(x) is the indicator function of set A

• f(x) is a convex combination of– f1(x) = ex I(-1, 0) and f2(x) = e-x I[0, 1)

–p1 = p2 = 0.5


29

Convolution

• desired random variable X can be expressed as sum of other random variables that are IID and can be generated more readily then X directly

• algorithm–generate Y1, Y2, … Ym IID each with distribution function G

– return X = Y1 + Y2 + + Ym

• example: m-Erlang random variable X with mean ¯–sum of m IID exponential random variables with common mean ¯/m


30

Generate Continuous Variates

• Uniform distribution X(a,b)X = a + (b-a)U• Exponential (mean ¯)

X = - ¯ ln (1-U) or X = -¯ ln U• m-Erlang


Random Number Generators

Documents

Transcript of Random Number Generators