Generating random numbers · cand mare prime number (distinct natural number divisors 1 and itself...

Generating random numbers

Lecturer: Dmitri A. Moltchanov

E-mail: [email protected]

http://www.cs.tut.fi/kurssit/ELT-53606/

Network analysis and dimensioning I D.Moltchanov, TUT, 2013

OUTLINE:

• Why do we need random numbers;

• Basic steps in generation;

• Uniformly distributed random numbers;

• Statistical tests for uniform random numbers;

• Random numbers with arbitrary distributions;

• Statistical tests for random numbers with arbitrary distribution;

• Multidimensional distributions.

Lecture: Generating random numbers 2


1. The need for random numbersExamples of randomness in telecommunications:

• interarrival times between arrivals of packets, tasks, etc.;

• service time of packets, tasks, etc.;

• time between failure of various components;

• repair time of various components;

• . . .

Importance for simulations:

• random events are characterized by distributions;

• simulations: we cannot use distribution directly.

For example, M/M/1 queuing system:

• arrival process: exponential distribution with mean 1/λ;

• service times: exponential distribution with mean 1/µ.



Discrete-event simulation of M/M/1 queue

INITIALIZATION

time:=0;

queue:=0;

sum:=0;

throughput:=0;

generate first interarrival time;

MAIN PROGRAM

while time < runlength do

case nextevent of

arrival event:

time:=arrivaltime;

add customer to a queue;

start new service if the service is idle;

generate next interarrival time;

departure event:

time:=departuretime;

throughput:=throughtput + 1;

remove customer from a queue;

if (queue not empty)

sum:=sum + waiting time;

start new service;

OUTPUT

mean waiting time = sum / throughput



2. General notesGeneral approach nowadays:

• transforming one random variable to another one;

• as a reference distribution a uniform distribution is often used.

Note the following:

• most simulators contain generator of uniformly distributed numbers in interval (0, 1).

• may not contain arbitrarily distributed random numbers you want.

The procedure is to:

• generate RN with inform distribution between a and b, b >>>> a;

• transform it somehow to random number with uniform distribution on (0, 1);

• transform it somehow to a random number with desired distribution.



2.1. Pseudo random numbers

All computer generated numbers are pseudo ones:

• we know the method how they are generated;

• we can predict any ”random” sequence in advance.

The goal is then: imitate random sequences as good as possible.

Requirements for generators:

• must be fast;

• must have low complexity;

• must have sufficiently long cycles;

• must allow to generate repeatable sequences;

• must be independent;

• must closely follow a given distribution.



2.2. Step 1: uniform random numbers in (a, b)

Basic approach:

• generate random number with uniform distribution on (a, b);

• transform these random numbers to (0, 1);


Uniform generators:

• old methods: mostly based on radioactivity;

• Von Neumann’s algorithm;

• congruential methods.

Basic approach: next number is some function of previous one

γi+1 = F (γi), i = 0, 1, . . . , (1)

• recurrence relation of the first order;

• γ0 is known and directly computed from the seed.



2.3. Step 2: transforming to random numbers in (0, 1)

Basic approach:

• generate random number with uniform distribution on (0, 1);



Uniform U(0, 1) distribution has the following pdf:

f(x) =

1, 0 ≤ x ≤ 1

0, otherwise. (2)



Mean and variance are given by:

E[X] =

∫ 1

0

xdx =x2

2

∣∣∣∣∣1

0

=1

2,

σ2[X] =1

12. (3)

How to get U(0, 1):

• by rescaling from U(0,m) as follows:

yi = γi/m, (4)

• where m is the modulo in linear congruential algorithm.

What we get:

• something like: 0.12, 0.67, 0.94, 0.04, 0.65, 0.20, . . . ;

• sequence that appears to be random...



2.4. Step 3: non-uniform random numbers

Basic approach:

• generate random number with uniform distribution on (a, b);



If we have generator U(0, 1) the following techniques are avalable:

• discretization: bernoulli, binomial, poisson, geometric;

• rescaling: uniform;

• inverse transform: exponential;

• specific transforms: normal;

• rejection method: universal method;

• reduction method: Erlang, Binomial;

• composition method: for complex distributions.



3. Uniformly distributed random numbersThe generator is fully characterized by (S, s0, f, U, g):

• S is a finite set of states;

• s0 ∈ S is the initial state;

• f(S → S) is the transition function;

• U is a finite set of output values;

• g(S → U) is the output function.

The algorithm is then:

• let u0 = g(s0);

• for i = 1, 2, . . . do the following recursion:

– si = f(si−1);

– ui = g(si).

Note: functions f(·) and g(·) influence the goodness of the algorithm heavily.



user choice s0

s0

s1

s2

s3

s4

u0

u1

u2

u3

u4

u0=g(s0)

u1=g(s1)

u2=g(s2)

u3=g(s3)

u4=g(s4)

s1=f(s0)

s2=f(s1) s3=f(s2)

s4=f(s3)

Figure 1: Example of the operations of random number generator.

Here s0 is a random seed:

• allows to repeat the whole sequence;

• allows to manually assure that you get different sequence.



3.1. Von Neumann’s generator

The basic procedure:

• start with some number u0 of a certain length x (say, x = 4 digits, this is seed);

• square the number;

• take middle 4 digits to get u1;

• repeat...

• example: with seed 1234 we get 1234, 5227, 3215, 3362, 3030, etc.

Shortcoming:

• sensitive to the random seed:

– seed 2345: 2345, 4990, 9001, 180, 324, 1049, 1004, 80, 64, 40... (will always < 100);

• may have very short period:

– seed 2100: 2100, 4100, 8100, 6100, 2100, 4100, 8100,... (period = 4 numbers).

To generate U(0, 1): divide each obtained number by 10x (x is the length of u0).

Note: this generator is also known as midsquare generator.



3.2. Congruential methods

There are a number of versions:

• additive congruential method;

• multiplicative congruential method;

• linear congruential method;

• Tausworthe binary generator.

General congruential generator:

ui+1 = f(ui, ui−1, . . . ) mod m, (5)

• ui, ui−1, . . . are past numbers.

For example, quadratic congruential generator:

ui+1 = (a1u2i + a2ui−1 + c) mod m. (6)

Note: if here a1 = a2 = 1, c = 0, m = 2 we have the same as midsquare method.



3.3. Additive congruential method

Additive congruential generator is given:

ui+1 = (a1ui + a2ui−1 + · · ·+ akui−k) mod m. (7)

The common special case is sometimes used:

ui+1 = (a1ui + a2ui−1) mod m. (8)

Characteristics:

• divide by m to get U(0, 1);

• maximum period is mk;

• note: rarely used.

Shortcomings: consider k = 2:

• consider three consecutive numbers ui−2, ui−1, ui;

• we will never get: ui−2 < ui < ui−1 and ui−1 < ui < ui−2 (must be 1/6 of all sequences).



3.4. Multiplicative congruential method

Multiplicative congruential generator is given:

ui+1 = (aui) mod m. (9)

Characteristics:


• theoretical maximum period is m;

• note: rarely used.

Shortcomings:

• can never produce 0.

Choice of a,m is very important:

• recommended m = (2p − 1) with p = 2, 3, 5, 7, 13, 17, 19, 31, 61 (Fermat numbers);

• if m = 2q, q ≥ 4 simplifies the calculation of modulo;

• practical maximum period is at best no longer than m/4.



3.5. Linear congruential method

Linear congruential generator is given:

ui+1 = (aui + c) mod m, (10)

• where a, c,m are all positive.

Characteristics:


• maximum period is m;

• frequently used.

Choice of a, c,m is very important. To get full period m choose:

• m and c have no common divisor;

• c and m are prime number (distinct natural number divisors 1 and itself only);

• if q is a prime divisor of m then a = 1, mod q;

• if 4 is a divisor of m then a = 1, mod 4.



The step-by-step procedure is as follows:

• set the seed x0;

• multiply x by a and add c;

• divide the result by m;

• the reminder is x1;

• repeat to get x2, x3, . . . .

Examples:

• x0 = 7, a = 7, c = 7, m = 10 we get: 7,6,9,0,7,6,9,0,... (period = 4);

• x0 = 1, a = 1, c = 5, m = 13 we get: 1,6,11,3,8,0,5,10,2,7,12,4,9,1... (period = 13);

• x0 = 8, a = 2, c = 5, m = 13 we get: 8,8,8,8,8,8,8,8,... (period = 1!).

Recommended values: a = 314, 159, 269, c = 453, 806, 245, m = 231 for 32 bit machine.



Complexity of the algorithm: addition, multiplications and division:

• division is slow: to avoid it set m to the size of the computer word.

Overflow problem when m equals to the size of the word:

• values a, c and m are such that the result axi + c is greater than the word;

• it may lead to loss of significant digits but it does not hurt!

How to deal with:

• register can accommodate 2 digits at maximum;

• the largest number that can be stored is 99;

• if m = 100: for a = 8, u0 = 2, c = 10 we get (aui + c) mod 100 = 26;

• if m = 100: for a = 8, u0 = 20, c = 10 we get (aui + c) mod 100 = 170;

– aui = 8 ∗ 20 = 160 causing overflow;

– first significant digit is lost and register contains 60;

– the reminder in the register (result) is: (60 + 10) mod 70 = 70.

• the same as 170 mod 100 = 70.



3.6. How to get good congruental generator

Characteristics of good generator:

• should provide maximum density:

– no large gaps in [0, 1] are produced by random numbers;

– problem: each number is discrete;

– solution: a very large integer for modulus m.

• should provide maximum period:

– achieve maximum density and avoid cycling;

– achieve by: proper choice of a, c, m, and x0.

• effective for modern computers:

– set modulo to power of 2.



3.7. Tausworthe generator

Tausworthe generator (case of linear congruential generator or order k):

zi = (a1zi−1 + a2zi−2 + · · ·+ akzi−k + c) mod 2 =

(k∑j=1

ajzi−j + c

)mod 2. (11)

• where aj ∈ {0, 1}, j = 0, 1, . . . , k;

• the output is binary: 0011011101011101000101...

Advantages:

• independent of the system (computer architecture);

• independent of the word size;

• very large periods;

• can be used in composite generators (we consider in what follows).

Note: there are several bit selection techniques to get numbers.



A way to generate numbers:

• choose an integer l ≤ k;

• split in blocks of length l and interpret each block as a digit:

un =l−1∑j=0

znl+j2−(j+1). (12)

In practice, only two ai are used and set to 1 at places h and k. We get:

zn = (zi−h + zi−k) mod 2. (13)

Example:

• h = 3, k = 4, initial values 1,1,1,1;

• we get: 110101111000100110101111...;

• period is 2k − 1 = 15;

• if l = 4: 13/16, 7/16, 8/16, 9/16, 10/16, 15/16, 1/16, 3/16...



3.8. Composite generator

Idea: use two generators of low period to generate another with wider period.

The basic principle:

• use the first generator to fill the shuffling table (address - entry (random number));

• use random numbers of second generator as addresses in the next step;

• each number corresponding to the address is replaced by new random number of first generator.

The following algorithm uses one generator to shuffle with itself:

1. create shuffling table of 100 entries (i, ti = γi, i = 1, 2, . . . , 100);

2. draw random number γk and normalize to the range (1, 100);

3. entry i of the table gives random number ti;

4. draw the next random number γk+1 and update ti = γk+1;

5. repeat from step 2.

Note: table with 100 entries gives fairly good results.



4. Tests for random number generatorsWhat do we want to check:

• independence;

• uniformity.

Important notes:

• if and only if tests passed number can be treated as random;

• recall: numbers are actually deterministic!

Commonly used tests for independence:

• runs test;

• correlation test.

Commonly used tests for uniformity:

• Kolmogorov’s test;

• χ2 test.



4.1. Independence: runs test

Basic idea:

• compute patterns of numbers (always increase, always decrease, etc.);

• compare to theoretical probabilities.

1/3 1/3 1/3

1/3 1/3 1/3

Figure 2: Illustration of the basic idea.



Do the following:

• consider a sequence of pseudo random numbers: {ui, i = 0, 1, . . . , n};

• consider unbroken subsequences of numbers where numbers are monotonically increasing;

– such subsequence is called run-up;

– example: 0.78,081,0.89,0.81 is a run-up of length 3.

• compute all run-ups of length i:

– ri, i = 1, 2, 3, 4, 5;

– all run-ups of length i ≥ 6 are grouped into r6.

• calculate:

R =1

n

∑1≤i,j≤6

(ri − nbi)(rj − nbj)aij, 1 ≤ i, j ≤ 6, (14)

where

(b1, b2, . . . , b6) =

(1

6,

5

24,

11

120,

19

720,

29

5040,

1

840

), (15)



Coefficients aij must be chosen as an element of the matrix:

Statistics R has χ2 distribution:

• number of freedoms: 6;

• n > 4000.

If so, observations are i.i.d.



4.2. Independence: correlation test

Basic idea:

• compute autocorrelation coefficient for lag-1;

• if it is not zero and this is statistically significant result, numbers are not independent.

Compute statistics (lag-1 autocorrelation coefficient) as:

R =N∑j=1

(uj − E[u])(uj+1 − E[u])/N∑j=1

(uj − E[j])2. (16)

Practice: if R is relatively big there is serial correlation.

Important notes:

• exact distribution of R is unknown;

• for large N : if uj uncorrelated we have: Pr{−2/√N ≤ R ≤ 2/

√N};

• therefore: reject hypotheses of non-correlated at 5% level if R is not in {−2/√N, 2/

√N}.

Notes: other tests for correlation Ljung and Box test, ’Portmanteau’ test, etc.



4.3. Uniformity: χ2 test

The algorithm:

• divide [0, 1] into k, k > 100 non-overlapping intervals;

• compute the relative frequencies of falling in each category, fi:

– ensure that there are enough numbers to get fi > 5, i = 1, 2, . . . , k;

– values fi > 5, i = 1, 2, . . . , k are called observed values.

• if observations are truly uniformly distributed then:

– these values should be equal to ri = n/k, i = 1, 2, . . . , k;

– these values are called theoretical values.

• compute χ2 statistics for uniform distribution:

χ2 =k

n

k∑i=1

(fi −

n

k

)2. (17)

– that must have k − 1 degrees of freedom.



Hypotheses:

• H0 observations are uniformly distributed;

• H1 observations are not uniformly distributed.

H0 is rejected if:

• computed value of χ2 is greater than one obtained from the tables;

• you should check the entry with k − 1 degrees of freedom and 1-a level of significance.



4.4. Kolmogorov test

Facts about this test:

• compares empirical distribution with theoretical ones;

• empirical: FN(x) – number of smaller than or equal to x, divided by N ;

• theoretical: uniform distribution in (0, 1): F (x) = x, 0 < x < 1.

Hypotheses:

• H0: FN(x) follows F (x);

• H1: FN(x) does not follow F (x).

Statistics: maximum absolute difference over a range:

R = max |F (x)− FN(x)|. (18)

• if R > Rα: H0 is rejected;

• if R ≤ Rα: H0 is accepted.

Note: use tables for N , α (significance level), to find Rα.



Example: we got 0.44, 0.81, 0.14, 0.05, 0.93:

• H0: random numbers follows uniform distribution;

• we have to compute:

0.130.210.04-0.05R(j) – (j-1)/N

0.07-0.160.260.15j/N – R(j)

1.000.800.600.400.20j/N

0.930.810.440.140.05R(j)

0.130.210.04-0.05R(j) – (j-1)/N

0.07-0.160.260.15j/N – R(j)

1.000.800.600.400.20j/N

0.930.810.440.140.05R(j)

• compute statistics as: R = max |F (x)− FN(x)| = 0.26;

• from tables: for α = 0.05, Rα = 0.565 > R;

• H0 is accepted, random numbers are distributed uniformly in (0, 1).



4.5. Other tests

The serial test:

• consider pairs (u1, u2), (u3, u4), . . . , (u2N−1, u2N);

• count how many observations fall into N2 different subsquares of the unit square;

• apply χ2 test to decide whether they follow uniform distribution;

• one can formulate M -dimensional version of this test.

The permutation test

• look at k-tuples: (u1, uk), (uk+1, u2k), . . . , (u(N−1)k+1, uNk);

• in a k-tuple there k! possible orderings;

• in a k-tuple all orderings are equally likely;

• determine frequencies of orderings in k-tuples;

• apply χ2 test to decide whether they follow uniform distribution.



The gap test

• let J be some fixed subinterval in (0, 1);

• if we have that:

– un+j not in J , 0 ≤ j ≤ k, and both un−1 ∈ J , un+k+1 ∈ J ;

– we say that there is a gap of length k.

• H0: numbers are independent and uniformly distributed in (0, 1):

– gap length must be geometrically distributed with some parameter p;

– p is the length of interval J :

Pr{gap of length k} = p(1− p)k. (19)

• practice: we observe a large number of gaps, say N ;

• choose an integer and count number of gaps of length 0, 1, . . . , h− 1 and ≥ h;

• apply χ2 test to decide whether they independent and follow uniform distribution.



4.6. Important notes

Some important notes on seed number:

• do not use seed 0;

• avoid even values;

• do not use the same sequence for different purposes in a single simulation run.

Note: these instruction may not be applicable for a particular generator.

General notes:

• some common generators are found to be inadequate;

• even if generator passed tests, some underlying pattern might still be undetected;

• if the task is important use composite generator.



5. Random numbers with arbitrary distributionDiscrete distributions:

• discretization;

– for any discrete distribution.

• rescaling:

– for uniform random numbers in (a, b).

• methods for specific distributions.

Continuous distributions:

• inverse transform;

• rejection method;

• composition method;

• methods for specific distributions.



5.1. Discrete distributions: discretization

Consider arbitrary distributed discrete RV:

Pr{X = xj} = pj, j = 0, 1, . . .,∞∑j=0

pj = 1. (20)

The following method can be applied:

• generate uniformly distributed RV;

• use the following to generate discrete RV:

• this method can be applied to any discrete RV;

• there are some specific methods for specific discrete RVs.



Figure 3: Illustration of the proposed approach.



The step-by-step procedure:

• compute probabilities pi = Pr{X = xi}, i = 0, 1, . . . ;

• generate RV u with U(0, 1);

• if u < p0, set X = x0;

• if u < p0 + p1, set X = x1;

• if u < p0 + p1 + p2, set X = x2;

• . . .

Note the following:

• this is inverse transform method for discrete RVs:

– we determine the value of u;

– we determine the interval [F (xi−1), F (xi)] in which it lies.

• complexity depends on the number of intervals to be searched.



Example: p1 = 0.2, p2 = 0.1, p3 = 0.25, p4 = 0.45:

• determine generator for Pr{X = xj} = pj.

Algorithm 1:

• generate u = U(0, 1);

• if u < 0.2, set X = 1, return;

• if u < 0.3, set X = 2;

• if u < 0.55, set X = 3;

• set X = 4.

Algorithm 2 (more effective):


• If u < 0.45, set X = 4;

• if u < 0.7, set X = 3;

• if u < 0.9, set X = 1;

• set X = 2.



5.2. Example of discretization: Poisson RV

Poisson RV have the following distribution:

pi = Pr{X = i} =λie−λ

i!, i = 0, 1, . . . . (21)

We use the property:

pi+1 =λ

i+ 1pi, i = 1, 2, . . . . (22)

The algorithm:

1. generate u = U(0, 1);

2. i = 0, p = eλ, F = p;

3. if u < F , set X = i;

4. p = λp/(i+ 1), F = F + p, i = i+ 1;

5. go to step 3.



5.3. Example of discretization: binomial RV

Binomial RV have the following distribution:

pi = Pr{X = i} =n!

i!(n− i)!pi(1− p)n−i, i = 0, 1, . . . . (23)

We are going to use the following property:

pi+1 =n− ii+ 1

p

1− ppi, i = 0, 1, . . . . (24)

The algorithm:

1. generate u = U(0, 1);

2. c = p/(1− p), i = 0, d = (1− p)n, F = d;

3. if u < F , set X = i

4. d = [c(n− i)/(i+ 1)]d, F = F + d, i = i+ 1;

5. go to step 3.



5.4. Continuous distributions: inverse transform method

Inverse transform method:

• applicable only when cdf can be inversed analytically;

• works for a number of distributions: exponential, unform, Weibull, etc.

Assume:

• we would like to generate numbers with pdf f(x) and cdf F (x);

• recall, F (x) is defined on [0, 1].

The generic algorithm:


• set F (x) = u;

• find x = F−1(u),

– F−1(·) is the inverse transformation of F (·).



Example:

• we want to generate numbers from the following pdf f(x) = 2x, 0 ≤ x ≤ 1;

• calculate the cdf as follows:

F (x) =

∫ x

0

2tdt = x2, 0 ≤ x ≤ 1. (25)

• let u be the random number, we have u = x2 or u =√x;

• get the random number.



5.5. Inverse transform method: uniform continuous distribution

Uniform continuous distribution has the following pdf and cdf:

f(x) =

1(b−a) , a < x < b

0, otherwise, F (x) =

(x−a)(b−a) , a < x < b

0, otherwise. (26)

The algorithm:


• set u = F (x) = (x− a)/(b− a);

• solve to get x = a+ (b− a)u.



5.6. Inverse transform method: exponential distribution

Exponential distribution has the following pdf and cdf:

f(x) = λe−λx, F (x) = 1− e−λx, λ > 0, x ≥ 0. (27)

The algorithm:


• set u = F (x) = e−λx;

• solve to get x = −(1/λ) log u.



5.7. Inverse transform method: Erlang distribution

Erlang distribution:

• convolution of k exponential distributions.

The algorithm:


• sum of exponential variables x1, . . . , xk with mean 1/λ;

• solve to get:

x =k∑i=1

xi = −1

λ

k∑i=1

log ui = −1

λlog

k∑i=1

ui. (28)



5.8. Specific methods: normal distribution

Normal distribution has the following pdf:

f(x) =1

σ√

2πe−

12

(x−µ)2

σ2 , −∞ < x <∞, (29)

• where σ and µ are the standard deviation and the mean.

Standard normal distribution (RV Z = (X − µ/)σ) has the following pdf:

f(z) =1√2πe−

12z2 , −∞ < z <∞, where µ = 0, σ = 1. (30)



Central limit theorem:

• if x1, x2, . . . , xn are independent with E[xi] = µ, σ2[xi] = σ2, i = 1, 2, . . . , n;

• the sum of them approaches normal distribution if n→∞: E[∑xi] = nµ, σ2[

∑xi] = nσ2.

The approach:

• generate k random numbers ui = U(0, 1), i = 0, 1, . . . , k − 1;

• each random numbers has: E[ui] = (0 + 1)/2 = 1/2, σ2[ui] = (1− 0)2/12 = 1/12;

• sum of these number follows normal distribution with:∑ui ∼ N

(k

2,k√12

), or

∑ui − k/2k/√

12∼ N(0, 1). (31)

• if the RV we want to generate is x with mean µ and standard deviation σ:

x− µσ∼ N(0, 1). (32)

• finally (note that k should be at least 10):

x− µσ

=

∑ui − k/2k/√

12, or x = σ

√12

k

(∑ui −

k

2

)+ µ. (33)



5.9. Specific method: empirical continuous distributions

Assume we have a histogram:

• xi is the midpoint of the interval i;

• f(xi) is the length of the ith rectangle.

Note: the task is different from sampling from discrete distribution.



Construct the cdf as follows:

F (xi) =∑

k∈{F (xi−1),F (xi)}

f(xk), (34)

• which is monotonically increasing within each interval [F (xi−1), F (xi)].



The algorithm:


• assume that u ∈ {F (xi−1), F (xi)};

• use the following linear interpolation to get:

x = xi−1 + (xi − xi−1)u− F (xi−1)

F (xi)− F (xi−1). (35)

Note: this approach can also be used for analytical continuous distribution.

• get (xi, f(xi)), i = 1, 2, . . . , k and follow the procedure.



5.10. Rejection method

Works when:

• pdf f(x) is bounded;

• x has a finite range, say a ≤ x ≤ b.

The basic steps are:

• normalize the range of f(x) by a scale factor such that cf(x) ≤ 1, a ≤ x ≤ b;

• define x as a linear function of u = U(0, 1), i.e. x = a+ (b− a)u;

• generate pairs of random numbers (u1, u2), u1, u2 = U(0, 1);

• accept the pair and use x = a+ (b− a)u1 whenever:

– the pair satisfies u2 ≤ cf(a+ (b− a)u1);

– meaning that the pair (x, u2) falls under the curve of cf(x).



The underlying idea:

• Pr{u2 ≤ cf(x)} = cf(x);

• if x is chosen at random from (a, b):

– we reject if u2 > cf(x);

– we accept if u2 ≤ cf(x);

– we match f(x).



Example: generate numbers from f(x) = 2x, 0 ≤ x ≤ 1:

1. select c such that cf(x) ≤ 1:

• for example: c = 0.5.

2. generate u1 and set x = u1;

3. generate u2:

• if u2 < cf(u1) = (0.5)2u1 = u1 then accept x;

• otherwise go back to step 2.



5.11. Convolution method

The basis of the method is the representation of cdf F (x):

F (x) =∞∑j=1

pjFj(x), (36)

• pj ≥ 0, j = 1, 2, . . . ,∑∞

j=1 pj = 1.

Works when:

• it is easy to to generate RVs with distribution Fj(x) than F (x);

– hyperexponential RV;

– Erlang RV.

The algorithm:

1. generate discrete RV J , Pr{J = j} = pj;

2. given J = j generate RV with Fj(x);

3. compute∑∞

j=1 pjFj(x).



Example: generate from exponential distribution:

• divide (0,∞) into intervals (i, i+ 1), i = 0, 1, . . . ;

• the probabilities of intervals are given:

pi = Pr{i ≤ X < i+ 1} = e−i − e−(i+1) = e−i(1− e−1), (37)

– gives geometric distribution.

• the conditional pdfs are fiven by:

fi(x) = e−(x−i)/(1− e−1), i ≤ x < i+ 1. (38)

– in the interval i(X − i) has the pdf e−x/(1− e−1), 0 ≤ x < 1.

The algorithm:

• get I from geometric distribution pi = e−i/(1− e−1), i = 0, 1, . . . ;

• get Y from e−x/(1− e−1), 0 ≤ x < 1;

• X = I + Y .



6. Statistical tests for RNs with arbitrary distributionWhat we have to test for:

• independence;

• particular distribution.

Tests for independence:

• correlation tests:

– ’Portmanteau’ test, modified ’Portmanteau’ test, ±2/√n, etc.

– note: here we test only for linear dependence...

Tests for distribution:

• χ2 test;

• Kolmogorov’s test.



7. Multi-dimensional distributionsTask: generate samples from RV (X1, X2, . . . , Xn).

Write the joint density function as:

f(x1, x2, . . . , xn) = f1(x1)f2(x2|x1) . . . f(xn|x1 . . . xn−1). (39)

• f1(x1) is the marginal distribution of X1;

• fk(xk|x1, . . . , xk−1) is the conditional pdf of Xk with condition on X1 = x1, . . . , Xk−1 = xk−1.

The basic idea: generate one number at a time:

• get x1 from f1(x1);

• get x2 from f2(x2|x1), etc.

The algorithm:

• get n random numbers ui = U(0, 1), i = 0, 1, . . . , n;

• subsequently get the following RVs:

F1(X1) = u1, F2(X2|X1) = u2, . . . Fn(Xn|X1, . . . , Xn−1) = un. (40)



Example: generate from f(x, y) = x+ y:

• marginal pdf and cdf of X are given by:

f(x) =

∫ 1

0

f(x, y)dy = x+1

2, F (x) =

∫ x

0

f(x′)dx′ =1

2(x2 + x). (41)

• conditional pdf and cdf of Y are given by:

f(y|x) =f(x, y)

f(x)=x+ y

x+ 12

, F (y|x) =

∫ y

0

f(y′|x)dy′ =xy + 1

2y2

x+ 12

. (42)

• by inversion we get:

x =1

2(√

8u1 + 1− 1),

y =√x2 + u2(1 + 2x)− x. (43)


Generating random numbers · cand mare prime number (distinct natural number divisors 1 and itself...

Documents

Transcript of Generating random numbers · cand mare prime number (distinct natural number divisors 1 and itself...