Use of moment generating functions. Definition Let X denote a random variable with probability...

38
Use of moment generating functions

Transcript of Use of moment generating functions. Definition Let X denote a random variable with probability...

Use of moment generating functions

DefinitionLet X denote a random variable with probability density function f(x) if continuous (probability mass function p(x) if discrete)

Then

mX(t) = the moment generating function of X

tXE e

if is continuous

if is discrete

tx

tx

x

e f x dx X

e p x X

The distribution of a random variable X is described by either

1. The density function f(x) if X continuous (probability mass function p(x) if X discrete), or

2. The cumulative distribution function F(x), or

3. The moment generating function mX(t)

Properties1. mX(0) = 1

0 derivative of at 0.k thX Xm k m t t 2.

kk E X

2 33211 .

2! 3! !kk

Xm t t t t tk

3.

continuous

discrete

k

kk k

x f x dx XE X

x p x X

4. Let X be a random variable with moment generating function mX(t). Let Y = bX + a

Then mY(t) = mbX + a(t)

= E(e [bX + a]t) = eatmX (bt)

5. Let X and Y be two independent random variables with moment generating function mX(t) and mY(t) .

Then mX+Y(t) = mX (t) mY (t)

6. Let X and Y be two random variables with moment generating function mX(t) and mY(t) and two distribution functions FX(x) and FY(y) respectively.

Let mX (t) = mY (t) then FX(x) = FY(x).

This ensures that the distribution of a random variable can be identified by its moment generating function

M. G. F.’s - Continuous distributions

Name

Moment generating function MX(t)

Continuous Uniform

ebt-eat

[b-a]t

Exponential t

for t <

Gamma t

for t <

2

d.f.

1

1-2t /2

for t < 1/2

Normal et+(1/2)t22

M. G. F.’s - Discrete distributions

Name

Moment generating

function MX(t)

Discrete Uniform

et

N etN-1et-1

Bernoulli q + pet Binomial (q + pet)N

Geometric pet

1-qet

Negative Binomial

pet

1-qet k

Poisson e(et-1)

Moment generating function of the gamma distribution

tX txXm t E e e f x dx

1 0

0 0

xx e xf x

x

where

tX txXm t E e e f x dx

1

0

tx xe x e dx

using

1

0

t xx e dx

1

0

1a

a bxbx e dx

a

1

0

a bxa

ax e dx

b

or

then

1

0

t xXm t x e dx

t

tt

Moment generating function of the Standard Normal distribution

tX txXm t E e e f x dx

2

21

2

x

f x e

where

thus

2 2

2 21 1

2 2

x xtxtx

Xm t e e dx e dx

We will use 2

22

0

11

2

x a

be dxb

2

21

2

xtx

Xm t e dx

2 2

21

2

x tx

e dx

22 2 2 22

2 2 2 21 1

2 2

x tx tx t t t

e e dx e e dx

2

2

t

e

Note:

2

2 32 2

22

2 21

2 2! 3!

t

X

t t

tm t e

2 3 4

12! 3! 4!

x x x xe x

2 4 6 2

2 31

2 2 2! 2 3! 2 !

m

m

t t t t

m

Also

2 33211

2! 3!Xm t t t t

Note:

2

2 32 2

22

2 21

2 2! 3!

t

X

t t

tm t e

2 3 4

12! 3! 4!

x x x xe x

2 4 6 2

2 31

2 2 2! 2 3! 2 !

m

m

t t t t

m

Also 2 33211

2! 3!Xm t t t t

momentth kk k x f x dx

Equating coefficients of tk, we get

21

for 2 then 2 ! 2 !

mm

k mm m

0 if is odd andk k

1 2 3 4hence 0, 1, 0, 3

Using of moment generating functions to find the distribution of

functions of Random Variables

ExampleSuppose that X has a normal distribution with mean and standard deviation .

Find the distribution of Y = aX + b

2 2

2

tt

Xm t e

Solution:

22

2

atatbt bt

aX b Xm t e m at e e

2 2 2

2

a ta b t

e

= the moment generating function of the normal distribution with mean a + b and variance a22.

Thus Z has a standard normal distribution .

Special Case: the z transformation

1XZ X aX b

10Z a b

22 2 2 21

1Z a

Thus Y = aX + b has a normal distribution with mean a + b and variance a22.

ExampleSuppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y

Find the distribution of S = X + Y

2 2

2X

Xt

t

Xm t e

Solution:

2 2

2Y

Yt

t

Ym t e

2 2 2 2

2 2X Y

X Yt t

t t

X Y X Ym t m t m t e e

Now

or

2 2 2

2

X YX X

tt

X Ym t e

= the moment generating function of the normal distribution with mean X + Y and variance

2 2

X Y

Thus Y = X + Y has a normal distribution with mean X + Y and variance 2 2

X Y

ExampleSuppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y

Find the distribution of L = aX + bY

2 2

2X

Xt

t

Xm t e

Solution:

2 2

2Y

Yt

t

Ym t e

aX bY aX bY X Ym t m t m t m at m bt Now

2 22 2

2 2X Y

X Yat bt

at bte e

or

2 2 2 2 2

2

X YX X

a b ta b t

aX bYm t e

= the moment generating function of the normal distribution with mean aX + bY and variance

2 2 2 2

X Ya b

Thus Y = aX + bY has a normal distribution with mean aX + BY and variance

2 2 2 2

X Ya b

Special Case:

Thus Y = X - Y has a normal distribution with mean X - Y and variance

2 22 2 2 21 1

X Y X Y

a = +1 and b = -1.

Example (Extension to n independent RV’s)Suppose that X1, X2, …, Xn are independent each having a normal distribution with means i, standard deviations i

(for i = 1, 2, … , n)

Find the distribution of L = a1X1 + a1X2 + …+ anXn

2 2

2i

i

i

tt

Xm t e

Solution:

1 1 1 1n n n na X a X a X a Xm t m t m t Now

22 221 1

1 1 2 2n n

n n

a ta ta t a t

e e

(for i = 1, 2, … , n)

1 1 nX X nm a t m a t

or

2 2 2 2 2

1 11 1

1 1

......

2

n nn n

n n

a a ta a t

a X a Xm t e

= the moment generating function of the normal distribution with mean

and variance

Thus Y = a1X1 + … + anXn has a normal distribution with mean a11 + …+ ann and variance

1 1 ... n na a 2 2 2 21 1 ... n na a

2 2 2 21 1 ... n na a

1 2

1na a a

n

1 2 n 2 2 2 21 1 1

In this case X1, X2, …, Xn is a sample from a normal distribution with mean , and standard deviations and

1 2

1nL X X X

n

the sample meanX

Special case:

Thus

2 2 2 2 21 1 ...x n na a

and variance

1 1 ...x n na a has a normal distribution with mean

1 1 ... n nY x a x a x

11 1... nx xn n

1 1...n n

2 2 2 22 2 21 1 1

... nn n n n

If x1, x2, …, xn is a sample from a normal distribution with mean , and standard deviations then the sample meanx

Summary

22x n

and variance

x has a normal distribution with mean

standard deviation xn

0

0.1

0.2

0.3

0.4

20 30 40 50 60

Population

Sampling distribution of x

If x1, x2, …, xn is a sample from a distribution with mean , and standard deviations then if n is large the sample meanx

The Central Limit theorem

22x n

and variance

x has a normal distribution with mean

standard deviation xn

We will use the following fact: Let

m1(t), m2(t), … denote a sequence of moment generating functions corresponding to the sequence of distribution functions:

F1(x) , F2(x), … Let m(t) be a moment generating function corresponding to the distribution function F(x) then if

Proof: (use moment generating functions)

lim for all in an interval about 0.ii

m t m t t

lim for all .ii

F x F x x

then

Let x1, x2, … denote a sequence of independent random variables coming from a distribution with moment generating function m(t) and distribution function F(x).

1 2 1 2

=n n nS x x x x x xm t m t m t m t m t

Let Sn = x1 + x2 + … + xn then

=n

m t

1 2now n nx x x Sx

n n

1or n

n

n

x SS

n

t tm t m t m m

n n

Let x n n

z x

n

then

nn n

t t

z x

nt ntm t e m e m

n

and ln lnz

n tm t t n m

n

Then ln lnz

n tm t t n m

n

2 2

2 2 2ln

t tm u

u u

2

2 2Let or and

t t tu n n

u un

2

2 2

ln m u ut

u

0

Now lim ln lim lnz zn u

m t m t

2

2 20

lnlimu

m u ut

u

2

2 0lim using L'Hopital's rule

2u

m u

m ut

u

2

22

2 0lim using L'Hopital's rule again

2u

m u m u m u

m ut

2

22

2 0lim using L'Hopital's rule again

2u

m u m u m u

m ut

22

2

0 0

2

m mt

222 2

2 2 2

i iE x E xt t

222thus lim ln and lim

2

t

z zn n

tm t m t e

2

2Now t

m t e

Is the moment generating function of the standard normal distribution

Thus the limiting distribution of z is the standard normal distribution

2

21

i.e. lim2

x u

zn

F x e du

Q.E.D.