Moment Generating Functions. Continuous Distributions The Uniform distribution from a to b.

Moment Generating Functions

0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

f x

x0

0.1

0.2

0.3

0.4

0 5 10 15

1

b a

a b

f x

x

1

b a

a b

f x

x

Continuous Distributions

The Uniform distribution from a to b

1

0 otherwise

a x bf x b a

The Normal distribution (mean , standard deviation )

2

221

2

x

f x e

0

0.1

0.2

-2 0 2 4 6 8 10

The Exponential distribution

0

0 0

xe xf x

x

Weibull distribution with parameters and.

Thus 1x

F x e

1and 0x

f x F x x e x

The Weibull density, f(x)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0 1 2 3 4 5

( = 0.5, = 2)

( = 0.7, = 2)

( = 0.9, = 2)

The Gamma distribution

Let the continuous random variable X have density function:

1 0

0 0

xx e xf x

x

Then X is said to have a Gamma distribution with parameters and .

Expectation of functions of Random Variables

X is discrete

i ix i

E g X g x p x g x p x

E g X g x f x dx

X is continuous

Moments of Random Variables

kk E X

-

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

The kth moment of X.

the kth central moment of X

0 k

k E X

-

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

where = 1 = E(X) = the first moment of X .

Rules for expectation

Rules:

1. where is a constantE c c c

2. where , are constantsE aX b aE X b a b

2023. var X E X

22 22 1E X E X

24. var varaX b a X

Moment generating functions

Moment Generating function of a R.V. X

if is discrete

if is continuous

tx

xtX

Xtx

e p x X

m t E ee f x dx X

Examples1. The Binomial distribution (parameters p, n)

tX txX

x

m t E e e p x

0

1n

n xtx x

x

ne p p

x

0 0

1n nx n xt x n x

x x

n ne p p a b

x x

1nn ta b e p p

0,1,2,!

x

p x e xx

The moment generating function of X , mX(t) is:

2. The Poisson distribution (parameter )

tX txX

x

m t E e e p x 0 !

xntx

x

e ex

0 0

using ! !

t

xt xe u

x x

e ue e e e

x x

1te

e

0

0 0

xe xf x

x


3. The Exponential distribution (parameter )

0

tX tx tx xXm t E e e f x dx e e dx

0 0

t xt x e

e dxt

undefined

tt

t

2

21

2

x

f x e


4. The Standard Normal distribution ( = 0, = 1)

tX txXm t E e e f x dx

2 22

1

2

x tx

e dx

2

21

2

xtxe e dx

2 2 2 22 2

2 2 21 1

2 2

x tx t x tx t

Xm t e dx e e dx

We will now use the fact that 2

221

1 for all 0,2

x b

ae dx a ba

We have completed the square

22 2

2 2 21

2

x tt t

e e dx e

This is 1

1 0

0 0

xx e xf x

x


4. The Gamma distribution (parameters , )

tX txXm t E e e f x dx

1

0

tx xe x e dx

1

0

t xx e dx

We use the fact

1

0

1 for all 0, 0a

a bxbx e dx a b

a

1

0

t xXm t x e dx

1

0

t xtx e dx

tt

Equal to 1

Properties of Moment Generating Functions

1. mX(0) = 1

0, hence 0 1 1tX XX Xm t E e m E e E

v) Gamma Dist'n Xm tt

2

2iv) Std Normal Dist'n t

Xm t e

iii) Exponential Dist'n Xm tt

1ii) Poisson Dist'n

te

Xm t e

i) Binomial Dist'n 1nt

Xm t e p p

Note: the moment generating functions of the following distributions satisfy the property mX(0) = 1

2 33212. 1

2! 3! !kk

Xm t t t t tk

We use the expansion of the exponential function:2 3

12! 3! !

ku u u u

e uk

tXXm t E e

2 32 31

2! 3! !

kkt t t

E tX X X Xk

2 3

2 312! 3! !

kkt t t

tE X E X E X E Xk

2 3

1 2 312! 3! !

k

k

t t tt

k

0

3. 0k

kX X kk

t

dm m t

dt

Now 2 332

112! 3! !

kkXm t t t t t

k

2 1321 2 3

2! 3! !kk

Xm t t t ktk

2 13

1 2 2! 1 !kkt t t

k

1and 0Xm

24

2 3 2! 2 !kk

Xm t t t tk

2and 0Xm continuing we find 0kX km

i) Binomial Dist'n 1nt

Xm t e p p

Property 3 is very useful in determining the moments of a random variable X.

Examples

11

nt tXm t n e p p pe

10 010 1

n

Xm n e p p pe np

2 11 1 1

n nt t t t tXm t np n e p p e p e e p p e

2

2

1 1 1

1 1

nt t t t

nt t t

npe e p p n e p e p p

npe e p p ne p p

2 221np np p np np q n p npq

1ii) Poisson Dist'n

te

Xm t e

1 1t te e ttXm t e e e

1 1 2 121t t te t e t e tt

Xm t e e e e

1 2 12 2 1t te t e tt t

Xm t e e e e

1 2 12 3t te t e tte e e

1 3 1 2 13 23t t te t e t e t

e e e

0 1 0

1 0e

Xm e

0 01 0 1 02 22 0

e e

Xm e e

3 0 2 0 0 3 23 0 3 3t

Xm e e e

To find the moments we set t = 0.

iii) Exponential Dist'n Xm tt

1

X

d tdm t

dt t dt

2 21 1t t

3 32 1 2Xm t t t

4 42 3 1 2 3Xm t t t

5 54 2 3 4 1 4!Xm t t t

1

!kk

Xm t k t

Thus

2

1

10Xm

3

2 2

20 2Xm

1 !

0 !kk

k X k

km k

2 33211

2! 3! !kk

Xm t t t t tk

The moments for the exponential distribution can be calculated in an alternative way. This is note by expanding mX(t) in powers of t and equating the coefficients of tk to the coefficients in:

2 31 11

11Xm t u u u

tt u

2 3

2 31

t t t

Equating the coefficients of tk we get:

1 ! or

!k

kk k

k

k

2

2t

Xm t e

The moments for the standard normal distribution

We use the expansion of eu.2 3

0

1! 2! 3! !

k ku

k

u u u ue u

k k

2 2 2

22

2

2 3

2 2 2

212! 3! !

t

kt t t

tXm t e

k

2 4 6 212 2 3

1 1 11

2 2! 2 3! 2 !k

kt t t t

k

We now equate the coefficients tk in:

2 222

112! ! 2 !

k kk kXm t t t t t

k k

If k is odd: k = 0.

2 1

2 ! 2 !k

kk k

For even 2k:

2

2 !or

2 !k k

k

k

1 2 3 4 2

2! 4!Thus 0, 1, 0, 3

2 2 2!

Summary

Moments

Moment generating functions

Moments of Random Variables

kk E X

-

if is discrete

if is continuous

k

x

k

x p x X

x f x dx X

The moment generating function

if is discrete

if is continuous

tx

xtX

Xtx

e p x X

m t E ee f x dx X

Examples

1 0,1,2, ,n xxn

p x p p x nx

1. The Binomial distribution (parameters p, n)

1n nt t

Xm t e p p e p q

0,1,2,!

x

p x e xx


1te

Xm t e

0

0 0

xe xf x

x


undefined

X

tm t t

t

2

21

2

x

f x e


2

2t

Xm t e

1 0

0 0

xx e xf x

x


Xm tt

6. The Chi-square distribution (degrees of freedom )

(

21

2 2

112

2

0

0 0

xx e xf x

x

2

2

12

12

1 2Xm t tt

1. mX(0) = 1

2 33212. 1

2! 3! !kk

Xm t t t t tk

0

3. 0k

kX X kk

t

dm m t

dt

Properties of Moment Generating Functions

1i.e. 0Xm

20Xm

30 , etcXm

Let lX (t) = ln mX(t) = the log of the moment generating function

Then 0 ln 0 ln1 0X Xl m

The log of Moment Generating Functions

1 X

X XX X

m tl t m t

m t m t

2

2 X X X

X

X

m t m t m tl t

m t

1

0 0

0X

XX

ml

m

2

2 22 12

0 0 0 0

0

X X X

X

X

m m ml

m

Thus lX (t) = ln mX(t) is very useful for calculating the mean and variance of a random variable

1. 0Xl

2 2. 0Xl

Examples1. The Binomial distribution (parameters p, n)

1n nt t

Xm t e p p e p q

ln ln tX Xl t m t n e p q

1 tX t

l t n e pe p q

10Xl n p np

p q

2

t t t t

Xt

e p e p q e p e pl t n

e p q

220X

p p q p pl n npq

p q


1te

Xm t e

ln 1tX Xl t m t e

tXl t e

tXl t e

0Xl

2 0Xl


undefined

X

tm t t

t

ln ln ln if X Xl t m t t t

11Xl t t

t

2

2

11 1Xl t t

t

22

1 1Thus 0 and 0X Xl l


2

2t

Xm t e

2

2ln tX Xl t m t

, 1X Xl t t l t

2Thus 0 0 and 0 1X Xl l


Xm tt

ln ln lnX Xl t m t t

1Xl t

t t

2

21 1Xl t tt

22

Hence 0 and 0X Xl l

6. The Chi-square distribution (degrees of freedom )

21 2Xm t t

ln ln 1 22X Xl t m t t

12

2 1 2 1 2Xl tt t

2

2

21 1 2 2

1 2Xl t t

t

2Hence 0 and 0 2X Xl l

Summary of Discrete Distributions

Name

probability function p(x)

Mean

Variance

Moment generating

function MX(t)

Discrete Uniform p(x) =

1N x=1,2,...,N

N+12

N2-112

et

N etN-1et-1

Bernoulli p(x) =

p x=1q x=0

p pq q + pet

Binomial p(x) =

N

x pxqN-x Np Npq (q + pet)N

Geometric p(x) =pqx-1 x=1,2,... 1p

qp2

pet

1-qet

Negative Binomial p(x) =

x-1

k-1 pkqx-k

x=k,k+1,...

kp

kqp2

pet

1-qet k

Poisson p(x) =

x

x! e- x=1,2,... e(et-1)

Hypergeometric

p(x) =

A

x

N-A

n-x

N

n

n

A

N n

A

N

1-AN

N-n

N-1 not useful

Summary of Continuous Distributions

Name

probability density function f(x)

Mean

Variance

Moment generating function MX(t)

Continuous Uniform

otherwise

bxaabxf

0

1)(

a+b2

(b-a)2

12 ebt-eat

[b-a]t

Exponential

00

0)(

x

xlexf

lx

1

12

t

for t <

Gamma

f(x) =

00

0)( f(x)

1

x

xexaG

l lxaa

2

t

for t <

2

d.f.

f(x) = (1/2)

(/2) x e-(1/2)x x ? 0

0 x < 0

1

1-2t /2

for t < 1/2

Normal f(x) =

1

2 e-(x-)2/22

2 et+(1/2)t22

Weibull

f(x) = x e-x x ? 0

0 x < 0

( )+1

( )+2 -[ ]( )+1

not avail.

Moment Generating Functions. Continuous Distributions The Uniform distribution from a to b.

Documents

Transcript of Moment Generating Functions. Continuous Distributions The Uniform distribution from a to b.