MAS113 Introduction to Probability and StatisticsStandard continuous probability distributions...

Standard continuous probability distributions

MAS113 Introduction to Probability and

Statistics

Dr Jonathan Jordan

School of Mathematics and Statistics, University of Sheffield

2019–20

Dr Jonathan Jordan MAS113 Introduction to Probability and Statistics

Standard continuous distributions

We consider three standard probability distributions forcontinuous random variables: the exponential distribution, theuniform distribution, and the normal distribution.

The exponential distribution

The exponential distribution is used to represent a ‘time to anevent’. Examples of ‘experiments’ that we might describeusing a exponential random variable are

a patient with heart disease is given a drug, and weobserve the time until the patient’s next heart attack;

a new car is bought and we observe how many miles thecar is driven before it has its first breakdown.

Definition

If a random variable X has an exponential distribution, withrate parameter λ, then its probability density function is givenby

fX (x) = λe−λx ,

for x ≥ 0, and 0 otherwise.

We writeX ∼ Exp(λ),

to mean “X has an exponential distribution with rateparameter λ”.

Definition

If a random variable X has an exponential distribution, withrate parameter λ, then its probability density function is givenby

fX (x) = λe−λx ,

for x ≥ 0, and 0 otherwise.We write

X ∼ Exp(λ),

to mean “X has an exponential distribution with rateparameter λ”.

C.d.f.

Theorem

Cumulative distribution function of an exponential randomvariableIf X ∼ Exp(λ), then for x > 0

FX (x) = 1− e−λx .

Valid p.d.f.

We can see that limx→∞ F (X )(x) = 1 (so that “FX (∞) = 1”).

So, as required of a p.d.f.,∫ ∞−∞

fX (x)dx =

∫ ∞0

λe−λxdx = 1.

Valid p.d.f.

We can see that limx→∞ F (X )(x) = 1 (so that “FX (∞) = 1”).

So, as required of a p.d.f.,∫ ∞−∞

fX (x)dx =

∫ ∞0

λe−λxdx = 1.

Mean and variance

Theorem

(Expectation and variance of an exponential random variable)If X ∼ Exp(λ), then

E (X ) =1

Var(X ) =1

Mean and variance

Theorem

(Expectation and variance of an exponential random variable)If X ∼ Exp(λ), then

E (X ) =1

Var(X ) =1

Lack of memory

Theorem

(The ‘lack of memory’ property of an exponential randomvariable)If X ∼ Exp(λ), then

P(X > x + a|X > a) = P(X > x).

Exponential random variables have the property that they‘forget’ how ‘old’ they are.

If the lifetime of some object has an exponential distribution,and the object survives from time 0 to time a, it will ‘carry on’as if it was starting at time 0.

Lack of memory

Theorem

P(X > x + a|X > a) = P(X > x).

Lack of memory

Theorem

P(X > x + a|X > a) = P(X > x).

Example

A computer is left running continuously until it first develops afault.

The time until the fault, X , is to be modelled with anexponential distribution.The expected time until the first fault is 100 days.

1 If X ∼ Exp(λ), determine the value of λ. What is thestandard deviation of X?

2 What is the probability that is the computer develops afault within the first 100 days?

3 If the computer is still working after 100 days, what is theprobability that it will still be working after 150 days?

Example

A computer is left running continuously until it first develops afault.The time until the fault, X , is to be modelled with anexponential distribution.

The expected time until the first fault is 100 days.

Example

A computer is left running continuously until it first develops afault.The time until the fault, X , is to be modelled with anexponential distribution.The expected time until the first fault is 100 days.

Example

Earthquakes

Example

Suppose the number of earthquakes Nt in an interval [0, t] hasa Poisson(φt) distribution, for any value of t.

Let T be the time until the first earthquake.What is the cumulative distribution function of T?What is the distribution of T?

Earthquakes

Example

Suppose the number of earthquakes Nt in an interval [0, t] hasa Poisson(φt) distribution, for any value of t.Let T be the time until the first earthquake.

What is the cumulative distribution function of T?What is the distribution of T?

Earthquakes

Example

Suppose the number of earthquakes Nt in an interval [0, t] hasa Poisson(φt) distribution, for any value of t.Let T be the time until the first earthquake.What is the cumulative distribution function of T?

What is the distribution of T?

Earthquakes

Example

Suppose the number of earthquakes Nt in an interval [0, t] hasa Poisson(φt) distribution, for any value of t.Let T be the time until the first earthquake.What is the cumulative distribution function of T?What is the distribution of T?

Probability density functions and histograms

In a probability density function area represents probability.

In a histogram area represents observed data.

We illustrate this connection using simulations of theexponential distribution.

R code

We can simulate a sample of n independent random valuesfrom an Exp(1) distibution and draw a histogram of themusing the R code

x<-rexp(n)

hist(x,freq=FALSE)

Results

With large values of n, the shape of the plot should show aclose resemblance to the probability density function ofExp(1), which is e−x for x > 0. For n = 10000:

Histogram of x

0 2 4 6 8 10

The uniform distribution

The uniform distribution is used to describe a random variablethat is constrained to lie in some interval [a, b], but has thesame probability of lying in any interval contained within [a, b]of a fixed width.

It is an important concept in probability theory, but it is lessuseful for modelling uncertainty in the real world:

It is not often plausible in real situations that all intervals ofthe same width are equally likely.

Definition

If a random variable X has a uniform distribution over theinterval [a, b], then its probability density function is given by

fX (x) =1

b − a,

for x ∈ [a, b], and 0 otherwise.

We writeX ∼ U[a, b],

to mean “X has a uniform distribution over the interval [a, b].”

Definition

If a random variable X has a uniform distribution over theinterval [a, b], then its probability density function is given by

fX (x) =1

b − a,

for x ∈ [a, b], and 0 otherwise.We write

X ∼ U[a, b],

to mean “X has a uniform distribution over the interval [a, b].”

Facts about uniform distributions

If X ∼ U[a, b], then

For x ∈ [a, b] FX (x) =x − a

b − a

E (X ) =a + b

Var(X ) =(b − a)2

Proofs on course website

For x ∈ [a, b] FX (x) =x − a

b − a

E (X ) =a + b

Var(X ) =(b − a)2

For x ∈ [a, b] FX (x) =x − a

b − a

E (X ) =a + b

Var(X ) =(b − a)2

Example

Let X ∼ U[−1, 1].

Calculate E (X ), Var(X ) and P(X ≤ −0.5|X ≤ 0).

Example

Let X ∼ U[−1, 1].Calculate E (X ), Var(X ) and P(X ≤ −0.5|X ≤ 0).

Gaussian integral

The normal distribution is very important distribution in bothprobability and statistics.

Before studying it, we first introduce the Gaussian integral:∫ ∞−∞

e−x2

dx =√π. (1)

(A proof is given in Applebaum (2008), though you will needto understand changes of variables within double integration).

Gaussian integral

e−x2

=√π. (1)

Gaussian integral

e−x2

dx =√π. (1)

Gaussian integral

e−x2

dx =√π. (1)

Standard normal

We first define the standard normal distribution, beforeconsidering the more general case.

Definition

If a random variable Z has a standard normal distribution,then its probability density function is given by

fZ (z) =1√2π

(−z2

We writeZ ∼ N(0, 1),

to mean “Z has the standard normal distribution.”

Standard normal

Definition

fZ (z) =1√2π

(−z2

Standard normal

Definition

fZ (z) =1√2π

(−z2

Valid p.d.f.

We can confirm that this is a valid p.d.f.

Starting with the Gaussian integral, we make the substitutionx = z/

√2, so that dx

dz= 1/

√2:∫ ∞

−∞

1√πe−x

dx = 1

and the substitution then gives∫ ∞−∞

1√2π

e−z2

2 dz = 1,

Valid p.d.f.

√2, so that dx

dz= 1/

√2:∫ ∞

−∞

1√πe−x

dx = 1

1√2π

e−z2

2 dz = 1,

Valid p.d.f.

√2, so that dx

dz= 1/

√2:∫ ∞

−∞

1√πe−x

dx = 1

1√2π

e−z2

2 dz = 1,

C.d.f.

The c.d.f. is

FZ (z) = P(Z ≤ z) =

−∞

1√2π

{−t2

We can’t evaluate this integral analytically, and have to usenumerical methods.

We will see how to calculate FZ (z) using R.

C.d.f.

The c.d.f. is

FZ (z) = P(Z ≤ z) =

−∞

1√2π

{−t2

C.d.f.

The c.d.f. is

FZ (z) = P(Z ≤ z) =

−∞

1√2π

{−t2

Notation

The notation Φ is used to represent the cdf, and φ torepresent the pdf:

Φ(z) := P(Z ≤ z) =

−∞

1√2π

(−t2

φ(z) :=1√2π

(−z2

dzΦ(z) = φ(z).

Notation

Φ(z) := P(Z ≤ z) =

−∞

1√2π

(−t2

φ(z) :=1√2π

(−z2

dzΦ(z) = φ(z).

Notation

Φ(z) := P(Z ≤ z) =

−∞

1√2π

(−t2

φ(z) :=1√2π

(−z2

dzΦ(z) = φ(z).

Notation

Φ(z) := P(Z ≤ z) =

−∞

1√2π

(−t2

φ(z) :=1√2π

(−z2

dzΦ(z) = φ(z).

Φ(z) and Φ(−z)

Theorem

(Relationship between Φ(z) and Φ(−z).)

Φ(−z) = 1− Φ(z).

Quantile function

We denote the quantile function by Φ−1.

If we want z such that P(Z ≤ z) = α, then we write

Φ(z) = α

z = Φ−1(α).

Quantile function

Φ(z) = α

z = Φ−1(α).

Quantile function

Φ(z) = α

z = Φ−1(α).

Mean and variance

Theorem

(Expectation and variance of a standard normal randomvariable)If Z ∼ N(0, 1), then

E (Z ) = 0

Var(Z ) = 1.

Mean and variance

Theorem

(Expectation and variance of a standard normal randomvariable)If Z ∼ N(0, 1), then

E (Z ) = 0

Var(Z ) = 1.

The normal distribution: the general case

The standard normal distribution is one example of the familyof normal distributions, in which the mean is 0 and thevariance is 1, but, in general, normal random variables canhave any values for the mean and variance

(though variancescannot be negative, of course).

The normal distribution: the general case

The standard normal distribution is one example of the familyof normal distributions, in which the mean is 0 and thevariance is 1, but, in general, normal random variables canhave any values for the mean and variance (though variancescannot be negative, of course).

The normal distribution: examples

Normal distributions are used very widely in many situations,for example:

many physical characteristics of humans and otheranimals, for example the distribution of heights of femalesin a particular age group, can be well represented with anormal distribution;

scientists often assume that ‘measurement errors’ arenormally distributed;

normal distributions are commonly used in finance tomodel changes in stock prices (though not alwayssensibly!).

Definition

If a random variable X has a normal distribution with meanµ and variance σ2, then its probability density function is givenby

fX (x) =1√

2πσ2exp

{− 1

2σ2(x − µ)2

We writeX ∼ N(µ, σ2),

to mean “X has a normal distribution with mean µ andvariance σ2.”

Definition

If a random variable X has a normal distribution with meanµ and variance σ2, then its probability density function is givenby

fX (x) =1√

2πσ2exp

{− 1

2σ2(x − µ)2

}We write

X ∼ N(µ, σ2),

to mean “X has a normal distribution with mean µ andvariance σ2.”

Standard as a special case

Immediately, we can see that by setting µ = 0 and σ2 = 1 in(12), we get the standard normal p.d.f. in (9).

Transformation

Theorem

(Definition of a general normal random variable viatransformation of a standard normal random variable)Let Z ∼ N(0, 1), and define

X = µ + σZ .

Then E (X ) = µ, Var(X ) = σ2 and

X ∼ N(µ, σ2),

Transformation

Theorem

(Definition of a general normal random variable viatransformation of a standard normal random variable)Let Z ∼ N(0, 1), and define

X = µ + σZ .

Then E (X ) = µ, Var(X ) = σ2 and

X ∼ N(µ, σ2),

Summary

It’s worth stating again the relationship between a standardnormal random variable Z and a ‘general’ normal randomvariable X .

Given Z ∼ N(0, 1), we can obtain X ∼ N(µ, σ2) bytransforming Z :

X = µ + σZ .

Given X ∼ N(µ, σ2), we can obtain Z ∼ N(0, 1) bytransforming X :

Z =X − µσ

and we refer to transforming X to get a standard N(0, 1)random variable as standardising X .

Summary

It’s worth stating again the relationship between a standardnormal random variable Z and a ‘general’ normal randomvariable X .

Given Z ∼ N(0, 1), we can obtain X ∼ N(µ, σ2) bytransforming Z :

X = µ + σZ .

Given X ∼ N(µ, σ2), we can obtain Z ∼ N(0, 1) bytransforming X :

Z =X − µσ

and we refer to transforming X to get a standard N(0, 1)random variable as standardising X .

Visualising the mean and variance

By plotting the density function, we can see the effect ofchanging the value of µ and σ2 and so interpret theseparameters more easily.

Starting with the mean, the maximum of the p.d.f. is at x = µ.

The variance parameter σ2 determines how ‘spread out’ thep.d.f. is. If we increase σ2, whilst leaving µ unchanged, thepeak of the p.d.f. is in the same place, but we get a flattercurve.

The normal distribution in R

R will calculate the p.d.f., c.d.f. and quantile functions, andwill also generate normal random variables.

Note that in R, we specify the standard deviationrather than the variance.

Calculate the pdf: dnorm(x,mu,sigma)

Calculate the cdf: pnorm(x,mu,sigma)

Invert the c.d.f. to find the α quantile:qnorm(alpha,mu,sigma)

Generate m random observations from a normaldistribution: rnorm(m,mu,sigma)

Example

If X ∼ N(3, 4), what is the 25th percentile of the distributionof X?

Paternity suit

Example

(from Ross, 2010).An expert witness in a paternity suit testifies that the length,in days, of human gestation is approximately normallydistributed, with mean 270 days and standard deviation 10days.

The defendant has proved that he was out of the countryduring a period between 290 days before the birth of the childand 240 days before the birth of the child, so if he is thefather, the gestation period must have either exceeded 290days, or been shorter than 240 days.How likely is this?

Paternity suit

Example

(from Ross, 2010).An expert witness in a paternity suit testifies that the length,in days, of human gestation is approximately normallydistributed, with mean 270 days and standard deviation 10days.The defendant has proved that he was out of the countryduring a period between 290 days before the birth of the childand 240 days before the birth of the child, so if he is thefather, the gestation period must have either exceeded 290days, or been shorter than 240 days.

How likely is this?

Paternity suit

Example

(from Ross, 2010).An expert witness in a paternity suit testifies that the length,in days, of human gestation is approximately normallydistributed, with mean 270 days and standard deviation 10days.The defendant has proved that he was out of the countryduring a period between 290 days before the birth of the childand 240 days before the birth of the child, so if he is thefather, the gestation period must have either exceeded 290days, or been shorter than 240 days.How likely is this?

The two-σ rule

For a standard normal random variable Z ,

P(−1.96 ≤ Z ≤ 1.96) = 0.95.

Since E (Z ) = 0 and Var(Z ) = 1, the probability of Z beingwithin two standard deviations of its mean value isapproximately 0.95 (ie P(−2 ≤ Z ≤ 2) = 0.9545 to 4 d.p.).

The two-σ rule

For a standard normal random variable Z ,

P(−1.96 ≤ Z ≤ 1.96) = 0.95.

Since E (Z ) = 0 and Var(Z ) = 1, the probability of Z beingwithin two standard deviations of its mean value isapproximately 0.95 (ie P(−2 ≤ Z ≤ 2) = 0.9545 to 4 d.p.).

The two-σ rule: general case

If we now consider any normal random variable X ∼ N(µ, σ2),the probability that it will lie within a distance of two standarddeviations from its mean is approximately 0.95.

This is straightforward to verify:

(on board)

Convention

In Statistics, there is a convention of using 0.05 as a thresholdfor a ‘small’ probability, though the choice of 0.05 is arbitrary.

However, the two-σ rule is an easy to remember fact aboutnormal random variables, and can be a useful yardstick invarious situations.

Convention

In Statistics, there is a convention of using 0.05 as a thresholdfor a ‘small’ probability, though the choice of 0.05 is arbitrary.

However, the two-σ rule is an easy to remember fact aboutnormal random variables, and can be a useful yardstick invarious situations.

MAS113 Introduction to Probability and StatisticsStandard continuous probability distributions...

Documents

Transcript of MAS113 Introduction to Probability and StatisticsStandard continuous probability distributions...