Statistical inference

Distribution of the sample mean

• Take a random sample of n independent observations from a population.

• Calculate the mean of these n sample values. This is know as the sample mean.

• Repeat the procedure until you have taken all possible samples of size n, calculating the sample mean of each.

• Form a distribution of all the sample means.

• The distribution that would be formed is called the sampling distribution of means.

In many cases we need to infer information about a population based on the values obtained from random samples of the population.

Consider the following procedure:

Mean and variance of the sampling distribution of means

2

.

Consider a population in which and .

Suppose we take a total of samples, each of independent observations, on the random variable

Each sample will have a first observation, a second

X E X Var X

m n X

observation, and so on. Denote by , the random

variable "the th observation".iX

i

1 2

1 2

1 1 1

Then, the sample distribution of means is

n

n

X X XX

n

X X Xn n n

1 2

1 2

1 2

1 1 1

1 1 1

1 1 1

1 1 1

(since )

n

n

n

E X E X X Xn n n

E X E X E Xn n n

E X E X E X E aX aE Xn n n

n n n

1 2

1 2

21 22 2 2

2 2 2

2

1 1 1

1 1 1

1 1 1

1 1 1

(since )

n

n

n

Var X Var X X Xn n n

Var X Var X Var Xn n n

Var X Var X Var X Var aX a Var Xn n n

n n n

n

2

The standard deviation of the sampling distribution is , sometimes written .

This is known as the .

n n

standard error of the mean

The distribution of for normal X X

2

2, ,If then X N X Nn

Read Example 9.13 & 9.14, pp.438-440

The distribution of for non-normal X X

Central limit theorem

2

2

,

For samples taken from a non-normal population with mean and variance ,

by the Central Limit Theorem, is approximately normal and

provided

X

X Nn

30 that the sample size, , is large ( , say).n n

This theorem holds when the population of X is discrete or continuous!!

Read Example 9.15, pp.442-443

Do Exercise 9C, pp.443-444

Unbiased estimates of population parameters

Suppose that you do not know the value of a particular population parameter of a distribution. It seems sensible to take a random sample from the distribution and use it in some way to make estimates of unknown parameters.

The estimate is unbiased if the average (or expectation) of a large number of values taken in the same way is the true value of the parameter.

Point estimates,

ˆ

ˆ

If the random sample taken is of size

the best unbiased estimate of , the population mean, is where

( is the mean of the sample)

the best unbiased estimate

n

xx x

n

2 2

2 2 2

ˆ

ˆ1

of , the population variance, is where

( is the variance of the sample) n

s sn

22

2 2 21ˆ ˆ

1 1Alternatively, use or

xx xx

n n n

Read Example 9.17, p.448

Unbiased estimates of population parameters

Interval estimates

Another way of using a sample value to estimate an unknown population parameter is to construct an interval, known as a confidence interval. This is an interval that has a specified probability of including the parameter.

( , )90% 95% 99%

The interval is usually written , with and known as .The probabilities most often used in confidence intervals are , and .

a b a b confidence limits

95%( , )

0.95

For example, to work out a confidence interval for the unknown mean of a particular population, we construct an interval such that

The interval constructed uses

a b

P a b

the value of the mean, , of a random sample of

size taken from the population.x

nThree questions

• Is the distribution of the population normal or not?

• Is the variance of the population known?

• Is the sample size large or small?

Confidence interval for µ(normal population, known population variance, any sample size)

The goal is to calculate the end-values of a 95% confidence interval. We can adapt this approach for other levels of confidence.

2

, ,For random samples of size , if , then .n X N X Nn

0,1Standardizing, , where .X

Z Z Nn

95% 95%For a confidence interval, we find the -values between which of thedistribution lies.

z

1

0.975

(0.975)

1.960

1.96

The values of are .

P Z z

z

z

1.96 1.96 0.95

1.96 1.96

So

i.e.

XP

n

P X Xn n

2 95%

1.96 , 1.96

If is the mean of a random sample of any size taken from a normal population with

known variance , then a confidence interval for is given by

x n

x xn

n

Read Example 9.19, pp.451-452

Confidence interval for µ(normal population, known population variance, any sample size)

This computer simulation shows 100 confidence intervals constructed at the 95% level. On average 5% do not include µ.

In other words, on average, 95% of the intervals constructed will include the true population mean.

Critical z-values in confidence intervals

The z-value in the confidence interval is known as the critical value.

90%

1.645 , 1.645

confidence interval

x xn n

95%

1.96 , 1.96

confidence interval

x xn n

99%

2.576 , 2.576

confidence interval

x xn n

Confidence interval for µ(non-normal population, known population variance, large sample size)

30In this case, since the sample size is large (say, ), the Central Limit Theorem

may be used.

n

2

,i.e. is approximately normal and .X X Nn

2

30

95%

If is the mean of a random sample of size , where is large ( ),

taken from a non-normal population with known variance , then a

confidence interval for is given by

x n n n

1.96 , 1.96 x xn n

• for a given sample size, the greater the level of confidence, the wider the confidence interval;

• for a given confidence level, the smaller the interval width, the larger the sample size required;

• for a given interval width, the greater the level of confidence, the larger the sample size required.

Note

Read Examples 9.20 – 9.22, pp.454-457

Confidence interval for µ(any population, unknown population variance, large sample size)

2 2ˆIn this case, is used as an estimate for . Ideally, the distribution of

should be normal, but an approximate confidence interval may also be given

when the distribution of is not normal.

X

X

2 2 2

2 2

30 95%

ˆ ˆ1.96 , 1.96

ˆ1

1ˆ

1

Provided that is large, ( , say), a confidence interval for is

,

where , ( is the sample variance)

or

n n

x xn n

ns s

n

xx

n

2

.n

Read Examples 9.23 & 9.24, pp.458-459

Do Exercise 9e, pp.460-461

Confidence interval for µ(normal population, unknown population variance, small sample size)

2

30

When calculating confidence intervals, we have already encountered

the situation when ( ) are taken from a population

with .

n

large samples normal

unknown variance

0,1ˆ

For large samples: , where X

Z Z Nn

30ˆ

But if the sample size is small ( ), no longer has a normal distribution.X

nn

For small samples:

ˆ, where has a -distribution.

XT T t

n

Confidence interval for µ(normal population, unknown population variance, small sample size)

The t-distribution

The distribution of is a member of a family of -distributions. All -distributions aresymmetric about zero and have a single parameter which is a positive integer.

T t t

( )

is known as the of the distribution and we write

or .T t T t

number of degrees of freedom

2 10 0,1

0,1

This diagram shows and along with .

Note that as increases, gets closer andcloser to .

t t N

t vN

ˆ( 1)

For samples of size , it can be shown that

follows a -distribution with degreesof freedom.

n

XT

nt n

Probability density function of the t-distribution

12 2

12

, 1

12

f t n

t

1

0

z tz t e dt

1 !, n n n

Confidence interval for µ(normal population, unknown population variance, small sample size)

2

2

30

95%

If and are the mean and variance of a small sample ( ) from a normal population

with unknown mean and unknown variance , then a confidence interval for is given by

x s n

2 2ˆ ˆˆ,

1

1 0.95

, 95% 1

, where

and is the value from a distribution such that ,

i.e. encloses of the distribution.

nx t x t s

nn n

t t n P t T t

t t t n

To find the required value of , known

as the , the -distribution

tables are used.

t

tcritical value

95%

0.975

90%

0.95

99%

0.995

For a confidence interval, look

under column .

For a confidence interval, look

under column .

For a confidence interval, look

under column .

Read Examples 9.26 & 9.27, pp.466-468 Do Exercise 9f, p.468

Confidence intervals for µ

Read Examples 9.30, pp. 474-475

Read Examples 9.32 & 9.33, pp. 476-478

Do Exercise 9h, Q1-Q4, Q7, Q10-Q12, Q15, Q17-Q21, pp.478-481

Hypothesis testing- exampleA machine fills ice-packs with liquid. The volume of liquid follows a normal distribution with mean 524 ml and standard deviation 3 ml. The machine breaks down and is repaired. It is suspected the machine now overfills each pack, so a sample of 50 packs is inspected. The sample mean is found to be 524.9 ml. Is the machine over-dispensing?

Is the sample mean high enough to say that the mean volume of all packs has increased? A hypothesis (or significance) test enables a decision to be made on this question.

2

.

, 3

Let be the volume of liquid now dispensed. Let the unknown mean of be

Assume the standard deviation has remained unchanged i.e , where .

X X

X N

0

524

: 524

The hypothesis is made that is ml, i.e. the mean is . This isknown as the , and is written H

0null hypothesis

unchangedH

1

524524

Since it is suspected the mean has , the , , isthat the mean is ml, written : H

1alternative hypothesisincreasedgreater than

H

The test is now carried out.

Hypothesis testing- example50

To carry out the test, the focus shifts from , the volume of liquid in a pack, to ,the mean volume of a sample of packs. In this test, is known as the and its distribution is need

X XX test statistic

ed. 2

2

,

3524 524,

50

We know, for a sample of size , . The test starts by assuming

the null hypothesis is true, so . Hence .

n X Nn

X N

The result of the test depends on the location in the sampling distribution of the test value 524.9 ml.

If it is close to 524 then it is likely to have come from a distribution with mean 524 ml and there would not be enough evidence to say the mean volume has increased.

If it is far away from 524 (i.e. in the upper tail of the distribution) then it is unlikely to have come from a distribution with mean 524 ml and the mean is likely to be higher that 524 ml.

A decision needs to be made about the cut off point, , knownas the , which indicates the boundary of the regionwhere values of would be considered from 524 mland therefo

c

xcritical value

too far awayre . The region is known as the

or .critical region rejection regionunlikely to occur

Hypothesis testing- example524 524

Here, if lies in the critical region, we , (that the meanis ml), , (that the mean is greater than ml).

If does not lie

x

x

0

1

reject the null hypothesisin favour of the alternative hypothesis

HH

0 in the critical region, there is not enough evidence to reject , so is . (In this case, is known as the .)

Hx c0 accepted acceptance regionH

%For a significance level of , if the sample mean lies in the critical (or rejection)region, the result is said to be .

%significant at the level

0,1 5%Since the distribution of is normal, instead of finding , the critical value, it is possibleto use the standardized distribution and find the -value that gives in the upper tail.

X c xN z

0.05 1.645We require , and the standard normal table yields .P Z z z 0 1.645

524

3 50

We can now state the : reject if , where

.

Note the rejection criterion should be established the sample is taken.

H zx x

zn

rejection criterion

before

0

1

524.9 524524.9 2.12 1.645

3 50For the sample taken , so . Since , is rejected

in favour of .

x z z H

H

Conclusion: There is evidence, at the 5% level, that the mean volume of liquid being dispensed by the machine has increased.

One-tailed and two-tailed tests

0Say that the null hypothesis is .

1

:In a , the alternative hypothesis looks for an increase or adecrease in

H

one - tailed test

1 0For an increase, is and the critical region is in the .H upper tail

1 0For a decrease, is and the critical region is in the .H lower tail

1

1 0

In a , the alternative hypothesis looks for a change withoutspecifying an increase or decrease and is . The critical region is intwo parts:

HH

two - tailed test

Critical z-values and rejection criteria

Example: At the 1% level

One-tailed tests

Two-tailed tests

Stages in the hypothesis test

Testing the mean, µ, of a population(normal population, known variance, any sample size)

2

2

0 ,

When testing the mean of a normal population with known variance

for samples of size , the test statistic is

, where .

In standardized form, the test statis

X

n

X X Nn

0 0,1

tic is

where .

XZ Z N

n

Read Examples 11.1 & 11.2, pp.514-517

Testing the mean, µ, of a population(non-normal population, known variance, large sample size)

2

2

0 ,

When testing the mean of a non-normal population with known variance ,

provided that the sample size is large, the test statistic is

, where is approximately normal,

X

n

X X X Nn

0 0,1

.

In standardized form, the test statistic is

where .

XZ Z N

n

Read Examples 11.3, pp.517-518

Testing the mean, µ, of a population(any population, unknown variance, large sample size)

2

2

0

ˆ,

When testing the mean of a non-normal population with unknown

variance , provided that the sample size is large, the test statistic is

, where and

X

n

X X Nn

2 2 2

2

2 2

0

ˆ1

1ˆ

1

0,1ˆ

, ( is the sample variance)

or .

In standardized form, the test statistic is

where .

ns s

n

xx

n n

XZ Z N

n

Read Examples 11.4, pp.519-520

Do Exercise 11A, Q1-Q6, Q8-Q10, Q13, pp.522-523

Testing the mean, µ, of a population(normal population, unknown variance, small sample size)

2

0 1ˆ

ˆ

When testing the mean of a normal population with unknown

variance , when the sample size is small, the test statistic is

, where and with

X

n

XT T T t n

n

2 2 2

2

2 2

1

1ˆ

1

, ( is the sample variance)

or .

ns s

n

xx

n n

Read Examples 11.6 &11.7, pp.524-526

Do Exercise 11b, pp.527-528

1 2Testing the difference in means, ,

of two normal populations

1 2 1 2

2 21 1 1 2 2 2, ,

Consider two normal populations and with unknown means, and .

So and and we want to test the difference

between the means of these populations.

X X

X N X N

0 1 2

1 1 2 1 2 1 2 )

The hypothesis might be:

:

: (or or

H

H

0 1 2

0 1 2

:

: 0

Often the test involves the null hypothesis that the means are the same i.e.

or .

H

H

1 1 1

2 2 2

.

.

Take a random sample of size from and find its sample mean

Take a random sample of size from and find its sample mean

n X x

n X x

1 2

1 2 1 2 1 2

2 21 2

1 2 1 21 2

The test statistic is and to consider this sampling distribution we use

X X

E X X E X E X

Var X X Var X Var Xn n

1 2

( )

Testing of two normal populations

2 2

2variances and known

2 21 2

2 21 2

1 2 1 2 1 21 2

1 2 1 2

2 21 2

1 2

,

0,1

If the variances and are known, the test statistic is

where .

In standardized form, the test statistic is

where .

X X X X Nn n

X XZ Z N

n n

2 21 2

1 2 1 21 2

95% 1.96The confidence limits for are .x xn n

1 2

( )

Testing of two normal populations

2

common known variance

2 2 21 2

2

21 2 1 2 1 2

1 2

1 1,

If there is a common population variance ( )

and is known, then the test statistic is

where .

In standardized form, the test statistic is

X X X X Nn n

Z

1 2 1 2

1 2

0,11 1

where .X X

Z N

n n

1 2 1 21 2

1 195% 1.96The confidence limits for are .x x

n n

2Pooled two-sample estimate of 2

2

2 22 2 21 1 2 2

1 21 2

ˆ

ˆ2

If the common population variance, , is unknown, then an unbiased

estimate, , is used instead. This is known as a

, where

( and are n s n s

s sn n

pooled two- sample

estimate

the sample variances)

2

2 22 22 2

1 1 2 2 1 22

1 2 1 2

ˆ

1 1

ˆ2 2

An alternative format for is

i i i ix x x x

x x x x n n

n n n n

1 2The distribution of depends on whether the samples taken

are large or small.

X X

1 2

( )

Testing of two normal populations ,

2

common unknown variance large sample

1 2

21 2 1 2 1 2

1 2

1 2

1 1ˆ,

For large samples the distribution of is approximately normal.

The test statistic is

where .

In standardized form, the test statistic is

X X

X X X X Nn n

X XZ

1 2

1 2

0,11 1

ˆ

where .Z N

n n

1 2 1 21 2

1 1ˆ95% 1.96The confidence limits for are .x xn n

1 2

( )

Testing of two normal populations ,

2

common unknown variance small sample

1 2

1 2 1 21 2

1 2

21 1

ˆ

For small samples the standardized form of the distribution of

follows a -distribution. The test statistic is

where .

X X

t

X XT T t n n

n n

1 2 1 21 2

1 2

1 1ˆ95%

0.975 2

The confidence limits for are ,

where is such that for .

x x tn n

t P T t t n n

Read Examples 11.11 - 11.15, pp.536-542

Do Exercise 11d, Sections A & B, pp.543-546

Paired samplesIt is widely thought that people’s reaction times are shorter in the morning and increase as the day goes on. A light is programmed to flash at random intervals and the experimental subject has to press a buzzer as soon as possible and the delay is recorded.

Experiment 1: Two random samples of 40 students are selected from the school register. One of these samples, chosen at random, uses the apparatus during the first period of the day, while the second sample uses the apparatus during the last period of the day. The means of the two samples are compared.

Experiment 2: A random sample of 40 students is selected from the school register. Each student is tested in the first period of the day, and again in the last period. The difference in reaction times between the two periods for each student is calculated. The mean difference is compared with zero.

Experiment 1 requires a standard two-sample comparison of means, assuming a common variance. There is nothing wrong with this procedure but we could be misled. Firstly, suppose that all the bookworms were in the first sample and all the athletes in the second- we might conclude that reaction times decrease over the course of the day! More subtly, the variations between the students may be much greater than any changes in individual students over the time of day; these changes may pass unnoticed.

In Experiment 2 the variability between the students plays no part. All that matters is the variability of the changes within each student’s readings. The problems with Experiment 1 have vanished! Experiment 2 is a paired-sample test.

The paired-sample comparison of means

Let denote the mean of the distribution of differences between the paired values.d

0

1

: 0

: 0 0 0

Let

(or or as appropriate)d

d d d

H

H

1 2

0

, , ,

0

We have a single set of pairs of values and are interested in the ,

which assuming , are a random sample from a population with mean .

An unbiased estimate of the unknown vari

nn d d d

H

differences

22 21ˆ

1

ance of the population is given by

i

d i

dd

n n

We have created a single sample situation, so previous methods now apply!

95%For example, if the differences can be assumed to have a normal distribution,or if is sufficiently large that a normal approximation can be used, then a confidence interval for is provided by:d

n2 2ˆ ˆ

1.96 , 1.96 , where id dd

d d dn n n

1Alternatively, if the differences can be presumed to have a normal distribution,

but is small, then a distribution can be used.n t n

Paired sample- an example0.001

Suppose that Experiment 2 on the reaction times is carried out, with the following results(in units of seconds):

1%We analyse these data at the significance level.2 2ˆ266 9574 6.650 200.1308The summary statistics are , so that and i i dd d d

0

1

: 0

: 0

The hypotheses being compared are:

d

d

H

H

2

6.6502.97

2.237ˆ

The test statistic is , given by:

d

z

dz

n

02.97 2.3261%

Since , is rejected.There is evidence, at the level,that reaction times increase through the day.

H

Distinguishing between thepaired-sample and two-sample cases

• If the two samples are of unequal size then they are not paired.

• Two samples of equal size are paired only if we can be certain that each observation from the second sample is associated with a corresponding observation from the first sample.

Questions

A significance test for the

product-moment correlation coefficient

0

1

: 0

: 0

(no correlation)

(positive correlation)

H

H

6n

0

1

: 0

: 0

(no correlation)

(negative correlation)

H

H

8n

0

1

: 0

: 0

(no correlation)

(some correlation)

H

H

10n

Read Examples 13.1, pp.603-604 Do Exercise 13a, p.604