Chapter 7Sampling and Point Estimation

Sample This

Chapter 7A

This Week in Prob & Stat

today

fine print warning: while today’s presentation is mostly conceptual,Thursday’s presentation will be much more mathematical.

7-1 Introduction

• The field of statistical inference consists of those methods used to make decisions or to draw conclusions about a population.

• These methods utilize the information contained in a sample from the population in drawing conclusions.

• Statistical inference may be divided into two major areas:

• Parameter estimation

• Hypothesis testing

Fundamental Problem: Given that X1, X2, …, Xn is a random samplefrom some unknown population, what can be said about the population?For example, what is the distribution, mean, variance, median, range, etc.

The Big Picture Again

population

descriptivestatistics

parameter(e.g. mean)

sample

descriptivestatistics

statistic(e.g. sample

mean)

probabilitytheory

deduction

induction (inferential statistics)

Statistics and Sampling Statistical Inference:

Draw conclusions about a population based on sample. Hypothesis tests and parameter estimation.

Population: Generally impossible or impractical to observe

an entire population. Be aware that population may change over

time. Sample:

A subset of observations from a population. Must be representative of the population. Must be chosen randomly to avoid bias.

Parameter Estimation

a population

Estimators

Sampling – A Pictorial Presentation

X

f(x)Population

Random SampleX1, X2, …, Xn

1

n

ii

XX

n

Sample ( ) XStd dev X

n

2

Xi ~ Population(,2)

Sampling Distributions

The probability distribution of a statistic is called a sampling distribution.

- Definition makes sense. Statistic is a property of a sample from a population.

- Depends on the population distribution, sample size, and method of sample selection.

- Key statistics are things like the sample mean, variance, proportion, and difference of two means.

Definition of a Statistic

21

21

1

222

1

ˆˆ

/ˆ

)(1

1ˆ

1ˆ

pp

xx

nxp

xxn

s

xn

x

n

ii

n

ii

Statistic – any function of the observations in a random sample. Examples of point estimates:

A Sampling Distribution is the probability distribution of a statistic.

Sampling Distributions cont’d

nn

n

n

XXXX

X

X

n

2

2

2222

21

... and

... that so

...

If the Xi have a normal distribution, then so does the sample mean. The Xi are I.I.D.R.V.

7.2 Sampling Distributions and the Central Limit Theorem

Statistical inference is concerned with making decisions about a population based on the information contained in a random sample from that population.

Definitions:

The Central Limit Theorem

Figure 7-1

Distributions of average scores from throwing dice. [Adapted with permission from Box, Hunter, and Hunter (1978).]

What does it taketo become normal?

Who are you calling

normal?

More Normalcy

The CLT in Action

Ten people from a population having a mean weight of 190 lb. with a variance of 400 lb2 get on an elevator having a weight capacity of 2000 lb. What is the probability that their average weight exceeds 200 lb. and they all fall to their death?

2approx 190, 400 /10

200 190Pr 200 Pr Pr 1.581 .0569

/ 20 / 10

x xX n

XX z

n

Some Normal Thoughts on the Central Limit Theorem (CLT)

CLT tells us that a distribution of means will always be nearly normal if the sample is large enough.

Heights of adults are a result of both genetic and environmental factors. they are polygenic – influenced by many different

genes also many environmental factors; e.g. nutrition and

childhood diseases eventual height is then an average sample from the

large population of “height factors” Height therefore has a normal distribution

The tall and the short of it.

However All is not Normal

Weights of individuals are not normally distribution It is not just the cumulative result of many small

factors In addition, there may be one or two dominant causes

of obesity; e.g. a glandular disturbance In a similar manner, income is not normally

distributed as a result of a dominant factor – e.g. inherited wealth

On the other hand, mental test scores tend to be normally distributed due to many determinants: e.g. genetics and long-

term environmental conditions

CLT Revisited

The Main Result (of all time!):

1 2

2

1

If , ,..., is a random sample of size taken

from a population (finite or infinite) with mean and

finite variance , and if Y= , the

limiting form of the distribution of

n

n

ii

X X X n

X

as , is the standard normal distribution.

y

y

YZ

n

The sum of all Random Variables

A doctor spends an average (mean) of 20 minutes with each patient with a standard deviation of 8 minutes. Today’s appointment book shows 10 patients scheduled this morning (8 – 12).

The good doctor has a luncheon appointment at noon before her afternoon golf outing.

What is the probability she will make the luncheon on time?

10

1

( 200, 640)

240 200Pr{ 240} Pr Pr{ 1.581} .9431

25.3

i y yi

y

y

Y X n

YY Z

Difference in Sample Means

Approximate Sampling Distribution

More Mean Differences

1 2

1 2 1 2 1 2

2 22 1 2

1 2 1 2

1 2 1 2

2 21 1 2 2

( )

/ /

y

y

y

y

Y X X

E Y E X X E X E X

V Y V X X V X V Xn n

Y X XZ

n n

2 21 1 1 2 2 2( , ); ( , )X n X n

A Normal Difference Example

The section 1 class in ENM 661 consisting of 24 students had an average score of 82.7 on their midterm while section 2 consisting of 16 students scored an average of 81.4.

What is the probability that their average scores would differ by at least 1.3 if 1 - 2 = 0. Assume the population standard deviations are known where 1 = 10 and 2 = 12.

1 2 1 2

2 21 1 2 2

82.7 81.4 0( ).358

100 144/ /24 16

Pr .358 .36

X Xz

n n

Z

Now begins the discussion on point estimation

The discussion on the central limit theorem has now ended.

Definition of Point Estimate

22 vs., vs. sSorxX

s2 is a population parameter, S2 is a point estimator of s2. The estimate of S2 is s2. S2 has a sampling distribution. But s2 does not – it is just a number.

Point Estimate

is a point estimate of some population parameter of a statistic .

is a point estimator of . After a sample has been selected takes on a particular value .

is a random variable, is not, e.g.

Properties of Estimators

What makes a good estimator? What is the best estimator for a population parameter?

• Bias - does it hit the target?

• Variance – estimate is based on a sample

• Standard Error and Estimated Standard Error

• Mean Squared Error and Efficiency

• Consistency – how does the estimator behave as the sample size increases?

• Sufficiency – does the estimator use all the information that is available?

Bias of the Estimator

Def: The point estimator is an unbiased estimator for the parameter if E. If the estimator is not unbiased, then the difference E is called the bias of the estimator .

Is the sample mean unbiased?

1

1 1

1 1[ ] [ ]

n

i n ni

i ii i

Xn

E X E E X E Xn n n n

7-3 General Concepts of Point Estimation

7-3.1 Unbiased Estimators

Definition

Example 7-1

Example 7-1 (continued)

A Biased Estimator

Define an estimator for the population variance to be:

2 2 2 2

1 1

2 2 2 2 2

1 1

22 2

1

22 2 2

22 2

1 1( )

1

n n

i ii i

n n

i ii i

S X X X nXn n

E nS E X nX E X E nX

nE X nn

nn

nn n

n

2 2 2using: [ ]E X

22 2 2

2 2 2

1

1

nE S

n n

nE S E S

n

7-3.2 Variance of a Point Estimator

Definition

Figure 7-5 The sampling distributions of two unbiased estimators

.ˆˆ21 and

Variance of Estimator

Sample mean is the MVUE for the population mean for a population with normal distribution.

Generally, the stat package you use is making the reasonable choices for you.

Example of bad choice: sample size n=2Method 1: estimate mean as (X1 + X2)/2Method 2: estimate mean as (X1 + 2X2)/3

Variance of method 1 is 2/2 Variance of method 2 is 52/9

7-3.2 Variance of a Point Estimator

I just knew it was going to be the sample mean.

BLUE Estimator

Best Linear Unbiased Estimator (BLUE) Best is defined as the minimum

variance estimator from among all unbiased linear estimators

Is the sample mean a BLUE estimator for the population mean?

1

n

ii

XX

n

An Engineering Management Bonus Round!!!!

A real world example of sampling, parameter

estimation, and fishing for the correct answer.

How many Fish are in the Lake?

Let N = the number of fish in the lake k = the number of fish caught and tagged, and

released back into the lake allow for the tagged fish to be uniformly dispersed

within the lake X = a RV, the number of tagged fish caught in the

follow-on sample of size n Then an estimate for the number of fish in the lake

is found by assuming

ˆThen

k X

N nk n

NX

Is N-hat unbiased, BLUE, or MVUE?

counting the fishin the lake

2

ˆ ;

1ˆ

1ˆ

k nN

Xk n

E N E kn EX X

k nVar N Var kn Var

X X

, ,X hypergeometric N n k

Point Estimation- to be continued

next time- making standard errors- those magic moments- maximizing likelihoods

ENM 500 students engaged in random sampling.

ENM 500 students caughtdiscussing the central limittheorem.