Sampling distributions and the Central Limit Theorem · Theorem Sampling and inference...

Central LimitTheorem

Sampling andinference

Probabilitiesand probabilitydistributions

Normaldistribution


References

Sampling distributions and the Central LimitTheorem

Johan A. ElkinkSchool of Politics & International Relations

University College Dublin

4 November 2019




Normaldistribution


References

1 Sampling and inference

2 Probabilities and probability distributions

3 Normal distribution

4 Central Limit Theorem




Normaldistribution


References

Outline








Normaldistribution


References

Sampling

Statistical inference (or inductive statistics) concernsdrawing conclusions regarding a population of cases on thebasis of a sample, a subset.

Sampling refers to the selection of an appropriate subset ofthe population.

The sampling frame refers to the identifiable list of membersof the population, from which the sample can be selected.

Simple random sample: each subject from a population hasthe exact same chance of being selected in the sample, i.e. thesampling probability for each subject is the same. This is notcalled a “representative sample”!

When the sampling probability correlates with a variable ofinterest, we get biased results, which is referred to as samplingbias.




Normaldistribution


References

Exercise

What is wrong with the following scenarios?

• Students in a class are asked to raise their hands if theyhave cheated on an exam one or more times within thepast year.

• To get information on opinions among students, 100students are surveyed at the start of a 9 am class.

• To get information on public opinion, you stand at theentrance of the Apple Store in a shopping street andinterview passers-by randomly.




Normaldistribution


References

Sampling methods

Simple random sampling: Each subject from a populationhas the exact same chance of being selected in the sample.

Systematic random sampling: Systematic sampling involvesselecting a random starting point and then select every kthcase.

Stratified sampling: Select groups you need and samplewithin groups to make sure each group is sufficientlyrepresented.

Clustered sampling: To reduce costs, clusters are (randomly)sampled first, before lower levels are clustered.

Snowball sampling: A non-probability sampling techniquewhere existing study subjects recruit future subjects fromamong their acquaintances.




Normaldistribution


References

Parameters

A parameter is number that describes a feature of thepopulation. A parameter is generally fixed and not observable.

A statistic is a number that describes a feature of a sampleand is fixed for a given sample, but varies across samples.

We can use statistics to estimate parameters.

(Moore, McCabe and Craig, 2012, 198)




Normaldistribution


References

Sampling distribution

Using probability theory, we can understand how samplesbehave on average, given some assumptions.

By comparing the sample at hand to samples on average, wecan draw probabilistic conclusions about the populationparameters.

“The sampling distribution of a statistic is the distribution ofvalues taken by the statistic in all possible samples of the samesize from the same population.”

The sampling error is the amount of error when a populationparameter is estimated or predicted by a sample estimate. Thebigger the sample, the lower the sampling error.

(Moore, McCabe and Craig, 2012, 201)




Normaldistribution


References

Estimates and uncertainty

When we estimate a parameter, we are uncertain what thetrue value is.

Besides an estimate of the parameter, we also need anestimate of how certain we are of this estimate.

The typical indicator of this is the standard error.

(Plot: http://ipi.tomlouwerse.nl/)




Normaldistribution


References

Outline








Normaldistribution


References

Probability: definition

Frequentist approach: a proba-bility is the proportion of times aparticular event will occur given along run of repeated experiments.

Subjective (Bayesian)approach: a probabilityis the formal quantifi-cation of the subjectivebelief about how likelya certain event will oc-cur when an experimentis performed.




Normaldistribution


References

Probability: properties

For any event A, 0 ≤ Pr(A) ≤ 1

If S contains all possible events, Pr(S) = 1

If B is “not A”, Pr(B) = 1− Pr(A)

If A and C are independent events, Pr(AC ) = Pr(A)Pr(C )




Normaldistribution


References

Probability: example

Pr(head) =1

2

Pr(tail) =1

2

Sum of all possible events:

Pr(head) + Pr(tail) =1

2+

1

2= 1

Probability of head and then tail:

Pr(head)Pr(tail) =1

4




Normaldistribution


References

Probability mass function

For nominal or ordinal variables, the probability distributionlooks like a relative frequency graph (histogram); “probabilitymass function” (p.m.f.).

All probabilities together sum to 1.

heads tails

0.0

0.1

0.2

0.3

0.4

0.5




Normaldistribution


References

Probability density function

For interval and ratio variables, the probability distribution is acontinuous function, the “probability density function” (p.d.f.).

The total area under the graph is 1.

−4 −2 0 2 4 6

0.0

00

.05

0.1

00

.15

0.2

0

x

y

For a continuous variables, you can only speak of the probabilitythat the observation will be within a certain range of values,not of the probability of a specific value. The probability isrepresented by the area under the graph over that range.




Normaldistribution


References

Bernoulli trial

A Bernoulli distribution is a distribution with two possibleoutcomes (e.g. heads and tails) and a probability attached toeach outcome.

A Bernoulli trial is “an experiment in which s trials are madeof an event, with probability p of success in any given trial.”

(Weisstein, Eric W. “Bernoulli Trial.” http://mathworld.wolfram.com/BernoulliTrial.html)




Normaldistribution


References

Binomial distribution

The binomial distribution is “the (...) probability distribution(...) of obtaining exactly n successes out of N Bernoulli trials.”

P(n|N) =N!

n!(N − n)!pn(1− p)N−n

(Weisstein, Eric W. “Binomial Distribution.”http://mathworld.wolfram.com/BinomialDistribution.html)




Normaldistribution


References

Normal distribution

P(n|N) =N!

n!(N − n)!pn(1− p)N−n

limN→∞

p(n) =1√

2πNpqe−

(n−Np)2

2Npq

=1√

2πσ2e−

(n−n)2

2σ2

i.e. the limiting distribution of the binomial distribution is thenormal distribution, with σ2 ≡ Npq.

Also called Gaussian distribution, but Gauss did not invent it.

(Weisstein, Eric W. “Binomial distribution.” http://mathworld.wolfram.com/binomialdistribution.html) and(Davidson and MacKinnon, 1999, 130–135)




Normaldistribution


References

χ2-distribution

The sum of squares of r independent normal distributions, isdistributed chi-squared with r degrees of freedom:

χ2(r) ≡r∑i

x2i

(Weisstein, Eric W. “Chi-squared distribution.” http://mathworld.wolfram.com/chi-squareddistribution.html)




Normaldistribution


References

Outline








Normaldistribution


References

Galton board




Normaldistribution


References

Normal distribution

−3 −2 −1 0 1 2 3

0.0

0.1

0.2

0.3

0.4

z

y

Figure: Normal distribution




Normaldistribution


References

Normal distribution

f (x |µ, σ) =1√

2πσ2e−

12σ2 (x−µ)2

Thus entirely determined by the mean µ and the standarddeviation σ.

Notation: yi ∼ N(µ, σ2), meaning Y is normally distributedwith mean µ and variance σ2.




Normaldistribution


References

Check for normality

0.0

2.5

5.0

7.5

10.0

12.5

0.0 2.5 5.0 7.5

neutrality

coun

t

Attitudes towards neutrality in Lisbon 2009 survey

●

●

● ● ● ● ●

● ● ● ●

● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ● ●

2.5

5.0

7.5

−2 −1 0 1 2

theoretical

sam

ple

Normal Q−Q Plot for neutrality

Example from the Lisbon 2009 post-referendum survey: Attitude

towards Irish neutrality, on a scale from 1 “Ireland should be willing

to accept limitations on its neutrality” to 9 “Ireland should do

everything it can to strengthen its neutrality”.




Normaldistribution


References

z-scores

If a variable follows a normal distribution, we can standardizeit, converting the values of the variable to z-scores.

Definition: the z-score of a value yi for variable Y is thenumber of standard deviations between yi and the mean of Y ,y .

zi = yi−ysy

zi ∼ N(0, 1)

The N(0, 1) distribution is called the standard normaldistribution.

Calculating z-scores for a variable is called standardizing thevariable.




Normaldistribution


References

Rules of thumb

The surface under the probability distribution between z = −1and z = 1 is approximately 68%.

Between z = −2 and z = 2 approximately 95%.

−3 −2 −1 0 1 2 3

0.0

0.1

0.2

0.3

0.4

z

y




Normaldistribution


References

t-distribution

When σ is not known, but estimated, we use the t-distribution.When N is sufficiently large, the t-distribution approximatesthe normal distribution.

−4 −2 0 2 4

0.0

0.1

0.2

0.3

0.4

N(0,1)t(1)t(2)t(5)




Normaldistribution


References

Outline








Normaldistribution


References

i.i.d.

We make three assumptions about our data to proceed:

• The observations are independent

• The observations are identically distributed

• The population has a finite mean and a finite variance

A variable for which the first two assumptions hold is calledi.i.d..




Normaldistribution


References

Independent observations

Intuitively: the value for one case does not affect the value foranother case on the same variable.More formally: P(x1 ∩ x2) = P(x1)P(x2).

Examples of dependent observations:

• grades of students in different classes;

• stock values over time;

• economic growth in neighbouring countries.




Normaldistribution


References

Identically distributed

All the observations are drawn from the same random variablewith the same probability distribution.

An example where this is not the case would generally be paneldata. E.g. larger firms will have larger variations in profits,thus their variance differs, thus these are not observations fromthe same probability distribution.




Normaldistribution


References

Law of large numbers

A proper random sample is i.i.d.

The law of large numbers and the Central Limit Theorem helpus to predict the behaviour of our sample data.

The law of large numbers (LLN) states that, if these threeassumptions are satisfied, the sample mean will approach thepopulation mean with probability one if the sample is infinitelylarge.




Normaldistribution


References

Central Limit Theorem

If these three assumptions are satisfied,

• The sample mean is normally distributed, regardless ofthe distribution of the original variable.

• The sample mean has the same expected value as thepopulation mean (LLN).

• The standard deviation (standard error) of the samplemean is: S .E .(x) = σx = σx√

n.

Note that the standard error depends only on the sample size,not on the population size.




Normaldistribution


References

Unknown σ

When the population variance, σ2, is unknown, we can use thesample estimate:

σx =σx√n

=

√σ2x

n

σ2x =

∑ni=1(xi − x)2

n − 1

Note that the population variance for a sample proportion of pcan be estimated as:

σ2x =

∑ni=1(xi − x)2

n= p(1− p)

There is no division by n − 1 here, because only one parameterdetermines both the mean and the variance.




Normaldistribution


References

Example

Suppose we have a random sample of 100 individuals and askeach what their first preference vote would be if there wereelections today. If 30 of them say they would vote Fianna Fail,what is the standard error of the estimate that the proportionis p = .3?

σp =

√p(1− p)√

n=

√0.21√100

= 0.0458




Normaldistribution


References

Exercise

Calculate the standard errors:

• A sample of 20 students has an average grade of 60, withan estimated population variance of 10.

• Out of a sample of 100 road accidents, 10 were fatal.

• Of the 1300 respondents in a survey, 48% voted “Yes” onthe Lisbon Treaty referendum.

• The average score on a 5-point political knowledge scale inthe same survey is 2.34, with an estimated populationstandard deviation of 0.3.




Normaldistribution


References

Davidson, Russell and James G. MacKinnon. 1999. Econometric theory and methods. Oxford: OxfordUniversity Press.

Moore, David S., George P. McCabe and Bruce A. Craig. 2012. Introduction to the practice of statistics. 7thinternational edition ed. New York: W.H. Freeman.

Sampling distributions and the Central Limit Theorem · Theorem Sampling and inference...

Documents

Transcript of Sampling distributions and the Central Limit Theorem · Theorem Sampling and inference...