Statistics for the Social Sciences Psychology 340 Fall 2006 Hypothesis testing.

Statistics for the Social Sciences

Psychology 340Fall 2006

Hypothesis testing


Outline (for week)

• Review of: – Basic probability– Normal distribution– Hypothesis testing framework

• Stating hypotheses• General test statistic and test statistic distributions

• When to reject or fail to reject


Hypothesis testing

• Example: Testing the effectiveness of a new memory treatment for patients with memory problems

– Our pharmaceutical company develops a new drug treatment that is designed to help patients with impaired memories.

– Before we market the drug we want to see if it works.

– The drug is designed to work on all memory patients, but we can’t test them all (the population).

– So we decide to use a sample and conduct the following experiment.

– Based on the results from the sample we will make conclusions about the population.


Hypothesis testing

• Example: Testing the effectiveness of a new memory treatment for patients with memory problems

Memory treatment

No Memorytreatment

Memory patients

MemoryTest

MemoryTest

55 errors60 errors

5 error diff

•Is the 5 error difference: – A “real” difference due to the effect of the treatment

– Or is it just sampling error?


Testing Hypotheses

• Hypothesis testing– Procedure for deciding whether the outcome of a study (results for a sample) support a particular theory (which is thought to apply to a population)

– Core logic of hypothesis testing• Considers the probability that the result of a study could have come about if the experimental procedure had no effect

• If this probability is low, scenario of no effect is rejected and the theory behind the experimental procedure is supported


Basics of Probability

• Probability– Expected relative frequency of a particular outcome

• Outcome– The result of an experiment

Probability = Possible successful outcomes

All possible outcomes


Flipping a coin example

What are the odds of getting a “heads”?

One outcome classified as heads=

1

2=0.5

Probability = Possible successful outcomes

All possible outcomes

Total of two outcomes

n = 1 flip



What are the odds of getting two “heads”?

Number of heads

2

1

1

0

One 2 “heads” outcomeFour total outcomes

=0.25

This situation is known as the binomial# of outcomes = 2n

n = 2



What are the odds of getting “at least one heads”?

Number of heads

2

1

1

0

Four total outcomes

=0.75

Three “at least one heads” outcome

n = 2



HHH

HHT

HTH

HTT

THH

THT

TTH

TTT

Number of heads3

2

1

0

2

2

1

1

2n= 23 = 8 total outcomes

n = 3



Number of heads3

2

1

0

2

2

1

1

X f p

3 1 .125

2 3 .375

1 3 .375

0 1 .125

Number of heads0 1 2 3

.1

.2

.3

.4

probability

.125 .125.375.375

Distribution of possible outcomes(n = 3 flips)




.1

.2

.3

.4

probability

What’s the probability of flipping three heads in a row?

.125 .125.375.375 p = 0.125


Can make predictions about likelihood of outcomes based on this distribution.




.1

.2

.3

.4

probability

What’s the probability of flipping at least two heads in three tosses?

.125 .125.375.375 p = 0.375 + 0.125 = 0.50






.1

.2

.3

.4

probability

What’s the probability of flipping all heads or all tails in three tosses?

.125 .125.375.375 p = 0.125 + 0.125 = 0.25




Hypothesis testing


Distribution of possible outcomes(of a particular sample size, n)

• In hypothesis testing, we compare our observed samples with the distribution of possible samples (transformed into standardized distributions)

• This distribution of possible outcomes is often Normally Distributed


The Normal Distribution

• The distribution of days before and after due date (bin width = 4 days).

0 14-14Days before and after due date



• Normal distribution



• Normal distribution is a commonly found distribution that is symmetrical and unimodal. – Not all unimodal, symmetrical curves are Normal, so be careful with your descriptions

• It is defined by the following equation:

€

1

2πσ 2e−(X −μ )2 / 2σ 2

1 2-1-2 0


The Unit Normal Table

z .00 .01

-3.4-3.3::0::

1.0::

3.33.4

0.0003

0.0005::

0.5000::

0.8413::

0.9995

0.9997

0.0003

0.0005::

0.5040::

0.8438::

0.9995

0.9997

• Gives the precise proportion of scores (in z-scores) between the mean (Z score of 0) and any other Z score in a Normal distribution

– Contains the proportions in the tail to the left of corresponding z-scores of a Normal distribution

• This means that the table lists only positive Z scores

• The normal distribution is often transformed into z-scores.


Using the Unit Normal Table

z .00 .01

-3.4-3.3::0::

1.0::

3.33.4

0.0003

0.0005::

0.5000::

0.8413::

0.9995

0.9997

0.0003

0.0005::

0.5040::

0.8438::

0.9995

0.9997

15.87% (13.59% and 2.28%) of the scores are to the right of the score100%-15.87% = 84.13% to the left

At z = +1:

13.59%2.28%

34.13%

50%-34%-14% rule

1 2-1-2 0

Similar to the 68%-95%-99% rule



z .00 .01

-3.4-3.3::0::

1.0::

3.33.4

0.0003

0.0005::

0.5000::

0.8413::

0.9995

0.9997

0.0003

0.0005::

0.5040::

0.8438::

0.9995

0.9997

1. Convert raw score to Z score (if necessary)

2. Draw normal curve, where the Z score falls on it, shade in the area for which you are finding the percentage

3. Make rough estimate of shaded area’s percentage (using 50%-34%-14% rule)

• Steps for figuring the percentage above of below a particular raw or Z score:



z .00 .01

-3.4-3.3::0::

1.0::

3.33.4

0.0003

0.0005::

0.5000::

0.8413::

0.9995

0.9997

0.0003

0.0005::

0.5040::

0.8438::

0.9995

0.9997

4. Find exact percentage using unit normal table5. If needed, add or subtract 50% from this percentage6. Check the exact percentage is within the range of the estimate from Step 3

• Steps for figuring the percentage above of below a particular raw or Z score:


Suppose that you got a 630 on the SAT. What percent of the people who take the SAT get your score or worse?

SAT Example problems

• The population parameters for the SAT are: = 500, = 100, and it is Normally distributed

€

z =X − μ

σ=

630 − 500

100=1.3 From the

table:

z(1.3) =.0968

-1-2 1 2

That’s 9.68% above this score

So 90.32% got your score or worse



• You can go in the other direction too– Steps for figuring Z scores and raw scores from percentages:

1. Draw normal curve, shade in approximate area for the percentage (using the 50%-34%-14% rule)2. Make rough estimate of the Z score where the shaded area starts3. Find the exact Z score using the unit normal table4. Check that your Z score is similar to the rough estimate from Step 25. If you want to find a raw score, change it from the Z score


Inferential statistics

• Hypothesis testing– Core logic of hypothesis testing

• Considers the probability that the result of a study could have come about if the experimental procedure had no effect

• If this probability is low, scenario of no effect is rejected and the theory behind the experimental procedure is supported

• Step 1: State your hypotheses• Step 2: Set your decision criteria• Step 3: Collect your data • Step 4: Compute your test statistics • Step 5: Make a decision about your null hypothesis

– A five step program


– Step 1: State your hypotheses: as a research hypothesis and a null hypothesis about the populations• Null hypothesis (H0)

• Research hypothesis (HA)

Hypothesis testing

• There are no differences between conditions (no effect of treatment)

• Generally, not all groups are equal

This is the one that you test

• Hypothesis testing: a five step program

– You aren’t out to prove the alternative hypothesis • If you reject the null hypothesis, then you’re left with support for the alternative(s) (NOT proof!)


In our memory example experiment:

Testing Hypotheses

Treatment > No Treatment

Treatment < No

Treatment

H0

:HA:

– Our theory is that the treatment should improve memory (fewer errors).

– Step 1: State your hypotheses


One -tailed


In our memory example experiment:

Testing Hypotheses

Treatment > No Treatment

Treatment < No

Treatment

H0

:HA:

– Our theory is that the treatment should improve memory (fewer errors).

– Step 1: State your hypotheses


Treatment = No Treatment

Treatment ≠ No

Treatment

H0

:HA:

– Our theory is that the treatment has an effect on memory.

One -tailed Two -tailedno directionspecifieddirection

specified


One-Tailed and Two-Tailed Hypothesis Tests

• Directional hypotheses– One-tailed test

• Nondirectional hypotheses– Two-tailed test


Testing Hypotheses

– Step 1: State your hypotheses– Step 2: Set your decision criteria


• Your alpha () level will be your guide for when to reject or fail to reject the null hypothesis. – Based on the probability of making making an certain type of error


Testing Hypotheses

– Step 1: State your hypotheses– Step 2: Set your decision criteria– Step 3: Collect your data



Testing Hypotheses

– Step 1: State your hypotheses– Step 2: Set your decision criteria– Step 3: Collect your data – Step 4: Compute your test statistics


• Descriptive statistics (means, standard deviations, etc.)• Inferential statistics (z-test, t-tests, ANOVAs, etc.)


Testing Hypotheses

– Step 1: State your hypotheses– Step 2: Set your decision criteria– Step 3: Collect your data – Step 4: Compute your test statistics – Step 5: Make a decision about your null hypothesis


• Based on the outcomes of the statistical tests researchers will either:– Reject the null hypothesis– Fail to reject the null hypothesis

• This could be correct conclusion or the incorrect conclusion


Error types

• Type I error (): concluding that there is a difference between groups (“an effect”) when there really isn’t. – Sometimes called “significance level” or “alpha level”

– We try to minimize this (keep it low)

• Type II error (): concluding that there isn’t an effect, when there really is.– Related to the Statistical Power of a test (1-)


Error types

Real world (‘truth’)

H0 is correct

H0 is wrong

Experimenter’s conclusions

Reject H0

Fail to Reject H0

There really isn’t an effect

There really isan effect


Error types


H0 is correct

H0 is wrong


Reject H0

Fail to Reject H0

I conclude that there is an effect

I can’t detect an effect


Error types


H0 is correct

H0 is wrong


Reject H0

Fail to Reject H0

Type I error Type

II error

α

β


Performing your statistical test

H0: is true (no treatment effect) H0: is false (is a treatment effect)

Two populations

One population

• What are we doing when we test the hypotheses?


XA

they aren’t the same as those in the population of memory patients

XA

the memory treatment sample are the same as those in the population of memory patients.


Performing your statistical test

• What are we doing when we test the hypotheses?– Computing a test statistic: Generic test

€

test statistic =observed difference

difference expected by chance

Could be difference between a sample and a population, or between different samples

Based on standard error or an estimate

of the standard error


“Generic” statistical test

• The generic test statistic distribution (think of this as the distribution of sample means)– To reject the H0, you want a computed test

statistics that is large– What’s large enough?

• The alpha level gives us the decision criterionDistribution of the test statistic

-level determines where these boundaries go



If test statistic is here Reject H0

If test statistic is here Fail to reject H0

Distribution of the test statistic

• The generic test statistic distribution (think of this as the distribution of sample means)– To reject the H0, you want a computed test

statistics that is large– What’s large enough?

• The alpha level gives us the decision criterion



Reject H0

Fail to reject H0


One -tailedTwo -tailedReject H0

Fail to reject H0

Reject H0

Fail to reject H0

= 0.05

0.025

0.025split up into the two tails



Reject H0

Fail to reject H0



Fail to reject H0

Reject H0

Fail to reject H0

= 0.050.05

all of it in one tail



Reject H0

Fail to reject H0



Fail to reject H0

Reject H0

Fail to reject H0

= 0.05

0.05

all of it in one tail



An example: One sample z-test

Memory example experiment:• We give a n = 16 memory patients a memory improvement treatment.

• How do they compare to the general population of memory patients who have a distribution of memory errors that is Normal, = 60, = 8?

• After the treatment they have an average score of = 55 memory errors.

€

X

• Step 1: State your hypotheses

H0

:the memory treatment sample are the same as those in the population of memory patients.HA: they aren’t the same as those in the population of memory patients

Treatment > pop > 60

Treatment < pop < 60







€

X

• Step 2: Set your decision criteria

= 0.05One -tailed

H0: Treatment > pop > 60 HA: Treatment < pop < 60







€

X

= 0.05One -tailed

• Step 3: Collect your data








€

X

= 0.05One -tailed• Step 4: Compute your

test statistics

€

zX

=X − μ

X

σX

€

=55 − 60

816

⎛ ⎝ ⎜

⎞ ⎠ ⎟

= -2.5








€

X

= 0.05One -tailed

€

zX

= −2.5

• Step 5: Make a decision about your null hypothesis

-1-2 1 2

5%

Reject H0








€

X

= 0.05One -tailed

€

zX

= −2.5

• Step 5: Make a decision about your null hypothesis

- Reject H0- Support for our HA, the evidence suggests that the treatment decreases the number of memory errors


Statistics for the Social Sciences Psychology 340 Fall 2006 Hypothesis testing.

Documents

Transcript of Statistics for the Social Sciences Psychology 340 Fall 2006 Hypothesis testing.