Chapter 8Chapter 8 Slide Slide 11

Psy B07

POWERPOWER

Chapter 8Chapter 8 Slide Slide 22

Psy B07

Chapter 4 flashbackChapter 4 flashback

H0 true H0 false

Reject H0 Type I error Correct

Fail to Correct Type II errorreject H0

Type I error is the probability of rejecting Type I error is the probability of rejecting the null hypothesis when it is really true.the null hypothesis when it is really true.

The probability of making a type I error The probability of making a type I error is denoted as is denoted as ..

Chapter 8Chapter 8 Slide Slide 33

Psy B07

Chapter 4 flashbackChapter 4 flashback

Type II error is the probability of Type II error is the probability of failing to reject a null hypothesis failing to reject a null hypothesis that is really falsethat is really false

The probability of making a type The probability of making a type II error is denoted as II error is denoted as ..

In this chapter, you’ll often see In this chapter, you’ll often see these outcomes represented with these outcomes represented with distributionsdistributions

Chapter 8Chapter 8 Slide Slide 44

Psy B07

DistributionsDistributions

To make these representations clear, To make these representations clear, let’s first consider the situation where Hlet’s first consider the situation where H00 is, in fact, true:is, in fact, true:

Now assume that H0 is false (i.e., that some “treatment” has an effect on our dependent variable, shifting the mean to the right).

correct failure to reject

Alpha Type I Error

Chapter 8Chapter 8 Slide Slide 55

Psy B07

DistributionsDistributions

DistributionUnder H0

Correct Rejection

Distribution Under H1

Power

Alpha

Type II error

Chapter 8Chapter 8 Slide Slide 66

Psy B07

Definition of PowerDefinition of Power

Thus, power can be defined as follows:Thus, power can be defined as follows:

Assuming some manipulation effects Assuming some manipulation effects the dependent variable, the dependent variable, powerpower is the is the probabilityprobability that the sample mean will that the sample mean will be sufficiently different from the mean be sufficiently different from the mean under Hunder H00 to allow us to reject H to allow us to reject H0.0.

As such, the power of an experiment As such, the power of an experiment depends on three (or four) factors:depends on three (or four) factors:

Chapter 8Chapter 8 Slide Slide 77

Psy B07

Factors affecting powerFactors affecting power

AlphaAlpha As alpha is moved to the left (for example, As alpha is moved to the left (for example,

if one used an alpha of 0.10 instead of if one used an alpha of 0.10 instead of 0.05), beta would decrease, power would 0.05), beta would decrease, power would increase ... but, the probability of making increase ... but, the probability of making a type I error would increase.a type I error would increase.

11 - - 22 : : The further that HThe further that H11 is shifted away from is shifted away from

HH00, the more power (and lower beta) an , the more power (and lower beta) an experiment will have.experiment will have.

Chapter 8Chapter 8 Slide Slide 88

Psy B07

Factors affecting powerFactors affecting power

Standard error of the meanStandard error of the mean The smaller the standard error of The smaller the standard error of

the mean (i.e., the less the two the mean (i.e., the less the two distributions overlap), the greater distributions overlap), the greater the power. As suggested by the the power. As suggested by the CLT, the standard error of the mean CLT, the standard error of the mean is a function of the is a function of the population population variancevariance and and NN. Thus, of all the . Thus, of all the factors mentioned, the only one we factors mentioned, the only one we can really control is can really control is NN..

Chapter 8Chapter 8 Slide Slide 99

Psy B07

Effect sizeEffect size

Most power calculations use a term called effect size Most power calculations use a term called effect size which is actually a measure of the degree to which which is actually a measure of the degree to which the Hthe H00 and H and H11 distributions overlap. distributions overlap.

As such, effect size is sensitive to both the As such, effect size is sensitive to both the difference between the means under Hdifference between the means under H00 and H and H11, and , and the standard deviation of the parent populations.the standard deviation of the parent populations.

Specifically:Specifically:

21d

Chapter 8Chapter 8 Slide Slide 1010

Psy B07

Effect sizeEffect size

In English then, d is the number of In English then, d is the number of standard deviations separating the standard deviations separating the mean of Hmean of H00 and the mean of H and the mean of H1.1.

Note: N has not been Note: N has not been incorporated in the above incorporated in the above formula. You’ll see why shortlyformula. You’ll see why shortly

Chapter 8Chapter 8 Slide Slide 1111

Psy B07

Estimating effect sizeEstimating effect size

As d forms the basis of all calculations of power, As d forms the basis of all calculations of power, the first step in these calculations is to estimate the first step in these calculations is to estimate d.d.

Since we do not typically know how big the effect Since we do not typically know how big the effect will be a priori, we must make an educated guess will be a priori, we must make an educated guess on the basis of:on the basis of:

1) Prior research.1) Prior research.

2) An assessment of the size of the effect that 2) An assessment of the size of the effect that would would be important.be important.

3) General Rule (small effect d=0.2, medium 3) General Rule (small effect d=0.2, medium effect d=0.5, large effect d = 0.8)effect d=0.5, large effect d = 0.8)

Chapter 8Chapter 8 Slide Slide 1212

Psy B07

Estimating effect sizeEstimating effect size

The calculation of d took into account The calculation of d took into account 1) the difference between the means 1) the difference between the means of Hof H00 and H and H11 and 2) the standard and 2) the standard deviation of the population.deviation of the population.

However, it did not take into account However, it did not take into account the third variable the effects the the third variable the effects the overlap of the two distributions; N.overlap of the two distributions; N.

Chapter 8Chapter 8 Slide Slide 1313

Psy B07

Estimating effect sizeEstimating effect size

This was done purposefully so that we have This was done purposefully so that we have one term that represents the relevant one term that represents the relevant variables we, as experimenters, can do variables we, as experimenters, can do nothing about (d) and another representing nothing about (d) and another representing the variable we can do something about; N. the variable we can do something about; N.

The statistic we use to recombine these The statistic we use to recombine these factors is called delta and is computed as factors is called delta and is computed as follows:follows:

where the specific where the specific ƒ(N)ƒ(N) differs depending on the differs depending on the type of t-test you are computing the power for.type of t-test you are computing the power for.

)]N([d ƒ

Chapter 8Chapter 8 Slide Slide 1414

Psy B07

Power calcs for one-Power calcs for one-sample tsample t

In the context of a one sample t-test, the In the context of a one sample t-test, the

ƒƒ(N)(N) alluded to above is simply: alluded to above is simply:

Thus, when calculating the power Thus, when calculating the power associated with a one sample t, you must associated with a one sample t, you must go through the following steps:go through the following steps:

1) Estimate d, or calculate it using:1) Estimate d, or calculate it using:

N

21d

Chapter 8Chapter 8 Slide Slide 1515

Psy B07

Power calcs for one-Power calcs for one-sample tsample t

Calculate Calculate δδ using:using:

3) Go to the power table, and find the 3) Go to the power table, and find the power associated with the calculated power associated with the calculated δδ given the level of given the level of αα you plan to use (or you plan to use (or used) for the t-test used) for the t-test

Nd

Chapter 8Chapter 8 Slide Slide 1616

Psy B07

Power calcs for one-Power calcs for one-sample tsample t

ExampleExample::

Say I find a new stats textbook and after looking Say I find a new stats textbook and after looking at it, I think it will raise the average mark of the at it, I think it will raise the average mark of the class by about 8 points. From previous classes, I class by about 8 points. From previous classes, I am able to estimate the population standard am able to estimate the population standard deviation as 15. If I now test out the new text by deviation as 15. If I now test out the new text by using it with 20 new students, what is my power using it with 20 new students, what is my power to reject the null hypothesis (that the new to reject the null hypothesis (that the new students marks are the same as the old students students marks are the same as the old students marks).marks).

How many new students would I have to test to How many new students would I have to test to bring my power up to .90?bring my power up to .90?

Note:Note: Don’t worry about the bit on “noncentrality Don’t worry about the bit on “noncentrality parameters” in the book.parameters” in the book.