1Chapter 11 Prepared by Samantha Gaies, M.A.

– Power is based on the Alternative Hypothesis Distribution (AHD)

– Usually, the Null Hypothesis Distribution (NHD) is specified exactly, but

– the alternative hypothesis is just stated as the complement to the null and is there-fore not specific

The Four Major Factors of Power Analysis:– The significance criterion, α– The sample size, N– The population effect size, d– Power, or 1 – β (the probability of

rejecting H0 when it is not true)

Chapter 11: Introduction to Power Analysis

Chapter 11 Prepared by Samantha Gaies, M.A. 2

Power for Testing the Mean of a Single Population

– Power Determination

• Specifying an exact H1 (i.e., µ1) will help to determine d:

where µ0 is the mean of the reference population (i.e., the value of the null hypothesis)

• If d needs to be estimated, the guidelines are:

– d = .2 small effect size– d = .5 medium effect size– d = .8 large effect size

Chapter 11 Prepared by Samantha Gaies, M.A. 3

Power for the Mean of a Single Population (cont.)

• Once d is determined (or estimated, or guessed at), δ for the one-sample case can be found from this formula:

• Given α and the number of tails, we can then use a table to look up power.

• Sample Size Determination– You need to specify d; the value of δ

needed for your desired level of power and α (and # of tails) is looked up in a table.

– .80 is commonly considered a reasonable level for power.

– The size of the sample needed for the desired level of power is found from the following formula:

Chapter 11 Prepared by Samantha Gaies, M.A. 4

Power for the Test of the Proportion of a Single

Population– Power determination: Specifying an

exact H1 (i.e., π1) and a null hypothesis proportion (π1) will allow you to calculate d:

– which in turn allows you to find δ, in order to look up power:

– If you decide on a desired level of power, you can look up δ, and then calculate the needed sample size:

Chapter 11 Prepared by Samantha Gaies, M.A. 5

The Power of a Test of Pearson’s r

– Power Determination: Decide on the value for the population correlation (ρ1) that you expect, and calculate δ, in order to look up power.

– Sample Size Determination: If you can decide on a desired level of power, and an expected value for (ρ1), the needed sample size is given by the following formula:

Chapter 11 Prepared by Samantha Gaies, M.A. 6

Power for Testing the Difference between Independent Means

– Power Determination: the effect size for the difference of population means is given by:

where θ = value of μ1 – μ2 specified by your alternative hypothesis (H1), and σ = the population standard deviation (a single value because it is assumed that σ1 = σ2).

– Then, δ is given by the following formula:

where n is the size of each of the two groups to be tested.

– If you are stuck with unequal ns, use the harmonic mean of your two ns in the above formula (see next slide).

Chapter 11 Prepared by Samantha Gaies, M.A. 7

Power for Testing Two Independent Means (cont.)

• Sample Size Determination: the n for each of two equal-sized groups needed to attain the power level corresponding to δ is given by the following formula:

• If you need to estimate d, the following guidelines from Cohen (1988) can be used:

– d = .2 small effect size– d = .5 medium effect size– d = .8 large effect size

• If you need to calculate the harmonic mean of two numbers (e.g., sample sizes), the following formula can be used:

Chapter 11 Prepared by Samantha Gaies, M.A. 8

Power for Testing the Difference between Two Matched

Populations• Power Determination:

where n is the size of each group, d is the same as the effect size for independent means, and ρ is the correlation between the two sets of scores in the population (measures the degree of matching).

– As ρ gets closer to +1.0, the term involving ρ gets larger, so δ gets larger, which means power increases. A ρ of .5 increases power as much as doubling the sample sizes.

– Sample Size Determination:

Chapter 11 Prepared by Samantha Gaies, M.A. 9

Estimating d for a Power Analysis Involving Independent Means

– You can use the sample effect size (g) from a previous similar study to estimate d; g can be found from the t value and n for the previous study:

– If the ns in the previous study are not equal, use the following formula, which has the harmonic mean built in:

– If you are stuck with two samples of size n, and you want to know the minimum effect size for adequate power, find δ for the power you want and use this formula:

21

21

NN

NNtg

10 Chapter 11 Prepared by Samantha Gaies, M.A.

Interpretation of t Values

• Significant t obtained with large Ns: implies a smaller effect size than if the same t value were obtained with small Ns

• Negative (i.e., nonsignificant) results with small Ns: less conclusive and less trustworthy than negative results with large Ns

• Comparing t values from different studies:– p values tell us nothing automatically

about effect sizes. However, with a series of similar tests based on the same Ns, smaller p values will be associated with larger estimates of effect size.

11 Chapter 11 Prepared by Samantha Gaies, M.A.

Four Ways to Manipulate Power– Change sample sizes– Change alpha and/or number of tails– Change the effect size – Add or improve matching between

two sets of scores

Try this two independent-group example…

Let’s say d = 1.47; n1 = n2 = 10

•How much power does your two-group t test have?

•For a .05, two-tailed test, power = .91