Research Methods in Psychology Repeated Measures Designs.

Research Methods in Psychology

Repeated Measures Designs

Repeated Measures Designs

Each individual participates in each condition of the experiment• completes the DV with each condition• hence “repeated measures”

Also called “within-subject” design• entire experiment is conducted “within” each

subject

Repeated Measures Designs, continued

Why Use a Repeated Measures Design?• no need to balance individual differences

across conditions of experiment all participants are in each condition

• fewer participants needed• convenient and efficient• more sensitive

Sensitivity

A sensitive experiment• can detect the effect of an independent

variable• even if the effect is small

Repeated measures designs are more sensitive than independent groups designs• “error variation” is reduced

same people participate in each condition variability due to individual differences eliminated

Practice Effects

Main disadvantage of repeated measures designs is practice effects• People change as they are tested repeatedly

performance may improve over time people may become bored or tired over time

Practice effects become a potential confounding variable if not controlled

Practice Effects, continued

Example:• Suppose a researcher compares two different

study methods, A and B Condition A: participants use a highlighter to mark

key points while reading a text, then take a test on the material

Condition B: participants read a text, then make up sample test questions and answers, then take a test on the material


• Suppose all participants first experience Condition A and

then Condition B results indicate test scores are higher in Condition

A compared to Condition B

• Is marking text with highlighter (A) better than writing sample questions/answers (B)? impossible to know

• confounding of IV with order of presentation• practice effects (boredom, fatigue) may account for

poorer performance in Condition B


Practice effects must be balanced, or averaged, across conditions• Counterbalancing the order of conditions

distributes practice effects equally across conditions half of the participants do Condition A, then B the remaining participants to Condition B, then A Conditions A and B then have equivalent practice

effects practice effects aren’t eliminated, but they are

averaged across the conditions of the experiment

Counterbalancing Practice Effects

Two types of repeated measures designs• Complete and Incomplete• purpose of each type of design is to

counterbalance practice effects• each design uses different procedures for

counterbalancing practice effects

Complete Design

Practice effects are balanced within each participant in the complete repeated measures design• each participant experiences each condition

several times, using different orders each time• a complete repeated measures design is used

when each condition is brief (e.g., simple judgments

about stimuli)

Complete Design, continued

Two methods for generating orders of conditions• block randomization• ABBA counterbalancing


Block randomization• a block consists of all conditions (e.g., 4

conditions: A, B, C, D)• generate a random order of the block (ACBD)• participant completes condition A, then C,

then B, then D• generate a new random order for each time

the participant completes the conditions of the experiment (e.g., DACB, CDBA, ADBD)


Block randomization• balances practice effects only when

conditions are presented many times• practice effects are averaged across the

many presentations of the conditions• practice effects are not balanced if conditions

are presented only a few times to each participant


ABBA counterbalancing• used when conditions are presented only a

few times to each participant• procedure: present one random sequence of

conditions (e.g., DABC), then present the opposite of the sequence (CBAD)

• each condition has the same amount of practice effects


ABBA counterbalancing• balance practice effects that are “linear”

linear practice effects • participants change in the same way following each

presentation of a condition

nonlinear practice effects• participants change dramatically following the

administration of a condition• example: participant experiences insight about how to

complete an experimental task (“aha … now I get it”)• likely to use this insight in subsequent conditions


• Example of linear practice effects suppose participants gain “one unit” of practice

with each administration (“trial”) of a condition • there are zero practice effects with the first

administration

Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6Condition A B C C B A

Practice Effects +0 +1 +2 +3 +4 +5

Practice effects are balanced because total practice effects is +5 for each condition:

A: 0 + 5 B: 1 + 4 C: 2 + 3


• Example of nonlinear practice effects: Suppose a participant figures out a method for

completing the task on the third trial, and then uses the new method for subsequent trials

Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6

Condition A B C C B A

Practice Effects +0 +1 +5 +5 +5 +5

Practice effects are not balanced across the conditions:

A: 0 + 5 = 5 B: 1 + 5 = 6 C: 5 + 5 = 10


• Nonlinear practice effects create a confounding differences in scores on the DV may not be caused

by the IV (conditions A, B, C) differences on DV may be due to different amounts

of practice effects associated with each condition

• ABBA counterbalancing should not be used when practice effects are likely to vary or change

over time (i.e., nonlinear practice effects) use block randomization instead


• ABBA counterbalancing should not be used when anticipation effects can occur participants develop expectations about which

condition will appear next in a sequence responses may be influenced by expectations

rather than actual experience of each condition if anticipation effects are likely, use block

randomization

Incomplete Design

Each participant experiences each condition of the experiment exactly once• complete design: more than once

Practice effects are balanced across participants in the incomplete design• complete design: practice effects balanced

within each subject

Incomplete Design, continued

General rule for balancing practice effects• each condition (e.g., A, B, C) must appear in

each ordinal position (1st, 2nd, 3rd) equally often

• if this rule is followed, practice effects will be balanced across conditions will not confound the experiment


Two techniques for balancing practice effects in an incomplete repeated measures design• all possible orders• selected orders


All possible orders• use when there are four or fewer conditions• two conditions (A, B) → two possible orders: AB, BA

half of the participants would be randomly assigned to do condition A first, followed by B

other half of participants would complete condition B first, followed by A

• three conditions (A, B, C) → six possible orders:ABC, ACB, BAC, BCA, CAB, CBA

participants would be randomly assigned to one of the six orders


• four conditions (ABCD) → 24 possible orders (ABCD, ABDC, ACBD, ACDB, ADBC, etc.)

• five conditions → 120 possible orders• six conditions → 720 possible orders• at least one participant must receive each

order of the conditions therefore, all possible orders is used for

experiments with four or fewer conditions of the IV


Selected orders• select particular orders of conditions to

balance practice effects• two methods

Latin Square random starting order with rotation

• each condition appears in each ordinal position exactly once

• each participant is randomly assigned to one of the orders of conditions


• Procedure for Latin Square randomly order the conditions of the experiment

(e.g., ABCD) number the conditions (A = 1, B = 2, C = 3, D = 4) use this rule for generating the 1st order:

1, 2, N, 3, N – 1, 4, N – 2, 5, N – 3, 6, etc.

where N = last number of conditions

• the first order of four conditions would be 1 2 4 3


to generate 2nd order of conditions add 1 to each number in the first order (1 2 4 3)

• “N” represents the number of conditions (e.g., 4); we can’t use N + 1 because this would create a 5th condition

• additional rule: N + 1 always is “1” -- the first condition• the second order of conditions is 2 3 1 4

to generate 3rd order of conditions add 1 to each number in the second order (again, N + 1 = 1)

• the third order of conditions is 3 4 2 1 follow the same procedure for each subsequent

order• The number of orders is the same as the number of

conditions (e.g., 4 conditions → 4 orders)


Match letters of conditions to their numbers to create the Latin Square

1st 2nd 3rd 4th 1st 2nd 3rd4th

1 2 4 3 A B D C

2 3 1 4 B C A D

3 4 2 1 C D B A

4 1 3 2 D A C B


• Each condition appears in each ordinal position equally often, which balances practice effects For example, condition “A” appears in each ordinal

position:

1st 2nd 3rd 4th

A B D C

B C A D

C D B A

D A C B


• Another advantage of Latin Square each condition precedes and follows every other

condition once (e.g., AB and BA, BC and CB)

1st 2nd 3rd 4th

A B D C

B C A D

C D B A

D A C B

this helps to control for potential order effects


Random starting order with rotation• generate a random order of conditions (e.g., ABCD)• rotate the sequence by moving each condition one position to

the left each time

1st 2nd 3rd 4th

A B C D

B C D A

C D A B

D A B C each condition appears in each ordinal position to balance

practice effects unlike Latin Square, order of conditions is not balanced

Data Analysis of Repeated Measures Designs

Complete repeated measures designs require an additional step• Because participants complete each condition

many times, a summary score (e.g., mean) is computed for each participant for each condition

• this represents each participant’s average performance in each condition

Data Analysis, continued

• Suppose two participants complete two conditions (A, B) of

an experiment four times each the DV is their rating on a 1–5 scale assume IV (conditions A, B) represents two types

of stimuli participants are asked to judge (e.g., size of the stimuli)


Suppose the following data are observed in an ABBA design:

Condition Participant 1 Participant 2

A 2 1

B 4 4

B 5 4

A 1 1

B 3 5

A 1 2

A 2 3

B 5 5


To analyze these data we first need to compute the average rating for each condition (A, B) for each participant:

Condition Participant 1 Participant 2

A 2 1

B 4 4

B 5 4

A 1 1

B 3 5

A 1 2

A 2 3

B 5 5

A: (2+1+1+2)/4 = 1.50 A: (1+1+2+3)/4 = 1.75

B: (4+5+3+5)/4 = 4.25 B: (4+4+5+5)/4 = 4.50


Next calculate the mean for each condition across all participants

In this example with two participants, the means for conditions A and B are

Condition A Condition Bparticipant 1 1.50 4.25participant 2 1.75 4.50

mean 1.625 4.375

Null hypothesis testing or confidence intervals would be used to determine whether this difference between means is reliable

The Problem of Differential Transfer

Repeated measures designs should not be used when differential transfer is possible• occurs when the effects of one condition

persist and affect participants’ experience of subsequent conditions

• use independent groups design instead• assess whether differential transfer is a

problem by comparing results for repeated measures design and random groups design

Comparison of Two Designs

Differences between repeated measures design and independent groups design• Independent variable

repeated measures: each participant experiences every condition of the IV

independent groups: each participant experiences only one condition of the IV

• What is balanced (averaged) across conditions to rule out alternative explanations for findings? repeated measures: practice effects independent groups: individual differences variables

Research Methods in Psychology Repeated Measures Designs.

Documents

Transcript of Research Methods in Psychology Repeated Measures Designs.