kxkxk 3-way Factorial Designs

• kxkxk 3-way designs, effects & causality

• 3-way interactions

• describing effects and checking if they are . “descriptive” or “misleading”

• an example

3-way Factorial Designs

The simplest factorial design is a 2x2, which can be expanded in two ways:

1) Adding conditions to one, the other, or both IVs

2) Add a 3rd IV (making a 3-way factorial design)

Learning Psyc Methods Learning Psyc Content

Ugrads Grads Ugrads Grads

Computer

Instruction

Lecture

Instruction

Identify the three IVs in this design . . .

Specify the properties of each condition/cell of this design . . .

3-Way Factorial Designs

There are 7 omnibus F-tests involved in a 3-way factorial

• 3 main effects F-tests (one for each IV)

• 3 2-way interaction F-tests (one for each pair of IVs)

• 1 3-way interaction F-test

For the example, these F-tests are…

• main effects

• 2-way interactions

• 3-way interaction

1. Topic 2. Instruction Method 3. Ed. level

1. Topic X Inst. Method 2 .Topic X Ed. Level

3. Inst. Method X Ed. Level

Topic X Instruction Method X Ed. level

Causal Interpretations of 3-way Designs

• When can a main effect be causally interpreted ?

• When can a 2-way interaction be causally interpreted ?

• When can a 3-way interaction be causally interpreted ?

When the conditions of that IV are RA, Manip, Etc.

• When MEs of both the involved IVs are causally interpretable

• When we can causally interpret the differences between causally interpretable simple effects.

• When MEs of all 3 IVs are causally interpretable

• When we can causally interpret the differences between simple 2-way interactions which each involve causally interpretable differences between causally interpretable simple effects.

A 2-way interaction… •“emerges” when there are 2 IVs (categorical or quant)•is the non-additive/joint effect of the 2 IVs•usually describes as “when the effect of one IV is different for different values of the 2nd IV”

Expanding this to define 3-ways…•“emerges” when there are 3 IVS (categorical or quant)•is non-additive/joint effect of the 3 IVs•usually identified as the “when the 2-way interaction of 2 IVs is different for different values of the 3rd IV”

What does a 3-way interaction look like?

• Remember that a 2-way interaction is, “when the effect of one IV is different for different levels of a 2nd IV”

• Extending this to a design with 3 IVs, a 3-way interaction is, “when the interaction of two IVs is different for different levels of a 3rd IV”

Practice

Difficulty 1 10

Easy 90 = 90

Hard 40 < 70

SE of Practice is different for Easy and Hard Tasks

Familiar Task

Practice

Difficulty 1 10

Easy 90 = 90

Hard 40 < 70

Unfamiliar Task

Practice

Difficulty 1 10

Easy 60 < 70

Hard 20 < 60

The 2-way interaction of Practice and Difficulty is different for Familiar and Unfamiliar Tasks

Remember what the 3-way F-test tells you !!!

Familiar Task

Practice

Difficulty 1 10

Easy 80 < 90

Hard 60 = 60

Unfamiliar Task

Practice

Difficulty 1 10

Easy 50 < 80

Hard 40 = 40

F(1, 152) = 4.234, p = .0235

The simple 2-way interaction pattern of Difficulty x Practice is the same for Familiar and Unfamiliar Tasks?!?

Where is the interaction?

1st – the significant 3-way tells you that the two simple 2-ways are different – that’s what a 3-way interaction means !!!

2nd – notice the Easy Practice effect is smaller for Familiar than for Unfamiliar – sometimes that’s all it takes for a sig 3-way !!!

Basic Rules for Follow-up Analyses:

#1 Any significant effect with k>2 conditions will require a follow-up significance test to compare the condition means of that effect to describe the pattern of that effect.

#2 Any lower-order efffect involved in a significant higher order effect will require follow-up significance tests to compare the means pattern of that lower-order effect with the means pattern of the corresponding simple effects, to determine if the means pattern of the lower-order effect is descriptive or misleading.

Remember…In a 3-way effect we have 6 lower-order effects •3 2-way interactions•3 Main effects

As we discussed… There are 2 situations when we will care!#1 if the aggregates compared in the effect represent a target population that we care about!#2 when the effect is descriptive, and can be used to generalize from the aggregate to all the separate conditions/values (in the current study/design)

For each lower-order effect we have to decide… Do we care???

If the aggregates represent meaningful populations, but the effect is misleading we’ll have to carefully describe the situation – when the pattern is descriptive and misleading, etc.

If the effect is neither representative not a useful general description, then we may not care about the effect !!!

“Descriptive” effects in a 3-way

The 3-way -- significant or not -- is always descriptive !

If the 3-way is significant, all 2-way & main effects are “suspect”

If the 3-way is significant, a 2-way is only descriptive if that 2-way has the same pattern for each condition of the 3rd IV

If the 3-way is significant, a main effect is only descriptive if that main effect has the same pattern for each combination of the other two IVs

If the 3-way is non-significant, all three 2-ways are descriptive

“Descriptive” effects in a 3-way

If a 2-way is significant, the main effects of those 2 IVs are “suspect”

If a 2-way is significant, the main effect of an IV involved in that interaction is only descriptive if that main effect has the same pattern for each condition of the other IV

The main effect of an IV that is not involved in any 2-way or 3-way interaction is always descriptive

kxkxk Stimulus Type Shape Texture

#Practices #Practices

1 10 1 10Modality

Vision

Touch

Bimodal

Effects for this Design

3-way interaction of x Modality x Practice

2-way interaction of Stimulus x Modality2-way interaction of Stimulus x Practice2-way interaction of Modality x Practice

Main effect of Stimulus Main effect of Modality Main effect of Practice

Simple interaction of Stimulus x Modality at dif values of PracticeSimple interaction of Stimulus x Practice at dif values of ModalitySimple interaction of Modality x Practice at dif values of Stimulus

Simple effect of Stimulus at dif combos of Modality & PracticeSimple effect of Modality at dif combos of Stimulus & PracticeSimple effect of Practice at dif combos of Modality & Practice

kxkxk

100

90

80

70

60

50

Vision Touch Bimodal

10 Practices

1 PracticeShape

Texture

kxkxk

100

90

80

70

60

50

1 Practices 10

VisionTouchBimodal

Shape

Texture

kxkxkPERFORMANCE

kxkxk