Covariates in Repeated-Measures Analyses

Repeated Measures What change has occurred (in response to a treatment)?

Mechanism Variables How much of the change was due to a change in whatever?

Individual Responses to a Treatment What's the effect of subject characteristics on the change?

Two Within-Subject Factors What's the effect of the treatment on a pattern of responses in

sets of trials (e.g., the effect on fatigue)?

Will G HopkinsAuckland University of TechnologyAuckland, NZ

Two or more measurements (trials) per subject longitudinal or monitoring studies interventions or experiments:

Data are means and standard deviations

Repeated Measures

Y Dependent variable Repeated measure

exptal

control

Group Between-subjects factor Different subjects on each level

Trial Within-subjects factor Same subjects on each level

pre mid post

Period oftreatment

Repeated Measures: Data for Mixed Modeling

One row per subject per trial:

Athlete Group Trial YChris exptal pre 66Chris exptal mid 68Chris exptal post 71Sam exptal pre 74Sam exptal mid .Sam exptal post 77

Jo control pre 71Jo control mid 72

Missing valuemeans loss ofonly one trialfor the subject.

Repeated Measures: Fixed Effects in Mixed Modeling

Fixed effects = means. Fixed effects model

without control group: Y Trial This model estimates means

of Y for each level of Trial(e.g., Trialpre, Trialmid, Trialpost).

Effect of the treatment = means of Y for Trialpost – Trialpre.

Fixed-effects model with a control or other groups: Y GroupTrial This model estimates means for each level of Group and Trial. Effect of treatment = means of Y for

[Trialpost – Trialpre]exptal – [Trialpost – Trialpre]control.

Athlete Group Trial YChris exptal pre 66Chris exptal mid 68Chris exptal post 71Sam exptal pre 74

Repeated Measures: Random Effects in Mixed Modeling

Random effects = standard deviations (SDs). In the simplest repeated-measures model, there is one

between-subject SD and one within-subject SD.

Y

Trial 1 Trial 2 Trial 3

The between-subject SD represents typical variation in the true score between subjects.

The within-subject SD represents typical variation in a subject's score between trials.

In SAS, random Athlete specifies and estimates the pure between-subject SD and a residual error representing the within-subject SD (the same SD for each trial).

Observed SD in any trial = (between SD2 + within SD2).

Repeated Measures: Data for ANOVA

One row per subject: You have to define which

columns represent your within-subjects factor.

The fixed-effects models are effectively the sameas for mixed models, but...

You have less control over the random effects.

If there is no control group, use a 1-way repeated-measures ANOVA (1 way = Trial) or a paired t test.

With a control group, use a 2-way (Trial, Group) repeated-measures ANOVA and analyze the interaction GroupTrial.

Ypre Ymid Ypost66 68 7174 . 77

71 72 72

Athlete GroupChris exptalSam exptal

Jo controlPat control 64 64 63

Measure = "Y"within-subjects factor = "Trial"

Missing value means loss of subject.

Repeated Measures: Data for T Test Calculate the most interesting change scores:

Athlete Group Ypre Ymid YpostChris exptal 66 68 71Sam exptal 74 . 77

Jo control 71 72 72Pat control 64 64 63

Use an unpaired t test to analyze the difference in the change score between exptal and control groups. Usually the post-pre change score, but… …can be for any parameter ("within-subject modelling").

• Missing values not a problem. More robust but less powerful than more complex analyses.

1-1

Ypost-Ypre

53

Mechanism Variables

Mechanism variable = something in the causal path between the treatment and the dependent variable. Necessary but not sufficient that it "tracks" the dependent.

Dependent variable

pre mid post

exptal

control

Mechanism variable

pre mid post

exptal

control

Trial

Important for PhD projects or to publish in high-impact journals. It can put limits on a placebo effect, if it's not placebo affected. Can't use ANOVA; can use graphs and mixed modeling.

Mechanism Variables: ANOVA?

For ANOVA, data have to be one row per subject:Mechanism variable =within-subjects covariate

Xpre Xmid Xpost8.4 8.7 9.19.0 9.2 9.7

7.9 7.7 7.87.1 7.1 7.2

You can't use ANOVA, because it doesn't allow you to match up trials for the dependent and covariate.

Athlete Group Ypre Ymid YpostChris exptal 66 68 71Sam exptal 74 75 77

Jo control 71 72 72Pat control 64 64 63

Measure = "Y"within-subjects factor = "Trial"

Mechanism Variables: Graphical Analysis - 1

Choose the most interesting change scoresfor the dependent and covariate:

Then plot the change scores…

Athlete Group Ypre Ymid Ypost Xpre Xmid XpostChris exptal 66 68 71 8.4 8.7 9.1Sam exptal 74 75 77 9.0 9.2 9.7

Jo control 71 72 72 7.9 7.7 7.8Pat control 64 64 63 7.1 7.1 7.2

1.50.7

-0.10.1

Xpost-Xpre

Change scorefor covariate

53

1-1

Ypost-Ypre

Change scorefor dependent

Mechanism Variables: Graphical Analysis - 2

Three possible outcomes with a real mechanism variable:

Ypost - Ypreexptal

control

Xpost - Xpre

0

0

1. Large individual responses…

…even in the control group.

The covariate is an excellent candidate for a mechanism variable.

…tracked by mechanism variable…

Mechanism Variables: Graphical Analysis - 3

Three possible outcomes with a real mechanism variable:2. Apparently poor tracking of individual responses…

… could be due to noise in either variable.

The covariate could still be a mechanism variable.

Ypost - Ypre

Xpost - Xpre0

0

Mechanism Variables: Graphical Analysis - 4

Three possible outcomes with a real mechanism variable:

The covariate is a good candidate for a mechanism variable.

Ypost - Ypre

Xpost - Xpre0

0

3. Little or no individual responses……but mechanism variable tracks mean response.

Relationship between change scores is often misinterpreted.

Ypost - Ypre

Xpost - Xpre

0

0

Mechanism Variables: Graphical Analysis - 5

0

"Overall, changes in X track changes in Y well, but… Noise may have obscured tracking of any individual responses. Therefore X could be a mechanism."

"The correlation between change scores for X and Y is trivial. Therefore X is not the mechanism."

Mechanism Variables: Mixed Modeling Overview Need to quantify the role of the mechanism variable, with

confidence limits. Mixed modeling with restricted maximum likelihood

estimation does the job. Data format is

one row per trial:

Athlete Group Trial YChris exptal pre 66Chris exptal mid 68Chris exptal post 71Sam exptal pre 74

X8.48.79.19.0

Mechanism variable =within-subjects covariate

No problem with aligning trials for the dependent and covariate.

Mechanism Variables: Mixed Modeling WITHOUT Covariate

To remind you… Fixed effects (= means) model

without control group: Y Trial This model estimates means

of Y for each level of Trial(e.g., Trialpre, Trialmid, Trialpost).

Effect of the treatment = means of Y for Trialpost – Trialpre.

Fixed-effects model with a control or other groups: Y GroupTrial This model estimates means for each level of Group and Trial. Effect of treatment = means of Y for

[Trialpost – Trialpre]exptal – [Trialpost – Trialpre]control.

Athlete Group Trial YChris exptal pre 66Chris exptal mid 68Chris exptal post 71Sam exptal pre 74

Mechanism Variables: Mixed Modeling WITH Covariate Fixed-effects model: Y [Group]Trial X

Estimates means for each level of Trial with X held constant. So, contrasts of interest derived from Trial represent effects of

treatment not explained by putative mechanism variable. Example:

Effect of treatment from usual model Y [Group]Trial:4.6 units (95% likely limits, 2.1 to 7.1 units).

Effect of treatment from model Y [Group]Trial X:2.5 units (95% likely limits, -1.0 to 7.0 units).

So, a little more than half the effect (2.5 units) is not explained by X, but we need a larger sample or more reliable Y and/or X to reduce the uncertainty (-1.0 to 7.0 units).

If changes in X can't be due to any placebo effect, the placebo effect is 2.5 units.

Mechanism Variables: Random Effects in the Mixed Model Simple random-effects (= standard deviations) model:

random Athlete In the Statistical Analysis System, this model specifies and

estimates a pure between-subject SD and a residual error representing within-subject SD (the same SD for each trial).

More complex model: random Athlete AthleteX This model implies X has a different effect for each subject. The coefficient of X in the fixed-effects model represents the

mechanism effect averaged over all subjects. The SD from AthleteX is the typical variation in this mechanism

effect between subjects. Need >2 trials to estimate this SD. All with confidence limits, which you interpret, of course.

Subjects may differ in their response to a treatment…

Individual Responses

…due to subject characteristics interacting with the treatment. It's important to measure and analyze their effect on the treatment. Use mixed modeling, ANOVA, or "within-subject modeling". Using Y for Trialpre as a characteristic needs a special approach to

avoid artifactual regression to the mean. See newstats.org.

Y

Trialpre mid post

Data arevalues forindividuals

pre mid post

boysgirls

Individual Responses: Mixed Modeling - 1

Data format is one row per trial:

Fixed-effects model… without a control group: Y Trial TrialCovariate with a control group or other groups:

Y GroupTrial GroupTrialCovariate

Group Trial Yexpt pre 66expt mid 68expt post 71expt pre 74

AthleteChrisChrisChrisSam

AthleteChrisChrisChrisSam

Sex AgeF 23F 23F 23M 19

Subject characteristics = between-subject covariates

Individual Responses: Mixed Modeling - 2 If Covariate is nominal (e.g., Sex), [Group]TrialCovariate

represents different means for each level of Sex and Trial. Y for (Trialpost – Trialpre)Sexfemale – (Trialpost – Trialpre)Sexmale

= difference between effect of treatment on females and males. Overall effects of treatment from [Group]Trial represent effects for

equal numbers of females and males, even if unequal in the study. If Covariate is numeric (e.g., Age), [Group]TrialCovariate

represents different slopes for each level of Trial. Y for (Trialpost – Trialpre)Age10

= increase in the effect per decade of age, e.g. 2.1 units.10y-1. Overall effects of treatment from [Group]Trial represent effects for

subjects on the mean age. Random-effects model: include special term to quantify individual

responses before and after adding covariate. See newstats.org.

Individual Responses: Repeated-Measures ANOVA

Data format is one row per subject:

If no control group, use repeated-measures ANOVA (ANCOVA) and analyze the interaction TrialCovariate.

With a control group, analyze GroupTrialCovariate.

Group Ypre Ymid Ypostexptal 66 68 71exptal 74 75 77

control 71 72 72control 64 64 63

Within-subjectsfactor = "Trial"

AthleteChrisSam

JoPat

AthleteChrisSam

JoPat

Sex AgeF 23M 19

F 19M 19

Covariates

Individual Responses: Within-Subject Modeling Calculate the most interesting change scores or other within-

subject parameters:

If no control group, analyze effect of Covariate on change score with unpaired t test, linear regression, or simple ANOVAs.

With a control group, analyze effect of GroupCovariate on the change score with a simple ANOVA.

Less powerful, more robust than mixed modeling or ANOVA.

Athlete Sex Age Group Ypre Ymid YpostChris F 23 exptal 66 68 71Sam M 19 exptal 74 75 77

Jo F 19 control 71 72 72Pat M 19 control 64 64 63

1-1

Ypost-Ypre

53

= sets of several measurements for each trial, e.g. 4 bouts:

Y

pre mid post

control

Trial

exptal

Two Within-Subject Factors

We want to estimate the overall increase in Y in the exptal group in the mid and post trials, and…

…the greater decline in Y in the exptal group within the mid and post trials (representing, for example, increased fatigue).

Use mixed modeling, ANOVA, or within-subject modeling.

Within Subject between TrialsWithin Subject within Trial

Between Subjects within BoutStandard deviations:

12

34

Bout

Two Within-Subject Factors: Mixed Modeling - 1

Data format is one row per bout per trial:

Athlete Sex Age Group Trial BoutChris F 23 expt pre 1Chris F 23 expt pre 2Chris F 23 expt pre 3Chris F 23 expt pre 4

Y68676564

Chris F 23 expt mid 1Chris F 23 expt mid 2Chris F 23 expt mid 3Chris F 23 expt mid 4

72706866

Two Within-Subject Factors: Mixed Modeling - 2 Fixed-effects model…

Bout can be nominal (like Sex) or numeric (like Age). In the example, Bout is best modeled as a linear numeric

effect. Polynomials are also possible. Model (without control group): Y Trial TrialBout Increase in the linear fatigue effect between Bouts 1 and 4

from pre to post = Y for (Trialpost – Trialpre)Bout3.• The change in Bout is 3 units.

Overall increase in Y pre to post = Y for Trialpost – Trialpre + (Trialpost – Trialpre)Bout2.5. • The middle of each trial corresponds to Bout = 2.5.

Add TrialCovariate and TrialBoutCovariate to the model to explore individual responses.

Two Within-Subject Factors: Mixed Modeling - 3 Random-effects models

It takes time to get used to random-effects models! Simplest is random Athlete AthleteTrial;

• Athlete gives the pure between-subjects variation.• The residual is the within-trial (between-bouts) error. • AthleteTrial gives the pure within-subject variation between

trials. Add the residual variance to get observed between-bouts between-trials variation.

If Bout is numeric, random Athlete AthleteBout AthleteTrial;implies Bout has a different slope for each subject. • The SD from AthleteBout is the typical variation in the slope

between subjects. • Other random effects are as above, sort of.

Two Within-Subject Factors: ANOVA; Within-Subject Modeling With sufficiently powerful ANOVA, you can specify two nominal

within-subject effects and take into account various within-subject errors (using adjustments for asphericity).

Specifying a linear or polynomial fatigue effect is possible but difficult (for me, anyway).

Within-subject modeling is much easier. In the example, derive the Bout slope (or any other parameter)

within each trial for each subject. Derive the change in the slope between pre and post for each

subject. Do an unpaired t test for the difference in the changes between

the exptal and control groups. Simple, robust, highly recommended!

This presentation was downloaded from:

A New View of Statistics

SUMMARIZING DATASUMMARIZING DATA GENERALIZING TO A POPULATIONGENERALIZING TO A POPULATIONSimple & Effect StatisticsSimple & Effect Statistics

Precision ofMeasurementPrecision ofMeasurement

Confidence LimitsConfidence Limits

StatisticalModelsStatisticalModels

DimensionReductionDimensionReduction

Sample-SizeEstimationSample-SizeEstimation

newstats.org