Prospective power analysis for multilevel designs using...
Transcript of Prospective power analysis for multilevel designs using...
Janeen LoehrDepartment of Psychology, University of Saskatchewan
Introduction
Prospective power analysis for multilevel designs using SIMR
Research questions: • Does joint performance influence joint agency? • Does explicit feedback enhance performance’s
effect?Experiment design: • Participants to be recruited in pairs• Continuous predictor: Pace Error (pair level)• Categorical predictor: Feedback (implicit, explicit)
(between pairs) • Outcome variable: Joint Agency (participant level)Effect size of interest:• Interaction between Pace Error and Feedback
Participants nested within pairs4
SIMR2,3
References:1. Lane & Hennes (2018). Power struggles: Estimating sample size for multilevel relationships research. J Soc Pers Relat, 35, 7–31. 2. Green & Macleod (2016). SIMR: An R package for power analysis of generalized linear mixed models by simulation. Methods Ecol Evol, 7, 493–498. 3. Green, MacLeod, & Alday, P. (2017). Package “simr.” Retrieved from https://cran.rproject.org/web/packages/simr/simr.pdf4. Loehr (2018). Shared credit for shared success: Successful joint performance strengthens the sense of joint agency. Conscious Cogn, 66, 79-90.
• SIMR2,3 estimates power using Monte Carlo simulation • makeLmer builds an artificial fitted mixed-
effects model with specified parameters • extend increases the number of observations
in the artificial fitted model• powerCurve estimates power at different
sample sizes by repeatedly:1. Simulating a new dataset using the fitted
model provided2. Refitting the model to the simulated dataset 3. Applying the statistical test of interest to the
simulated fit. • Power = proportion of successful tests @ Step 3.
Parameter Estimates:• Fixed effects:
• Random effects:
• Pace error: 0-65 ms over 40 trials
Code: Simulated power:
• Prospective power analysis is necessary given problems caused by low statistical power
• Multilevel designs and mixed-effects model analyses are becoming more common
• Challenges for prospective power analysis for multilevel designs:• Accommodate the structure of the multilevel
model of interest• Estimate values for each of the model’s many
parameters1
• Solution: Simulate power using parameter estimates from analyses of existing datasets1
Dataset b Int b PErr1 (like implicit) 33.30 0.122 (like explicit) 30.90 0.18Difference -2.4 0.06
Dataset 1 Dataset 2Param (level) Estimate (% var) Estimate (% var)Residual 289 (43%) 345 (37%)Int (Dyad) 38 (6%) -Int (Part.) 345 (51%) 549 (58%)PErr (Part.) 0.04 (0.006%) 0.11 (0.01%)Int (Trial) 5 (0.6%) 49 (5%)
Trials repeated within participants
Research question: • Does perceptual overlap between action
outcomes influence the sense of joint agency?Experiment design: • Participants to be recruited in pairs• Categorical predictor: Accompaniment
Distance (near,far) (within pairs)• Outcome variable: Joint Agency (participant
level)Effect size of interest:• Effect of Accompaniment Distance on Joint
Agency
Parameter Estimates:• Fixed effects:
• Random effects:
• 36 trials per condition
Parameter bIntercept 63Accomp. Dist. -5.9
Dataset 1 Param (level) Estimate
Residual 325Int (Dyad) --Int (Part.) 183Accomp. Dist. (Part.) 18Int (Trial) --
Code: Simulated power:
Detailed methods
…
Poster + R code
Poster + R code