Issues withMixed Models

Model doesn’t converge…

OR

Convergence

Likelihood Landscape

Likelihood Landscape

Maximum Likelihood Estimation

Mea

n

Maximum Likelihood Estimation

Mea

n

Maximum Likelihood Estimation

Mea

n

Maximum Likelihood Estimation

Mea

n

Maximum Likelihood Estimation

Likelihood = the probability of seeing the data we actually collected given a particular model

Maximum Likelihood Estimates = those values that make the observed data most likely to have happened

Sources of Convergence Problems

• You estimate more parameters than data (or, in general, too many parameters

• Severe collinearity (e.g., two predictors are exactly correlated)

• Missing cells in your design

• Predictors of vastly different metrics

Failure to converge

GENDERATTITUDE male

femalepolite 16

0informal 1632

… and then trying to test the ATTITUDE*GENDER interaction

How can this happen?

“Death by Design”

(coined byRoger Mundry)

designanalysis

Solutions to Convergence Problems

• Drop a random slope(not preferred, should be reported)

• Drop subjects/items for which there is not enough data (not preferred, should be reported)

• Rescale variables so that they lie range between 0 and 1; or make them on similar metrics overall

• Center continuous predictors

• Nonlinear transformations of skewed predictors

Solutions to Convergence Problems

• Change order of variable names in model formula

• Have a balanced and complete design

p-values

The p-value conundrum

What are the degrees of freedom?

How to get p-values out ofmixed models is not entirely straightforward…

DouglasBates

“There are a number of ways to compute p-values from LMEMs,

none of which is uncontroversially the best.”

Barr et al. (2013)

Ways to get p-values

• t-test/F-test with normal approximation• Likelihood Ratio Test• Boostrapping• Permutation• Markov Chain Monte Carlo (MCMC)

Getting p-vals with normal approximation

xmdlcoefs=data.frame(summary(xmdl)@coefs)coefs$p = 2*(1-pnorm(abs(coefs$t.value)))coefs

Function for getting p-vals with normal approximation

create.sig.table = function(x){

coefs=data.frame(summary(x)@coefs)coefs$p = 2*(1-pnorm(abs(coefs$t.value)))coefs$sig = character(nrow(coefs))coefs[which(coefs$p < 0.05),]$sig = "*"coefs[which(coefs$p < 0.01),]$sig = "**"coefs[which(coefs$p < 0.001),]$sig = "***"return(coefs)

}

Likelihood Ratio Test

First model needs to be nested in second

Likelihood RatioThe likelihood ratio expresses how many times more likely the data are under one model than the other

Mea

n

Mea

n

Likelihood Ratio Test

Likelihood Ratio Test

Important whendoing likelihood ratio tests

lmer(…,REML=FALSE)

http://anythingbutrbitrary.blogspot.com/2012/06/random-regression-coefficients-using.html

Final issue:Random slopes

DANGEROUS!!!

Random intercept onlymodels are known to bevery anti-conservativein many circumstances

(cf. Barr et al., 2013,Schielzeth & Forstmeier, 2008)

Schielzeth & Forstmeier (2008)

Random intercept only

Type I error simulation

10 subjects

10 data points each

5 of those in condition A,5 in B

LRT intercept ML 0.052LRT slope ML 0.035LRT intercept REML 0.052LRT slope REML 0.035

z-test intercept ML 0.053z-test slope ML 0.039z-test intercept REML 0.054z-test slope REML 0.042

Add to this explicit subject slopesfor A/B

10 subjects

10 data points each

5 of those in condition A,5 in B

LRT intercept ML 0.24LRT slope ML 0.15LRT intercept REML 0.24LRT slope REML 0.069

z-test intercept ML 0.24z-test slope ML 0.079z-test intercept REML 0.25z-test slope REML 0.091

Add to this explicit subject slopesfor A/B

10 subjects

10 data points each

5 of those in condition A,5 in B

LRT intercept ML 0.24LRT slope ML 0.15LRT intercept REML 0.24LRT slope REML 0.069

z-test intercept ML 0.24z-test slope ML 0.079z-test intercept REML 0.25z-test slope REML 0.091

Add to this explicit subject slopesfor A/B + take item slopes

10 subjects

10 data points each

5 of those in condition A,5 in B

LRT intercept ML 0.18LRT slope ML 0.085LRT intercept REML 0.18LRT slope REML 0.052

z-test intercept ML 0.21z-test slope ML 0.064z-test intercept REML 0.23z-test slope REML 0.08

LRT intercept ML 0.18LRT slope ML 0.085LRT intercept REML 0.18LRT slope REML 0.052

z-test intercept ML 0.21z-test slope ML 0.064z-test intercept REML 0.23z-test slope REML 0.08

Add to this explicit subject slopesfor A/B + take item slopes

10 subjects

10 data points each

5 of those in condition A,5 in B

“Keep it maximal”

“Keep it maximal”

random effectsjustified by the design

vs.random effects

justified by the dataBarr et al.

(2013)

“Keep it maximal”

“for whatever fixed effects are of critical interest, the

corresponding random effects should be in that analysis”

Barr et al. (2013)

That’s it(for now)