Introduction to Linear Mixed EffectsKiran PedadaPhD Student (Marketing)

March 26, 2015

Correlated Data

Forms of correlated data: Time Series data Repeated measurements Longitudinal data Spatial data

Source: http://www.stat.missouri.edu/~spinkac/stat8320/LinearMixedModels.pdf

Linear Mixed Effects Models

Mixed model analysis provides a general, flexible approach in the situations of correlated data.

•Mixed model consists of two components: Fixed effects – usually the conventional linear

regression part Random effects – associated with individual

experimental units produced at random from the data generating process.

Source: http://www.stat.cmu.edu/~hseltman/309/Book/chapter15.pdfhttp://www.mathworks.com/help/stats/linear-mixed-effects-models.html

Linear Mixed Effects Models

The standard form of a linear mixed effects model:

Y=βX+Zb+u

Source: http://www.mathworks.com/help/stats/linear-mixed-effects-models.html

Fixed effect

Random effect

Error

Y is the n x1 response vector, and n is the number of observations.X is an n x p fixed-effects design matrix.β is a p x 1 fixed-effects vector.Z is an n x q random-effects design matrix.b is a q x 1 random-effects vector.u is the n x 1 observation error vector.

Random Effect and Error Vectors

Random effects vector, b, and the error vector, ε are assumed to be independent and distributed as follows :

b ~ N (0, σ2D(θ))ε ~ N (0, σ2I)

Where D is a symmetric and positive semi definite matrix, parameterized by a variance component vector θ, I is an n x n identity matrix, and σ2 is the error variance.

Source: http://www.mathworks.com/help/stats/linear-mixed-effects-models.html

Bodo Winter Example

General Form of Linear Mixed Model:Y=βX+Zb+u

Bodo Winter Fixed Effect Model:

Pitch ~ politeness + sex + uBased on the general form of Linear Mixed Model, we can write the Bodo Winter Example as follows:

Y= β1X1 + β2X2 + uWhere, Y is the response variable, i.e., Pitch and X1 and X2 are the fixed effects, i.e., politeness and sex. β1 and β2 are fixed effect parameters.

Source: http://www.bodowinter.com/tutorial/bw_LME_tutorial.pdf

Mixed Effect Model

If we add one or more random effects to the fixed effect model, then model will become a Mixed Effect Model.

Let us add one random effect (for subject).

Thus, the Mixed Effect Model will look like the following:

Y= β1X1 + β2X2 + Zb + u εWhere, Z is the random effect, i.e., multiple responses per subject. And b is random effect parameter.

Matrix Notation of the Bodo Winter Mixed Model

Y = X β + Z b + u

200 X 1200 X 1 200 X 1

200 X 1

200 X 2 2 X 1 200 X 40 40 X 1

X = β = 200 X 2

2 X 1

Source: Dr. Westfall Notes

To make the example simple, let us consider 1 fixed effect and one random effect.

Let us say, there are 40 female subjects with 5 repetitions on each subject. Half of the subjects are observed in formal case (1) and other half in informal case (0).

1

200

6

Matrix Notation of the Bodo Winter Mixed Model

X β = 200 X 2 2 X 1

=

200 X 1

Source:Source: Dr. Westfall Notes

x

Matrix Notation of the Bodo Winter Mixed Model

Z = 200 X 40

Source: Dr. Westfall Notes

1

5

6

200

Matrix Notation of the Bodo Winter Mixed Model

Z b = 40 X 1200 X 40

Source: Dr. Westfall Notes

Variance-covariance Matrix

Y= β1X1 + β2X2 + Zb + u εCov(ε) = Cov (Zb + u) = Cov (Zb)+ Cov(u) = Z Cov (b) ZT+ σ2I

Source: Dr. Westfall Notes

Cov(ε) = Z Cov (b) ZT+ σ2I

Variance-covariance Matrix

Z Cov (b) ZT+ σ2I

200 X 200

Source: Dr. Westfall Notes

b b b b

b b b

b b

b

R simulation

R Simulation