Multilevel Linear Modeling

aka HLM

The Design

• We have data at two different levels• In this case, 7,185 students (Level 1)• Nested within 160 Schools (Level 2)• We want to investigate effects at both

levels.

Get the Data

• Download data file in XLS format• Print the Cheat-Sheet• Boot up SAS and import the data• name the imported member “HLM”• The first analysis will be have no

predictors, only means.

Level 1 Equation

• The math achievement of the ith student at the jth school =

• the intercept for the jth school (the mean at that school)

• + error.Note: We are using “0” instead of “a” to stand for intercept.

ijjij eY 0

Level 2 Equation

• The intercept (mean match achievement) at the jth school =

• The average intercept (mean) across schools

• + (group j intercept) – (overall intercept) – the effect of being in the jth school.

jj 0000

Combine the Two Equations

• Substitute (00 + 0j), from the Level 2 equation, for 0j in the Level 1 equation

• A student’s score = • The average intercept across schools• + the effect of being at the jth school• + other stuff

ijjij eY 000

SAStitle 'Model 1: Unconditional Means Model, Intercepts Only'; options formdlim='-' pageno=min nodate; proc mixed data = covtest noclprint; class School; model MathAch = / solution; random intercept / subject = School;run;

Fixed Effects

• Effects that are constant across schools.• model MathAch =/• No effect follows "=," the only parameter

estimated will be mean across schools.

Solution for Fixed EffectsEffect Estimate Standard

ErrorDF t Value Pr > |t|

Intercept 12.6370 0.2443 159 51.72 <.0001

Random Effects• Effects that vary across schools.• random intercept / subject = School;

• There is significant variance across schools (8.6097) in intercepts.

• And among students within schools (39.1487)

Covariance Parameter Estimates

Cov Parm Subject Estimate Standard Error

Z Value Pr > Z

Intercept School 8.6097 1.0778 7.99 <.0001

Residual   39.1487 0.6607 59.26 <.0001

Intraclass Correlation

• the proportion of the variance in MathAch that is due to differences among schools

• = (that due to schools) / (total variance)• = 8.6097 / (8.6097 + 39.1487) = 18%.

Next, we are going to add a Level 2 predictor, MeanSES. Do note that this variable has been centered to mean 0.

Add a Level 2 Predictor

• The intercept (mean math achievement) at the jth school =

• The average intercept across schools when all predictors have value 0 (the mean, since we centered MeanSES).

• + the effect of being in a school with the MeanSES of school j

• + the effect of everything else on which j differs from the other schools.

jjj 001000 MeanSES

title 'Model 2: Including Effects of School (Level 2) Predictors'; title2 '-- predicting MathAch from MeanSES'; run;proc mixed covtest noclprint; class school; model MathAch = MeanSES / solution ddfm = bw; random intercept / subject = school;run;• “bw” specifies between/within partitioning of df

Fixed Effects• MeanSES was centered about zero. That is,

transformed to mean zero.

• Math Achievement = 12.6495 + 5.8635(School MeanSES – GrandMean SES)

• Each 1 point increase in School’s mean SES increases achievement by 5.86 points.

Solution for Fixed Effects

Effect Estimate Standard Error

DF t Value Pr > |t|

Intercept 12.6495 0.1492 158 84.77 <.0001

MeanSES 5.8635 0.3613 158 16.23 <.0001

Random Effects

• Including school mean SES in model reduced variance in intercepts from 8.6097 to 2.6357 = a drop of 5.974.

• School mean SES accounts for 5.974/8.6097 = 69% of the variance among schools.

Covariance Parameter Estimates

Cov Parm Subject Estimate Standard Error

Z Value Pr > Z

Intercept School 2.6357 0.4036 6.53 <.0001

Residual   39.1578 0.6608 59.26 <.0001

Unexplained Variance

• After accounting for SES, MathAch intercepts (means) still differ significantly across schools (z = 6.53)

• Residual variance = 2.6357 (among schools) + 39.1578 (within schools) = 41.7935.

• 2.6357/41.7935 = 6.3% remains to be explained by some other Level 2 predictor.

Use a Level 1 Predictor

• Score for the ijth student = the intercept for the jth school

+ the effect of this student’s SES

+ other things involving that student.• Student SES will be centered by

subtracting from it the mean SES at the student’s school

ijijjjij eY SES10

Level 2 Equations

• Have dropped the MeanSAS predictor• Need a random intercept and a random

slope.

• Intercept for School j =Grand intercept

+ effect of being in School j

jj 0000

• Slope (for relating student’s SES to MathAch) at School j =The grand slope

+ the effect (on slope) of being at School j

jj 1101

Combined Equation

• The fixed effects

• The random effects

])SES([ SES1000 ijcijij eMYj

ijijjj eMj

)SES([ SES10

title 'Model 3: Including Effects of Student-Level Predictors';title2 '--predicting MathAch from cSES'; data HLM2; set HLM; cSES = SES - MeanSES; run; proc mixed data = hsbc noclprint covtest noitprint; class School; model MathAch = cSES / solution ddfm = bw notest; random intercept cSES / subject = School type = un; run;

Fixed Effects

• estimated MathAch for a student whose SES is average for his or her school is 12.6493

• average slope, across schools, for predicting MathAch from student SES is 2.1932, which is significantly different from zero

Solution for Fixed EffectsEffect Estimate Standard

ErrorDF t Value Pr > |t|

Intercept 12.6493 0.2445 159 51.75 <.0001cSES 2.1932 0.1283 7024 17.10 <.0001

Random Effects

• The estimated variance in intercepts, across schools, is a significant 8.6769, even after controlling for student SES.

Covariance Parameter EstimatesCov Parm Subject Estimate Standard

ErrorZ Value Pr Z

UN(1,1) School 8.6769 1.0786 8.04 <.0001

• The correlation between School Intercept and School Slope is a nonsignificant .051.

Covariance Parameter EstimatesCov Parm Subject Estimate Standard

ErrorZ Value Pr Z

UN(2,1) School 0.05075 0.4062 0.12 0.9006

• The variance in slopes (for predicting MathAch from student SES) is a significant .694.

• The slopes differ significantly across schools.

Covariance Parameter EstimatesCov Parm Subject Estimate Standard

ErrorZ Value Pr Z

UN(2,2) School 0.6940 0.2808 2.47 0.0067

• There remains significant within-school variance, 36.7, after controlling for student SES.

• SES accounted for 39.1487-36.7006 – 2.4481 units of variance, or 2.4881/39.1487* = 6.25% of the within-school variance.

*See Slide 9

Covariance Parameter EstimatesCov Parm Subject Estimate Standard

ErrorZ Value Pr Z

Residual   36.7006 0.6258 58.65 <.0001

Predictors at Both Levels

• Level 1: Student’s SES• Level 2: School mean SES• And a new Level 2 predictor, whether the

school is in the public sector (0) or is Catholic (1).

title 'Model 4: Model with Predictors From Both Levels and Interactions';proc mixed noclprint covtest noitprint; class School; model mathach = MeanSES sector cSES MeanSES*Sector MeanSES*cSES Sector*cSES MeanSES*Sector*cSES / solution ddfm = bw notest; random intercept cSES /subject = School type = un;run;

Nonsignificant Fixed Effects

• Without further comment, I shall drop these two nonsignificant interactions from the model

Solution for Fixed Effects

Effect Estimate Standard Error

DF t Value Pr > |t|

MeanSES*Sector -1.1276 0.7355 156 -1.53 0.1273

MeanSES*Sector*cSES -0.1888 0.5997 7021 -0.31 0.7528

Our Reduced Model

• Level 1• Level 2, intercepts

• Level 2, slopes

ijjjij ecSESY 10

jjjj SectorMeanSES 00201000

jjjj SectorMeanSES 11211101

.

Combined Equation• Fixed Effects

– Grand intercept– Overall slope for School SES (Predictor 1)– Overall slope School Sector (Predictor 2)– Overall slope for Student SES– Interaction between School SES and Student SES– Interaction between School Sector and Student

SES

ijjjj

jjjj

jjij

ecSES

cSESSectorcSESMeanSEScSES

SectorMeanSESY

10

121110

020100

]

[

• Random Effects

• Effect on intercept of being at School j• Effect on slope being at School j• Everything else affecting Student i at

School j

][ 10 ijjjj ecSES

title 'Model 5: Model with Two Interactions Deleted'; title2 '--predicting mathach from meanses, sector, cses and ';title3 'cross level interaction of meanses and sector with cses'; run; proc mixed noclprint covtest noitprint; class School; model MathAch = MeanSES Sector cSES MeanSES*cSES Sector*cSES /solution ddfm = bw notest; random intercept cSES / subject = School type = un;proc means mean q1 q3 min max skewness kurtosis; var MeanSES Sector cSES; run;

Fixed EffectsSolution for Fixed EffectsEffect Estimate Standar

d ErrorDF t Value Pr > |t|

Intercept 12.1136 0.1988 157 60.93 <.0001MeanSES 5.3391 0.3693 157 14.46 <.0001Sector 1.2167 0.3064 157 3.97 0.0001cSES 2.9388 0.1551 7022 18.95 <.0001MeanSES*cSES 1.0389 0.2989 7022 3.48 0.0005

Sector*cSES -1.6426 0.2398 7022 -6.85 <.0001

Interpret New Effects

• Sector: Math achievement is higher at Catholic Schools

• MeanSES x cSES: the slopes for predicting MathAch from cSES differ across levels of MeanSES.

• Sector x cSES: the slopes for predicting MathAch from cSES differ between public and Catholic schools

Sector x cSES

• In the combined equation, substitute 0 for value of sector to get equation for public schools

• And 1 to get equation for Catholic schools• Public: 12.11 + 5.34(MeanSES) +

2.94(cSES) + 1.04(MeanSES)(cSES)• Catholic: 13.33 + 5.34(MeanSES) +

1.30(cSES) + 1.04(MeanSES)(cSES)

• Intercept higher for Catholic than for public – MathAch higher at Catholic schools.

• Slope for student SES higher at public schools than at Catholic schools.

MeanSES x cSES

• Find Q1, Q2, and Q3 for School SES

• Substitute the quartile values into the combined equation to get one equation for each quartile.

• For each of two values (-3, +3) of cSES, predict MathAch at each value with each equation.

• Prepare table and plot of predicted values.

• Notice that the slope increases as MeanSES increases

MeanSEScSES

Diffrnc-3 +3

Q1 5.57 16.43 10.86Q2 6.23 19.25 13.02Q3 7.06 21.88 14.82

Random Effects

• UN(1,1): The intercepts still differ significantly across schools.

Covariance Parameter Estimates

Cov Parm Subject Estimate Standard Error

Z Value Pr Z

UN(1,1) School 2.3817 0.3717 6.41 <.0001

UN(2,1) School 0.1926 0.2045 0.94 0.3464

UN(2,2) School 0.1014 0.2138 0.47 0.3177

Residual   36.7212 0.6261 58.65 <.0001

• UN(2,1): No significant correlation between intercepts and slopes.

• UN(2,2): The slopes (predicting MathAch from cSES) do not differ significantly across schools.

• I shall drop cSES from the random effects.

Trimmed Modeltitle 'Model 6: Simpler Model Without cSES Slopes';proc mixed noclprint covtest noitprint; class School; model MathAch = MeanSES Sector cSES MeanSES*cSES Sector*cSES / solution ddfm = bw notest; random intercept / subject = School; run;

Effects of Trimming

• All of the fixed effects are still significant.• Intercepts still differ significantly across

schools.• The Log Likelihood statistic has increased

from 46503.7 to 46504.8, indicating slightly poorer fit.

• We can evaluate the difference in Log Likelihood statistics via Chi-square on 2 df. p = .58.