Testing Multilevel Theories Through Multilevel Structural Equation Modeling

Christopher R. Beasley

Midwest Eco 2013

Digital Copy of Slides

crbeasley.info

Structural Equation Modeling

Social

Desirability

Value

Congruence

Demands-

Abilities

Interpersonal

Similarity

Needs-

Supplies

Satisfaction

Commitment

OH TenureP-E Fit

Nesting

• Dependence

– Time

– Complex Sampling

Study Design

• Random sample by house

• Oxford House residents

– 95% of houses agreed to assist

• nj = 82

– 48% individual response rate

• ni = 296

Why Multilevel

• Disaggregation of data (Byrne, 2012)

– Biased estimates and standard errors

• Aggregated data (Byrne, 2012)

– Lack of individual variance may exaggerate group

effects

MSEM

• Extension of Multi-Group SEM

– Covariance matrices at within and between level

instead of for different groups

• Assumptions

– General Linear Model assumptions

• Linearity

• Normality

• Homoscedasticity

• Independence

MSEM Alternatives

• Segregated Approach (Yuan & Bentler, 2007)

– More established

– Modification indices

• Partially Saturated Approach (Ryu & West, 2009)

– MLR

– Random coefficients

– 2-1-1, 2-1-2, 2-2-2, 1-1-2, 1-2-1, 1-1-1

– Better power for level 2

– Greater n at L1, so more influence on model fit statistics (Hox, 2002)

Software Packages

• Mplus

• LISREL

• EQS

• GLLAMM

Process (Stapleton, 2013)

1. Consider single-level model

2. Baseline models for w/i & b/t levels while saturating the other level

3. Theoretical w/i model with b/t saturated for model fit

4. Theoretical b/t model with w/i saturated for model fit

5. Combined w/i & b/t theoretical model for parameters

6. Evaluate random coefficients

1. Consider Single Model

• Consider single-level model

– Null model for descriptives

• Define ICC < 0.02 as within unless very large cluster sizes, then possibly lower threshold

WITHIN ARE OHFITVC;

Between-Group Variance

ANALYSIS: TYPE IS TWOLEVEL RANDOM;

MODEL:

%WITHIN%

OHFITVC;

%BETWEEN%

OHFITVC;

Observed VariablesNull

ICC

Value Congruence 0.03

Commitment 0.11

Citizenship Behavior 0.04

1.1 Multilevel Reliability

ANALYSIS: TYPE IS TWOLEVEL;

MODEL:

%WITHIN%

VCw@1;

OHGEF3R OHGEF7R OHGEF15 (WR1-

WR3);

VCw BY OHGEF3R* (WL1)

OHGEF7R (WL2)

OHGEF15 (WL3);

%BETWEEN%

VCb@1;

OHGEF3R OHGEF7R OHGEF15 (BR1-

BR3);

VCb BY OHGEF3R* (BL1)

OHGEF7R (BL2)

OHGEF15 (BL3);

MODEL CONSTRAINT: NEW(NUMW

DENOMW OMEGAW NUMB DENOMB OMEGAB);

NUMW = (WL1+WL2+WL3)**2;

DENOMW = NUMW+(WR1+WR2+WR3);

OMEGAW = NUMW/DENOMW;

NUMB = (BL1+BL2+BL3)**2;

DENOMB = NUMB+(BR1+BR2+BR3);

OMEGAB = NUMB/DENOMB;

WR1 > 0; BR1 > 0;

WR2 > 0; BR2 > 0;

WR3 > 0; BR3 > 0;

Geldhof, Preacher, & Zyhur (in press)

1.1 Measurement Reliability

Measures ωw ωb

Satisfaction 0.77 0.92

Demands-Abilities Fit 0.78 0.97

Needs-Supplies Fit 0.79 0.92

Interpersonal Similarity 0.79 0.98

Social Desirability 0.88 0.96

Value Congruence 0.90 0.93

Tenure 0.91 0.93

Commitment 0.91 0.99

1.2 Descriptive Statistics

DEFINE: SDESIR=SDESIR/3;

Observed Variables n Min Max Mean SD SE

Social Desirability 292 0 13 6.67 3.23 0.19

Tenure 291 1 5 1.78 0.81 0.05

Commitment 292 2 7 5.26 1.10 0.06

Satisfaction 293 2 7 6.14 1.01 0.06

Interpersonal Similarity 291 1 5 3.42 0.96 0.06

Value Congruence 291 1 5 3.88 0.76 0.04

Demands-Abilities 293 1 5 3.96 0.80 0.05

Needs-Supplies Fit 293 1 5 4.01 0.70 0.04

2. Baseline Models

Baseline w/i Model

MODEL:

%WITHIN%

OHFITVC with OHCit@0 OHCOM@0;

OHCit with OHCOM@0;

%BETWEEN%

OHFITVC with OHCit OHCOM;

OHCit with OHCOM;

Baseline b/t Model

MODEL:

%WITHIN%

OHFITVC with OHCit OHCOM;

OHCit with OHCOM;

%BETWEEN%

OHFITVC with OHCit@0 OHCOM@0;

OHCit with OHCOM@0;

Commitment

3. Theoretical w/i Model

Value

Congruence

Citizenship

Behavior

CommitmentValue

Congruence

Citizenship

Behavior

3. Theoretical w/i Model

ANALYSIS: TYPE IS TWOLEVEL;

MODEL:

%WITHIN%

OHCOM on OHFITVC;

OHCOM on OHCit;

%BETWEEN%

OHCOM with OHFITVC OHCit;

OHFITVC with OHCIT;

3. Theoretical w/i Model

Chi-Square Test of Model Fit

Value 38.019*

Degrees of Freedom 1

P-Value 0.0000

Scaling Correction Factor

RMSEA

Estimate 0.359

CFI/TLI

CFI 0.895

TLI 0.373

SRMR

Value for Within 0.169

Value for Between 0.010

Commitment

4. Theoretical b/t Model

Value

Congruence

Citizenship

Behavior

CommitmentValue

Congruence

Citizenship

Behavior

4. Theoretical b/t Model

ANALYSIS: TYPE IS TWOLEVEL;

MODEL:

%WITHIN%

OHCOM with OHFITVC OHCit;

OHFITVC with OHCIT;

%BETWEEN%

OHCOM OHFITVC OHCit;

4. Theoretical b/t Model

Chi-Square Test of Model Fit

Value 5.192*

Degrees of Freedom 3

P-Value 0.1583

Scaling Correction Factor

RMSEA

Estimate 0.050

CFI/TLI

CFI 0.994

TLI 0.988

SRMR

Value for Within 0.022

Value for Between 0.693

Commitment

3. Theoretical Combined Model

Value

Congruence

Citizenship

Behavior

CommitmentValue

Congruence

Citizenship

Behavior

CommitmentValue

Congruence

Citizenship

Behavior

5. Combined Model

ANALYSIS: TYPE IS TWOLEVEL;

MODEL:

%WITHIN%

OHCOM on OHFITVC;

OHCOM on OHCit;

%BETWEEN%

OHCOM OHFITVC OHCit;

5. Combined Model Two-Tailed

Estimate S.E. Est./S.E. P-Value

Within Level

OHCOM ON

OHFITVC 0.711 0.079 9.025 0.000

OHCIT 0.338 0.055 6.168 0.000

Between Level

Means

OHCOM 5.250 0.076 68.744 0.000

OHCIT 5.870 0.058 101.094 0.000

OHFITVC 3.887 0.046 83.593 0.000

Commitment

6. Random Coefficients

Value

Congruence

Citizenship

Behavior

CommitmentValue

Congruence

Citizenship

Behavior

CommitmentValue

Congruence

Citizenship

Behavior

Random

Slope

6. Random Coefficients

LISTWISE=ON; (or Monte Carlo integration)

ANALYSIS: TYPE IS TWOLEVEL;

MODEL:

%WITHIN%

OHCOM on OHFITVC;

s1 | OHCOM ON OHCit;

%BETWEEN%

OHCOM OHFITVC OHCit;

6. Random Coefficients

LISTWISE=ON;

ANALYSIS: TYPE IS TWOLEVEL;

MODEL:

%WITHIN%

OHCOM on OHFITVC;

s1 | OHCOM ON OHCit;

%BETWEEN%

OHCOM OHFITVC OHCit;

s1 on OHFITVC;

6. Random CoefficientsLoglikelihood

H0 Value -1074.838

H0 Scaling Correction Factor 0.9723

for MLR

Information Criteria

Akaike (AIC) 2175.675

Bayesian (BIC) 2223.249

Sample-Size Adjusted BIC 2182.024

6. Random CoefficientsTwo-Tailed

Estimate S.E. Est./S.E. P-Value

Between Level

S1 ON

OHFITVC 0.327 0.112 2.905 0.004

Intercepts

S1 -1.099 0.454 -2.421 0.015

Unresolved Issues

• Sample & Power Estimation

• Model Fit Approaches

• Model Fit Statistics

• MLR Performance in Multilevel Model

• 3+ Levels

• Balancing of Cluster Sizes

Unresolved Issues - Saturation

• “In the MODEL command, the following variable

is a y-variable (endogenous) on the BETWEEN level and an x-variable (exogenous) on the WITHIN level.

• This variable will be treated as a y-variable on both levels: OHGEFIS”

• Any discrepancy is treated as a y-variable on both levels

Other Models

• Longitudinal Multilevel Models

• Latent Models

• Multilevel EFA

• Multilevel CFA

• 3+ Levels

Other Topics

• Convergence Problems

• Power Analysis & Sample Size

• Alternative Estimators (MLR default)

– MUML, Bayes

• Random Starting Seeds

• Interval Estimates

– Bayes

– Monte Carlo

Resources

• Barbara Byrne

Structural Equation Modeling with Mplus

• Hancock & Mueller Structural Equation Modeling: A Second Course

• Little, Bovarid, & Card Modeling Contextual Effects in Long. Studies

• Kris Preacher http://www.quantpsy.org/pubs.htm

• Steve Miller “Things Statistical” http://personalityandemotion.com/