Detecting local dependence in latent class models

Example LCA Local dependence BVR and MI EPC Conclusions References

Detecting local dependence inlatent class models

Daniel Oberski

Department of methodology and statistics

(Based on joint work with Jeroen Vermunt and Geert Van Kollenburg)

Detecting local dependence in latent class models Daniel Oberski


Latent class analysis

• Latent class analysis (LCA) used for: model-basedclassification, clustering, latent structure analysis,estimating false positives and false negatives, (specificityand sensitivity) of error-prone variables;

• assumes local independence;

• If local dependence, severe bias can occur.



Latent class analysis

• Latent class analysis (LCA) used for: model-basedclassification, clustering, latent structure analysis,estimating false positives and false negatives, (specificityand sensitivity) of error-prone variables;

• assumes local independence;• If local dependence, severe bias can occur.



Detecting local dependence in LCA

• Local dependence in LCA leads to model misfit;• Usual reaction: increase number of classes;

This talk:• Do not increase number of classes;• Model the local dependencies directly;

• Use specifically tailored measures to detect which localdependencies should be modeled:

• Bivariate residuals (BVR),• Score test or ``modification index'' (MI),• Expected parameter change (EPC).





This talk:• Do not increase number of classes;• Model the local dependencies directly;• Use specifically tailored measures to detect which localdependencies should be modeled:







• Bivariate residuals (BVR),

• Score test or ``modification index'' (MI),• Expected parameter change (EPC).






• Bivariate residuals (BVR),• Score test or ``modification index'' (MI),

• Expected parameter change (EPC).



..1 Example LCA

..2 Local dependence

..3 BVR and MI

..4 EPC

..5 Conclusions



Example LCA: dentists rating caries in xrays




Here is an xray of a person who possibly has caries:




y1 : Yes (1)

y2 : No (0)

y3 : No (0)

y4 : Yes (1)



Actual dentists' ratings collected by Espeland &Handelman (1989)

> dentistsVar1 Var2 Var3 Var4 Var5 Observed

1 0 0 0 0 0 18802 0 0 0 0 1 7893 0 0 0 1 0 434 0 0 0 1 1 755 0 0 1 0 0 236 0 0 1 0 1 637 0 0 1 1 0 88 0 0 1 1 1 229 0 1 0 0 0 18810 0 1 0 0 1 19111 0 1 0 1 0 17... etc.



2-class local independence model for dentists



Local independence latent class model

Patternwise likelihood:

Pr(Y = y) =∑t

Pr(Y = y|ξ = t)Pr(ξ = t), (1)

where the conditional probability of the response patterns is

Pr(Y = y|ξ = t) = exp(Xθ)

1T exp(Xθ)(2)

With X a design matrix containing:• Observed variables main effects (intercepts);• Latent class × observed variables interaction.



Estimation of local independence latent class model

Log-likelihoodℓ(θ) = nT logPr(Y = y),

with vector n the observed frequency of each responsepattern. Total sample size is N. Maximum likelihood estimates

θ̂ = argmaxθ∈Rq

ℓ(θ)

by expectation-maximization, quasi-Newton, or a combination.This gives expected frequencies µ̂ := N · Pr(Y = y|θ = θ̂)



2-class local independence model for dentists' ratings

Possible goals:

Goal: How LCA is useful:* See how well dentists rate xrays ``sensitivity'' & ``specificity''

(conditional probabilities)from intercepts and slopes

* Classify each photograph Pattern posteriors giveprobabilistic classification

* Estimate ``true'' caries prevalence Latent class proportions

Local dependence biases all of these results.



Questions for you

• What does the latent class variable represent?• And if the number of classes is increased?

Model fit:

L2 X2 df BIC AIC AIC3 CAIC129.85 132.00 20 -35.37 89.85 69.85 -35.37

• The model does not fit. What could be a reason?• What should be done?



Local dependence latent class models



2-class local dependence model for dentists



Local dependence latent class model

Conditional probability of the response patterns still

Pr(Y = y|ξ = t) = exp(Xθ)

1T exp(Xθ)

But now X is a design matrix containing:• Observed variables main effects (intercepts);• Latent class × observed variables interaction;• (Some) observed variables × observed variables 2-wayinteractions;



• Can we find a locally dependent two-class model that fits?• Which local dependence parameters are needed?

•• Goal: Examine fit of each pair of variables (dentist ratings)to the local independence assumption without fitting allpossible alternative models.



Detecting local dependencies with the BVR and MI



Fit of two-way cross-table between dentists 1 and 3

Observed

No YesNo 3250 280Yes 123 216

Expected

No YesNo 3217 313Yes 156 183

Bivariate residuals

No YesNo 32.6 -32.6Yes -32.6 32.6

BVR1,3 = r112∑k,l

µ̂−1kl = (32.6)2

∑k,l

µ̂−1kl ≈ 1063(0.0154) ≈ 16.3

MI1,3 = r112 Var(r11)−1 = (32.6)2/(31.3) ≈ 1063(0.0320) ≈ 34.0



Local dependencies between five dentists' x-rayratings for caries.

Dentist dependence MI BVR1 ↔ 2 3.1 1.41 ↔ 3 34.0 ** 16.31 ↔ 4 13.1 ** 7.71 ↔ 5 2.7 0.82 ↔ 3 6.8 * 1.72 ↔ 4 1.8 0.62 ↔ 5 16.4 ** 4.73 ↔ 4 2.7 1.03 ↔ 5 5.1 * 0.84 ↔ 5 3.5 1.0

Note: ** Sig. (α = 0.05) with Bonferroni correction for multipletesting, * without correction



BVR and MI

``Bivariate residual'' (BVR):• Pearson "chi-square" residual in cross-table of a pair ofobserved variables: ∑(observed - expected)2/expected

• Surprise! Not a chi square statistic!



●

●●

●

●●

● ●●

● ●●

●●●●●●●●●●●●●●●●●●●

● ●●

●●

●

●

●

●

●

●

0 20 40 60 80 100

020406080

100

MI equals chi−square improvement...

χ2

MI

●

●●

●

●

●

●●●

● ●●

●●●●●●●●●●●●●●●●●● ● ● ●● ●●

● ● ●●●

●

0 20 40 60 80 100

020406080

100

... BVR does not.

χ2

BV

R



BVR and MI

``Bivariate residual'' (BVR):• Pearson residual in cross-table of a pair of observedvariables: (observed - expected)2/expected;

• Not a chi square statistic.

``Modification index'' (MI):• ``Score test'' (``Lagrange multiplier'' test) for introducing alocal dependency between a pair of variables;

• Turns out to use the same residual as the BVR, but withthe correct variance! (thanks Jeroen)

• Under the null hypothesis, chi-square distributed (1 df)



BVR not even close to chi-square distributed, MI is.

Monte Carlo simulation under the null hypothesisCondition α for nominal 5% Empirical distribution

BVR MI BVRλ n Naive Boot MI Mean Var Mean Var

0.5 200 0.000 0.050 0.051 0.97 1.7 0.33 0.20.5 500 0.000 0.020 0.050 1.06 2.3 0.36 0.20.5 1000 0.000 0.060 0.065 0.96 1.9 0.33 0.20.5 5000 0.000 0.085 0.055 0.97 2.0 0.34 0.20.8 200 0.000 0.065 0.040 1.04 1.7 0.25 0.10.8 500 0.000 0.070 0.060 1.05 2.0 0.25 0.10.8 1000 0.000 0.060 0.090 1.22 2.6 0.30 0.20.8 5000 0.000 0.035 0.060 1.16 3.1 0.28 0.2Should be: 0.050 0.050 0.050 1.00 2.0 1.00 2.0



BVR and MI: main findings

• Many (most?) published applied studies using BVRpretend it is a chi-square statistic*;

• BUT, if pretend BVR is chi-square:

• BVR does not provide nominal α levels;• BVR power to detect local dependencies miserably low*.

• Referring MI to chi-square:

• MI provides approximately nominal α levels;• Power of MI is adequate except for very smalldependencies (±0.05)*.

•• Can bootstrap BVR values, leading to performance similarto, though slightly below, MI*

* See paper.





• BUT, if pretend BVR is chi-square:• BVR does not provide nominal α levels;

• BVR power to detect local dependencies miserably low*.• Referring MI to chi-square:



* See paper.





• BUT, if pretend BVR is chi-square:• BVR does not provide nominal α levels;• BVR power to detect local dependencies miserably low*.

• Referring MI to chi-square:



* See paper.






• Referring MI to chi-square:• MI provides approximately nominal α levels;

• Power of MI is adequate except for very smalldependencies (±0.05)*.


* See paper.






• Referring MI to chi-square:• MI provides approximately nominal α levels;• Power of MI is adequate except for very smalldependencies (±0.05)*.


* See paper.



Assessing substantive size of local dependencies with the EPC



Assessing substantive size besides statisticalsiginificance

``Modification index'' (MI):• Statistical ``score'' test of the hypothesis that the localdependence between two variables, say ψ, equals zero.

• ψ corresponds to the observed variable × observedvariable 2-way interaction in conditional prob. model;

• Only statistical significance → MI depends on:

• Sample size;• Size of the loadings of the two variables;• Size of the loadings of the other variables;• Latent variable intercept(s);• Observed variable intercepts;• Regression coefficients of covariates if present.






• Only statistical significance → MI depends on:• Sample size;

• Size of the loadings of the two variables;• Size of the loadings of the other variables;• Latent variable intercept(s);• Observed variable intercepts;• Regression coefficients of covariates if present.






• Only statistical significance → MI depends on:• Sample size;• Size of the loadings of the two variables;

• Size of the loadings of the other variables;• Latent variable intercept(s);• Observed variable intercepts;• Regression coefficients of covariates if present.






• Only statistical significance → MI depends on:• Sample size;• Size of the loadings of the two variables;• Size of the loadings of the other variables;

• Latent variable intercept(s);• Observed variable intercepts;• Regression coefficients of covariates if present.






• Only statistical significance → MI depends on:• Sample size;• Size of the loadings of the two variables;• Size of the loadings of the other variables;• Latent variable intercept(s);

• Observed variable intercepts;• Regression coefficients of covariates if present.






• Only statistical significance → MI depends on:• Sample size;• Size of the loadings of the two variables;• Size of the loadings of the other variables;• Latent variable intercept(s);• Observed variable intercepts;

• Regression coefficients of covariates if present.






• Only statistical significance → MI depends on:• Sample size;• Size of the loadings of the two variables;• Size of the loadings of the other variables;• Latent variable intercept(s);• Observed variable intercepts;• Regression coefficients of covariates if present.




``Expected parameter change'' (EPC):• Estimate of ψ, the strength of the local dependencebetween two variables;

• No need to fit alternative model;

• Assesses substantive rather than statistical significance.




``Expected parameter change'' (EPC):• Estimate of ψ, the strength of the local dependencebetween two variables;

• No need to fit alternative model;• Assesses substantive rather than statistical significance.



Local dependencies between five dentists' x-rayratings for caries.

Dentist dependence EPC MI BVR1 ↔ 2 -0.081 3.1 1.41 ↔ 3 -0.261 34.0 ** 16.31 ↔ 4 -0.146 13.1 ** 7.71 ↔ 5 -0.117 2.7 0.82 ↔ 3 -0.140 6.8 * 1.72 ↔ 4 -0.058 1.8 0.62 ↔ 5 -0.157 16.4 ** 4.73 ↔ 4 0.074 2.7 1.03 ↔ 5 -0.191 5.1 * 0.84 ↔ 5 -0.104 3.5 1.0

Note: ** Sig. (α = 0.05) with Bonferroni correction for multipletesting, * without correction



Based on EPC and MI,• keep two-classes, but...• ...free five out of ten possible bivariate localdependencies.

•• 15 degrees of freedom (was 20)• L2 = 28.4 (p = 0.07) (was 129.9)• BIC is −95.5 (was −35.37)• Best two-class model in lit. (Qu et al. 1996): BIC −83.4




•• 15 degrees of freedom (was 20)

• L2 = 28.4 (p = 0.07) (was 129.9)• BIC is −95.5 (was −35.37)• Best two-class model in lit. (Qu et al. 1996): BIC −83.4




•• 15 degrees of freedom (was 20)• L2 = 28.4 (p = 0.07) (was 129.9)

• BIC is −95.5 (was −35.37)• Best two-class model in lit. (Qu et al. 1996): BIC −83.4




•• 15 degrees of freedom (was 20)• L2 = 28.4 (p = 0.07) (was 129.9)• BIC is −95.5 (was −35.37)

• Best two-class model in lit. (Qu et al. 1996): BIC −83.4




•• 15 degrees of freedom (was 20)• L2 = 28.4 (p = 0.07) (was 129.9)• BIC is −95.5 (was −35.37)• Best two-class model in lit. (Qu et al. 1996): BIC −83.4



Conclusions



Conclusions

• Local dependence is a serious problem in LCA;• Sometimes modeling dependence directly is preferable toincreasing no. classes;

• Need to monitor local dependence.

•• Introduced MI and BVR, tests for local dependence:

• MI works well as chi-square statistic;• BVR does not, but can bootstrap p-values.

• Introduced EPC: local dependence substantive size.•• Dentist example: obtained more parsimonious andeasy-to-interpret two-class model than current literature.



Conclusions



•• Introduced MI and BVR, tests for local dependence:• MI works well as chi-square statistic;

• BVR does not, but can bootstrap p-values.• Introduced EPC: local dependence substantive size.•• Dentist example: obtained more parsimonious andeasy-to-interpret two-class model than current literature.



Conclusions



•• Introduced MI and BVR, tests for local dependence:• MI works well as chi-square statistic;• BVR does not, but can bootstrap p-values.




Conclusions




• Introduced EPC: local dependence substantive size.

•• Dentist example: obtained more parsimonious andeasy-to-interpret two-class model than current literature.



Conclusions







• Class-specific local dependence;• Multivariate MI, EPC, (polytomous items);• Effect of freeing local dependencies on params of interest;• Problem of parameter dependence;• Problem of multiple comparisons;• (Post-hoc) power of MI (Satorra);• MI for other params than locdeps (e.g. item bias).



Thank you for your attention!

Daniel [email protected]

Working papers on this topic:Oberski, D., Van Kollenburg, G., and Vermunt, J. (submitted). AMonte Carlo evaluation of three methods to detect localdependence in binary data latent class models.

Oberski, D. and Vermunt, J. (submitted). The Expected ParameterChange (EPC) for local dependence assessment in binary datalatent class models.

Oberski, D. (submitted). Change in SEM parameters of interest asa criterion for partial measurement invariance: TheEPC-interest.


Appendix

Appendix


Appendix

0.5 0.8

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

−0.4

−0.2

−0.05

0.050.2

0.4

200

500

1000

5000200

500

1000

5000

Log(sample size)

Pow

er

p−value type

BVR

BVR_bootstrap

MI


Appendix

Convergence to noncentral chi-square in worstcondition

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●●

●●

●

●

Sample size: 200Q−Q plot, noncentral χ1

2(4.7)

Theoretical quantiles

MI

0

10

20

30

40

0 5 10 15 20 25

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●

● ●


2(11.8)


MI

010203040506070

0 10 20 30 40

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●

●●●●●●●

●●●●●

●


2(23.7)

Theoretical quantilesM

I

0

20

40

60

80

10 20 30 40 50 60

●●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●

●●

●


2(118)


MI

50

100

150

200

80 120 160


Appendix

Local dependencies between indicators of Hispanicethnicity in the U.S. Census

Local dependence EPCL TL EPCgs TGS ψ̃ WaldAncestry-re ↔ Language-in 0.92 5.0 1.45 7.9 0.22 0.2Ancestry-re ↔ Origin-in -0.76 2.5 -1.23 4.1 -0.23 0.1Ancestry-re ↔ Origin-re 2.94 45.6 1.32 20.5 1.82 18.7Language-in ↔ Origin-in 4.14 97.1 1.59 37.2 3.52 53.4Language-in ↔ Origin-re -1.08 7.9 -1.76 12.8 1.33 7.1

Origin-in ↔ Origin-re 1.10 6.1 2.20 12.2 0.52 0.3


Detecting local dependence in latent class models

Science

Transcript of Detecting local dependence in latent class models