Sun-Joo CHO Allan COHEN Brian BOTTGE
description
Transcript of Sun-Joo CHO Allan COHEN Brian BOTTGE
![Page 1: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/1.jpg)
Detecting Intervention Effects Using a Multilevel Latent
Transition Analysis with a Mixture IRT model
Sun-Joo CHO
Allan COHEN
Brian BOTTGE
![Page 2: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/2.jpg)
2
Context
• If you are asked by national Department of Education to test the efficacy of a new instructional method which was designed to improve learning disabilities students’ performance. Students’ performance were measured in two time points.
• People may concern that • (1) How much progress do student make? • (2) What’s the intervention effects?
![Page 3: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/3.jpg)
3
• (3) What does the progress depend on? (intervention, teacher or both)
• (4) How do children’s response process change over time?
• (5) Do student perform differently?
![Page 4: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/4.jpg)
4
Measuring Change
• CTT for measuring change
• 1. repeated ANOVA or MANOVA: mean change of a total score as a manifest variable across time points
• However, the assumption of sphericity (homogeneous and all correlation between any pair of measures equal) is often violated.
![Page 5: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/5.jpg)
5
LTA-LCM
• Latent Transition analysis with Latent Class Model
• LTA-LCM assumes that there is no variability on the latent trait within classes.
LCM
LTA-LCM
![Page 6: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/6.jpg)
6
LTA-MixIRTM
• Latent transition model with a mixture response model
Mixture Model
LTA_MixIRTM
![Page 7: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/7.jpg)
7
Multilevel LTA-MixIRTMMixture Proportion in teacher level latent class
Mixture Proportion in individual level latent class
Transition proportion IRT model
![Page 8: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/8.jpg)
8
• Proportion of the population in group-level latent class
• Proportion of the population in individual-level latent class at Time 1
• Transition proportion for t=2,…,T
![Page 9: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/9.jpg)
9
![Page 10: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/10.jpg)
10
Estimation
• Assumption: Item parameters are invariant across time and across group-level latent class (item parameters are estimated using post-test data)
• Anchor items (scale comparability among latent class?)
• Mplus (TWOLEVEL MIXTURE)
![Page 11: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/11.jpg)
11
Empirical Study Results
• 310 students nested under 49 teachers answer 20 items at both pre-test and post-test occasion
![Page 12: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/12.jpg)
12
• (1) Multilevel 1PL
ICC= 0.117 (pre-), 0.406 (post-)
(2) Detection of Latent Classes
A. Only one latent class at pre-test (very few students answer correctly)
B. Two latent class at post-test for student-level
C. Two latent class at post-test for teacher-level after the latent class at student level were determined
![Page 13: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/13.jpg)
13
![Page 14: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/14.jpg)
14
![Page 15: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/15.jpg)
15
![Page 16: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/16.jpg)
16
• Comparison between multilevel LTA-MixIRTM and LTA-MixIRTM: students were clustered into four transition patterns in multilevel LTA MixIRTM whereas there was two in the LTA MixIRTM.
![Page 17: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/17.jpg)
17
Do students engage in problem solving differently?
![Page 18: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/18.jpg)
18
How do students’ response change over time?
• On the post-test, 27.4% of the students were classified into student-level Class 2 (pattern 112 and 212). Higher transition proportion of pattern 212 indicate intervention effect.
![Page 19: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/19.jpg)
19
• Pattern 212 has larger mean and large variance, indicating much progress.
How much progress? Teachers/intervention?
![Page 20: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/20.jpg)
20
• In, Pattern 111 (pho=0.856), students tended to retain their relative positions.
• In Pattern 211 (pho=0.004), little variability for these student on post-test.
![Page 21: Sun-Joo CHO Allan COHEN Brian BOTTGE](https://reader033.fdocuments.in/reader033/viewer/2022051517/5681599e550346895dc6e9e4/html5/thumbnails/21.jpg)
21
• Questions:
• 1. How about item parameter recovery?
• 2. Anchor item is necessary?
• 3. Achilles’ heel in mixture model