Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann
description
Transcript of Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann
![Page 1: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/1.jpg)
1
Uses of a Statewide Longitudinal Data System to Evaluate and Inform Programs,
Policies, and Resource Allocations
Presented February 13, 2013
26th Annual Management Information Systems [MIS] Conference
Deborah T. Carran, Jacqueline Nunn, Sara HooksJohns Hopkins University
Stacey N. DammannYork College
![Page 2: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/2.jpg)
2013 MIS Conference 2
Background: Linking Data Sets
Part C Early Intervention Services data (Birth – 3)Part B Special Education Services data (4, 5 – 21)General Education Services dataK Children assessed for K Readiness
WSS-K, scaled then scores at 3 levels Readiness (Developing, Approaching, Fully Ready)
Maryland State Assessment (MSA), scaled then scored at 3 levels (Advance, Proficient, Basic)– Math and Reading assessments administered annually in
grades 3 through 8
![Page 3: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/3.jpg)
2013 MIS Conference 3
![Page 4: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/4.jpg)
2013 MIS Conference 4
Benefits of Using the Maryland IDEA Scorecard
• Data at the State, District, and School levels that drills down to the student level
• Data that allows users to identify students in need of targeted interventions in the alert categories (attendance, academics, suspension, and mobility)
• Data to create an action plan to monitor student progress within targeted interventions
• Reporting functions to support monitoring of progress towards targets of interventions
![Page 5: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/5.jpg)
2013 MIS Conference 5
How to use these data at the state level?
• Descriptive– Who do we serve?
• Longitudinal– What is the educational placement of children served in EIS by K
and Grade 3?– Is Fall K WSS is a successful predictor of later standardized test
performance?• Comparative– Is there a difference in standardized achievement performance
within educational services (Gen Ed and Sp Ed) for children served in EIS?
![Page 6: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/6.jpg)
2013 MIS Conference 6
Tracking Plan
Birth Cohort09/01/2001-08/31/2002
Yes EIS
Gen Ed Grade 3
Sp Ed Grade 3
No EIS
Gen Ed Grade 3
Sp Ed Grade 3
![Page 7: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/7.jpg)
2013 MIS Conference 7
Method
• PARTICIPANTS– 42% tracking match– Missing data
• Outcomes– WSS-K (2006-07)– RSAA and MSAA (Spring 2011)
• Procedure• Three studies presented
![Page 8: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/8.jpg)
2013 MIS Conference 8
Missing Data
• N = 5,328 participants received EIS– 42.1% (n = 2,245) matched – 57.9% (n = 3,083) unmatched (missing data)
• Is there a significant difference between matched and unmatched participants on characteristics: gender, Part C eligibility category, Part C MA, Race, age of entry, age of exit, and/or months in EIS?
![Page 9: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/9.jpg)
2013 MIS Conference 9
Gender
Female33%
Male67%
Unmatched
Female34%
Male66%
Matched
Results of the Chi-Square statistic assuming equal probabilities indicated no significant difference between Unmatched and Matched participants for gender [X2 (1, N = 5,328) = 0.93, p > .05]. Inspection of cell counts and percentages indicated that the distribution for gender was similar for both groups.
![Page 10: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/10.jpg)
2013 MIS Conference 10
Part C Eligibility 25% Delay Atypical Develp/Behav High Probability
N (%) N (%) N (%)
Unmatched 1,902 (61.9%) 463 (15.1%) 707 (23.0%)
Matched 1523 (67.8%) 376 (16.7%) 346 (15.4%)
Total 3,425 (64.4%) 839 (15.8%) 1053 (19.8%)
Results of the Chi-Square statistic assuming equal probabilities indicated a significant difference between Unmatched and Matched participants for Part C Eligibility [X2 (2, N = 5,317) = 47.23, p < .001]. Inspection of cell counts and percentages indicated that a greater proportion of participants in the Unmatched group were eligible due to a Condition with a High Probability than Matched participants. Conversely, a greater proportion of Matched participants were eligible based on 25% Delay than Unmatched participants.
![Page 11: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/11.jpg)
2013 MIS Conference 11
Part C Eligibility cont.
62%15%
23%
Unmatched25% Developmental DelayAtypical Devel or BehaviorHigh Probability
68%
17%
15%
Matched25% Developmental DelayAtypical Devel or BehaviorHigh Probability
![Page 12: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/12.jpg)
2013 MIS Conference 12
Part C Medical Assistance
Yes MA39%
No MA61%
Unmatched
Yes MA33%
No MA67%
Matched
Results of the Chi-Square statistic assuming equal probabilities indicated a significant difference between Unmatched and Matched participants for Part C Medical Assistance [X2 (1, N = 5,278) = 17.49, p = .001]. Inspection of cell counts and percentages indicated that a greater proportion of participants in the Unmatched group were eligible for Medical Assistance while receiving EIS.
![Page 13: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/13.jpg)
2013 MIS Conference 13
Race
Am Ind/Alaska Native
Asian or Pacific
Islander Black/AfAm Hispanic Multiple White
Unmatched 2 (0.1%) 121 (3.9%) 971 (31.5%) 237 (7.7%) 112 (3.6) 1640 (53.2%)
Matched 5 (.2%) 91 (4.1%) 606 (27%) 201 (9%) 75 (3.3%) 1267 (56.4%)
Total 7 (.1%) 212 (4%) 1577 (29.6%) 438 (8.2%) 187 (3.5%) 2907 (54.6%)
Results of the Chi-Square statistic assuming equal probabilities indicated a significant difference between Unmatched and Matched participants for Race [X2 (5, N = 5,328) = 16.76, p < .05]. Inspection of cell counts and percentages indicated that a greater proportion of participants in the Unmatched group were Black or African American while a smaller proportion of participants in the Unmatched group were Hispanic than in the Matched group.
![Page 14: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/14.jpg)
2013 MIS Conference 14
Entry/Exit Age in Months
EntryAge ExitAge Months in Program0
5
10
15
20
25
30
35
16.53
30.38
13.7412.23
28.9
16.71 UnmatchedMatched
![Page 15: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/15.jpg)
2013 MIS Conference 15
Summary – Missing Data• Gender – the two groups are similar• Eligibility – significantly greater proportion of Unmatched
participants in High Probability Category• Medical Assistance – significantly greater proportion of
Unmatched participants received MA• Race – significantly greater proportion of Black or African
American participants in Unmatched group with significantly greater proportion of Hispanic participants in Matched group
• Age – Matched participants were significantly younger at age of entry than Unmatched participants. Matched participants received EIS longer than Unmatched participants. Age of exit was similar for both groups.
![Page 16: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/16.jpg)
2013 MIS Conference 16
Outcome Instruments• Work Sampling System-Kindergarten (WSS-K)
Assesses 7 Domains, scaled then scored at 3 levels (Proficient, In Process, Needs Development)• Personal and Social Development• Language and Literacy• Mathematical Thinking• Scientific Thinking• Social Studies• The Arts• Physical Development
• Reading State Accountability Assessment (RSAA)– scaled then scored at 3 levels (Basic, Proficient, Advanced)
• Math State Accountability Assessment (MSAA)– scaled then scored at 3 levels (Basic, Proficient, Advanced)
![Page 17: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/17.jpg)
2013 MIS Conference 17
Procedure• Student data tracking system links records of students
in General Ed, Special Ed (Part B) and EIS (Part C)• Identify children in 3rd Grade 2010-11 school year
with birth dates between Sept. 1, 2001 and Aug. 31, 2002
• At the state level, student identifiers entered in student data tracking system to obtain student service level (Gen Ed or Sp Ed), student outcomes, and demographic info
• Deidentified data provided to researchers
![Page 18: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/18.jpg)
2013 MIS Conference 18
Studies and RQ for this Presentation1. Impact of Early Intervention on K Readiness– Who was served by EIS?– Does EIS impact later K Readiness Scores?
2. Tracking Children Receiving Early Intervention Services (Part C Services Birth to 3) into Elementary School to Grade 3– What is the educational placement at Grade 3 for
children who received EIS?– What is the SAA performance at Grade 3 for
children who received EIS?
![Page 19: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/19.jpg)
2013 MIS Conference 19
Different Data Set
3. K Readiness and Grade 3 SAA Performance– Is there a relationship between K Readiness and
Grade 3 RSAA and MSAA scoring?– What subscales of the WSS predict RSAA and
MSAA scoring?
![Page 20: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/20.jpg)
2013 MIS Conference 20
Study 1: Impact of Early Intervention Services on K Readiness
• Does level of service provided to children (Birth – 3) enrolled in early intervention services (EIS) programs enhance their later performance on the Kindergarten Work Sampling System (WSS-K)?
![Page 21: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/21.jpg)
2013 MIS Conference 21
K Readiness: Method• 2,245 children – Who were eligible and received EIS services in MD
linked with MD MMSR scores– Born between Sept 1, 2001 and Aug 31, 2002
• EIS services• WSS-K– Summary scaled composite score
• Hierarchical Linear Regression
![Page 22: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/22.jpg)
2013 MIS Conference 22
Participants
Birth Cohort09/01/2001-08/31/2002
Yes EISKindergarten
2006-07
![Page 23: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/23.jpg)
2013 MIS Conference 23
EIS Services & WSS-K Average ScoresService Description Children WSS-K WSS-K
N Mean SD
Audiology 474 73.2 13.1
Family Counseling/Training 375 71.7 13.2
Occupational Therapy 467 71.0 14.4
Physical Therapy 565 73.0 13.9
Special Instruction 1145 71.8 13.5
Speech/Language Therapy 1553 74.4 12.9
Other Services Than Above 335 73.1 13.6
At least 1 Service 2245 74.5 12.8
Max WSS-K score of 90
![Page 24: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/24.jpg)
2013 MIS Conference 24
VariablesOutcome: WSS-KPredictors:• Demographics: FaRMs, Gender, Minority
• Earliest Age of Child that Part C Services Begin(-) receive services earlier better prepares child to enter K
• Time in Program (+) indicates longer time in program better prepares child to enter K
• Total Minutes Services (+) indicates longer time in program better prepares child to enter K
![Page 25: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/25.jpg)
2013 MIS Conference 25
Hierarchical Regression ResultsVariables Model 1 Model 2
β β
FaRMs -.170** -.171**
Gender -.048* -.044*
Minority -.120** -.103**
Age Svc Began (months) -.069*
Time in Program (days) -.066*
Total Minutes Services -.138**
R2 .061 .086
R2 Change .025**
![Page 26: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/26.jpg)
2013 MIS Conference 26
EIS – WSS Conclusions• Demographic Controls: WSS-K was higher for students not
economically disadvantaged, higher for girls, and for White students
• Age Svc Began (-): For every month earlier a child starts receiving services, he/she is expected to score .017 SD increase on the WSS-K– Supports previous findings
• Time in Program (-): inconclusive; possible that children who are in the program for longer times have more severe disabilities
• Total Minutes Services (-): inconclusive, possible that children with more severe disabilities will have more/longer services– Correlation Time in Program & Total Min Svc r = .37, p < .001
![Page 27: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/27.jpg)
2013 MIS Conference 27
Study 2
Tracking Children Receiving Early Intervention Services (Birth to 3) into Elementary School by Service Level:
How do they Compare with their Peers?
![Page 28: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/28.jpg)
2013 MIS Conference 28
Background• Part C Services (IFSP)• Part b/B Services (IEP)• Little research examining outcomes of children
receiving Part C services, mostly Part B services (Cole, Dale, Mills & Jenkins, 1993; Daley & Carlson, 2009; Peterson et al., 2004; Walker et al., 1988)
– Focused on developmental progress over short term; not longitudinal or growth trajectories
– Enrollment in special education changes for children as they move through elementary school
– Limitations (disability, small samples, covariates)
![Page 29: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/29.jpg)
2013 MIS Conference 29
First Steps: Descriptive Analyses
• Children in 3rd grade (within birth cohort)– What is their Education Service (Gen Ed, Sp Ed)?– Who were the children receiving EIS?
![Page 30: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/30.jpg)
2013 MIS Conference 30
Participants
Birth Cohort09/01/2001-08/31/2002
Yes EIS
Gen Ed Grade 3
Sp Ed Grade 3
No EIS
Gen Ed Grade 3
Sp Ed Grade 3
![Page 31: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/31.jpg)
2013 MIS Conference 31
Total Sample, Gen Ed and HI Sp Ed
Characteristic N %Total Sample 52,584 100.0
General Education 47,928 91.1
Special Education (Part B) 4,656 8.9
Disability Codes 4, 6, 8, and 9 3,994 86.7
04-Speech or Language Impairments 1,898 40.7
06-Emotional Disturbance 224 4.8
08-Other Health Impairments 721 15.5
09-Specific Learning Disabilities 1,151 24.7
All Other Disability Codes* 665 14.3
*Autism, Deaf, Deaf-Blindness, Developmental Delay, Hearing Impaired, Intellectual Disability, Multiple Disabilities, Orthopedic Impairment, Traumatic Brain Injury, Visual Impairment
![Page 32: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/32.jpg)
2013 MIS Conference 32
Gender N %
Males 26,449 50.3
Females 26,135 49.7
New Race Descriptions
American Indian/Alaskan 140 0.3
Asian 3,108 5.9
Black/African American 17,044 32.4
Hispanic 6,611 12.6
Multiple Races 2,258 4.3
Native Hawaiian/Pac Islander 33 0.1
White 23,390 44.5
Eligible for Free and Reduced Meals
No 29,892 56.8
Yes 22,692 43.2
Limited English Proficiency Identified
No 47,081 89.5
Yes 4,056 7.7
Exited 1,447 2.8
![Page 33: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/33.jpg)
2013 MIS Conference 33
Students in Grade 3 Historically Tracked to EISCharacteristic N %
Third Grade Students Received EIS Part C 2,482 100.0 General Education Grade Three 1,628 65.6
Special Education Grade Three 854 34.4
Disability Codes 4, 6, 8, and 9 (SL, ED, OHI, SLD) 617 72.2
All Other Disability Codes 237 27.8
Gender
Males 1,646 66.3
Females 836 33.7
Limited English Proficiency Identified
No 2,353 94.8
Yes 97 3.9
Exited by Grade Three 32 1.3
Free and Reduced Meals in Grade Three
No 1,627 65.6
Yes 855 34.4
Minority Status
Yes, Minority 1,086 43.8
No, Minority 1,396 56.2
![Page 34: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/34.jpg)
34
Students in Grade 3 Tracked to EIS, EIS Characteristics
Part C Eligibility of Grade 3 Students Born Sept. 1, 2001 – Aug. 31, 2002
25% Delay Atypical Development
High Probability Condition
Student Placement Grade Three 2011 (n = 1,674) (n = 400) (n = 408)
N % N % N %
General Education Grade Three 1,093 65.3 284 71.0 251 61.5
Special Education Grade Three 581 34.7 116 29.0 157 38.5
2013 MIS Conference
65.5%, n = 1,628 enrolled in general education at grade three
![Page 35: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/35.jpg)
2013 MIS Conference 35
Average State Assessment Scores by Educational Service at K and Grade 3
2008 WSS-K 2011 RSAA 2011 MSAA
N M SD M SD M SD
General Ed Gr 3 47928 77.9 10.9 430.8 38.2 429.9 41.1
No EIS 46300 77.9 10.9 430.9 38.2 429.9 41.1
Yes EIS 1628 77.2 11.1 427.8 39.1 428.6 41.7
Special Ed Gr 3 3994 67.8 13.5 368.0 120.6 364.6 114.4
No EIS 3377 68.0 13.4 371.5 117.3 367.1 111.2
Yes EIS 617 67.2 13.9 349.2 135.9 350.9 129.8
![Page 36: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/36.jpg)
2013 MIS Conference 36
Hierarchical Regression Analyses
Outcome: RSAA Grade 3, MSAA Grade 3Three Models:
Model 1, 2, 3Demographics: FaRMs, Gender, MinorityModel 2, 3Part C EISModel 3EIS x FaRMsEIS x GenderEIS x Minority
![Page 37: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/37.jpg)
Hierarchical Regression Results: WSS-K
WSS-K General Ed Special Ed
Model 1 Model 2 Model 3 Effect Size Model 1 Model 2 Model 3 Effect Size
Variable B (SE) B (SE) B (SE) Partial ŋ2 B (SE) B (SE) B (SE) Partial ŋ2
Constant 81.57**(.10) 81.60**(.10) 81.60**(.10) .592 72.95** (.50) 73.20** (.52) 73.65** (.55) .248
FARMS -3.57**(.12) -3.58**(.12) -3.56**(.12) .003 -3.73** (.50) -3.83** (.50) -3.79** (.56) .002
Gender -2.46**(.11) -2.45**(.11) -2.49**(.11) .000 -1.89** (.51) -1.86** (.51) -2.54** (.55) .004
Minority -1.98**(.12) -1.99**(.12) -1.98**(.12) .001 -3.62** (.50) -3.61** (.50) -3.63** (.56) .009
EIS -.80** (.29) -1.16* (.56) .000 -1.28* (.62) -4.37** (1.37) .003
EIS * FARMS -.47 (.68) .000 -.35 (1.32) .000
EIS * Gender 1.09 (.60) .000 4.22** (1.37) .001
EIS * Minority -.40 (.66) .000 .53 (1.31) .000
R2.057 .057 .057 .056 .057 .060
R2 Change .0001** .000 .001* .003*
2013 MIS Conference 37
*p ≤ .05, **p ≤ .01
![Page 38: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/38.jpg)
2013 MIS Conference 38
WSS-K ResultsGeneral Ed
• WSS-K higher for students not FaRMs, for girls, and for White students
• EIS students scored lower (.7 M diff)
• No interaction effects
Special Ed• WSS-K higher for students
not FaRMs, for girls, and for White students
• EIS students scored lower (.8 M diff)
• Sig interaction EIS x Gender– F > M NO EIS (2.9 M diff)– M > F EIS (1.7 M diff)
![Page 39: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/39.jpg)
Hierarchical Regression Results: RSAA
RSAA Grade 3 General Ed Special Ed
Model 1 Model 2 Model 3 Effect Size Model 1 Model 2 Model 3 Effect Size
Variable B (SE) B (SE) B (SE) Partial ŋ2 B (SE) B (SE) B (SE) Partial ŋ2
Constant 451.50**(.29) 451.70**(.30) 451.76**(.30) .592 399.95**(4.10) 404.91**(4.19) 406.34**(4.44) .248
FARMS -22.31**(.35) -22.34**(.35) -22.31**(.36) .003 -19.31**(3.99) -21.19**(3.99) -20.24**(4.36) .002
Gender -10.15**(.32) -10.04**(.32) -10.14**(.32) .000 5.16 (4.06) 5.79 (4.05) 4.08 (4.39) .004
Minority -11.26**(.35) -11.35**(.35) -11.39**(.36) .001 -46.56** (4.00) -46.75** (3.99) -48.16** (4.37) .009
EIS -5.49**(.89) -7.64**(1.71) .000 -27.86**(5.16) -39.27**(11.48) .003
EIS * FARMS -1.00 (2.08) -5.64 (10.86)
EIS * Gender 3.08 (1.84) 12.72 (11.48)
EIS * Minority 1.30 (1.98) 9.38 (10.75)
R2.153 .154
.154 .054 .061 .060
R2 Change .001** .000 .007** .000
2013 MIS Conference 39
*p ≤ .05, **p ≤ .01
![Page 40: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/40.jpg)
2013 MIS Conference 40
RSAA ResultsGeneral Ed
• RSAA higher for students not FaRMs, for girls, and for White students
• EIS students scored lower (3.1 M diff)
• No interaction effects
Special Ed• RSAA higher for students
not FaRMs and for White students
• EIS students scored lower (22.3 M diff)
• No interaction effects
![Page 41: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/41.jpg)
Hierarchical Regression Results: MSAA
MSAA Grade 3 General Ed Special Ed
Model 1 Model 2 Model 3 Effect Size Model 1 Model 2 Model 3 Effect Size
Variable B (SE) B (SE) B (SE) Partial ŋ2 B (SE) B (SE) B (SE) Partial ŋ2
Constant 448.58**(.32) 448.77**(.32) 448.81**(.33) .550 390.64**(3.84) 394.63**(3.92) 396.59**(4.16) .248
FARMS -22.64**(.38) -22.67**(.38)-22.57**(.39) .003 -18.99**(3.74) -20.50**(3.74) -20.10**(4.08) .002
Gender -2.24**(.35) -2.13**(.35) -2.28**(.35) .000 18.68**(3.81) 19.19**(3.79) 15.52**(4.11) .004
Minority -14.36**(.38) -14.45**(.38)-14.47**(.39) .001 -53.53**(3.75) -53.68**(3.74) -53.01**(4.09) .009
EIS -5.35**(.96) -7.68**(1.85) .000 -22.53**(4.84) -37.81**(10.76) .003
EIS * FARMS -3.46 (2.24) -3.77 (10.18)
EIS * Gender 4.85* (1.99) .000
25.01* (10.76) .001
EIS * Minority 1.02 (2.15) -2.25 (10.08)
R2 .142 .142
.143 .078 .083 .084
R2 Change .001** .000* .005** .000
2013 MIS Conference 41
*p ≤ .05, **p ≤ .01
![Page 42: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/42.jpg)
2013 MIS Conference 42
MSAA ResultsGeneral Ed
• WSS-K higher for students not FaRMs, for girls, and for White students
• EIS students scored lower (1.3 M diff)
• Sig interaction EIS x Gender– F > M No EIS (1.9 M diff)– M > F EIS (4.2 M diff)
Special Ed• WSS-K higher for students
not FaRMs, for girls, and for White students
• EIS students scored lower (16.2 M diff)
• Sig interaction EIS x Gender– M > F No EIS (10.4 M diff)– M > F EIS (42.9 M diff)
![Page 43: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/43.jpg)
2013 MIS Conference 43
General Ed EIS SAA Conclusions• Demographic Controls: WSS-K, RSAA, & MSAA
was higher for students not economically disadvantaged, higher for girls, and for White students
• EIS: WSS-K, RSAA, & MSAA when FaRMs, Gender, and Minority are controlled, students who received EIS scored lower than their Gen Ed peers.
• EIS Interactions: WSS-K & RSAA no significant interactions; MSAA EIS x Gender (F EIS)
![Page 44: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/44.jpg)
2013 MIS Conference 44
Special Ed EIS SAA Conclusions• Demographic Controls: WSS-K, RSAA, & MSAA
was higher for students not economically disadvantaged and for White students; females scored higher on WSS-K & MSAA
• EIS: WSS-K, RSAA, & MSAA when FaRMs, Gender, and Minority are controlled, students who received EIS scored lower than their Sp Ed peers
• EIS Interactions: WSS-K & MSAA significant EIS x Gender (F EIS lower in K and more at Grade 3)
![Page 45: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/45.jpg)
2013 MIS Conference 45
Youth in EIS have lower average scores than their peers in Gen Ed and Sp Ed
WSS K RSAA G3 MSAA G3 0
5
10
15
20
25
0.67
3.1
1.30.8
22.3
16.2
Gen EdSp Ed (HI)
Aver
age
Mea
n Di
ffere
nce
Betw
een
Non
EIS
and
EIS
![Page 46: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/46.jpg)
2013 MIS Conference 46
Female EIS in Sp Ed (HI) Services have lower average scores than their peers
WSSK RSAA G3 MSAA G30
50
100
150
200
250
300
350
400
F EISM EISFM
![Page 47: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/47.jpg)
2013 MIS Conference 47
Study 3: K Readiness and Grade 3 MSA Performance
Is performance on the WSS-K predictive of Grade 3 high stakes testing (Reading MSA and
Math MSA)?
![Page 48: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/48.jpg)
2013 MIS Conference 48
Background
Maryland Model for School Readiness (MMSR) Pearson Work Sampling System Assessment
Assesses 7 Domains, scaled then scored at 3 levels (Proficient, In Process, Needs Development)• Personal and Social Development• Language and Literacy• Mathematical Thinking• Scientific Thinking• Social Studies• The Arts• Physical Development
![Page 49: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/49.jpg)
2013 MIS Conference 49
Method• Participants/Procedure– General Ed Fall K Work Sampling System student scores
(2003, 2004, 2005) N = 152,105Matched to – Grade 3 Spring MSA Math, MSA Read student scores (2006,
2007, 2008) N = 100,958Match Rate 66% for Reading & Math
• Instruments– WSS is a 30 item instrument with scaled items coded 1 - 3– MSA is a high stakes test coded 1 - 3
![Page 50: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/50.jpg)
2013 MIS Conference 50
Kindergarten MMSR Composite Score Distribution by Grade 3 MSA Performance Level
MSA N % WSS-K M WSS-K SD Math
Basic 7,876 17.71% 63.98 5.71 Proficient 55,777 55.25% 72.21 5.52 Advanced 27,304 27.05% 78.38 5.25
Total 100,957 100.00% 72.42 3.51
Reading Basic 17,695 17.52% 64.78 5.67 Proficient 64,907 64.28% 72.63 5.62 Advanced 18,376 18.20% 79.02 4.91
Total 100,978 100.00% 72.42 3.85
![Page 51: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/51.jpg)
2013 MIS Conference 51
Matching Students K to Grade 3Kindergarten School Year
N WSS-K Scores
Grade 3 MSAA
Grade 3 MSAA
Grade 3 RSAA
Grade 3 RSAA
N Students Match Rate (%)
N Students Match Rate (%)
2002-2003 54,452 36,032 66.17% 36,026 66.16%
2003-2004 50,024 33,986 67.94% 34,008 67.98%
2004-2005 47,629 30,940 64.96% 30,944 64.97%
Total 152,105 100,958 66.37% 100,978 66.39%
![Page 52: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/52.jpg)
2013 MIS Conference 52
Method
• Causal Comparative Descriptive Design• CART analysis– Binary decision trees representing a series of rules
that lead to membership in a class or value– Variables in the dataset are analyzed and split on a
value to best predict outcome – By following the splits in the decision tree to the
targeted outcome, observers can discern the patterns and combinations of variables that best predict either the presence or absence of the desired outcome
![Page 53: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/53.jpg)
MIS DC 2013 Data Conference 53
Grade 3 Math Decision Tree
All Participants
![Page 54: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/54.jpg)
MIS DC 2013 Data Conference 54
Grade 3 Math Decision Tree
IIA3: Begin Phonemic Aware
![Page 55: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/55.jpg)
MIS DC 2013 Data Conference 55
Grade 3 Math Decision Tree
IIA3: Begin Phonemic Aware
IIA3: Begin Phonemic Aware
![Page 56: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/56.jpg)
MIS DC 2013 Data Conference 56
Grade 3 Math Decision Tree
IIA3: Begin Phonemic Aware
IIIB1: Number & Quantity
IIA3: Begin Phonemic Aware
![Page 57: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/57.jpg)
MIS DC 2013 Data Conference 57
Grade 3 Math Decision Tree
IIIC2:Recog/Dup/Extend Patterns
IIA3:Begin Phonemic Aware
IIA3:Begin Phonemic Aware
IIIB1: Number & Quantity
![Page 58: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/58.jpg)
MIS DC 2013 Data Conference 58
Grade 3 Math Decision Tree
IIA3: Begin Phonemic Aware
IIIB1: Number & Quantity
IIA3: Begin Phonemic Aware
IIIC2: Recog/Dup/Extend Pattern
IIIA1: Strategy Use for Math
![Page 59: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/59.jpg)
MIS DC 2013 Data Conference 59
Grade 3 Math Decision Tree
IIA3: Phonemic Aware
IIA3: Phonemic Aware
IIIB1: Number & Quantity IIIC2: Patterns
IIIA1: Strategy Use Math IIA1: Listening
![Page 60: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/60.jpg)
MIS DC 2013 Data Conference 60
Grade 3 Math Decision Tree
IIA3: Phonemic Aware
IIIC2: Patterns
![Page 61: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/61.jpg)
MIS DC 2013 Data Conference 61
Grade 3 Math Decision Tree
IIIC2: Patterns
IBI: Rules & Routines
IIA3: Phonemic Aware
![Page 62: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/62.jpg)
MIS DC 2013 Data Conference 62
Grade 3 Math Decision Tree
IIIA1: Strategy Use for Math
IBI: Rules & Routines
IIIC2: Patterns
IIA3: Phonemic Aware
![Page 63: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/63.jpg)
MIS DC 2013 Data Conference 63
Grade 3 Math Decision Tree
IIIA1: Strategy for Math
IBI: Rules & RoutinesIIIC2: Patterns
IIA3: Phonemic Aware
IIA3: Phonemic Aware
IIA3: Phonemic Aware
IIA1: ListeningIIIA1: Strategy for Math
IIIC2: Patterns
![Page 64: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/64.jpg)
2013 MIS Conference 64
Math Decision Tree Nodes DescriptionsTerminal
NodeSegment: Predicted Outcome
# Students Average Math Score
Demonstrates beginning phonemic awareness
(IIA3)
Shows understanding of number and quantity (IIIB1)
Recognizes, duplicates, and
extends patterns (IIIC2)
Begins to use and explain strategies to
solve mathematical
problems (IIIA1)
Gains meaning by listening
(IIA1)
Follows classroom rules
and routines (IB1)
1 3 2,838 1.495 3 3 {1,2,3} {1,2,3} {1,2,3} {1,2,3}
2 3 4,833 1.709 1 {2,3} {1,2,3} {1,2,3} {1,2,3} {1,2,3}
3 3 2,173 1.723 {2,1} {1,2,3} {3,2} 1 {1,2,3} {1,2,3}
4 2 14,620 1.927 {2,1} {1,2,3} {3,2} {2,3} {1,2,3} {1,2,3}
5 2 4,647 2.027 {2,1} {1,2,3} 1 {1,2,3} {3,2} {1,2,3}
6 2 4,857 2.188 {2,1} {1,2,3} 1 {1,2,3} 1 {1,2,3}
7 2 4,091 2.143 1 {1,2,3} {1,3} {1,2,3} {1,2,3} {1,2,3}
8 1 3,908 2.246 1 {1,2,3} 1 {1,2,3} {1,2,3} {1,2}
9 1 4,164 2.324 1 {1,2,3} 1 {1,2,3} {1,2} 1
10 1 10,019 2.453 1 {1,2,3} 1 {1,2,3} 1 1
![Page 65: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/65.jpg)
2013 MIS Conference 65
Ranked (Highest to Lowest) Significant WSS-K Indicators used in the Predictive Math Model
SAA MathDemonstrates beginning phonemic awareness (IIA3)Shows understanding of number and quantity (IIIB1)Recognizes, duplicates, and extends patterns (IIIC2)Begins to use and explain strategies to solve mathematical problems (IIIA1)Gains meaning by listening (IIA1)Follows classroom rules and routines (IB1)
![Page 66: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/66.jpg)
2013 MIS Conference 66
Grade 3 Math Score Distribution: Comparing Training & In Time Validation
Predicted Outcome
Training Average
MSA Score
In Time Average
MSA Score
Actual Training Outcome: % Students
Per Segment
Actual In Time Validation Outcome:
% Students Per Segment
1 (Basic)
2 (Proficient)
3 (Advanced)
1 (Basic)
2 (Proficient)
3 (Advanced)
Advanced 435 434 6.16% 49.80% 44.04% 6.99% 48.34% 44.67%
Proficient 411 411 19.08% 59.88% 21.04% 19.14% 59.48% 21.38%
Basic 385 384 42.05% 50.82% 7.13% 43.63% 49.50% 6.88%
Total 414 414 18.95% 55.04% 26.01% 19.46% 54.17% 26.37%
![Page 67: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/67.jpg)
2013 MIS Conference 67
Conclusion
• Fall WSS-K is a moderately successful predictor of later standardized Math test performance
![Page 68: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/68.jpg)
MIS DC 2013 Data Conference 68
Grade 3 Reading Decision Tree
![Page 69: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/69.jpg)
MIS DC 2013 Data Conference 69
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
![Page 70: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/70.jpg)
MIS DC 2013 Data Conference 70
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IIA3: Phonemic Aware
![Page 71: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/71.jpg)
MIS DC 2013 Data Conference 71
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IIA3: Phonemic Aware
IIIA1: Strategy f/Math
![Page 72: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/72.jpg)
MIS DC 2013 Data Conference 72
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IIA3: Phonemic Aware
IIIA1: Strategy f/Math IIA1: Listening
![Page 73: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/73.jpg)
MIS DC 2013 Data Conference 73
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IIA3: Phonemic Aware
IIIA1: Strategy f/Math IIA1: Listening
IIIA1: Strategy f/Math
![Page 74: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/74.jpg)
MIS DC 2013 Data Conference 74
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IIA3: Phonemic Aware
IIIA1: Strategy f/Math IIA1: Listening
IIIA1: Strategy f/Math IIIB1: Number & Quantity
![Page 75: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/75.jpg)
MIS DC 2013 Data Conference 75
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IB1: Rules & Routines
![Page 76: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/76.jpg)
MIS DC 2013 Data Conference 76
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IB1: Rules & Routines
IIC2: Print Concepts
![Page 77: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/77.jpg)
MIS DC 2013 Data Conference 77
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IB1: Rules & Routines
IIIC2: Patterns
IIB1: Effective Comm
![Page 78: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/78.jpg)
MIS DC 2013 Data Conference 78
Grade 3 Reading Decision Tree
IIA3: Phonemic Aware
IB1: Rules & Routines
IIIC2: Patterns
IIB1: Effective Comm
IIA3: Phonemic Aware
IIIA1: Strategy f/Math IIA1: Listening
IIIA1: Strategy f/ Math IIIB1: Number & Quantity
![Page 79: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/79.jpg)
2013 MIS Conference 79
Read Decision Tree Nodes DescriptionsTerminal
NodeSegment: Predicted Outcome
# Students Average Reading Score
Demonstrates beginning phonemic awareness
(IIA3)
Begins to use and explain strategies to
solve mathematic
problems (IIIA1)
Gains meaning by
listening (IIA1)
Shows understanding of number and quantity
(IIIB1)
Follows classroom rules and
routines (IB1)
Shows some understanding of concepts about print
(IIC2)
Speaks clearly and conveys
ideas effectively
(IIB1)
1 3 4,164 1.566 3 3 {1,2,3} {1,2,3} {1,2,3} {1,2,3} {1,2,3}
2 3 3,466 1.708 3 {2,1} {1,2,3} {1,2,3} {1,2,3} {1,2,3} {1,2,3}
3 3 2,176 1.732 2 3 {1,2} {1,2,3} {1,2,3} {1,2,3} {1,2,3}
4 2 15,345 1.875 2 {2,3} {1,2} {1,2,3} {1,2,3} {1,2,3} {1,2,3}
5 2 4,419 1.949 2 {1,2,3} 1 {1,3} {1,2,3} {1,2,3} {1,2,3}
6 2 4,274 2.084 2 {1,2,3} 1 1 {1,2,3} {1,2,3} {1,2,3}
7 2 5,384 2.083 1 {1,2,3} {1,2,3} {1,2,3} {1,2} {1,2,3} {1,2,3}
8 2 2,609 2.123 1 {1,2,3} {1,2,3} {1,2,3} 1 {1,2} {1,2,3}
9 2 1,601 2.156 1 {1,2,3} {1,2,3} {1,2,3} 1 1 {3,2}
10 1 12,545 2.307 1 {1,2,3} {1,2,3} {1,2,3} 1 1 1
![Page 80: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/80.jpg)
2013 MIS Conference 80
Ranked (Highest to Lowest) Significant WSS-K Indicators used in the Predictive Reading Model
SAA ReadingDemonstrates beginning phonemic awareness (IIA3)*Gains meaning by listening (IIA1)*Begins to use and explain strategies to solve mathematical problems (IIIA1)*Shows understanding of number and quantity (IIIB1)*Follows classroom rules and routines (IB1)*Recognizes, duplicates, and extends patterns (IIIC2)*Speaks clearly and conveys ideas effectively (IIB1)
*also significant predictors in MSA math model
![Page 81: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/81.jpg)
2013 MIS Conference 81
Grade 3 Reading Score Distribution: Comparing Training & In Time Validation
Predicted Outcome
Training Average
MSA Score
In Time Average
MSA Score
Actual Training Outcome: % Students
Per Segment
Actual In Time Validation Outcome:
% Students Per Segment
1 (Basic)
2 (Proficient)
3 (Advanced)
1 (Basic)
2 (Proficient)
3 (Advanced)
Advanced
444
444 5.35% 58.62% 36.03% 5.50% 58.44% 36.06%
Proficient
421
421 17.94% 66.41% 15.65% 17.87% 66.32% 15.81%
Basic
398
398 39.03% 56.66% 4.31% 38.27% 57.89% 3.84%
Total
422
422 18.81% 62.96% 18.23% 18.76% 63.08% 18.16%
![Page 82: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/82.jpg)
2013 MIS Conference 82
Additional Analyses/Conclusions
• Sex– For math and reading, similar performance
• Ethnicity– For math and reading, Whites and Asian/Pac Islanders
realized predicted performance at higher levels; African-American, Hispanic, & Native American realized predicted performance at lower levels
• FaRMs– For math and reading, student receiving FaRMs realized
predicted performance at lower levels
![Page 83: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/83.jpg)
2013 MIS Conference 83
Conclusions • Fall K WSS is a modestly successful predictor of later
standardized test performance• Math SAA performance was better predicted than
Reading SAA• Overidentification of middle group (Proficient)• Findings support the importance of early literacy
skills• Findings support the persistent ‘gap’ in student
achievement, evident from K
![Page 84: Deborah T. Carran, Jacqueline Nunn, Sara Hooks Johns Hopkins University Stacey N. Dammann](https://reader035.fdocuments.in/reader035/viewer/2022062812/5681637f550346895dd460b4/html5/thumbnails/84.jpg)
2013 MIS Conference 84
Summary QuestionsAre early intervention services effective in preparing
children to enter kindergarten?Value of EIS?Continue funding of Birth to five model?
Is level of K preparedness predictive of grade 3 high stakes testing performance?Value of high quality preK programs (QRIS)?Fund additional preK programs?
Are early intervention services effective in preparing children to be successful in school?Value of EIS?Further tracking studies looking at child outcomes.