5-talk symposium on DRDP - UC Berkeley BEAR Center...Decision Making Josh Sussman and Perman...
Transcript of 5-talk symposium on DRDP - UC Berkeley BEAR Center...Decision Making Josh Sussman and Perman...
-
5-talk symposium on DRDP
Symposium title: Supporting Children’s Kindergarten and Early Learning Goals with a Standards-Aligned Assessment -Effective Practices in Statewide Aggregate Reporting
1. Policy implications
2. The correspondence between the foundations/standards and DRDP
3. This talk– use of DRDP data by states4. the Criterion Zone Boundary process
5. DRDP reports
-
The Use of DRDP Data for Statewide
Aggregate Reporting and Data-Based
Decision Making
Josh Sussman and Perman Gochyyev
BEAR Center
UC Berkeley
-
Objective
Demonstrate how the DRDP – designed as a formative assessment --also produces valid and reliable information that supports the needs of states for
1. Information about child development and kindergarten readiness (using Criterion Zone Boundaries)
2. Disaggregated data
3. Actionable evidence to support data based policymaking
-
Some typical obstacles to the use of data in
educational settings
Availability &
Quality
• Technical infrastructure & data fidelity (Ackerman, 2018)
-
Some typical obstacles to the use of data in
educational settings
Availability &
Quality
Analysis &
Interpretation
• Technical infrastructure & data fidelity (Ackerman, 2018)
• Measurement issues -- reliability, validity, and relevance of data to educational practice (Russo et al., 2019)
• Technical skills
-
Some typical obstacles to the use of data in
educational settings
Availability &
Quality
Analysis &
Interpretation
Data Use
• Technical infrastructure & data fidelity (Ackerman, 2018)
• Measurement issues -- reliability, validity, and relevance of data to educational practice (Russo et al., 2019)
• Technical skills
• Clarity on what the data can be used for (Shen & Cooley, 2008)
• Data leadership (Roegman et al., 2018)
-
Remainder of the presentation
I. Features of DRDP measurement that support data use
II. DRDP data for understanding, reporting, and accountability
III. Additional possibilities for DRDP data to support state-level DBDM
-
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
-
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
-
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
Designed as a formative
assessment (e.g., Akers,
2014; Black & Wiliam,
1998)
-
Criterion Zone Boundaries:
(1) Information about
students’ readiness for kindergarten
(2) Information to support
state-level policymaking
Approaches to
Learning
Social and
Emotional
Development
Language and
LiteracyCognition
Perceptual, Motor,
and Physical
Development
Designed as a formative
assessment (e.g., Akers,
2014; Black & Wiliam,
1998)
-
Reporting
• What percent of eligible kindergarteners are ready for kindergarten?• What about disaggregated groups
-
Data analysis
• 1.5 million cases of simulated DRDP data “based on a true story”
wletc’ = wletc + r + s, where
r = N(0.4, 1)
s = 0.4
• Panel data, unbalanced, cross-nested, complex (e.g., students join and leave)
-
Statewide information and reporting
What percent of eligible
kindergarteners are ready for
kindergarten?
• Use of criterion zone boundaries to define K-
readiness
-
III. More advanced analytics has the
possibility to support state-level DBDM
1. At what age does inequality begin and in which domains is it greatest?
2. Which types of preschool programs best support student development?
3. How much longer, on average, does it take for students classified as ELLs to become K-ready than those classified as proficient?
-
Average development of K-eligible children
Do
ma
in S
core
-
When does equity begin to be a concern?
-
When does equity begin to be a concern?
-
When does equity begin to be a concern?
800
300
Lan
gu
ag
e a
nd
Lit
era
cy D
ev
elo
pm
en
t
Infancy
End of K
570
i1
i2
i3
i4
-
When does equity begin to be a concern?
800
300
Lan
gu
ag
e a
nd
Lit
era
cy D
ev
elo
pm
en
t
Infancy
End of K
570
i1
i2
i3
i4
-
Which types of preschool programs best support
child development?
-
Advanced analytics can help contextualize
gaps in an actionable metric
• Survival models: how much longer do ELLs, a group who are known to struggle, take to become K-ready?
• The answers have the promise to inform policy decisions about the timing and intensity of interventions
-
Survival models in education
• It is about time• From whether to when• How many months does it take for children to become kindergarten-ready?
• Why?• The observations are censored
• Some kids may not be categorized as K-ready within the period of data collection• If a child reached K-ready level after the data collection, researchers will not know that
• Singer & Willett (1991):• We believe that researchers avoid asking questions about time-to-event ("When?") because of
methodological difficulties introduced when members of the sample do not experience the target events during the data collection period.
• One approach: dichotomize at a particular time and model the binary outcome• Does not eliminate censoring• Does not answer: how long does it take for an average child to reach• How to incorporate time-varying covariates
-
Survival models: K-readiness
• Time to K-readiness (instead of K-readiness itself)• We can also identify factors predicting students’ K-readiness
• ELL vs. non-ELL: differences in proportions of K-ready children• if ELL status is varies across time, survival models will accommodate that • Investigate gaps between ELLs and non-ELLs across ages as a function of probability of being K-
ready?
• Hazard probability: conditional probability that a child will become K-ready in that month, given that he or she stayed “not K-ready” up to that point.
• A chronological summary of the probability of becoming K-ready• States (“K-ready”) can be absorbing or recurring• Accommodate ”delayed entry:” kids becoming parts of the “snapshot” observation at different
ages
• “Of all the survival methods available, we believe that discrete-time survival analysis offers the most promise for exploring educational transitions.” (Willett and Singer, 1991)