What Works Clearinghouse: Education · Today’s Presentation Early childhood development and child...
37
What Works Clearinghouse: Early Childhood Education Elizabeth W. Cavadel Mathematica Policy Research
Transcript of What Works Clearinghouse: Education · Today’s Presentation Early childhood development and child...
What Works
Clearinghouse:
Early Childhood EducationElizabeth W. Cavadel
Mathematica Policy Research
Today’s Presentation Early childhood development and child care
Understanding the What Works Clearinghouse (WWC)
Introduction to the WWC
Discussion of the backbone of the WWC
Examining products released by the WWC, including relevant examples
Identifying whether a study was reviewed, and its WWC rating
Searching for particular evidence from Intervention Reports
Group discussion
Early care and education
Approximately 60% of children ages 3 to 5 are in a “care
arrangement” during the day (Child Trends, 2012)
Estimated that 25 - 41% are in full-time, center-based,
care
38% of children under 3 spend at least 35 hours/week in
non-parental care (NCLS-B, 2002)
Research shows quality matters and yes, the quality of care
is highly variable, across both income and types of care
What quality means in early childhood:
Strong adult-child interactions: supporting social-emotional
development
Reciprocal caregiving
Increasing complexity
Rich environmental exploration: supporting cognitive development
Varied experiences and exposure
The ability to interact with, and act upon, the environment
Language: supporting language and literacy development
Amount and diversity of talk
Types of talk
Conversations
Domain specific knowledge: science, math, early literacy skills
What is the WWC?
Problem and Context
• Substantial amount of research on education interventions• Large variation in quality• No common standards• Synthesizing findings is not
simple• Studies summarized many ways
• How does that affect choices?• Often based on personal
experience and ideology • Infrequently on strong research
WWC Mission
To be a central and trusted source of scientific evidence for what works in education
Develops and implements standards for reviewing and synthesizing education research
Assesses the credibility of research evidence on the effectiveness of interventions (does not directly assess interventions)
Transparent, objective, and high quality
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_what_06
1015.pdf
What is the WWC?
The WWC reviews evidence of
effectiveness of programs, policies, or
practices by using a consistent and
transparent set of standards. The WWC
doesn’t rank, evaluate, or endorse
interventions.
WWC: Influencing Research and Practice
9
Producers of Information
• Universities
• Research organizations
• Developers
Consumers of
Information
• State and federal govt.
• Education departments
• Superintendents
• School boards
• Principals and teachers
• Parents and the public
• Funders and
consultants
Research
Standards Effectiveness Information
Evidence-Based
Decisions
Improved Student
Outcomes
Research
Topics
What Works
Clearinghouse• Develop standards
• Train reviewers
• Identify research
• Review research
• Summarize evidence
• Disseminate findings
The Review: Backbone of a WWC
Product
Review of publication to determine whether it supports causal
inferences
Three sets of design standards
Group design standards
Pilot single-case design standards
Pilot regression-discontinuity design standards
Four ratings
Meets WWC design standards without reservations
Meets WWC design standards with reservations
Does not meet WWC design standards
Not eligible for review
Review Protocols: Sets Parameters for Review
http://ies.ed.gov/
ncee/wwc/Publica
tions_Reviews.aspx
?f=All%20Publicatio
n%20and%20Produc
t%20Types,5;#pubs
earch
Are the groups formed randomly?
Is sample attrition high or low?
Were groups similar before the intervention
began?
Are there confounding factors or concerns with
outcomes?
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_rates_061015.pdf
Key questions for a group design review
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_reporting_061
015.pdf
Improvement Index
Effectiveness Rating
Extent of Evidence
What does the WWC report?
WWC Products
WWC Website: All Resources and
Products
http://whatworks.ed.gov
http://ies.ed.gov/ncee/w
wc/
Intervention Reports
An intervention report is a summary of findings of
the most reliable research on a given curriculum,
program, practice, or policy in education.
http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_fasttrack_100714.p
df
http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_socialskills_020513.
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_info_making_061
015.pdf
1. Define scope of review
2. Conduct a comprehensive
search
3. Screen and review studies
4. Combine findings
5. Summarize the review
Steps in Writing an Intervention Report
Searching for an intervention report
http://ies.ed.gov/ncee/wwc/
Publications_Reviews.aspx?f=A
ll+Publication+and+Product+Ty
pes%2c5%3b
Example Intervention Report
What can you learn from an
intervention report?
http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid
=578
Parts of an Intervention Report
Program description
Research
Effectiveness
http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_careeracademies_09
2215.pdf
Inside an Intervention Report
Program Information
Research Summary
Effectiveness Summary
References
Appendices
http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_careeracademies_09
2215.pdf
Single Study Reviews
Assess one study against WWC standards
Not a systematic review on that particular intervention
First stage for some single study reviews is a “Quick Review”
Email blast giving sense of whether study will meet standards, ideally one week after media attention
What can you learn from a quick
review?
http://ies.ed.gov/ncee/wwc/quickreview.aspx?sid=239
What can you learn from a single
study review?
http://ies.ed.gov/ncee/wwc/SingleStudyReview.aspx?sid=236
http://ies.ed.gov/ncee/wwc/pdf/single_study_reviews/wwc_fitkids_082515.pd
f
WWC Practice Guides
A practice guide is a set of recommendations for educators to address challenges in their classrooms and schools. The level of research evidence supporting each recommendation is provided.
Expert panel to develop recommendations
Conduct a comprehensive literature search
Review each study against WWC standards
Summarize the review
http://ies.ed.gov/ncee/wwc/Publications_Reviews.aspx?f=All%20Publication%20and%20Prod
uct%20Types,3;#pubsearch
What is in a practice guide?
http://ies.ed.gov/ncee/wwc/pdf/practice_guides/wwc_algebra_040715.p
df
http://ies.ed.gov/ncee/wwc/pdf/practice_guides/early_math_pg_111313
Recommendations
How-to steps
Lots of Examples
What can an example do for you?
If you are a researcher,
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_gsa_v1.pdf
Has a study been reviewed by the WWC?
http://ies.ed.gov/ncee/wwc/ReviewedStudies.as
px
Find What Works
http://ies.ed.gov/ncee/wwc/findwhatworks.aspx
Contact InformationThe What Works Clearinghouse
http://whatworks.ed.gov
On Facebook at
www.facebook.com/whatworksclearinghouse
On Twitter at
www.twitter.com/WhatWorksED
And a widget you can add to your own pages
http://ies.ed.gov/ncee/wwc/Widget.aspx
Elizabeth Cavadel
M. C. Bradley
