Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to...
-
Upload
sophia-griffith -
Category
Documents
-
view
222 -
download
0
Transcript of Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to...
![Page 1: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/1.jpg)
Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta
Center for Response to Intervention in Early Childhood
Division of Early Childhood
October 30, 2012
Using Programmatic Research to Develop
Feasible, Effective Language and Early
Literacy Interventions
http://www.crtiec.org
![Page 2: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/2.jpg)
CRTIEC
IES Research and Development Center funded in 2008 Objectives
Conduct a focused program of research to develop and evaluate intensive interventions for preschool language and early literacy skills that supplement core instruction
Develop and validate an assessment system aligned with these interventions for universal screening and progress monitoring
Carry out supplementary research responsive to the needs of early childhood education and special education practitioners and policy makers.
Provide outreach and leadership Annual Preschool RTI Summit Website and Resources (http://www.critec.org)
![Page 3: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/3.jpg)
3
DynamicMeasurement GroupSupporting School Suc cess One Step at a Time
The Forest Friends
![Page 4: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/4.jpg)
Acknowledgments
In addition to the authors, this work has been coordinated by: Gabriela Guerrero, along with Jane Atwater, Tracy Bradfield, Annie Hommel, Naomi Schneider, Sean Noe, Alisha Wackerle-Hollman, and a host of dedicated research assistants, students, and postdocs at University of Kansas, University of Minnesota, the Ohio State University, and the Dynamic Measurement Group.
We want to acknowledge the partnership of the many early education programs that collaborated with us on this important study.
![Page 5: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/5.jpg)
Big Idea!
Studies Should Build on Each Other! Research is a process where one
conducts several studies programmatically to nail down an effect and reveal the extensions and limitations of an intervention (Robinson, 2004)
![Page 6: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/6.jpg)
Today’s Topic: Programmatic Intervention Development Research
Overview (Greenwood) Tier 2 Intervention Design Planning
Phase: Goldstein/Spencer Iterative Testing and Development
Phase: Goldstein/Spencer Efficacy and Effectiveness Phase:
Greenwood/Goldstein Implications/Discussion (Carta)
![Page 7: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/7.jpg)
Challenges Young Children Face?
Many children enter kindergarten with limited oral language skills that place them at risk for later reading difficulties (Dickinson & Snow, 1997)
These children become struggling readers because they lack the necessary language and early literacy experiences needed to learn these skills prior to kindergarten.
![Page 8: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/8.jpg)
What Should We Teach in Preschool?
Adequate early literacy experience before kindergarten enables children to acquire knowledge of two related domains of information needed to learn to read. First, children need sources of information that will
directly support their understanding of the meaning of print in school. 1. These are: vocabulary knowledge, oral language skills,
language comprehension, and conceptual knowledge leading to reading comprehension (Biemiller, 2006).
Second, children need familiarity with the alphabet, the ability to translate print into sounds and sounds into print (Treiman, Tincoff, & Richmond-Welty, 1997), and print awareness (Badian, 2000)
![Page 9: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/9.jpg)
Challenges Preschools Face?
The field continues struggling to improve instructional quality and outcomes for all children (Justice, Hamre, & Pianta, 2008; Greenwood, Carta, Atwater et al., 2012)
The field is just beginning to consider intentional instruction and differentiating instruction for individual children (NAEYC, DEC, & NHSA, 2012)
There is a lack of evidence-based Tier 2 and 3 interventions and aligned measurement tools for screening and progress monitoring (Greenwood, Bradfield et al., 2011)
![Page 10: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/10.jpg)
Is There a Solution?: RTI and Multi-Tiered Support Systems
Universally screen frequently to identify children not making expected progress
Provide these children more intensive supplementary (Tier 2) or alternative (Tier 3) experiences
Monitor progress and adapt instructional support as needed
Improve the quality of Tier 1, core instruction in the language and early literacy
![Page 11: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/11.jpg)
How is CRTIEC Approaching It?
Developing evidence-based practice through programmatic research? Interventions developed teach skills with
evidence that they are precursors of later learning to read
Interventions are delivered through practices containing effective components
Efficacy of the intervention is confirmed by testing in rigorously designed studies
Measures are developed with evidence of sensitivity, validity, accuracy, and reliability
![Page 12: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/12.jpg)
IES Programmatic Research
Goal 1 Goal 2 Goal 3 Goal 4 Goal 5
Exploration Development/Innovation
Efficacy/Replication
Scale Up Measurement
Explore the association between (a) education outcomes and alterable factors and (b) conditions that may mediate or moderate these relations
Develop new interventions based on a theoretical framework through a process of test, evaluate, improve and retest (i.e., iterative development
Evaluate fully developed interventions in authentic educational settings
Evaluations to determine whether or not fully developed interventions are effective when they are implemented under conditions that would be typical if a school district or other education delivery setting were to implement them as routine practice
Research to develop and validate (a) new measures and to (b) adapt and improve original measures for broader use in educational settings
![Page 13: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/13.jpg)
Tier 2 Intervention: Design and Planning Phase
![Page 14: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/14.jpg)
Tier 2 Embedded Storybook Interventions
As part of an RTI model, there is a need for high-quality interventions to improve early language and literacy skills for preschool children who are falling behind.
Overview of design and development work on interventions feasible for high fidelity implementation in preschool classrooms.
How findings from early efficacy studies have informed our development.
![Page 15: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/15.jpg)
To effectively implement response to intervention in early childhood…
![Page 16: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/16.jpg)
…how should we design Tier 2 interventions?
![Page 17: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/17.jpg)
We need interventions that work in classrooms…
![Page 18: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/18.jpg)
…and that don’t place additional demands on teachers.
![Page 19: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/19.jpg)
Children learn best when we teach explicitly…
![Page 20: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/20.jpg)
… and when we give children opportunities to respond.
![Page 21: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/21.jpg)
So we designed an intervention.
![Page 22: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/22.jpg)
Story Friends Program
![Page 23: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/23.jpg)
Small groups of children participate in ‘listening centers.’
![Page 24: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/24.jpg)
Prerecorded storybooks and explicit embedded lessons are delivered under headphones.
![Page 25: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/25.jpg)
Multiple listens provide repeated exposures to instruction and many opportunities to respond.
MONDAY
TUESDAY
WEDNESDAY
![Page 26: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/26.jpg)
Ellie’s First Day Leo’s Brave Face Jungle Friends Go to the Beach
Vocabulary Words
enormous brave soaked
different grin gorgeous
Comprehension Questions How do you think Ellie
feels about meeting new friends? [Why?]
How do you think Leo feels about going to the
dentist? [Why?]What do you think will happen in this story?
Where did Ellie go in our story?
What did Leo learn from the
dentist?
How did Tanisha feel when the wave knocked
over her sandcastle?
At the end of the story, Ellie was happy. Why was
Ellie happy?
At the beginning of the story, Leo was afraid of the dentist. What do you do when you
are afraid?
Do you think the Jungle Friends will go to the beach again? [Why or
why not?]
![Page 27: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/27.jpg)
enormous, different brave, grin soaked, gorgeous
reckless, ignore unusual, greet ill, discover
leap, pause speedy, unique ridiculous, tumble
![Page 28: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/28.jpg)
The Forest Friends are thrilled! They are excited to go to the carnival. Thrilled. Say thrilled. (2) Thrilled means excited. Tell me, what word means excited? (2) Thrilled! Good work! When are you thrilled? (2) What about… when you get a present! …Or your friends come over to play! I bet that makes you feel excited. Now, lift the flap. Look! These boys are at a birthday party. They are excited. They are thrilled! Tell me, what does thrilled mean? (3) Excited! That’s right.
![Page 29: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/29.jpg)
Marquez Monkeys Around
The friends all tried to help Ellie Elephant. Why did they help Ellie? (3) Because she couldn’t get out by herself. She was stuck! The friends were worried, so they worked together to get Ellie out.
![Page 30: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/30.jpg)
Measures of instructional content are administered periodically.
![Page 31: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/31.jpg)
Tier 2 Intervention: Iterative Testing and Development Phase
![Page 32: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/32.jpg)
Timeline
Year 1, 2008-2009: intervention development Year 2, 2009-2010: pilot study Year 3, 2010-2011: early efficacy study with
single case design, implemented by research staff
Year 4: 2011-2012: early efficacy study, group design with embedded single case design, implemented by research staff
Year 5: 2012-2013: efficacy trial with randomized cluster design, implemented by classroom staff
![Page 33: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/33.jpg)
2010-2011
Year 3 VC Early Efficacy Study
![Page 34: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/34.jpg)
Participants
9 preschool children in 3 classrooms were identified with limited oral language skills in fall.
Multiple gating procedures for identification that included a teacher survey, Picture Naming IGDI 2.0, norm-referenced tests.
![Page 35: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/35.jpg)
Characteristics of Participants
School Child Age GenderPicture Naming
2.0PPVT-IV CELF-P2
School A
A1 4;9 Female 6 80 86
A2 4;9 Male 10 78 88
A3 4;6 Male 7 88 77
School B
B1 4;11 Male 5 83 73
B2 4;10 Female 9 87 86
B3 4;11 Female 6 96 90
School C
C1 4;10 Male 11 83 90
C2 4;5 Male 6 80 94
C3 4;3 Female 5 84 94
PPVT-IV: M = 84.3, Range 78 – 96; CELF-P2: M = 86.4, Range 73 - 94
![Page 36: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/36.jpg)
Method
Single-case repeated acquisition design Intervention was 9 books with embedded
vocabulary and comprehension lessons. Implemented by research staff Measures:
Mastery monitoring probes at pretest and posttest for each book
2 outcomes: Vocabulary and Comprehension
![Page 37: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/37.jpg)
Mastery Monitoring Items and Scoring
Taught Vocabulary Maximum score at pretest and posttest
was 4 Untaught Vocabulary
Maximum score at pretest and posttest was 2
2 points possible per word "Tell me, what does enormous mean?“
"Really big" 2 points “means a big building” 1 points “I don’t know” 0 points
![Page 38: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/38.jpg)
Mastery Monitoring Items and Scoring
Comprehension Maximum score at pretest and posttest
was 6. Three 2-point comprehension questions
"At the end of the story, Ellie is happy. Why is Ellie happy?“ “Because she made new friends” 2
points “Because she likes playing” 1 point “Her big” 0 points
![Page 39: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/39.jpg)
1 2 3 4 5 6 7 8 90
1
2
3
4
Book
Mas
ter
Mon
itor
Sco
re
Child A2
Posttest
Pretest
Untaught Word
![Page 40: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/40.jpg)
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child A1
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child A2Posttest
Pretest
ControlWord
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child A3
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child B1
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child B2
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child B3
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child C1
0
1
2
3
4
1 2 3 4 5 6 7 8 9
Child C2
0
1
2
3
4
1 2 3 4 5 6 7 8 9
ChildC3
School A
School B
School C
Book
Mas
tery
Mon
itori
ng P
robe
Sco
re
Year 3 Results: Vocabulary
![Page 41: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/41.jpg)
Year 3 Results: Vocabulary
Average number of words learned (per child) = 8.11, Range 3 - 13
Average number of children who learned each word = 4.06, Range 0 – 8
Lowest “unusual” - no children learned Highest “ill” – 8 children learned
![Page 42: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/42.jpg)
Year 3 Results: Comprehension
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child A1
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child A2
Posttest
Pretest
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child A3
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child B1
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child B2
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child B3
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child C1
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child C2
0
1
2
3
4
5
6
1 2 3 4 5 6 7 8 9
Child C3
School A
School B
School C
Books
Mas
tery
Mon
itorin
g Pro
be S
core
for C
ompr
ehen
sion
![Page 43: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/43.jpg)
Year 3 Results: Comprehension
Criterion for treatment effect: Pretest-posttest difference of at least 2
Treatment effects for most participants for many books (Range: 0 - 6 books).
Average gain score per book was 1.1 points (SD = 1.66, Range = -4 - 4)
![Page 44: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/44.jpg)
We learned a lot…
![Page 45: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/45.jpg)
…but there was still work to be done.
![Page 46: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/46.jpg)
Revisions for Year 4
Replaced 5 words, rewrote 1 story, revised 7 embedded lessons.
Words that were replaced were the lowest performing words (e.g., unusual).
Lessons that were revised were for lower performing words and were based on observations from the facilitators. EXAMPLE: picture for ‘ridiculous’ was
changed from an illustration in the story to a photo of a ridiculous dog. (next slide)
![Page 47: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/47.jpg)
![Page 48: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/48.jpg)
Revisions for Year 4
Inclusion of simple unit review books Repeat of lessons from a set of 3 books
Development of Unit Tests Measure of vocabulary learning in 3 books plus a
review book Designed to be administered ~ once per month
Refinements to training materials, staff manuals, fidelity procedures, scoring reliability
Development of the Assessment of Story Comprehension
![Page 49: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/49.jpg)
Year 4 Study
Randomized group design with embedded single case design
3 classrooms with 6 children in each, children randomly assigned to treatment or delayed treatment.
Intervention implemented by research staff
![Page 50: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/50.jpg)
Year 4 Participants
N = 18; 11 girls, 7 boys African American Recruited from public pre-K settings Identified as having limited oral
language skillsPPVT-IV CELF-P
M Range M Range
Treatment 83.44 77-90 89.11 79-98
Comparison
83.44 78-89 83.89 67-96
No significant difference between groups on these measures.
![Page 51: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/51.jpg)
Year 4 Measurement
Group Design Unit Test of vocabulary words taught in 3
books Assessment of Story Comprehension
Embedded Single Case Design Mastery Monitoring Probes administered at
pretest and posttest for each book.
![Page 52: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/52.jpg)
Year 4 Results: Vocabulary
0
2
4
6
8
10
12
UT1 Pre UT1 Post UT2 Pre UT2 Post UT3 Pre UT3 Post
Participant Comparison
![Page 53: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/53.jpg)
ANOVA of gain scores for each Unit Test Significant differences favoring the
treatment group at each time point Average gain of 4.44 – 6.33 points per
Unit Test Effect sizes between 1.37 – 2.84
Year 4 Results: Vocabulary
![Page 54: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/54.jpg)
Word Level Data from Year 4
Average number of words learned (per child) = 10, Range 3-17
Average number of children who learned each word = 4.94, Range 4-8.
Lowest “enormous”, “brave”, “soaked”, comfort” “speedy” “ridiculous - 3 children learned
Highest “ill” – 8 children learned
![Page 55: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/55.jpg)
Year 4 Results: Comprehension
0
2
4
6
8
10
12
14
16
ASC Pre ASC 2 ASC 3 ASC Post
Participant Comparison
![Page 56: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/56.jpg)
Tier 2 Intervention: Efficacy and Effectiveness Phase
![Page 57: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/57.jpg)
Efficacy and Effectiveness Phase
Kansas cross-site replication of the Tier 2 Ohio Vocabulary/Comprehension intervention
Can others using the same intervention replicate similar results with another group of children identified with weak skills? Ohio produces original intervention and
findings Kansas seeks to replicate
![Page 58: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/58.jpg)
Participants: 2010-11 Replication Sample(s)
State Classrooms
Children Non-WhiteNon-
English Home
Language
Individual
Education
Program
Ohio 3 9 8 0 0
Kansas
3 9 8 5 3
Total 6 18 16 5 3Note. Both samples were predominately non-White. The KS participants included dual language learners and students with IEPs
![Page 59: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/59.jpg)
Student Risk Status at Start
State Measure M Range
Ohio Vocabulary IGDI (Max = 15) 7.2 of 15
5-11
Peabody Picture Vocabulary Test (PPVT) (M = 100, SD = 15)
84.3 78-96
Clinical Evaluation of Language Fundamentals (CELF) (M = 100, SD = 15)
86.4 73-94
Kansas
Vocabulary IGDI (Max = 15) 7.3 of 15
5-13
Peabody Picture Vocabulary Test (PPVT) (M = 100, SD = 15)
86.9 73-107
Clinical Evaluation of Language Fundamentals (CELF) (M = 100, SD = 15)
72.6 50-102
Note. IGDI = Individual Growth and Development Indicator
Lower Language
![Page 60: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/60.jpg)
Tier 2 Vocabulary and Comprehension Storybooks with Embedded Instruction Intervention
![Page 61: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/61.jpg)
Organization of Replicaiton Results
Mean Results Across Storybooks Mean acquisition of words taught and
comprehension before and after the listening intervention across storybooks
Cumulative Word Mastery View Students with the Best and Worst
response to intervention Overall Effect Size
![Page 62: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/62.jpg)
![Page 63: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/63.jpg)
Mean Cumulative Mastery of All Vocabulary Taught
MasteryGoal
![Page 64: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/64.jpg)
Least and Most Responsive Student
![Page 65: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/65.jpg)
Overall Effect Sizes
Ohio Standard Mean Difference (SMD)1
1. Vocabulary, d = 1.712. Comprehension, d = 0.59
Kansas Standard Mean Difference (SMD)1. Vocabulary, d = 1.522. Comprehension, d = 0.57
Note. [d = ((Xafter – Xbefore)/SDbefore)]
Note. 1Spencer et al. (in press)
![Page 66: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/66.jpg)
Evidence Produced
The Vocabulary and Comprehension Tier 2 intervention was fully developed, implementable with fidelity
Efficacy was demonstrated in Kansas in replication by a different team in different schools, serving children with weak skills including some dual language learners and students with IEPs
A range of student response to the intervention was observed
Future work needs to focus on achieving larger student effects demonstrating similar findings with implementation by
preschool personnel in larger samples
![Page 67: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/67.jpg)
Year 5 Story Friends Efficacy Trial
2012-2013 school year, 24 classrooms in OH, 8 in KS
Cluster randomized design: classrooms randomly assigned to Treatment and Comparison Treatment: Story Friends Program Comparison: Story Friends books with no
embedded interventions Implemented by educational staff Research staff provides assistance to
teachers and administers assessments
![Page 68: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/68.jpg)
Implications
• New evidence-based interventions are not developed in a single stand alone study!
• Instead, intervention development requires an iterative process that includes planning, piloting, evaluation, improvement and re-testing prior to testing at large scale.
![Page 69: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/69.jpg)
Programmatic research: What did we learn along the way?
Pilot: Intervention was feasible. Efficacy studies with single-case design in
OH: Intervention produced weekly vocabulary gains across multiple children.
Replication of single case design studies in KS: Intervention resulted in the same effect (weekly vocabulary gains) in another location
![Page 70: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/70.jpg)
Efficacy group design with researchers as implementers: Children in intervention group did better than controls on standardized measures.
Efficacy group design with teachers as implementers: Children in intervention group did better than controls on standardized measures.
Next step: An independent evaluation of effectiveness showing that the intervention works.
At each step we learn something that we move forward to the next.
![Page 71: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/71.jpg)
Value of Replication Studies
Replication helps us test whether the intervention is effective when implemented by different researchers; external validity of the intervention
Replication helps us see if intervention can systematically be changed and whether effects still hold up: How much can you vary an intervention and still see positive outcomes?
Replication is an important step prior to large scale up.
But replication studies are still fairly rare
![Page 72: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/72.jpg)
Implications: Importance of Programmatic Research
• This type of programmatic research is important for at least three big reasons:• It makes it more likely that only our
strongest interventions will be going forward for large efficacy and effectiveness trials.
• It minimizes the risk of weak interventions being tested in large scale studies.
![Page 73: Charlie Greenwood, Elizabeth Spencer, Howard Goldstein, & Judy Carta Center for Response to Intervention in Early Childhood Division of Early Childhood.](https://reader036.fdocuments.in/reader036/viewer/2022062304/56649d845503460f94a6aad6/html5/thumbnails/73.jpg)
It reinforces the idea that educational researchers should be aiming to nail down an intervention’s effect and extend the effect to different learning environments and student characteristics.