1
Lessons Learned from OSEP’s Model Demos: Creating Change to Promote Children’s
Success
Lessons Learned from Model Demonstration Projects
Mary Wagner, Ph.D., Principal InvestigatorPhyl Levine, Ph.D., Director
SRI International
OSEP Project Directors’ ConferenceJuly 19, 2010 Washington, DC
Model Demonstration Project Cohorts
• Cohort 1: Progress monitoring/RtI for struggling readers K-5– Lehigh University, University of Pittsburgh– University of of Minnesota, Minneapolis Public Schools– University of Oregon
• Cohort 2: Tertiary behavior interventions K-8– University of Kansas, Illinois PBIS Network– University of Oregon– University of Washington
• Cohort 3: Early childhood language development– Orelena Hawks Puckett Institute – University of Kansas – Vanderbilt University, Florida State University
• Cohort 4: Writing proficiency in high school– University of Kansas– University of Hawaii-Manoa
The Model Demonstration Coordination Center (MDCC) was launched in 2005 to:
• Identify characteristics of an effective implementation/evaluation/refinement process that moves a practice from early testing to being ready for wider adoption
• Coordinate the evaluation of each cohort of Model Demonstration Project’s (MDPs) and synthesize and analyze their findings to maximize the strength of evidence produced
MDCC activities:• Facilitate a collaborative partnership with the MDPs
to create opportunities for learning• Contribute and/or broker methodological expertise• Develop a data system to assemble
MPD-provided data• Conduct cross-MDP and cross-cohort
analyses• Communicate implementation and
evaluation findings to promote adeeper understanding of the modeldemonstration process and its results
Framework for
understanding model
implementation and outcomes
Characteristics of the “source” in progress monitoring models*
SourceThe Model
Core intervention componentsMeasurement model and proceduresHow data are used to:▪ Monitor student progress
▪ Adapt instruction for strugglingreaders
▪ Inform eligibility determination forspecial education and developmentand monitoring of IEP goals
*Using cohort 1 as an example
Characteristics of progress monitoring “purveyors”
PurveyorThe MDP Grantee
Core implementation components▪ Introducing model to schools/teachers
▪ Formal professional development
▪ Ongoing coaching
▪ MDP staff selection and staffingstrategy
Characteristics of the progress monitoring “destination” organizations
DestinationParticipating Schools and Teachers
Characteristics of participating schools,classrooms, and teachers
Implementation outcomes―changes in:▪ Staff knowledge, attitudes, and actions
▪ Organizational structures, processesand culture
▪ External relationships
Sustained implementation
Influences on destination organizations and their implementation
• District-level (e.g., other initiatives, superintendent turnover, history with grantee university)
• State level (e.g., RtI initiatives, testing requirements)
• Other factors (e.g., union power/influence)
Intervention outcomes of progress monitoring models Student Outcomes
Increased:Oral reading fluencySAT-10 scoresPercentage reaching
benchmarksPercentage proficient on
state tests
Systems OutcomesFewer students referred
to special educationfor reading disabilities
Increased use of studentPM data in determin-ing special educationeligibility and settingand monitoring IEPgoals
Feedback on progress monitoring model implementation and effectiveness
• Reflect on lessons learned within cohorts
• Fidelity data
• Social validity data
Analysis and reporting
• Describe variations in each component of the conceptual framework for the three MDPs in a cohort
• Generate hypotheses from implementation/ innovation research regarding how variations may shape implementation experiences
• Hold hypotheses up to implementation experiences and outcomes– Across MDPs in a cohort– Across cohorts
• Derive principles regarding an effective and efficient model demonstration process
14
Cohort I:The University of Minnesota: Teri
Wallace Minneapolis Public Schools: Douglas
Marston
Lehigh University: Edward Shapiro and The University of Pittsburgh: Naomi
Zigmond
The University of Oregon: Gerald Tindal
15
Project MP3: Monitoring Progress In Pennsylvania Pupils
Lehigh University
16
What Makes RTI(I) Work?
• Consensus• Infrastructure• Implementation
17
Lessons Learned: What Made It Work?Consensus
• District committed to sustainability from outset
• District support at highest level of administration
• Willingness of schools to own the process
18
Lessons Learned: What Made It Work?Infrastructure Support
• Professional Development Specialist in district• District redirected resources to support implementation• District partnered with Project to secure new instructional
resources• Schools had similar context• Each school had identified principal leadership, from the
district perspective• Presence of well established core reading program• Willingness to modify schedules• Willingness to seek needed professional development• Inclusion of parent advisory component
19
Lessons Learned: What Made It Work?Implementation Support
• Universal screening measures already in place for several years
• Use of data, use of progress monitoring, not evident despite universal screening
• Schools modified professional development schedule to meet project specifications
• Schedules included recognized time for core and grade level meetings
20
Process Outcomes
• Implementation fidelity high for model in all schools• Data use and data based decisions high in all schools• District expanded the project to all elementary buildings
in district within 2 years of project ending• Maintained presence of parent advisory group to process• Maintained process despite change in building principal
at one building• Staff able to assume data management task after support
for a year• Model fit within the statewide initiatives
21
Some Not So Good Outcomes
• Despite strong implementation in all buildings, one building had much poorer student outcomes
• Instructional leadership of building principal level becomes crucial
22
DPM
University of Minnesota and Minneapolis Public Schools
http://progressmonitoring.net/ - see link to video
23
Context
• District had long history of commitment to using data for instructional decision-making though not implemented fully in all schools
• Implementation school’s leadership was committed to RTI, Reading Initiatives, Professional Development, etc.
• Schools had numerous and varied initiatives needing alignment
• Teachers’ knowledge of RTI and its various components varied across the schools
24
Implementation - Tools
• System Supports– Data Management System– Time – Master Calendar
• Professional Development• Leadership and Changing Roles
25
Process Outcomes
• Procedures were developed to assess teachers’ fidelity of implementation related to interventions, progress monitoring procedures, and data review meetings
• Professional development strategies were targeted based on the needs shown in student data. For example, PLCs focused on Tier 2 and 3 interventions needed to meet student needs. Coaching responded to needs identified through fidelity checks, and more…
26
Process Outcomes, continued
• Data meetings met to review progress of all students and focused on students receiving Tier 2 and 3 interventions. A RTI coordinator was hired to facilitate these meetings, organize data, etc. Originally supported through grant dollars, this position remains despite district cuts.
• Teacher attitudes were measured to determine their view of the components of RTI. They were supportive and positive.
27
Teacher Testimonial
I thought I was an “OK” teacher and having been through the whole process of learning more about instruction, understanding the developmental aspects of reading, knowing what to do because I have good assessment data – I feel like a great teacher.
28
Lessons Learned from Model Demos:
Creating Change to Promote Children’s
Success
Cohort II
Tertiary Intervention: The K-I Center
Lucille Eber Co-Principal Investigator
Illinois PBIS Network
Wayne Sailor
Principal Investigator University of Kansas
OSEP Project Director’s Conference
July 19-21, 2010
29
Cohort II- cont.
Intensive Positive Behavior Support (IPBS) The University of Oregon: Cynthia Anderson
Scaling the Pyramid: A Model of TertiaryIntervention Services to Students withChallenging BehaviorUniversity of WA: Carol Davis Ilene Schwartz I
30
K-I Center Leadership
• Jamie Bezdek• Amy McCart• Holly Sweeney
• Kimberli Breen• Kelly Hyde• Sheri Luecking• Diane McDonald• Jen Rose
31
Implementation Outcomes: Change in Knowledge/Skills of Adults
• Role of Building Administrators with behavior support
• Change in role of Special Education personnel
• Role of District Administrators in guiding the systems change process
32
Example: Principal Role(no longer a “Special Education issue”)
Administrators being taught the system features, the data/tools, and the practices well enough to guide/lead any “corrections” needed within FBA/BIP and wraparound plans.
Administrator Training Skill set example:
If an individual behavior intervention plan
is not working, what should a principal look for or ask?
33
Administrator Role Example
School personnel should not be able to choose NOT to provide students with evidence-based interventions.
Academic ‘analogy’:
Personnel are expected to provide evidence-based academic instruction; same expectations need to be established for behavior support.
34
Implementation Outcomes: Change in Organizational Structure/Culture
• Systems need to establish competency and confidence with fidelity of behavior interventions within general education.
• All staff “owning” success of ALL students
• Shift in system habits: from “test/place” to interventions and progress monitoring
35
Implementation Outcomes: Changes in External Relationships
• District Leadership Teams
• District External Coaches
• Special Education Directors/Organizations
• Community Partners
• Families
36
Implementation Outcomes: Sustainability of Model
Replication of Tertiary Demos Moving Faster…
IL Phases of Implementation: Secondary Phase I (n=8 Replication Schools)
0%10%20%30%40%50%60%70%80%90%
Team meets
regularly
Sec/Ter.
tracking tool
used
Students are
referred for
tier 2
interventions
DPR used 70% success
rate for simple
tier 2
interventions
Sch
oo
ls w
ith It
em
in P
lac
e
Fa ll 2008 Spring 2009
38
Implementation Outcomes: Sustainability of Model
• Policy changes are addressed
– transition support to ensure behavioral success
– Job Description and supervision changes
• Training and technical assistance for behavior skill sets are routinely scheduled in districts
• Use of data by district teams (risk ratio, LRE, etc) becomes ‘business as usual’.
39
Policy Examples
• Summer FTE of School Social Workers to support some students w/Tier 3 plans
• Transition planning support expected as students change grade levels
• Job Descriptions include expectations for evidence-based behavior support
40
Training Examples
• Administrator role with behavior support at all three tiers
• Changing role of special education personnel in facilitating teams/plans
• Teacher expectations with behavior support at all three tiers
• Using RtI ‘framework’ with family engagement
41
Data Examples
Review data by ethnicity and disability routinely at building and district levels
– Discipline data
– Academic data
– Restrictiveness of placement data
Students with IEPs Served in Separate Placements
102122
7.48
4.91 4.58
9.01
0
2
4
6
8
10
FY07 FY08
Ra
tio
90
100
110
120
130
# o
f St
ud
ents
Students w/IEPs in separate placementsDistrict Ratio State Target
43
Lessons Learned from Model Demos:
Creating Change to Promote Children’s Success
Cohort 3Early Childhood Language
Interventions
OSEP Project Director’s Conference Washington DCJuly 19-21, 2010
44
The Center on Everyday Child Language Learning
Orelena Hawks Puckett Institute Carl Dunst
Carol Trivette
University of Kansas Dale Walker
Jane Atwater Kathy Bigelow
Vanderbilt University Florida State University
Ann KaiserJuliann Woods
45
Common Features of Cohort 3Process
Models of naturalistic evidence-based language interventions for children ages birth through 5
Implement the model in 3 sites representing typical settings
Enroll children in Part C programs and follow them into Part B programs
Provide professional development to providers implementing the model
Assess impacts using some common measures
46
Coaches build the capacity of parents and early educators to promote communication
Parents/teachers use evidence-based practices
Build on strengths and collaborate with parents about strategies and routines Embed intervention into home and school routines Collaborate with Part C and Part B during transitions
47
Parents are their child’s first communication partners
Teachers and service providers support child communication across settings
KTTP Communication teams include families, providers across agencies, and communication coaches
Communication coaches facilitate parent and service providers learning
Coordinated transition and continuity in communication intervention between Parts C and B is lead by the parent in the communication team
48
Identify children’s interests and everyday activities that are suited for learning communication skills Increase child participation in these interest-based everyday activities
Embed instructional practices for supporting and strengthening communication in the contexts of activities
Part C providers facilitate parent learning and use of strategies
Approach fits within existing programs
Strengthen parents’ abilities to enhance the transition to Part B
49
Strategies to Change in External Relationships
Use the team that includes parents to collaborate with the Part B preschool services during transition
Empower parents to lead the team to support the transition to Part B preschool services
Empower parents to promote continuity for children from Part C to Part B preschool services
50
Process Reflections on TransitionsProjects differ in the people they are trying to include in
the transition from Part C to Part B Projects all include parents in the transition but the
emphasis of the role of the parent variesProjects differ in terms how direct their involvement is
in the transition process
These differences lead to different transition challenges: joining an already formed team, development of parent leadership, development of transition skills in parents
Top Related