Scaling Up SW PBS Implementation: Different Journeys to the · PDF fileScaling Up SW PBS...
Transcript of Scaling Up SW PBS Implementation: Different Journeys to the · PDF fileScaling Up SW PBS...
Scaling Up SW PBS Implementation:
Different Journeys to the Same
Destination
Heather Peshak George, Ph.D.
APBS Conference, St. Louis, MO: March 26, 2010
2
Florida’s PBS Project
Our Mission• Increasing the capacity of Florida’s school districts to
address problem behavior using positive behavior support through a Response to Intervention framework.
What we do…• Provide training and technical assistance to districts across
the state in the development and implementation of positive behavior supports at the school-wide, targeted group, classroom and individual student levels
3
Exploration Phase
• Is there a need for change?
• What current practices and initiatives exist that are facilitators or barriers?
• What is innovation and does it address our problem?
• How do we plan for implementation?
• Is the team ready to begin installation of innovation?
4
Installation Phase
• What individuals have authority to reallocate resources/facilitate implementation and connect with state improvement plan?
• Who will guide implementation?
• What does implementation of the innovation involve?
5
Initial Implementation Phase
• Who are initial implementers?
• How do we begin implementation?
• How do we monitor fidelity and outcomes?
6
Full Implementation Phase
• How do we build local training/coaching capacity?
• What resources do we need to add more schools?
• How do we sustain fidelity and positive outcomes over time?
7
Innovation and Sustainability Phase
• How do we braid innovation with new/existing initiatives?
• How do we become more efficient and effective?
• How do we share what we have learned?
9
Funding
• IDEA Part B Discretionary Grant
• Around 1997: $33,000/year
• Current: Almost $1.8 million/year
10
Visibility
Before:
• Present at every state conference
• Special education
• Expanded across disciplines
• ABA, Safe Schools, RtI, Reading, School
Psychologists, Superintendent, Evaluation
Now:
• Present when requested or “invited”
11
Click Here…
12
http://flpbs.fmhi.usf.edu/resources_newsletter.asp
Newsletter:
Read directly
from the website
or download to
print.
Use to find useful
tips, data,
important dates,
etc.
13
Political Support and Policy
• Expanded funding of project
• State RtI Implementation Plan
• Active member on the FL DOE’s State Transformation Team for RtI
• Title of Project to include RtI:B
• RtI
• Bullying & Harassment Policy
• Zero Tolerance
State Infrastructure• State Management Team
– Role: Provide leadership and facilitate policy-level changes to support implementation of effective educational practice
• State Transformation Team
– Role: Analyze progress toward statewide efforts, recommend actions for improvement, and support District Based Leadership Teams DBLT to build the capacity of districts to implement evidence-based practices and to establish integrated RtI academic and behavior systems in each school
• District Based Leadership Team
– Role: Provide leadership, advisement, and training at the district level and assist schools in their implementation efforts
• School Based Leadership Team
– Role: Develop a school implementation plan. The school based team willbecome “trainers” and “coaches” for the school staff and will be responsible for school wide implementation
• Advisory Group
– Role: Provide on-going stakeholder input
– pp. 11-13 of the Statewide RtI Implementation Plan
District Responsibilities
Based on selfBased on self--assessment results, and in conjunction assessment results, and in conjunction
with the student progression plan and Kwith the student progression plan and K--12 12
Comprehensive Reading Plan, develop an RtI Comprehensive Reading Plan, develop an RtI
implementation plan organized around building implementation plan organized around building
consensus, infrastructure, and implementation. Plans consensus, infrastructure, and implementation. Plans
should also address:should also address:
–– How current resources will be used to implement How current resources will be used to implement
RtI and identify additional resources neededRtI and identify additional resources needed
–– How stakeholders will be educatedHow stakeholders will be educated
–– How stakeholders will be involvedHow stakeholders will be involved
p. 8 of the Statewide RtI Implementation Plan
Alignment with Existing InitiativesThe basic elements of RtI are required by NCLB The basic elements of RtI are required by NCLB
and IDEA, therefore, they are included in all and IDEA, therefore, they are included in all
broadbroad--based initiatives for schools striving to based initiatives for schools striving to
meet (AYP), such as:meet (AYP), such as:
–– KK--12 Reading Plan and Reading First12 Reading Plan and Reading First
–– FloridaFlorida’’s Continuous Improvement Model (FCIM)s Continuous Improvement Model (FCIM)
–– FloridaFlorida’’s Positive Behavior Supports (PBS)s Positive Behavior Supports (PBS)
–– PS/RtI Pilot ProjectPS/RtI Pilot Project
–– Bright BeginningsBright Beginnings
p. 1 of the Statewide RtI Implementation Plan
Tiered Model of School Supports & the Problem-Solving Process
ACADEMIC and BEHAVIOR SYSTEMS
Tier 3: Intensive, Individualized
Interventions & Supports The most intense (increased time, narrowed focus,
reduced group size) instruction and intervention
based upon individual student need provided in
addition to and aligned with Tier 1 & 2 academic
and behavior instruction and supports.
Tier 2: Targeted, Supplemental
Interventions & Supports.More targeted instruction/intervention and
supplemental support in addition to and aligned
with the core academic and behavior curriculum.
Tier 1: Core, Universal
Instruction & Supports.General academic and behavior instruction and
support provided to all students in all settings.
FL RtI State Transformation Team, Dec. 2009)
18
19
Training
• School-Wide/Initial (Tier 1)
• Booster/Retraining (Tier 1)
• Classroom (Tiers 1 & 2) – online only
• Targeted Group (Tier 2)
• Individual Student (Tier 3)
• Team Leader
• Principal/Administrator
• SWIS
• Coaches’Italics indicates both on-site & online modules available
20
Tier 1 Training Sequence
Day 1
Intro to RtI:B
Teaming
Developing Expectations & Rules
Developing a System for Teaching
Day
3
Evaluation
Classroom PBS
Facilitated work time
Day
2
Developing a Reward System
Effective Discipline Procedures
•Definitions
•Forms
•Process
•Responses
Implementing PBS
# Florida Schools' Initial PBS Training
12
61
8781
53
77
119
219
0
50
100
150
200
250
2001-2002 2002-2003 2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
Year
Total Schools Trained
22
Number of Schools Trained(As of March, 15 2010)
6 Pre-K schools
387 Elementary schools
167 Middle schools
82 High schools
56 Alt/Center schools
50 Other (e.g. K-8)
748 TOTAL SCHOOLS
Coaching & Behavioral ExpertiseRtI:B Support Structure
Project Contact
����
District Coordinator
(District Leadership Team)
����
COACH����
School-based Team Leader
����
PBS Team Members
25
Factors to Consider in Developing Comprehensive Evaluation Systems
1) Systems Preparation– Readiness activities
2) Service Provision– Training and technical assistance
3) Identification and Assessment of Behavior Problems– Possible data sources
4) Evaluation Process– Timelines, data systems
5) Evaluation Data (Across all three Tiers)– Implementation Fidelity, Impact on Students, Attrition, Client
Satisfaction
6) Products and Dissemination– Reports, materials, presentations, etc.
(modified from Childs, Kincaid & George, in press)
26
Florida’s Evaluation Model
Training
On-going
technical
assistance
FLPBS
↓
Districts
↓
Coaches
↓
Schools
End-
Year
Impact
Outcome data (ODR, ISS, OSS)
FL Comprehensive Assessment Test
Benchmarks of Quality
School Demographic Data
PBS Walkthrough
Daily Progress Reports
Behavior Rating Scales
Climate Surveys
Implementation Fidelity
PBS Implementation Checklist (PIC)
Benchmarks of Quality (BoQ)
Benchmarks for Advanced Tiers (BAT)
School Demographic Data
School-wide Implementation Factors
Tier 3 plan fidelity checklist
BEP Fidelity checklist
Project Impact
Attrition Survey/Attrition Rates
District Action Plans
Client Satisfaction
School-Wide Implementation Factors
District Coordinator’s Survey
Training Evaluations
�Annual Reports
�Revisions to training
and technical
assistance process
�National, State,
district, school
dissemination
activities
�Website
�On-line training
modules
Systems
Preparation
Service
ProvisionEvaluation
Process
Evaluation
Data
Products and
Dissemination
MidY
ear
I
MidY
ear
II
Identification/
Assessment
�Discipline Records
�ESE Referrals
�Surveys
�Walkthroughs
�PIC
�Classroom Assessment
Tool
�Student rank/rating
�Teacher requests
�Lack of response
�BAT
�Behavior Rating Scale
�Daily Progress Report
Charts
•District Action
Plan
•District
Readiness
Checklist
•School
Readiness
Checklist
•New School
Profile (includes
ODR, ISS, OSS)
27
(1) Systems Preparation• Tier 1
• District Readiness Checklist• District Action Plan
– District baseline and goals
• School Readiness Checklist• Baseline data
• Tier 2• School Readiness
• Implementation of Tier 1
• School Infrastructure
• Tier 3– District Action Plan
• Systems change
• Evaluation of products and processes
• Establish vision and goals
28
29
(2) Service Provision
Training and ongoing technical assistance
FLPBS
Districts
Coaches
Schools
•Training
•Tier 1- District and multi-district
on-site
•Tier 2 –District, multi-district,
web-based
•Tier 3- Post assessment, goal
setting, systems/process established
(3) Identification & Assessment
• Tier 1
– Discipline records, attendance, ESE referrals, baseline BoQ, action plans, climate surveys, coaches surveys, walkthrough (mini SET), PBS Implementation Checklist (PIC)
• Classroom
– Discipline records, teacher requests, student rankings/ratings, ESE referrals, observations, Classroom Assessment Tool
• Tier 2
– Discipline records, teacher requests, student rankings/ratings(SSBD, TRF, etc…), lack of response to Tier 1, Daily Progress Reports, PBS Implementation Checklist (PIC), Benchmarks for Advanced Tiers (BAT)
• Tier 3
– Above items, lack of response to Tier 2, Behavior Rating Scale, observation data, intervention fidelity checklist, PBS Implementation Checklist (PIC), Benchmarks for Advanced Tiers (BAT)
31
Referrals per Student
0
10
20
Nu
mb
er
of
Re
ferr
als
pe
r S
tud
en
t
Students
Office Discipline Referrals
Student Initials Grade/Period I or E (Step 2)
_________ ________ ________
_________ ________ ________
_________ ________ ________
Academic Personally Taught Personally Given Externalizing Concerns Expectations SW Reward 1. __________________ ____ Yes ____ Yes ____ Yes
2. __________________ ____ Yes ____ Yes ____ Yes
3. __________________ ____ Yes ____ Yes ____ Yes
•Rank top 3 externalizing and top 3 internalizing
students
•Check “YES” if personally taught expectations to
the student
•Check “YES” if personally given a SW-PBS
reward to student
Teacher Nomination
Tier 2 Progress Monitoring
Behavior Rating ScaleBehavior Date
Hitting 8 or more6-7 times4-5 times2-3 times0-1 times
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
Profanity 16 or more times12-15 times8-11 times4-7 times0-3 times
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
Requesting Attention/Assistance
55% or more40-55%25-40%10-25%0-10%
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
54321
(4) Evaluation Process
• Timelines for State Evaluation– Baseline (due date varies)
– Mid Year I – due 10/31• School Profile
• PBS Implementation Checklist (PIC) (Tiers 1-3)
– Mid Year II – due 2/28• PBS Implementation Checklist (PIC) (Tiers 1-3)
– End Year – due 6/15• Benchmarks of Quality (BoQ) (Tier 1)
• Benchmarks for Advanced Tiers (BAT) (Tiers 2-3)
• Outcome Data Summary
• School-wide Implementation Factors (SWIF)
• Web-based Data Entry and Reporting• PBSES
• Statewide Student Database – Academic/Behavior
36
(5) Evaluation Data
a) Implementation Fidelity
– PIC (All Tiers)
– BoQ (Tier 1)
– BAT (Tiers 2-3)
– SWIF (All Tiers)
– Walkthrough (Tier 1)
– Tier 2 & 3 intervention specific fidelity measures
c) Attrition
– Attrition Survey (All)
b) Impact on Students
– Outcome data (ODR, ISS, OSS)
– Academic achievement
– School Demographic Data (e.g. ethnicity)
– Attendance
– DPR charting
– Behavior Rating Scale
d) Client Satisfaction
– SWIF
– Climate surveys
– Social validation
37
(a) Implementation Fidelity
1. Are schools trained in SWPBS implementing with fidelity? Across years? Across school types?
– BoQ, BAT, School Demographic Data
2. What factors are related to implementing with fidelity?
– SWIF survey, BoQ, BAT
38
Tier 1 Critical
Element
Implementation
Level chart
39
PBS
Implementation
Level chart
40
High Implementing Florida PBS Schools(Scoring 70 or Above on BoQ)
57 5660
7774
0
10
20
30
40
50
60
70
80
90
100
School Year
Percent of Schools Scoring 70+ 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
Average
Score
66
Average
Score
66
Average
Score
72
Average
Score
77
Average
Score
76
BoQ Totals by School Type Across Years
Average BoQ Total Score by School Type
69
64
70 6971
66 66
7674 73
63
8178
74 75
8080
74
67
80
0
10
20
30
40
50
60
70
80
90
Elementary Middle High Alt/Center
School Type
Average BoQ Total Score
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
38Schools
77Schools
104Schools
27Schools
54Schools
55Schools
6Schools
14Schools
17Schools
15Schools
26Schools
28Schools
149Schools
221Schools
59Schools
93Schools
17Schools
25Schools
32Schools
37Schools
Brief Walk-through
Benchmarks for Advanced TiersScore Summary for BAT Sections
0
10
20
30
40
50
60
70
80
90
100
Tie
r 1
T2
&3
:Co
mm
it
T2
&3
:Stu
Id
ent
T2
&3
:Mo
nit
or
T2
:Su
pp
ort
T2
:Mai
n I
mp
l
T2
:Mai
n M
on
ito
r
T2
:2n
d I
mp
l
T2
:2n
d M
on
ito
r
T2
:3rd
Im
pl
T2
:3rd
Mo
nit
or
T2
:4th
Im
pl
T2
:4th
Mo
nit
or
T3
:Su
pp
ort
T3
:Ass
ess
T3
:Mo
nit
or
To
tal
BAT Sections
Percent Possible Score
44
(b) Impact on Student Behavior
1. Do schools implementing SWPBS decrease ODRs, days of ISS, and days of OSS?
– ODRs, ISS, OSS
2. Do schools implementing SWPBS realize an increase in academic achievement?
– FCAT scores
3. Is there a difference in outcomes across school types?
– ODRs, ISS, OSS, FCAT scores, school demographic data
4. Do schools implementing with high fidelity have greater outcomes implementers with low fidelity?
– BoQ, ODRs, ISS, OSS
5. Do teams that work well together have greater outcomes than those that don’t work as well together?
– Team Process Evaluation, ODRs, ISS, OSS
Percentage Change in Discipline Outcome
Between Baseline and Year 1
-24%
-18%
-8%
-30%
-25%
-20%
-15%
-10%
-5%
0%
ODR ISS OSS
Percentage Change
110Schools
89Schools
108Schools
higher implementing schools reported 37% fewer ODRs per 100 students
162
114115
94
105100
107
85
115
73
0
20
40
60
80
100
120
140
160
180
Low (BoQ <70) High (BoQ >=70)
Implementation Level
Average ODRs per 100 Students
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
35
Schools
77
Schools
83
Schools
39
Schools
76
Schools
107Schools
67
Schools
162
Schools
ODR Rates by Implementation Level
Across School Years
102
Schools
256
Schools
ISS Rates by Implementation Level
Across School Years
52
29
45
28
43
32
54
25
39
26
0
10
20
30
40
50
60
Low (BoQ <70) High (BoQ >=70)
Implementation Level
Average Days ISS per 100 Students
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
35Schools
77Schools
83Schools
39Schools
76Schools
107Schools
67Schools
162Schools
67Schools
162Schools
overall average difference of 40% across the five school years
73
43
61
34
56
37
51
28
50
25
0
10
20
30
40
50
60
70
80
Low (BoQ <70) High (BoQ >=70)
Implementation Level
Average Days OSS per 100 Students 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
35Schools
77Schools
83Schools
39Schools
76Schools
107Schools
67Schools
162Schools
OSS Rates by Implementation Level
Across School Years
67Schools
162Schools
overall average difference of 43% across the five school years
49
(c) Attrition
1. Why do schools discontinue implementation of SWPBS?
– Attrition Survey
50
(d) Consumer Satisfaction
1. Are our consumers satisfied with the training, technical assistance, products and support received?
– SWIF survey
– District Coordinators survey
– Training evaluation
– Climate surveys
School-Wide Implementation Factors (SWIF)
Higher Implementing Lower Implementing
(70+ on BoQ) (-70 on BoQ)
90% + respondents from high implementing
schools identified these factors as Helpful:
80%+ respondents from low implementing
schools identified these factors as Helpful:
Factors
MOST
Helpful to
Implementation of
SWPBS
•Expectations and rules clearly defined
•Administrator committed to PBS,
willing to teach and model PBS, willing
to reward students
•Representative and committed PBS
Team
•Reward system works
•PBS Coach’s guidance with process
•Students responses to rewards and
activities
•Expectations and rules clearly defined
•Administrator willing to reward
students
•Representative PBS Team
25%+ respondents from high implementing
schools identified these factors as Problematic:
50%+ respondents from low implementing
schools identified these factors as Problematic:
Factors
MOST
Problematic to
Implementation of
SWPBS
•Adequate funding
•Team recognizes faculty participation
•Staff stability from year to year
•Student stability from year to year
•Staff time for PBS
•Staff belief about effectiveness of PBS
•Staff philosophy
•Staff consistency in teaching
•Staff consistency in discipline
procedures
53
(6) Products and Dissemination
• Annual Reports
• Revisions to Training
• Revisions to Technical Assistance process
• Dissemination activities:
– National, state, district, school levels
• Revisions to Website
• On-line training modules
Model SchoolsFlorida's Model Schools
1739 56
91133
95
184221
280
410
0
50
100
150
200
250
300
350
400
450
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009
Number of Model Schools Number of Active Schools
32% of eligible schools achieved model school status in 2008-
2009. Of the 133 schools = 27 Gold, 72 Silver, 34 Bronze
55
What’s New
• Train-the-Trainer Evaluation Process
• Expansion of Online Training Modules
• State-wide Behavioral Database
• Possible State RtI Center
• High Schools
• Further Research
• ???
56
Current Research
• PBS Implementation Checklist (PIC)
• Benchmarks of Quality (Revised)
• Benchmarks for Advanced Tiers (BAT)
• Walkthrough
• Tier 3
• Model Demonstrations at Tier 2/3
57
Challenge
58
Contact
Heather Peshak George, Ph.D.
• Phone: (813) 974-6440
• Fax: (813) 974-6115
• Email: [email protected]
• State Website: http://flpbs.fmhi.usf.edu
• National Website: www.pbis.org
• Association on PBS: www.apbs.org