Using PBIS Assessment to Create School and District PBIS Evaluation Plans
description
Transcript of Using PBIS Assessment to Create School and District PBIS Evaluation Plans
![Page 1: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/1.jpg)
Using PBIS Assessment to Create School and District
PBIS Evaluation Plans
Northern California PBIS SymposiumNovember 18, 2013
Are we doing what we said we would do?
![Page 2: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/2.jpg)
Agenda
Overview of available surveys/tools on the PBIS Assessment website
Developing an Evaluation Action PlanDecisions on which tools to chooseDecisions on timelines
Assessment Example Plans
Developing Evaluation Reports
Questions/Comments
![Page 3: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/3.jpg)
Acknowledgments
School-wide Positive Behavior Support: Implementers' Blueprint and Self-Assessment
Sugai, G., Horner, R.H., Algozzine, R., et.al. (2010).
Evaluation Blueprint for School-Wide Positive Behavior Interventions & Support
Algozzine, R., Horner, R.H, Sugai, G., et.al. (2010).
Fidelity Measures to Improve Implementation of Positive Behavioral Support
Tobin, T. J., Vincent, C. G., Horner, R. H., Rossetto Dickey, C., & May, S. A. (2012).
![Page 4: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/4.jpg)
PBIS Evaluation Blueprint
“A hallmark of School-wide Positive Behavior Support (SWPBS) is a commitment to formal evaluation.”
“Evaluation is the process of collecting and using information for decision making.
![Page 5: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/5.jpg)
Foundations
Evaluation is the process of gathering data toAnswer questionsMake decisions
Define the core questions/decisions, before selecting measures.
Provide an adequate context for interpreting the data.
Effective evaluation is cyclical (repeated)Use the evaluation process for improvement
![Page 6: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/6.jpg)
Evaluation Cycle
Core Questions
Evaluation Plan
Data Collection
Evaluation Report
![Page 7: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/7.jpg)
Core Evaluation QuestionsContext
What are/were the goals and objectives for SWPBIS implementation? (state/district capacity; school adoption; student outcomes)
Who delivered the training and technical assistance for SWPBIS implementation?
InputWhat professional development events were part of SWPBIS implementation support?
Was projected level of TA capacity provided (training/coaching)?
Who received the professional development (schools/ cohorts)?
![Page 8: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/8.jpg)
Core Evaluation QuestionsFidelity
To what extent was Tier I SWPBIS implemented with fidelity ?
To what extent were Tier II and Tier III SWPBIS implemented with fidelity?
To what extent is the Leadership Team establishing core functions?
![Page 9: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/9.jpg)
Leadership TeamActive Coordination
FundingVisibility Political
Support
Training Coaching Evaluation
Local School/District Teams/Demonstrations
BehavioralExpertise
Policy
Sugai et al., www.pbis.org
![Page 10: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/10.jpg)
Core Evaluation QuestionsImpact
To what extent is SWPBIS associated with changes in student behavioral outcomes?
To what extent is SWPBIS associated with changes in academic performance, and dropout rates?
To what extent is district/state capacity established? (local training, coaching, evaluation, behavioral expertise)
To what extent is leadership and policy structure established?
![Page 11: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/11.jpg)
Core Evaluation Questions
Replication, Sustainability, and ImprovementTo what extent do Fidelity and Impact outcomes sustain across time?
To what extent does initial SWPBIS implementation affect Implementation with later cohorts?
To what extent did SWPBIS implementation change educational/behavioral capacity/policy?
To what extent did SWPBIS implementation affect systemic educational practice?
![Page 12: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/12.jpg)
Evaluation Questions
FidelityTo what extent was SWPBS implemented as designed?
To what extent was SWPBS implemented with fidelity?
“Are we doing what we said we would do?”
![Page 13: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/13.jpg)
To What Extent Can
PBIS Applications.orgPBIS AssessmentPBIS Evaluation
Help us answer the question:“Are we doing what we said we would do?”
![Page 14: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/14.jpg)
Current Tools in PBIS Assessment
Research Tool
Annual Assessment
Tool
Progress Monitoring
Tool
UniversalSystem
School-wide Evaluation Tool
(SET)
Benchmarks of Quality (BoQ)
Early Childhood BoQ (ECBoQ)
PBIS Self-Assessment Survey (SAS)
Team Implementation
Checklist (TIC)
Secondary &TertiarySystems
Benchmarks of Advanced Tiers (BAT)
PBIS Self-Assessment Survey (SAS)
Monitoring Advanced Tiers
Tool (MATT)
Outcome Measures: School Safety Survey
![Page 15: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/15.jpg)
Current Tools in PBIS Assessment
Research Tool
Annual Assessment
Tool
Progress Monitoring
Tool
UniversalSystem
School-wide Evaluation Tool
(SET)
Benchmarks of Quality (BoQ)
Early Childhood BoQ (ECBoQ)
PBIS Self-Assessment Survey (SAS)
Team Implementation
Checklist (TIC)
Secondary &TertiarySystems
Individual Student School-wide Evaluation
Tool(ISSET)
Benchmarks of Advanced Tiers (BAT)
PBIS Self-Assessment Survey (SAS)
Monitoring Advanced Tiers
Tool (MATT)
Outcome Measures: School Safety Survey
![Page 16: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/16.jpg)
www.pbisapps.org
A Tour of
at
![Page 17: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/17.jpg)
Evaluation Action PlanningDecisions:
Evaluation ToolsWhich tools answer your district/schools’ evaluation questions?
Evaluation TimelineWhen/how often will you administer the tools?
Evaluation ReportingHow will you report outcomes to schools and to the District?How often will you report outcomes to schools and to the District?
![Page 18: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/18.jpg)
Evaluation Action PlanningDecisions about Evaluation Tools
Does your District/School need a research tool?(SET/ISSET)
Which annual assessment tool(s) will you use?BoQ, SAS, SSS, BAT, ECBoQ
Which progressing monitoring tool(s) will you use?TIC, MATT
Does your District/School need to evaluate Tier I or Tier II & III implementation or all three?
![Page 19: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/19.jpg)
Evaluation Action Planning
It is about…
CONTEXTYour settings => Your needs
![Page 20: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/20.jpg)
Which Surveys Meet Your
District’s Needs?
![Page 21: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/21.jpg)
Example District A
Evaluation Questions What is our baseline, and what has been our progress through the year?
Are we improving on our implementation of PBIS?
How do our individual staff members rate our implementation?
Are our self-assessments consistent with data from outside evaluators?
Newly implementing, grant needing specific research data as well as fidelity of implementation data
![Page 22: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/22.jpg)
District A: Evaluation PlanMeasure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
SET X X X X
TIC X X X X X X X X X
Safety Survey X X X X
SAS X X X X
![Page 23: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/23.jpg)
Example District B
Evaluation Questions In what specific PBIS areas are we strong, and in which areas do we need more training?
Are we improving on our implementation of PBIS?
How do our staff members actually rate our PBIS implementation?
How is our implementation of Tiers II and III?
Implementing PBIS for three years, have been using the TIC/SET, and have been consistently over 80%.
![Page 24: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/24.jpg)
District B: Evaluation Plan
Measure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
BoQ X X X
SAS X X X
Safety Survey X X X
ATT X X X X X X X X X
![Page 25: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/25.jpg)
Example District C
Evaluation Questions Which schools are not meeting the SET benchmark?
In what specific PBIS areas are we strong, and in which areas do we need more training?
How can we tailor our training and evaluation to meet the needs of various schools?
Are we improving on our implementation of PBIS?
How do our individual staff members rate our implementation?
Implementing for five years, some schools are doing very well, but others are not making progress.
![Page 26: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/26.jpg)
District C: Evaluation Plan: Cohort 1
Measure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
SASSurvey X
SET X X X
Safety Survey X X X
TIC X X X X X X X X X
Schools still not performing well
![Page 27: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/27.jpg)
District C: Evaluation Plan: Cohort 2
Measure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
SASSurvey X
BoQ X X X
Safety Survey X X X
MATT X X X X X X X X X
Schools performing well
![Page 28: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/28.jpg)
Example District D
Evaluation Questions In what specific PBIS areas are we strong, and in which areas do we need more training?In what academic areas are we strong, and in which areas do we need more improvement?How can we tailor our training/evaluation to meet the needs of various schools?
How can we schedule our evaluations so that we are not overwhelming our schools?
The district wants to bring RtI Behavior and Academics together and create one evaluation plan.
![Page 29: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/29.jpg)
Assessment Instrument
Fall (August/Sept/Oct)
Winter (Dec/Jan)
Spring(April/May) Purpose
Team Implementation Checklist (TIC)
X X X Fidelity of Implementation
Self-Assessment Survey
X Fidelity of implementation
School Safety Survey
X Outcomes related to
Protective and Risk Factors
SET X Fidelity of Implementation
AIMsWeb X X XScreening
Progress Monitor
SAT10 X Summative Evaluation
DIBELS X X X Screening
Progress monitoring
Nieves Flores, 2012
![Page 30: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/30.jpg)
Task: Developing Your PlanQuestions to Consider:
Which surveys are your schools/districts currently using?Do they answer your evaluation questions:
Are we implementing SWPBIS with fidelity?Is there more information that we need?
Pbisapps.org > PBIS Assessment > Find Out More about Surveys
Do your surveys assess the Tiers you are implementing?Do you need a research tool?
![Page 31: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/31.jpg)
Task: Developing Your Plan
Are you “topping out” of the surveys you are currently taking?
Do you need to customize your evaluation plan for different schools, or will one evaluation plan fit for all schools?
Do you want to assess all tiers of PBIS implementation?
Is your Evaluation Plan manageable for schools?
![Page 32: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/32.jpg)
District Evaluation Planning Template
Measure Year One Year Two Year Three
Fall Winter Spring Fall Winter Spring Fall Winter Spring
![Page 33: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/33.jpg)
SWPBIS School or District Evaluation Reports
![Page 34: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/34.jpg)
Basic Contents of SWPBIS Evaluation Reports
From the PBIS Evaluation BlueprintContext
Input
Impact on SWPBIS Fidelity
Impact on Student Outcomes
Implications for the Future
![Page 35: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/35.jpg)
Context
Define SWPBIS
Define goals of the specific project
Define stakeholders and evaluation questions
![Page 36: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/36.jpg)
Input
Who received what support and from whom?Leadership teamLocal Capacity Building
Training, Coaching, Behavioral Expertise, and Evaluation
School teams
![Page 37: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/37.jpg)
Impact on SWPBIS Fidelity
Leadership TeamSWPBIS District Leadership Team Self-Assessment
Capacity DevelopmentNumber of trainers/coaches available to support school teams/districtsBehavioral expertise available to support Tier II and Tier III implementationEvaluation capacity
School TeamsTier I Implementation (TIC, BoQ, SET, SAS)
Collectively and/or by training cohortTier II / Tier III Implementation (MATT, BAT, ISSET)
Collectively and/or by training cohort
![Page 38: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/38.jpg)
Impact on Student Outcomes
Behavioral OutcomesOffice Discipline ReferralsSuspension/ExpulsionAttendanceDrop Out/GraduationDisproportionality in ODR and suspension/expulsion
Academic outcomesCurriculum Based Measures
Oral Reading Fluency, Dibels, AimsWebStandardized Measures
State and NationalMath, reading, writing assessments
![Page 39: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/39.jpg)
Impact and Future Directions
Advances to datePlans for next three yearsAreas in need of adaptation
Narrowing of evaluation questions and concerns
![Page 40: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/40.jpg)
Sharing Evaluation Reports
State and District examples at PBIS.orgSend in your district examples, so we can all learn!
![Page 41: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/41.jpg)
Support and Resources
Portal Support SectionSearch FunctionCommunity ForumChat with us online
Account Questions:[email protected]
Technical Assistance Questions:
New phone number1-855-455-8194
![Page 42: Using PBIS Assessment to Create School and District PBIS Evaluation Plans](https://reader036.fdocuments.in/reader036/viewer/2022062521/56816674550346895dda0bc8/html5/thumbnails/42.jpg)
Questions & Answers
When you leave the webinar, please complete the survey.
THANK YOU!