Post on 22-Dec-2015
ActivitySelf Perception
Coaching teams to use data for decision making
Coaching
What is a PBIS Coach?
What do we mean by “Coach”PBIS Coaches are not “trainers”, they support teams who have basic training in PBIS. PBIS Coaches support teams to make data-based decisions toward quality improvement of student, staff and family outcomes.
What do I need to be doing as a coach?
1. Prevent team members from launching into solutions before they are ready by engaging in active problem solving
2. Get team members to ask questions, even if they don’t have all the information
3. Don’t move forward until a measureable goal is identified and a solution is designed
What do I need to know to coach well?
1. Desired Outcomes – how will we know if what we’re doing is having a positive effect on students, staff, and families?
2. Practices – what PBIS Interventions are in place? 3. Data – what data do we have and what tools do we have to collect &
summarize data? 4. Systems – what do we have in place to support teams to look at our
data and use it for quality improvement?
Data-Based Decision Making
Here’s what we know…
Decisions are more likely to be effective and efficient when they are based on data.
The quality of decision making depends most on the first step (defining the problem to be solved).
Define problems with precision and clarity
Data help us ask the right questions…they do not provide the answers. Use data to:
Identify problemsRefine problemsDefine the questions that lead to solutions
Data help place the “problem” in the context rather than in the students.
School-wide PBIS
Primary Prevention: School-wide & Classroom-wide systems for all students and all staff in all settings.
Universal, Tier I
Secondary Prevention: Systems for targeted or group-based interventions for students needing additional support beyond the Universal or Tier I system.
Targeted, Tier II
Tertiary Prevention: System for students requiring more intensive & individualized supports for academic, social, or mental health services.
Individualized,Tier III
What is DBDM?
The process of planning for student success (both academic and behavioral) through the use of ongoing progress monitoring and analysis of data
Douglas County School District (Colorado)
Why Do it?
The value of Data-Based Decision Making is: Quality Improvement Cycle of continuous Improvement
Improving what? fidelity of implementation, social climate, learning environment, student learning, attendance, grades)
How do we do it?
Right Data/Format/Time/PeopleRight QuestionsSolution Development & Action Planning
Hallway Noise Study
A brief vignette to demonstrate how SWIS data is used to support data-based decision making.
Kartub, D., Taylor-Greene, S., March, R., Horner, R.H. (2000). Reducing Hallway Noise: A Systems Approach. Journal of Positive Behavior Interventions, 2(3). 179-182
Using SWIS Data for Active Decision Making
Problem
Staff at a middle school (Grades 6-8) in a rural school district with 520 students have identified an issue with student noise in the hallways.
Teachers complain that hallway noise is significantly disruptive around lunch.
Three lunch periods (by grade)
Students required to walk past classrooms still in session to access cafeteria.
Problem Solving Process
a. Team Assesses the Extent of the Problem• Vote during faculty meeting confirmed as a priority
to address
b. Review Existing Practices• Students were taught school-wide expectations• Teaching Assistant in hall gives out detentions &
office referrals for loud noise.
c. Review Existing Data• Referrals by location• Hallway ODR per student
d. Build a hypothesisNoise is occurring because • Students have been in class all morning (low blood
sugar) and want to socialize (peer attention)• Hallway is loud at beginning and end of day
e. Define problem-solving logic• Small number of kids = address group/individually
Large number of kids = address system• Define, teach, monitor, and reward BEFORE
increasing use of punishment.
Problem Solving Process (cont.)
Office Referrals by Location
Students: 173 Referrals: 530
Office Discipline Referrals by Student
Drill Down into the Problem
Who? Large number of students across grade levels
What? Disruptive (loud, rowdy) behavior
When? After morning class
Where? Hallway
Why? (a) To gain peer attention, and (b) behavior is similar to what they do before and after school.
*Teaching Assistant’s consequences are not proving effective
Solution (keep it simple)Make lunch hallways look different from hallways in morning and afternoon.
Change lighting
Review school-wide expectations for hallwayFive-minute review of “quiet”
Build reward for valued behaviorThree days of quiet in hallway results in an extra five minutes of social time (at lunch or at end of school)
Remind students to be quiet just before they are released for lunch
Measure and ImplementUse a decibel meter to measure noise levelPublic posting of results
Build Action Plan
Actions Who When1. Build “Quiet” Curriculum Ben and
MaryNov 12
2. Buy Decibel Meter Rob Nov 10
3. Teach Hallway Expectations/ Reminders
Team Dec 2-3
4. Collect and Post Data Reiko Ongoing
5. Schedule Lunch Times Ms. Green Ongoing
6. Graph and Report Data Reiko Ongoing
7. Report to Staff Team Staff Meeting
Sixth Grade Lunch Noise
Seventh Grade Lunch Noise
Eighth Grade Lunch Noise
Improving Decision Making
Problem Solution
From
To
Problem
Problem
Solving
SolutionAction Plannin
g
How?
Right Data/Format/Time/PeopleWhat is the right data? What would be the right format? What is the right time (schedule) to bring the data? Who are the right people to be discussing and using this data to address issues?
How?
Right Questions
The statement of a problem is important for team-based problem solving.
Everyone must be working on the same problem with the same assumptions.
Problems often are framed in a “Primary” form. That form creates concern, but is not useful for problem-solving.
Frame primary problems based on initial review of data
Use more detailed review of data to build “Solvable Problem Statements.”
What are the data we need for a decision?
Precise problem statements include information about the following questions:
What is the problem behavior?How often is the problem happening?Where is the problem happening?Who is engaged in the behavior?When is the problem most likely to occur?Why is the problem sustaining?
Primary versus Precision Statements
Primary StatementsToo many referrals
September has more suspensions than last year
Gang behavior is increasing
The cafeteria is out of control
Student disrespect is out of control
Precision StatementsThere are more ODRs for aggression on the playground than last year. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.
Primary versus Precision Statements
Precision Statement:
What? More ODRs for disruption.Where? In the hallway.Who? A large number of students across grade levels.When? After morning class.Why? To get access to peer attention.
There are more ODRs for disruption (loud, rowdy behavior) in the hallway. These are most likely to occur during after morning class, with a large number of students across grade levels, and the disruption is related to getting peer attention.
How? Solution Development & Action Planning
Prevention— how can we avoid the problem context?Who? When? Where?Schedule change, curriculum change, etc.
Teaching— how can we define, teach, and monitor what we want?Teach appropriate behaviorUse problem behavior as negative example
Recognition— how can we build in systematic rewards for positive behavior?
Extinction— how can we prevent problem behavior from being rewarded?
Consequences— what are efficient, consistent consequences for problem behavior?
How will we collect and use data to evaluate:Implementation fidelity?Impact on student outcomes?
Solution DevelopmentSolution
Component Action Step(s)
PreventionHow can we avoid the problem context?
Example: Schedule Lunch Times, Change Lighting
TeachingHow can we define, teach, and monitor what we want?
Example: Build “Quiet” Curriculum, Buy Decibel Meter, Teach Hallway Expectations/ Reminders
RecognitionHow can we build in systematic rewards for positive behavior?
Example: Three days of quiet in hallway results in an extra five minutes of social time (at lunch or at end of school)
ExtinctionHow can we prevent problem behavior from being rewarded?
Example: Public posting of results
Corrective Consequence
Consequences—what are efficient, consistent consequences for problem behavior?Example: Continue current system (Minor/Major ODR)
Data collection
Implementation fidelity?Example: Walkthrough report, observation, self-assessment
Impact on student outcomes?Example: SWIS ODR Data
Solution Components
What are the action steps? Who is Responsible? By When? How will fidelity be
measured? Notes/Updates
Prevention
Schedule Lunch Times, Change Lighting
Custodial staff to adjust lightingPrincipalto adjust schedule
Ongoing
Nov 10
New lunch scheduleWalkthrough report
Teaching
Build “Quiet” Curriculum, Buy Decibel Meter, Teach Hallway Expectations/ Reminders
Ben & Mary Nov 12 Permanent productStaff Self Assessment
Recognition
Continue current acknowledgment system and add an extra five minutes of social time (at lunch or at end of school) after three days of quiet in hallway
Reiko & Principal Nov 9 (announcements & chart up)
Announcement madeChart made
Extinction Public posting of results of decibel readings
Reiko Ongoing Posted chart
Corrective Consequence
Continue current system (Minor/Major ODR)
Hallway and Cafeteria supervisors
Ongoing SWIS ODR Reports
What data will we look at?
Who is responsible for gathering the data?
When/How often will data be gathered?
Where will data be shared? Who will see the data?
Data Collection
ODR recordSupervisor weekly report
SWIS Data Entry person and Principal share report with supervisors
Weekly In supervisor meeting and posted in the faculty lounge on PBIS bulletin board
All staff
Precise Problem Statement: Many students across grade levels are engaging in disruptive (loud, rowdy) behavior in the hallway after morning class, and the behavior is maintained by peer attention.
Goal: Reduce hallway ODRs by 50% per month (currently 24 per month average)
Most frequently misunderstood and overlooked component!Example: public posting of results will reduce likelihood of
payoff that previously reinforced this behavior
Value of this work/process
Quality Improvement is Continuous Improvement
The value of Data-Based Decision Making is: Quality Improvement Cycle of continuous Improvement
Improving what? fidelity of implementation, social climate, learning environment, student learning, attendance, grades)
Example: Hallway Study, recoup teaching time, improving social climate (for staff and students)
Show Videos
ActivityReflection Activity - Shapes
Lunch BreakGo eat!
ActivityData Treasure Hunt
Different Data for Different Decisions
Decision-Making for Quality Improvement
Outcome DataDiscipline Data for Short-Term Improvement
Progress Monitoring (formative)Universal Screening
Discipline Data for Long-Term ImprovementAnnual summarization and review of Strengths, Weaknesses, and Planning (summative)
Decision-Making for Quality Improvement
Fidelity DataFidelity Data for Short-Term Improvement
Progress MonitoringUniversal Screening
Fidelity Data for Long-Term ImprovementAnnual Assessments
Decision-Making for Quality Improvement
Outcome Data• Discipline (e.g., referrals)• Academic• Attendance• Climate/Culture• School Safety
Fidelity Data• Team/Self Assessments• Walk-through reports• PBIS Assessment (e.g.,
SET, Self Assessment, BoQ, TIC)
Decision-Making for Quality Improvement
Outcome Data• Discipline Data for Continuous
System Improvement and Progress Monitoring (formative)
• Discipline Data for Universal Screening of Student Needs
• Discipline Data for Evaluation (summative) of Strengths, Weaknesses, and Planning
Fidelity Data• Fidelity Data for Continuous
Improvement• Fidelity Data for Evaluation
Connecting Outcomes & Fidelity
Lucky Sustaining
Positive outcomes, low understanding of how we achieved them
Replication of success unlikely
Positive outcomes, high understanding of how we achieved them
Replication of success likely
Losing Ground Learning
Undesired outcomes, low understanding of how we achieved them
Replication of failure likely
Undesired outcomes, high understanding of how we achieved them
Replication of mistakes unlikely
Out
com
es
Fidelity
Sustaining
Positive outcomes, high understanding of how we achieved them
Replication of success likely
Discipline Data for a cycle of Continuous quality improvement
Short-Term vs. Long-Term Quality Improvement
Quality improvement requires two levels of analysis/use:
Short-term (sometimes called progress monitoring) is using data regularlyLong-term (sometimes called evaluation or annual assessment) is summarizing data for a big picture
Harbor Haven Middle School
565 studentsGrades 6, 7, 8
Harbor Haven Middle School
Is there a problem?
If so, what is it?
Problem
Dashboard
Harbor Haven Middle School
Median
Harbor Haven Middle School
HallwayCaféBusGym
10:0011:00-12:30
DefianceHarassmentTheft
School-wide Data
Harbor Haven Middle School
School-wide Data
Majority of referrals come from 6th and 7th grades.
11 students have more than 2 referrals.
What Do I Know?
What? Defiance and harassment.
Where? Hallways and cafeteria.
Who? A large number of students. Majority of referrals are from 6th grade.
When? 10:00 and 11:00-12:30.
What Do I Know?
I know pieces of information.
But, I do not know if any of this information is connected.
I need to drill down to look for connections.
Data Drill Down
Use the information from the SWIS Dashboard to drill
down and analyze data.
Change the graph type to change the
analysis.
Data Drill Drown
Cafeteria and harassment are connected.
Change the graph type to change the
analysis.
Data Drill Down
Cafeteria, harassment, and the time range 11:30-12:00
are connected.
Change the graph type to change the
analysis.
Many students are engaging in harassment in the cafeteria during the 11:30-12:00 time range (6th grade lunch), and the behavior is maintained by peer attention.
Data Drill Down for Connections
Precise Problem Statement & Solution Development
Cafeteria Harassment 11:30-12:00
Obtain Peer Attention
Solution DevelopmentTarget Area(s): Harassment in the cafeteriaGoal: Reduce referrals for harassment in the hall & cafeteria by 50%
Solution Component Action Step(s)
Prevention Maintain the current lunch schedule, but shift 6th grade classes to balance numbers.
Teaching Teach behavioral expectations in cafeteria
Recognition Establish “Friday Five”—an extra 5 minutes of lunch on Friday for five good days.
Extinction Encourage all students to work for “Friday Five”, making a reward for problem behavior less likely.
Corrective Consequence Active supervision and continued early consequence (ODR)
Data collection Maintain ODR record and supervisor weekly report
Solution Components
What are the action steps? Who is Responsible? By When? How will fidelity be
measured? Notes/Updates
Prevention
Maintain current lunch schedule, but shift 6th grade classes to balance numbers.
Principal to adjust schedule and send to staff
January 10 Email to staff
Teaching
Teach behavioral expectations in the cafeteria
Teachers will take class to cafeteria; cafeteria staff will teach expectations
Rotating schedule on January 15
Sign-up sheet for scheduled times
Recognition
Establish “Friday Five”—extra 5 min. of lunch on Friday for 5 good days
School counselor and Principal will create a chart and staff extra recess
Principal to give announcement on intercom on Monday
Announcement madeChart made
Extinction
Encourage all students to work for “Friday Five”—make reward for problem behavior less likely
All staff Ongoing
Corrective Consequence
Active supervision and continued early consequence (minor/major ODR)
Hallway and Cafeteria supervisors
Ongoing
What data will we look at?
Who is responsible for gathering the
data?
When/How often will data be gathered?
Where will data be shared?
Who will see the data?
Data Collection
ODR recordSupervisor weekly report
SWIS Data Entry person and Principal share report with supervisors
Weekly In supervisor meeting and posted in the faculty lounge on PBIS bulletin board
All staff
Precise Problem Statement: Many students are engaging in harassment in the cafeteria during the 11:30-12:00 time range (6th grade lunch), and the behavior is maintained by peer attention.
Goal: Reduce cafeteria ODRs by 50% per month (currently 24 per month average)
Discipline Data for Universal Screening
Using SWIS Data for Decision Making
Universal Screening ToolProportion of students with
0-1 Office Discipline Referrals (ODRs)2-5 ODRs6+ ODRs
Progress Monitoring Tool
Compare data across timePrevent previous problem patterns
Using the Referrals by Student as a Universal Screening Tool
Research Study on Early Intervention
Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun0
2
4
6
8
10
12
0-12-56+
Cum
ulati
ve M
ean
OD
Rs
Cumulative Mean ODRs Per Month for 325+ Elementary Schools 08-09
Jennifer Frank, Kent McIntosh, Seth May
Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun0
2
4
6
8
10
12
0-12-56+
Research Study on Early InterventionCu
mul
ative
Mea
n O
DRs
Cumulative Mean ODRs Per Month for 325+ Elementary Schools 08-09
Jennifer Frank, Kent McIntosh, Seth May
The “October Catch”
Discipline Data for Long-Term Continuous Improvement
SWIS Year-End Report
Fidelity Data for Short-Term Continuous Quality Improvement
PBIS Assessment Reports
Fidelity Data for Universal Screening
How would we ensure that the Universal Screening occurred?
Who is responsible to gather the data for the teamWhat is the schedule for reviewing this data (for purposes of universal screening)?
Fidelity Data for Long-Term Continuous Improvement
Two-fold
TBD
Activity