Post on 01-Apr-2015
IT KEEPS GETTING BETTER!Using School-wide Data for Continuous Quality Improvement
Kelsey R. Morris, EdD—University of OregonNadia K. Sampson, MA—University of Oregon
Session C13
Session Overview• Value of data-based decision making• Nexus of data-based decision making and SWPBIS• Cycle of continuous quality improvement
• Student Outcomes• Implementation Fidelity
Goal
• Craft precise problem statements• Craft solution-based action plans• Engage in continuous quality improvement
Maximizing Your Session Participation
• Where are you in your implementation of the concepts presented?• Exploration & Adoption
• Installation
• Initial Implementation
• Full Implementation
• What do you hope to learn?
• What new learning do you take away from the session?
• What will you do with your new learning?
Data Collection & Analysis• Data are “the numerical results of measuring some
quantifiable aspect of behavior” (Mayer, Sulzer-Azaroff, & Wallace,
2012, p. 130).
• Data collection involves:• Observation of a behavior
• Notation of the behavior characteristics and context
• Data analysis involves:• Converting numerical results into graphs
• Using graphed results for instructional decision making
Value and Utility of Data
• Repeatedly giving people the right information, at the right time, in the right format is the single most effective way to improve decision making and achieve valued outcomes (Gilbert, 1978).
Data
PracticesSystems
Valued Outcomes
Performance Gap & Cause Analysis
Cur
rent
Rea
lity Valued
Outcom
es
Environment/System
1—Information • Clear expectations• Timely, specific
feedback
2—Resources • Materials, tools• Time• Processes
3—Incentives • Financial & non-
financial encouragement
Individual Persons
6—Knowledge • Requisite knowledge
and skill base
5—Capacity • Ability to learn
and do
4—Motives • Desire to work
and excel
Data-based Decision Making• Effective teams use data to document progress and outcomes, guide
decisions, and inform stakeholders (Boudett, City, & Murnane, 2006; Burke, 2010; Deno, 2005; Hill 2010; Newton, Algozzine, Algozzine, Horner, & Todd, 2011; Newton, Horner, Algozzine, Todd, & Algozzine, 2009; Pidgeon & Gregory, 2004; Renfro & Grieshaber, 2009)
• A critical predictor of sustained implementation of SWPBIS (Coffey & Horner, 2012; McIntosh et al., 2013)
• Fidelity and student outcome data are essential (Fixsen, Blase, Metz, & Van Dyke, 2013)
• Continues to be a struggle for schools (Dunn, Airola, Lo, & Garrison, 2013; Schildkamp, Ehren, & Lai, 2012; Telzrow, McNamara, & Hollinger, 2000)
• Advances in computer technology could provide efficient means for data management (Wayman, 2005)
Components of SWPBIS
Defined Behavior Expectations
Teaching of Behavior Expectations
Acknowledgment Systems
Consequence Systems
Evaluation
Evaluation of Effectiveness• Essential Question
• Is the student successful with this level of support?
• Intensity is a two-way street.• Improved student outcomes are
the result of continually monitoring and modifying (as needed) instruction, interventions, and supports.
Goal:Increase prosocial behavior and enhance quality of life
Continuous Quality Improvement
Plan
Implement
Evaluate
Adult Behaviors Cause Student Change
Outcomes Fidelity
School-wide Behavior Data• Critical Questions
• How often are problem behaviors occurring?
• When are problem behaviors frequently occurring?
• Where are problem behaviors frequently occurring?
• What problem behaviors are frequently occurring?
• Who is frequently engaging in problem behaviors?
School-wide Behavior Data
September = rate of 3.50 average referrals/day/month October = rate of 4.53 average referrals/day/month
Spikes at 9:45 AM and 1:00 PM – 2:30 PM
Top 3 non-classroom locations = bathroom, playground, & hallway
School-wide Behavior Data
Tuesday and Wednesday are the school days with the highest frequency.
Top 3 Problem Behaviors = inappropriate language, defiance, disruption
5th & 8th grades have the highest frequency
25 students have more than 1 referral
Problem Solving with Precision• The statement of a problem is important for team-based
problem solving.• Everyone must be working on the same problem with the same
assumptions.
• Problems are often framed in the “primary” form. • Raises awareness
• Not useful for problem solving
• Precise problem statements result from a detailed data review and are solvable.
Problem Solving with Precision
Primary Statements
• There are too many referrals
• Gang behavior is increasing
• The cafeteria is out of control
• Student disrespect is a big problem
Precision Statement
• There are more ODRs for aggression on the playground than last month. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.
Problem Solving with Precision• There are more ODRs for aggression on the playground
than last month. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.
What? Where? When? Who? Why?
Aggression Playground 1st RecessLarge number
of students
To get new playground equipment
Data Analysis for Precision
Location
Problem Behavior
Time of Day
Persons Involved
Motivation
Precise Problem
Statement
Continuous Quality Improvement
Plan
Implement
Evaluate
Solution Development & Action PlanningEssential Elements
Explanation
1. PreventionHow can we avoid the problem context?• Who? What? When? Where?
2. TeachingHow can we define, teach, and monitor what we want?• Teach appropriate behavior, use problem behavior as the
non-example
3. RecognitionHow can we build in systematic acknowledgment/rewards for positive behavior?
4. ExtinctionHow can we prevent the problem behavior from continuing to pay off? (tied to motivation/function of behavior)
5. ConsequencesWhat are efficient, consistent consequences for problem behavior?
6. EvaluationHow will we collect and use data to evaluate our fidelity and outcomes?
Solution Development & Action Planning
Remember to include the
precise problem statement as well as a statement of
the goal.
Continuous Quality Improvement
Plan
Implement
Evaluate
Jan-14 May-14 Sep-140
10
20
30
40
50
60
70
80
90
100
39
54
32
Hamlin Middle SchoolTFI Subscales
Tier ITier IITier III
Evaluation—SWIS was installed for
data collection and analysis
Hamlin Action Planning• PBIS Team used TFI results to identify items of importance
1.3 & 1.4 Behavioral Expectations & Teaching Expectations
1.6 Discipline Policies (e.g., flowchart)
1.9 Feedback & Acknowledgement (e.g., menu of reinforcers)
• PBIS Team worked on items during team meetings• Solicited feedback from whole faculty
• Revised as needed
• Created lesson plans to teach all staff the aspects of school-wide PBIS during in-service week
Jan-14 May-14 Sep-140
10
20
30
40
50
60
70
80
90
100
39
71
5446
32
65
Hamlin Middle School TFI Subscales
Tier ITier IITier III
Jan-14 May-14 Sep-140
10
20
30
40
50
60
70
80
90
100
39
71
90
5450
32
44
Hamlin Middle SchoolTFI Subscales
Tier ITier IITier III
Enablers of Sustainability
Admin
istra
tor S
upport
Staff
Buy-in
Fidel
ity
Data
Team
ing
Resourc
es
Stake
holder
Invo
lvem
ent
Train
ing
SWPBS P
hiloso
phy
Motiv
atio
n
Distri
ct S
upport
0
10
20
30
40
50
60
70
Nu
mb
er o
f R
esp
on
ses
McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014).
Barriers to Sustainability
0102030405060708090
Nu
mb
er o
f R
esp
on
ses
McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014).
What is most related to high sustainability?
McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014).
IT KEEPS GETTING BETTER!Using School-wide Data for Continuous Quality Improvement
Kelsey R. Morris, EdD—University of OregonNadia K. Sampson, MA—University of Oregon
Session C13