Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session...
Transcript of Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session...
![Page 1: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/1.jpg)
National Center on Response to Intervention
National Center on Response to Intervention
Things to Consider When Developing an RTI Evaluation Plan
![Page 2: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/2.jpg)
National Center on Response to Intervention
Session Agenda
Rationale for RTI evaluation Defining RTI and the essential components of an RTI
model Measuring fidelity of implementation Things to consider when evaluating RTI Ways to assess success indicators and examples of
data
2
![Page 3: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/3.jpg)
National Center on Response to Intervention
Session Objectives Recognize why it is important to evaluate the RTI model and
what the evaluation might tell you.
Develop an understanding of the essential components of an RTI model and whether they are being implemented fully.
Increase understanding of key considerations for developing an RTI evaluation plan.
Increase awareness of available evaluation tools and resources for measuring fidelity and implementation.
Have a team discussion on success indicators and considerations for developing an evaluation plan.
3
![Page 4: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/4.jpg)
National Center on Response to Intervention
RATIONALE FOR AN RTI EVALUATION
4
![Page 5: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/5.jpg)
National Center on Response to Intervention
Why Evaluate RTI? Justify RTI Implementation to stakeholders
• Resources, training efforts, funding
Understand whether the essential components of RTI are being implemented with fidelity and refine the model as needed
Assess the impact of the implementation of the RTI model or its components on student outcomes
Empirically support statements about the effectiveness of RTI for stakeholders
5
![Page 6: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/6.jpg)
National Center on Response to Intervention
Why Evaluate RTI?
Identify areas for improvement or targeted technical assistance
Assess implementation integrity and fidelity of • The RTI model • RTI model components • Implementation process
6
![Page 7: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/7.jpg)
National Center on Response to Intervention
Key Evaluation Questions
1. How do I know whether my RTI model and the components of RTI are being implemented with fidelity?
2. How do I know if my RTI model is working and to what extent is it working?
3. If my RTI model is working, what will change for students, for schools, for districts?
7
![Page 8: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/8.jpg)
National Center on Response to Intervention
WHAT IS RTI?
8
![Page 9: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/9.jpg)
National Center on Response to Intervention 9
Response to Intervention (RTI) integrates assessment and intervention within a school-wide, multi-level prevention system to maximize student achievement and reduce behavior problems.
Defining RTI
(National Center on Response to Intervention)
![Page 10: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/10.jpg)
National Center on Response to Intervention 10
With RTI, schools identify students at risk for poor learning outcomes, monitor student progress, provide evidence-based interventions, and adjust the intensity and nature of those interventions based on a student’s responsiveness; and
RTI may be used as part of the determination process for identifying students with specific learning disabilities or other disabilities.
Defining RTI
(National Center on Response to Intervention)
![Page 11: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/11.jpg)
National Center on Response to Intervention
RTI as a Preventive Framework
RTI is a multi-level instructional framework aimed at improving outcomes for all students.
RTI is preventive, and provides immediate support to students who are at risk for poor learning outcomes.
RTI may be a component of a comprehensive evaluation for students with learning disabilities.
11
![Page 12: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/12.jpg)
National Center on Response to Intervention
Essential Components of RTI Screening Progress Monitoring School-Wide, Multi-Level Prevention System
• Primary level • Secondary level • Tertiary level
Data-based decision making for • Instruction • Evaluating effectiveness • Movement within the multi-level system • Disability identification (in accordance with state law)
12
![Page 13: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/13.jpg)
National Center on Response to Intervention
Essential Components of RTI
13
![Page 14: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/14.jpg)
National Center on Response to Intervention
Screening
PURPOSE: Identify students who are at risk of poor learning outcomes
FOCUS: All students TOOLS: Brief assessments that are valid, are reliable,
and demonstrate diagnostic accuracy for predicting learning or behavioral problems
TIMEFRAME: Administered more than one time per year (e.g., fall, winter, spring)
14
![Page 15: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/15.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE How are you implementing screening in your RTI
model?
15
![Page 16: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/16.jpg)
National Center on Response to Intervention
Progress Monitoring PURPOSE: Monitor students’ response to primary,
secondary, or tertiary instruction to estimate rates of improvement, identify students who are not demonstrating adequate progress, and compare the efficacy of different forms of instruction
FOCUS: Students identified through screening as at risk for poor learning outcomes
TOOLS: Brief assessments that are valid, reliable, and evidence based
TIMEFRAME: Students are assessed at regular intervals (e.g., weekly, biweekly, or monthly)
16
![Page 17: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/17.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE How are you implementing progress monitoring in
your RTI model?
17
![Page 18: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/18.jpg)
National Center on Response to Intervention 18
Multi-Level Prevention System
~15%
~5%
Tertiary level: specialized individualized
instruction for students with intensive needs
Secondary level: supplemental group
instruction for students with at-risk response to
primary level
Primary level: School-wide/classroom
instruction for all students, including
differentiated instruction
~80% of Students
![Page 19: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/19.jpg)
National Center on Response to Intervention
Changing the Intensity and Nature of Instruction
Intervention Duration Frequency Interventionist Group size
19
![Page 20: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/20.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE What does your multi-level prevention system look
like? Are you fully implementing all three levels?
20
![Page 21: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/21.jpg)
National Center on Response to Intervention
Data-Based Decision Making: The Basics Analyze data at all levels of RTI implementation (e.g.,
state, district, school, grade level) and all levels of prevention (i.e., primary, secondary, or tertiary).
Establish routines and procedures for making decisions. Set explicit decision rules for assessing student progress
(e.g., state and district benchmarks, level, and/or rate). Use data to compare and contrast the adequacy of the
core curriculum and the effectiveness of different instructional and behavioral strategies.
21
![Page 22: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/22.jpg)
National Center on Response to Intervention
Data-Based Decision Making: Types of Decisions
Instruction Evaluate effectiveness Movement within the multi-level prevention system Disability identification (in accordance with state
law)
22
![Page 23: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/23.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE How are you using data to make decisions within
your RTI model? • Are there data teams? • Do you have explicit decision rules? • Are there routines and procedures in place for collecting
and reviewing data?
23
![Page 24: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/24.jpg)
National Center on Response to Intervention
Essential Components of RTI
24
![Page 25: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/25.jpg)
National Center on Response to Intervention
ASSESSING FIDELITY OF IMPLEMENTATION
25
![Page 26: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/26.jpg)
National Center on Response to Intervention
What Is Fidelity of Implementation?
26
![Page 27: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/27.jpg)
National Center on Response to Intervention
Indicators of Successful Implementation Fidelity of implementation of essential components Effectiveness of the scale-up process Impact of RTI on service delivery (e.g., staff attitudes,
delivery of instruction and supplemental supports, or design of instructional supports)
27
![Page 28: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/28.jpg)
National Center on Response to Intervention
Monitor Fidelity Ways to Measure Fidelity Self-report data Observation Logs, lesson plans, and student work
28
![Page 29: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/29.jpg)
National Center on Response to Intervention
Self-Report Data May provide an indicator of teacher knowledge, context
of implementation Types
• Questionnaires
• Surveys • Interviews
Considerations • Often unreliable when used alone
• Efficient
29
![Page 30: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/30.jpg)
National Center on Response to Intervention
Observation Types
• Spot checks
• Peer or administrator observations
• Peer coaching
• Item-by-item checklists of lesson components/rubrics
Considerations • Develop checklists of critical implementation components
• Record and listen to sessions at random
• Least efficient but most reliable
30
![Page 31: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/31.jpg)
National Center on Response to Intervention
Logs, Lesson Plans, Student Work Allows evaluation of what was done
• Content covered
• Student progress
Considerations • Moderately efficient
• Moderately reliable
• Less information about delivery, dosage, adherence to scripts or lesson components (if applicable)
31
![Page 32: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/32.jpg)
National Center on Response to Intervention
Measuring Fidelity Handout Work with your team to A. Choose an essential component to focus on during
this activity. B. Fill out the handout and discuss the last two
columns relating to the essential component your team selected: • What are you currently doing to measure fidelity?
• What can you implement to measure fidelity?
32
![Page 33: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/33.jpg)
National Center on Response to Intervention
Fidelity-of-implementation forms Self-assessment rubrics Interview protocols Product review forms Free RTI surveys and checklists Direct observation instructions or rubrics
Examples of Measures and Tools for Evaluating Implementation of RTI Components and the RTI Process
33
![Page 34: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/34.jpg)
National Center on Response to Intervention
NCRTI Integrity Rubric and Worksheet Organized according to the essential components of
RTI as identified by the NCRTI: • Screening, progress monitoring, multi-level prevention
system, data-based decision making
Includes overarching factors: • Staff qualification, cultural and linguistic competence,
leadership, communication and involvement of parents, prevention focus
34
![Page 35: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/35.jpg)
National Center on Response to Intervention
NCRTI Integrity Rubric Intended for use by individuals responsible for
monitoring school-level fidelity of RTI implementation or as a measure of self-assessment.
Provides descriptions of three levels of potential ratings (1, 3, or 5) across each factor.
35
![Page 36: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/36.jpg)
National Center on Response to Intervention
![Page 37: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/37.jpg)
National Center on Response to Intervention 37
Insufficient evidence that the screening tools are reliable; or that correlations between the instruments and valued outcomes are strong; or that predictions of risk status are accurate.
![Page 38: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/38.jpg)
National Center on Response to Intervention
NCRTI Integrity Framework Worksheet Intended for use by RTI coordinators or evaluators
with extensive RTI experience Provides space to develop a narrative rationale for
each rating Data collected through interviews or site visits
38
![Page 39: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/39.jpg)
National Center on Response to Intervention 39
![Page 40: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/40.jpg)
National Center on Response to Intervention 40
1. Screening Tools
What tools do you use for universal screening? When your school selected the screening tool(s), how much attention was paid to the evidence from the vendor regarding the validity, reliability, and accuracy of the tool? Does your school have documentation from the vendor that these tools have been shown to be valid, reliable and accurate (including with sub-groups)? Do you have reasons to believe that the screening tools(s) that you use may have issues with validity, reliability, or accuracy (including with sub-groups)? If so, please explain.
![Page 41: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/41.jpg)
National Center on Response to Intervention 41
RTI State Database
(http://state.rti4success.org/)
![Page 42: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/42.jpg)
National Center on Response to Intervention 42
Example—Colorado’s RTI Implementation Rubric
http://www.cde.state.co.us/rti/
![Page 43: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/43.jpg)
National Center on Response to Intervention 43
Example—Connecticut’s Evaluation of Districts’ RTI Procedures: Self-Assessment
http://www.sde.ct.gov/sde/lib/sde/word_docs/cali/srbi_self_assessment_csde_october_2008.doc
![Page 44: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/44.jpg)
National Center on Response to Intervention
Example—Connecticut’s Evaluation of Districts’ RTI Procedures: Interview
Questions May Include •What is the purpose of your team? • What are the activities of the team? • What is the expectation of the team? • How often do you meet? How long is the meeting? • How is the agenda for the meeting determined? • How are decisions determined? • How are strategies for student improvement determined? How are they evaluated? • How does the data team influence classroom/school-wide practice (e.g., coaching teachers, support personnel)? • Give an example of how the data team supports improvement in student outcomes (e.g., academic or behavioral)?
http://www.sde.ct.gov/sde/lib/sde/word_docs/cali/srbi_observation_interview_protocol_october_2008.doc. This document was developed in 2008.
44
![Page 45: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/45.jpg)
National Center on Response to Intervention 45
Example—Connecticut’s Evaluation of Districts’ RTI Procedures: Permanent Product Review
http://www.sde.ct.gov/sde/lib/sde/word_docs/curriculum/cali/assessment_inventory.doc
![Page 46: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/46.jpg)
National Center on Response to Intervention
Example—Connecticut’s Evaluation of Districts’ RTI Procedures: Building/Classroom Tour • The observer would
write down evidence of practice, notes, or comments for each these key areas.
• The observer also may record the readiness level (beginning, developing, proficient, or exemplary) observed for each of these key areas.
http://www.sde.ct.gov/sde/lib/sde/word_docs/cali/srbi_observation_interview_protocol_october_2008.doc
46
![Page 47: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/47.jpg)
National Center on Response to Intervention 47
Example—Connecticut’s Evaluation of Districts’ RTI Procedures: Data Team Observation
In this example, observers would check the box if they saw evidence of the key area during the observation period.
http://www.sde.ct.gov/sde/lib/sde/word_docs/cali/srbi_observation_interview_protocol_october_2008.doc
![Page 48: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/48.jpg)
National Center on Response to Intervention 48
Example—Delaware’s District RTI Planning Guide
http://www.doe.k12.de.us/infosuites/staff/profdev/rti_files/District%20RTI%20Planning%20Guide.doc
![Page 49: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/49.jpg)
National Center on Response to Intervention
CONSIDERATIONS FOR DEVELOPING AN EVALUATION PLAN
49
![Page 50: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/50.jpg)
National Center on Response to Intervention
Things to Consider
Data collection Staffing and expertise Funding Evaluation tools Timeframe Data analysis Sharing of results
50
![Page 51: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/51.jpg)
National Center on Response to Intervention
Examples of Types of Data Compliance with state and federal regulations Components of the RTI process (i.e., screening and
progress monitoring data) Fidelity of implementation Effectiveness of RTI (student outcome measures) Financial impact of RTI
51
![Page 52: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/52.jpg)
National Center on Response to Intervention
Data Collection: Key Questions
What data do we need to collect? Can we use existing data? If so, what sources? Do we need to collect additional data? If so, how will
we collect the data? Can we use similar data sources across all sites?
52
![Page 53: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/53.jpg)
National Center on Response to Intervention
Data Collection: Methods Consistency across schools’ data reporting systems
and data collected • Use a common language across all schools • Collect consistent data across schools • Use consistent cut scores and benchmarks
Multi-method/multi-measure assessment is preferred
53
![Page 54: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/54.jpg)
National Center on Response to Intervention
Data Collection: Methods
Evaluation Question
Success Indicators
Type of Data Needed
Submission or Collection Methods
Timeframe of Data Collection
What is it we want to know?
What is an indicator of what we want to know?
What type of data will provide information for the success indicator?
How will the data be collected and analyzed?
What is the timeframe for data collection?
54
![Page 55: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/55.jpg)
National Center on Response to Intervention
Staffing and Expertise
Who will oversee the evaluation process? Who will be responsible for collecting and analyzing
data? Do we have sufficient internal staff expertise to
design and conduct an evaluation? Do we need external expertise? What is available?
55
![Page 56: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/56.jpg)
National Center on Response to Intervention
Funding What funding resources are available to support the
ongoing evaluation of RTI? What funding is available to support knowledge
development or engage experts? What funding is available to address any necessary
changes to the RTI process?
56
![Page 57: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/57.jpg)
National Center on Response to Intervention
Evaluation Tools What tools are currently available? Can we use existing tools? Do we need to adapt or
develop new tools? What resources do we have for analyzing and
reporting the results? Do we have sufficient training on how to use the
tools and reporting information we select? Note: This will be covered further in later sections of the presentation
57
![Page 58: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/58.jpg)
National Center on Response to Intervention
Timeframe
How much time is needed to plan for evaluation? When will data collection begin? Will multiple years or a single year of data be used
for analysis? When will data analysis occur (single time point or
multiple time points) and when will the results be reviewed?
58
![Page 59: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/59.jpg)
National Center on Response to Intervention
Data Analysis How will we aggregate data (across students, classes,
grades, schools)? Do we have consistent benchmarks (across grades,
within the district)? How many years of data will we consider? Will we use a data system to support analysis? If so,
which one?
59
![Page 60: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/60.jpg)
National Center on Response to Intervention
Sharing of Results How will we report the results? How will we disseminate the results and next steps?
• Parents • School board • Teachers • Administration • School staff • Others
60
![Page 61: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/61.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE How will we share the results of our evaluation, or
how do we currently share our evaluation data? • What methods will/do we use? • Who will be/is our target audience? • Will /do we use different methods for different audiences?
61
![Page 62: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/62.jpg)
National Center on Response to Intervention
Reflection: Guiding Questions for Evaluation
What do we want to know? What data do we have that we can use to evaluate
(e.g., fidelity data, student outcome data)? Do we need to collect additional data? If so, what
type?
62
![Page 63: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/63.jpg)
National Center on Response to Intervention
ASSESSING STUDENT, SCHOOL, DISTRICT OUTCOMES
63
![Page 64: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/64.jpg)
National Center on Response to Intervention
Individuals, Grades, Schools, Districts • Aggregation of data
o Mean (average) across students, grades, schools o Range (spread) of scores from low to high
Different Levels or Types of Analysis
64
![Page 65: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/65.jpg)
National Center on Response to Intervention
Different Levels or Types of Analysis
Single year or across multiple years of implementation • Comparing cohorts of students
Schools implementing RTI ,or comparison of schools implementing RTI with schools not implementing RTI
65
![Page 66: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/66.jpg)
National Center on Response to Intervention
Data Systems
Published data systems • Pros: Data can be accessed by multiple users; many
systems provide easy-to-interpret printouts • Cons: Cost; lack of knowledge on how to use the system
Excel spreadsheet • Pros: No cost; accessible; easy to use for basic analyses • Cons: Access to data is often limited to a few members;
lack of skills in creating graphs and analyzing data
66
![Page 67: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/67.jpg)
National Center on Response to Intervention 67
Overarching Indicators of Success All measures of success are related to data-based decision making, but two common indicators of interest are
• Student outcomes (e.g., scores on end-of-year assessments, number of office referrals)
• Patterns of special education referral
![Page 68: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/68.jpg)
National Center on Response to Intervention
Student Outcomes Student outcomes can include
• State assessments • Discipline/behavioral referrals • Graduation rates • Retention rates • Dropout rates
The percentage of students achieving, nearly meeting, or exceeding standards on outcome measures should increase across years.
68
![Page 69: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/69.jpg)
National Center on Response to Intervention
Ways to Look at Changes in Student Outcomes
1. Changes in student outcomes by cohort across multiple years of RTI Implementation
2. Comparison of student outcomes for schools implementing RTI and schools not implementing RTI
69
![Page 70: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/70.jpg)
National Center on Response to Intervention 70
Example 1—Comparison of Cohorts Across Years in Grades 3–5
45%
55%
45%
55%
43%
57% 60%
40%
57%
43% 44%
56%
71%
29%
75%
25%
40%
60%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Proficient Not Proficient
Proficient Not Proficient
Proficient Not Proficient
2008–09 2009–10 2010–11
Per
cent
age
of S
tude
nts
![Page 71: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/71.jpg)
National Center on Response to Intervention 71
THINK-PAIR-SHARE
45%
55%
45%
55%
43%
57% 60%
40%
57%
43% 44%
56%
71%
29%
75%
25%
40%
60%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Proficient Not Proficient
Proficient Not Proficient
Proficient Not Proficient
2008–09 2009–10 2010–11
What does the graph tell you about student achievement in the following grades?
What trends do you see for students in third grade? What about fifth grade?
Per
cent
age
of S
tude
nts
![Page 72: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/72.jpg)
National Center on Response to Intervention 72
Example 2—Comparison of Proficiency on State Assessment Across Schools
48% 52%
45%
55% 50% 50%
57%
43%
53% 47%
75%
25%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Proficient Not Proficient Proficient Not Proficient
2008–09 2009–10 2010–11
Comparison Schools RTI Schools
Per
cent
age
of S
tude
nts
![Page 73: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/73.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE
73
48% 52% 45%
55% 50% 50% 57%
43%
53% 47%
75%
25%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Proficient Not Proficient
Proficient Not Proficient
2008–09 2009–10 2010–11
What does the graph tell you about student proficiency at schools implementing RTI in comparison to those not implementing RTI?
Comparison Schools RTI Schools
Per
cent
age
of S
tude
nts
![Page 74: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/74.jpg)
National Center on Response to Intervention
Evaluation Plan Graphic Organizer Work with your team to fill out section 1 of the graphic organizer, focusing on these questions: What are you currently doing to evaluate student
outcome measures? What processes can you implement to evaluate
student outcome measures?
74
![Page 75: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/75.jpg)
National Center on Response to Intervention
Fewer inappropriate referrals to special education • Efficiency of evaluation • Accuracy of referrals
Patterns of Special Education Referrals
75
![Page 76: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/76.jpg)
National Center on Response to Intervention
Disability Identification To ensure that underachievement in a child suspected of having a specific learning disability is not due to a lack of appropriate instruction in reading or math, the group must consider, as part of the evaluation, what is described in 34 CFR 300.304 through 300.306: Data demonstrate that prior to, or as a part of, the referral
process, the child was provided appropriate instruction in regular education settings, delivered by qualified personnel.
Data-based documentation of repeated assessments of achievement at reasonable intervals reflect formal assessment of student progress during instruction, which was provided to the child’s parents.
76
(www.idea.ed.gov)
![Page 77: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/77.jpg)
National Center on Response to Intervention
State Regulations for Disability Identification
States have different regulations for referrals for disability identification.
Contact the State Department of Education for additional information.
77
![Page 78: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/78.jpg)
National Center on Response to Intervention
Evaluation Plan Graphic Organizer Work with your team to fill out section 2 of the graphic organizer, focusing on these questions: What are you currently doing to evaluate patterns
of special education referrals? What additional information do you need to
evaluate patterns of special education referrals?
78
![Page 79: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/79.jpg)
National Center on Response to Intervention 79
Indicators of Success Using Screening Data
Identifying changes in the number of students identified as at risk or not at risk based on screening benchmarks
![Page 80: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/80.jpg)
National Center on Response to Intervention
Percentage of students achieving the benchmark should increase over the year. • Example: Oral Reading Fluency benchmarks (AIMSweb,
DIBELS, local norms)
Percentage of students achieving benchmark across years should increase toward ceiling levels. • The goal is to have at least 80% of students identified
as not at risk
Indicators of Success: Using Screening Data
80
![Page 81: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/81.jpg)
National Center on Response to Intervention
Why ~80%? The goal of at least 80% is based on the public health
model. This goal may not apply to all schools
• For example, schools with populations that are highly transient, high poverty, and/or have high percentages of English language learners (ELLs)
• These schools should still have high expectations for their students
• While they may use baseline data as starting points they should strive for the goal of 80%
81
![Page 82: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/82.jpg)
National Center on Response to Intervention
Example: Establishing Goals for Benchmark at School A A small urban community, high poverty, failing school
When an RTI model was introduced into that school • First universal screening revealed that 16% of students in the early
grades met benchmark; 60–70% did not.
• Rather than establish an unrealistic expectation that at least 80% of students reach benchmark over the course of several years, they established a goal of 50% at benchmark in year 1, and 60% at benchmark in year 2.
• While the school did not reach 80% at benchmark , it improved over a three-year period to about 55% at low risk and only 20% at risk.
82
![Page 83: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/83.jpg)
National Center on Response to Intervention
Ways to Look at Changes Using Screening Data 1. Changes in risk level aggregated across schools
within the same school year (need a consistent cut score across the schools)
a) Mean b) Range
2. Changes in risk level by cohort across multiple years of RTI implementation
3. Comparison of risk-level changes for schools implementing RTI and schools not implementing RTI
83
![Page 84: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/84.jpg)
National Center on Response to Intervention 84
Example 1a—Average Within the Same Year
73% 77%
84%
27% 23%
16%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Fall Winter Spring
Not at risk
At risk
Grade 1—Word Identification Fluency (WIF)
Per
cent
age
of S
tude
nts
Risk Level: Aggregated Mean for Seven Schools Across One Year
![Page 85: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/85.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE
85
0%
10%
20%
30%
40%
50%
60%
70%
Fall Winter Spring
At risk Not at risk
Grade 2 Oral Reading Fluency (ORF)
What does the graph tell you about second graders’ progress in these 7 elementary schools? What questions
might you raise based on the data?
Per
cent
age
of S
tude
nts
![Page 86: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/86.jpg)
National Center on Response to Intervention 86
29%23%
51%
85%
74%
94%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Fall Winter Spring
Perc
enta
ge o
f Stu
dent
s
Grade 1—Word Identification Fluency
Aggregated Range for Students Identified as Not at Risk Across Five Schools in One Year
![Page 87: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/87.jpg)
National Center on Response to Intervention 87
3% 3%0%
37%31%
10%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Fall Winter Spring
Perc
enta
ge o
f Stu
dent
s
Aggregated Range for Students Identified as at Risk Across Five Schools in One Year
Grade 1—Word Identification Fluency
![Page 88: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/88.jpg)
National Center on Response to Intervention 88
74%
26%
68%
32%
84%
16%
74%
26%
90%
10%
80%
20%
0%
20%
40%
60%
80%
100%
Not at Risk At Risk Not at Risk At Risk
Spring 2008
Spring 2009
Spring 2010
Example 2—Comparison of Cohorts Across Years in the Same Grade
Grade 1—Word Identification Fluency Grade 2—Oral Reading Fluency
Per
cent
age
of S
tude
nts
Percentage of First- and Second-Grade Students Identified as at Risk or Not at Risk Across Three Years
![Page 89: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/89.jpg)
National Center on Response to Intervention 89
Example 3—Comparison Across Schools and Years
18%
53%
36%
43%
14%
59%
17%
55%
26%
45%
12%
60%
0%
10%
20%
30%
40%
50%
60%
70%
At Risk Not at Risk At Risk Not at Risk
2008–09
2009–10
2010–11
Comparison Schools RTI Schools
Per
cent
age
of S
tude
nts
Percentage of First-Grade Students Identified as at Risk or Not at Risk on the Word Identification Fluency Screener Across Schools
Implementing and Not Implementing RTI for Three Years
What does the graph say about the change in risk level across years of RTI implementation for schools implementing RTI?
![Page 90: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/90.jpg)
National Center on Response to Intervention
Evaluation Plan Graphic Organizer Work with your team to fill out section 3 of the graphic organizer, focusing on these questions: What are you currently doing to evaluate changes
in student risk levels using screening data? What can you implement to evaluate changes in
student risk levels using screening data?
90
![Page 91: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/91.jpg)
National Center on Response to Intervention 91
Indicators of Success Using Progress Monitoring Data
Determine if students are making enough progress within prevention levels to close the gap
![Page 92: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/92.jpg)
National Center on Response to Intervention
Rate of Improvement (ROI) • Typical ROI (National Norm)
• Targeted ROI (Goal)
• Attained ROI (Actual)
Movement Within Levels: Using Progress Monitoring Data
92
![Page 93: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/93.jpg)
National Center on Response to Intervention
Example: National Norms or Typical ROI for Weekly Improvement
Grade Reading—
Slope Computation CBM—Slope for
Digits Correct
Concepts and Applications CBM—
Slope for Points
1 1.8 (WIF) .35 No data available
2 1.5 (PRF) .30 .40
3 1.0 (PRF) .30 .60
4 .40 (Maze) .70 .70
5 .40 (Maze) .70 .70
6 .40 (Maze) .40 .70
Note: These figures may change with further RTI research and are specific to one assessment tool.
93
![Page 94: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/94.jpg)
National Center on Response to Intervention
Ways to Look at Movement Within Levels of Prevention
1. Comparisons of typical, targeted, and attained ROI for a single student
a) Not closing the gap b) Closing the gap
2. Aggregated data comparing targeted and attained ROI by grade level for students in a single school
94
![Page 95: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/95.jpg)
National Center on Response to Intervention
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks of Instruction
Prob
lem
s C
orre
ct in
7 M
inut
es
X X
X
Targeted ROI/goal line
Actual ROI/trend line
Example 1a—Student Not Closing the Gap (Actual and Targeted ROI)
95
Wor
ds R
ead
Corr
ectly
X
![Page 96: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/96.jpg)
National Center on Response to Intervention
Example 1b—Student Closing the Gap (Actual and Targeted ROI)
96
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Wor
ds R
ead
Cor
rect
ly
Weeks of Instruction
X X
X X
Targeted ROI/goal line
Actual ROI/trend line
![Page 97: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/97.jpg)
National Center on Response to Intervention 97
Example 2—Targeted Versus Attained Levels of Progress Monitoring for Students at Secondary
Prevention 1.5
1.25 1.14
1.66
0 0.2 0.4 0.6 0.8
1 1.2 1.4 1.6 1.8
Grade 2—Passage Reading Fluency
Grade 3—Passage Reading Fluency
Targeted Attained
Typical
Rat
e of
Impr
ovem
ent
![Page 98: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/98.jpg)
National Center on Response to Intervention
THINK-PAIR-SHARE
1.8
2.07
1.6
1.7
1.8
1.9
2
2.1
Grade 1—Word Identification Fluency
Targeted
Attained
Think about the data for first-grade students. The typical ROI for first-grade students is 1.8. How are the students in
first grade doing? What does this say about
the secondary model for first grade?
98
Typical
Rat
e of
Impr
ovem
ent
![Page 99: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/99.jpg)
National Center on Response to Intervention
Evaluation Plan Graphic Organizer Work with your team to fill out section 4 of the graphic organizer, focusing on these questions: What are you currently doing to evaluate student
progress within prevention levels? What processes can you implement to evaluate
student progress within prevention levels?
99
![Page 100: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/100.jpg)
National Center on Response to Intervention 100
Indicators of Success Using Multi-Level Prevention System Data
Movement between prevention levels
• Movement to a higher level of intensity
• Movement to a lower level of intensity
• No movement
![Page 101: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/101.jpg)
National Center on Response to Intervention
Change in the level of intensity. Reflects growth in desired direction. Assume a general similarity in definition of levels of
intensity. Uses multiple measures (not single metric).
Change Within the Year
101
![Page 102: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/102.jpg)
National Center on Response to Intervention
Types of Changes
May need to look primarily at changes from • Primary to Secondary • Secondary to Tertiary • Secondary to Primary • Tertiary to Secondary • Tertiary to Primary • No movement from Secondary
102
![Page 103: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/103.jpg)
National Center on Response to Intervention 103
Example of an Effective RTI Model
![Page 104: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/104.jpg)
National Center on Response to Intervention 104
Example of an Ineffective RTI Model
![Page 105: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/105.jpg)
National Center on Response to Intervention
Ways to Look at Movement Between Levels of Prevention
1. Overall changes in the percentage of students within each level across the year
2. Percentage of students moving from a higher to lower level of intensity or lower to higher level of intensity
a) Individual school
b) Aggregated across schools
105
![Page 106: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/106.jpg)
National Center on Response to Intervention 106
Example 1—Percentage of Students by Level at School A (Fall to Spring)
61%
25% 14%
79%
15% 6%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Only Primary Secondary Tertiary
Fall Spring
Perc
enta
ge o
f stu
dent
s
![Page 107: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/107.jpg)
National Center on Response to Intervention 107
Example 2a—Changes Between Levels for First Graders in School A (Fall to Winter)
6.67% 5.26%
42.11%
28.57%
14.29%
0.00% 5.00%
10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% 45.00%
Primary to Secondary
Secondary to Tertiary
Secondary to Primary
Tertiary to Secondary
Tertiary to Primary
Perc
enta
ge o
f Stu
dent
s
Moved from less intensive to more intensive
Moved from more intensive to less intensive
![Page 108: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/108.jpg)
National Center on Response to Intervention 108
Example 2b—Changes Between Levels Across RTI Sites (Fall to Winter)
5%
15%
23%
13%
0%
5%
10%
15%
20%
25%
Primary to Secondary
Secondary to Tertiary
Secondary to Primary
Tertiary to Secondary
Moved from less intensive to more intensive
Moved from more intensive to less intensive
Perc
enta
ge o
f Stu
dent
s
![Page 109: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/109.jpg)
National Center on Response to Intervention
Evaluation Plan Graphic Organizer Work with your team to fill out section 5 of the graphic organizer, focusing on these questions: What are you currently doing to evaluate
movement of students between levels of prevention?
What processes can you implement to evaluate movement of students between levels of prevention?
109
![Page 110: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/110.jpg)
National Center on Response to Intervention
Guiding Questions Handout As a team, come to a consensus on the following guiding questions.
• What do we want to know (e.g., evaluation questions)?
• What is an indicator of what we want to know (e.g., success indicators)?
• What type of data do we need to collect (e.g., format, existing or new)?
• How will it be submitted or collected (e.g., by whom, data system use)?
• How frequently will we collect this data? When will we collect and analyze?
• Are there any potential challenges or concerns?
• What fidelity data do we already have?
• Do we need to adjust our process to make future evaluation efforts more successful? If so, what changes do we need to make?
110
![Page 111: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/111.jpg)
National Center on Response to Intervention
Review Activity 1. What type of data analysis would you use to find out if
a) Your students are making adequate progress within a prevention level?
b) Schools that are implementing RTI are outperforming schools that are not implementing RTI?
c) Your RTI model is working?
2. Why is fidelity important? 3. Provide three examples of data that can be used in an RTI
evaluation.
111
![Page 112: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/112.jpg)
National Center on Response to Intervention
Closing Good data in, good data out Evaluation plans help you understand whether your
RTI model is working, and should include • Measures of fidelity • Measures of student, school, and district success
There are multiple ways to evaluate your RTI model and analyze the data. Developing an evaluation plan is important.
112
![Page 113: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/113.jpg)
National Center on Response to Intervention
National Center on Response to Intervention
www.rti4success.org
Questions?
![Page 114: Things to Consider When Developing an RTI Evaluation Plan · Response to Intervention Session Objectives Recognize why it is important to evaluate the RTI model and what the evaluation](https://reader034.fdocuments.in/reader034/viewer/2022042316/5f0507987e708231d410e8bb/html5/thumbnails/114.jpg)
National Center on Response to Intervention 114
This document was produced under U.S. Department of Education, Office of Special Education Programs Grant No. H326E070004. Grace Zamora Durán and Tina Diamond served as the OSEP project officers. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service, or enterprise mentioned in this publication is intended or should be inferred. This product is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be: www.rti4success.org.
National Center on Response to Intervention