Response to Intervention RTI: Student Assessment & Progress-Monitoring Jim Wright .

Response to Intervention

www.interventioncentral.org

RTI: Student Assessment & Progress-Monitoring

Jim Wrightwww.interventioncentral.org



RTI: Student Assessment & Progress-Monitoring

How can we collect data in the classroom to evaluate Tier 1 interventions?

How can we be sure that we are asking the right questions and collecting data from enough sources to make good decisions about students?

How can we use data to proactively find students who are struggling?

How can we judge if the intervention has been carried out correctly?



Workshop Materials Available at:

• http://www.jimwrightonline.com/SWBOCES.php

3


www.interventioncentral.org 4

RTI Assumption: Struggling Students Are ‘Typical’ Until Proven Otherwise…

RTI logic assumes that:– A student who begins to struggle in general education is typical,

and that– It is general education’s responsibility to find the instructional

strategies that will unlock the student’s learning potential

Only when the student shows through well-documented interventions that he or she has ‘failed to respond to intervention’ does RTI begin to investigate the possibility that the student may have a learning disability or other special education condition.



Use Time & Resources Efficiently By Collecting Information Only on ‘Things That Are Alterable’

“…Time should be spent thinking about things that the intervention team can influence through instruction, consultation, related services, or adjustments to the student’s program. These are things that are alterable.…Beware of statements about cognitive processes that shift the focus from the curriculum and may even encourage questionable educational practice. They can also promote writing off a student because of the rationale that the student’s insufficient performance is due to a limited and fixed potential. “ p.359

Source: Howell, K. W., Hosp, J. L., & Kurns, S. (2008). Best practices in curriculum-based evaluation. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp.349-362). Bethesda, MD: National Association of School Psychologists.



Formal Tests: Only One Source of Student Assessment Information

“Tests are often overused and misunderstood in and out of the field of school psychology. When necessary, analog [i.e., test] observations can be used to test relevant hypotheses within controlled conditions. Testing is a highly standardized form of observation. ….The only reason to administer a test is to answer well-specified questions and examine well-specified hypotheses. It is best practice to identify and make explicit the most relevant questions before assessment begins. …The process of assessment should follow these questions. The questions should not follow assessment. “ p.170

Source: Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 159-176). Bethesda, MD: National Association of School Psychologists.



Relevant Academic Information: Sources and Purpose

• Tier 1: Instructional information. Teachers do classroom assessments (both formal and informal). Results are used to make day-to-day decisions about pacing of instruction, to determine students who need additional support, etc.

• Tier 1/Tier 2: Schoolwide screenings. Brief universal screenings are administered to all students at a grade level to measure academic skills that predict future school success. Results reflect on quality of core instruction and drive recruitment for Tier 2 programs.

• Tier 3: Analytic/diagnostic instructional assessment. Struggling students with more severe needs picked up in screenings may be administered a more detailed assessment (using qualitative and/or quantitative measures) to map out pattern of deficits in basic academic skills. Results are used to create a customized intervention plan that meets that student’s unique needs.

7



Methods of Classroom Data Collection




RTI ‘Pyramid of Interventions’

Tier 1

Tier 2

Tier 3

Tier 1: Universal interventions. Available to all students in a classroom or school. Can consist of whole-group or individual strategies or supports.

Tier 2 Individualized interventions. Subset of students receive interventions targeting specific needs.

Tier 3: Intensive interventions. Students who are ‘non-responders’ to Tiers 1 & 2 are referred to the RTI Team for more intensive interventions.



Interventions: Potential ‘Fatal Flaws’Any intervention must include 4 essential elements. The absence of any one of the elements would be considered a ‘fatal flaw’ (Witt, VanDerHeyden & Gilbertson, 2004) that blocks the school from drawing meaningful conclusions from the student’s response to the intervention:

1. Clearly defined problem. The student’s target concern is stated in specific, observable, measureable terms. This ‘problem identification statement’ is the most important step of the problem-solving model (Bergan, 1995), as a clearly defined problem allows the teacher or RTI Team to select a well-matched intervention to address it.

2. Baseline data. The teacher or RTI Team measures the student’s academic skills in the target concern (e.g., reading fluency, math computation) prior to beginning the intervention. Baseline data becomes the point of comparison throughout the intervention to help the school to determine whether that intervention is effective.

3. Performance goal. The teacher or RTI Team sets a specific, data-based goal for student improvement during the intervention and a checkpoint date by which the goal should be attained.

4. Progress-monitoring plan. The teacher or RTI Team collects student data regularly to determine whether the student is on-track to reach the performance goal.

Source: Witt, J. C., VanDerHeyden, A. M., & Gilbertson, D. (2004). Troubleshooting behavioral interventions. A systematic process for finding and eliminating problems. School Psychology Review, 33, 363-383.



Existing Records• Description: The teacher uses information already being

collected in the classroom that is relevant to the identified student problem.

• Examples of existing records that can be used to track student problems include:– Grades– Absences and incidents of tardiness– Homework turned in

12



Skills Checklists• Description: The teacher selects a global skill.

The teacher then breaks that global skill down into specific, observable ‘subskills’. Each subskill can be verified as ‘done’ or ‘not done’.

13



Skills Checklists: Example• The teacher selects the global skill ‘organizational skills’.• That global skill is defined as having the following

components, each of which can be observed:arriving to class on time; bringing work materials to class; following teacher directions in a timely manner; knowing how to request teacher assistance when

needed; having an uncluttered desk with only essential work

materials.

14



Behavioral Frequency Count• Description: The teacher observes a student behavior

and keeps a cumulative tally of the number of times that the behavior is observed during a given period.

• Behaviors that are best measured using frequency counts have clearly observable beginning and end points—and are of relatively short duration. Examples include:– Student call-outs.– Requests for teacher help during independent

seatwork.– Raising one’s hand to make a contribution to large-

group discussion.

15



Behavioral Frequency Count: How to Record

Teachers can collect data on the frequency of student behaviors in several ways:

• Keeping a mental tally of the frequency of target behaviors occurring during a class period.

• Recording behaviors on paper (e.g., simple tally marks) as they occur.

• Using a golf counter, stitch counter, or other mechanical counter device to keep an accurate tally of behaviors.

16



Behavioral Frequency Count: How to Compute• If student behaviors are being tallied during a class period, frequency-

count data can be reported as ‘X number of behaviors per class period’.

• If frequency-count data is collected in different spans of time on different days, however, schools can use the following method to standardize frequency count data : – Record the total number of behaviors observed.– Record the number of minutes in the observation period.– Divide the total number of behaviors observed by total minutes in

the observation period. Example: 5 callouts observed during a 10 minute period = 0.5 callouts per minute.

17



Behavior Log• Description: The teacher makes a log entry each time

that a behavior is observed. An advantage of behavior logs is that they can provide information about the context within which a behavior occurs.(Disciplinary office referrals are a specialized example of a behavior log.)

• Behavior logs are useful for tracking ‘low-incidence’ problem behaviors.

18



Behavior Log: Sample Form

19



Rating Scales• Description: A scale is developed that a rater can use to

complete a global rating of a behavior. Often the rating scale is completed at the conclusion of a fixed observation period (e.g., after each class period).

• Daily / Direct Behavior Report Cards are one example of rating scales.

20



Student Work Samples• Description: Work samples are collected for information

about the student’s basic academic skills, mastery of course content, etc.

• Recommendation: When collecting work samples:– Record the date that the sample was collected– If the work sample was produced in class, note the

amount of time needed to complete the sample (students can calculate and record this information).

– If possible, collect 1-2 work samples from typical students as well to provide a standard of peer comparison.

21



Work Performance Logs• Description: Information about student academic

performance is collected to provide insight into growth in student skills or use of skills in appropriate situations.

Example: A teacher implementing a vocabulary-building intervention keeps a cumulative log noting date and vocabulary words mastered.

• Example: A student keeps a journal with dated entries logging books read or the amount of ‘seat time’ that she spends on math homework.

22



Timed Tasks (e.g., Curriculum-Based Measurement)

• Description: The teacher administers structured, timed tasks to assess student accuracy and fluency.

• Example: The student completes a 2-minute CBM single-skill math computation probe.

• Example: The student completes a 3-minute CBM writing probe that is scored for total words written.

23



Combining Classroom Monitoring Methods• Often, methods of classroom data collection and

progress-monitoring can be combined to track a single student problem.

• Example: A teacher can use a rubric (checklist) to rate the quality of student work samples.

• Example: A teacher may keep a running tally (behavioral frequency count) of student callouts. At the same time, the student may be self-monitoring his rate of callouts on a Daily Behavior Report Card (rating scale).

24



Activity: Classroom Methods of Data CollectionIn your teams:

• Review the potential sources of classroom data that can be used to monitor Tier 1 interventions.

• What questions do you have about any of these data sources?

• How can your school make full use of these data sources to ensure that every Tier 1 intervention is monitored?

Classroom Data Sources:• Existing records• Skills checklist• Behavioral frequency count• Behavioral log• Rating scales• Student work samples• Work performance logs• Timed tasks (e.g., CBM)



Establishing RTI Guidelines to Diagnose Learning Disabilities: What Schools Should Know




Using RTI to Determine Special Education Eligibility: Building the Foundation

• Ensure Tier 1 (Classroom) Capacity to Carry Out Quality Interventions. The classroom teacher is the ‘first responder’ available to address emerging student academic concerns. Therefore, general-education teachers should have the capacity to define student academic concerns in specific terms, independently choose and carry out appropriate evidence-based Tier 1 (classroom) interventions, and document student response to those interventions.



Tier 1 (Classroom) Interventions: Building Your School’s Capacity

Train Teachers to Write Specific, Measureable, Observable ‘Problem Identification Statements.

Inventory Tier 1 Interventions Already in Use. Create a Standard Menu of Evidence-Based Tier 1

Intervention Ideas for Teachers. Establish Tier 1 Coaching and Support Resources. Provide Classroom (Tier 1) Problem-Solving Support to

Teachers. Set Up a System to Locate Additional Evidence-Based Tier 1

Intervention Ideas. Create Formal Guidelines for Teachers to Document Tier 1

Strategies. Develop Decision Rules for Referring Students from Tier 1 to

Higher Levels of Intervention.




• Collect Benchmarking/Universal Screening Data on Key Reading and Math (and Perhaps Other) Academic Skills for Each Grade Level. Benchmarking data is collected on all students at least three times per year (fall, winter, spring). Measures selected for benchmarking should track student fluency and accuracy in basic academic skills that are key to success at each grade level.




• Hold ‘Data Meetings’ With Each Grade Level. After each benchmarking period (fall, winter, spring), the school organizes data meetings by grade level. The building administrator, classroom teachers, and perhaps other staff (e.g., reading specialist, school psychologist) meet to:– review student benchmark data.– discuss how classroom (Tier 1) instruction should be changed

to accommodate the student needs revealed in the benchmarking data.

– select students for Tier 2 (supplemental group) instruction/intervention.



Tier 2: Supplemental (Group-Based) Interventions

Tier 2 interventions are typically delivered in small-group format. About 15% of students in the typical school will require Tier 2/supplemental intervention support.

Group size for Tier 2 interventions is limited to 4-6 students.

Students placed in Tier 2 interventions should have a shared profile of intervention need.

The reading progress of students in Tier 2 interventions are monitored at least 1-2 times per month.

Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools. Routledge: New York.



Using RTI to Determine Special Education Eligibility: Creating Decision Rules

• Establish the Minimum Number of Intervention Trials Required Prior to a Special Education Referral. Your district should require a sufficient number of intervention trials to definitively rule out instructional variables as possible reasons for student academic delays. Many districts require that at least three Tier 2 (small-group supplemental) and/or Tier 3 (intensive, highly individualized) intervention trials be attempted before moving forward with a special education evaluation.




• Determine the Minimum Timespan for Each Tier 2 or Tier 3 Intervention Trial. An intervention trial should last long enough to show definitively whether it was effective. One expert recommendation (Burns & Gibbons, 2008) is that each academic intervention trial should last at least 8 instructional weeks to allow enough time for the school to collect sufficient data to generate a reliable trend line.




• Define the Level of Student Academic Delay That Will Qualify as a Significant Skill Discrepancy. Not all students with academic delays require special education services; those with more modest deficits may benefit from general-education supplemental interventions alone. Your district should develop guidelines for determining whether a student’s academic skills should be judge as significantly delayed when compared to those of peers:– If using local Curriculum-Based Measurement norms, set an appropriate

‘cutpoint’ score (e.g., at the 10th percentile). Any student performing below that cutpoint would be identified as having a significant gap in skills.

– If using reliable national or research norms (e.g., reading fluency norms from Hasbrouck & Tindal, 2004), set an appropriate ‘cutpoint’ score (e.g., at the 10th percentile). Any student performing below that cutpoint would be identified as having a significant gap in skills.




• Define the Rate of Student Progress That Will Qualify as a Significant Discrepancy in Rate of Learning. The question of whether a student has made adequate progress when on intervention is complex. While each student case must be considered on its own merits, however, your district can bring consistency to the process of judging the efficacy of interventions by discussing the following factors…




• Define the Rate of Student Progress That Will Qualify as a Significant Discrepancy in Rate of Learning (Cont.). – Define ‘grade level performance’. The goal of academic

intervention is to bring student skills to grade level. However, your district may want to specify what is meant by ‘grade level’ performance. Local CBM norms or reliable national or research norms can be helpful here. The district can set a cutpoint that sets a minimum threshold for ‘typical student performance’ (e.g., 25th percentile or above on local or research norms). Students whose performance is above the cutpoint would fall within the ‘reachable, teachable range’ and could be adequately instructed by the classroom teacher.



`Estimate the academic skill gap between the target student and typically-performing peers:

There are three general methods for estimating the ‘typical’ level of academic performance at a grade level:

• Local Norms: A sample of students at a school are screened in an academic skill to create grade norms (Shinn, 1989)

• Research Norms: Norms for ‘typical’ growth are derived from a research sample, published, and applied by schools to their own student populations (e.g., Shapiro, 1996)

• Criterion-Referenced Benchmarks: A minimum level, or threshold, of competence is determined for an skill. The benchmark is usually defined as a level of proficiency needed for later school success (Fuchs, 2003)




• Define the Rate of Student Progress That Will Qualify as a Significant Discrepancy in Rate of Learning (Cont.). – Set ambitious but realistic goals for student improvement.

When an intervention plan is put into place, the school should predict a rate of student academic improvement that is ambitious but realistic. During a typical intervention series, a student usually works toward intermediate goals for improvement, and an intermediate goal is reset at a higher level each time that the student attains it.

The school should be able to supply a rationale for how it set goals for rate of student improvement.

• When available, research guidelines (e.g., in oral reading fluency) can be used.

• Or the school may use local norms to compute improvement goals.




• Define the Rate of Student Progress That Will Qualify as a Significant Discrepancy in Rate of Learning (Cont.). – Decide on a reasonable time horizon to ‘catch’ the student up

with his or her peers. Interventions for students with serious academic delays cannot be successfully completed overnight. It is equally true, though, that interventions cannot stretch on without end if the student fails to make adequate progress. Your district should decide on a reasonable span of time in which a student on intervention should be expected to close the gap and reach grade level performance (e.g., 12 months). Failure to close that gap within the expected timespan may be partial evidence that the student requires special education support.




• Define the Rate of Student Progress That Will Qualify as a Significant Discrepancy in Rate of Learning (Cont.). – View student progress-monitoring data in relation to peer

norms. When viewed in isolation, student progress-monitoring data tells only part of the story. Even if students shows modest progress, they may still be falling farther and farther behind their peers in the academic skill of concern. Your district should evaluate student progress relative to peers. If the skill gap between the student and their peers (as determined through repeated school-wide benchmarking) continues to widen, despite the school’s most intensive intervention efforts, this may be partial evidence that the student requires special education support.




• Define the Rate of Student Progress That Will Qualify as a Significant Discrepancy in Rate of Learning (Cont.). – Set uniform expectations for how progress-monitoring data

are presented at special education eligibility meetings. Your district should adopt guidelines for schools in collecting and presenting student progress-monitoring information at special education eligibility meetings. For example, it is recommended that curriculum-based measurement or similar data be presented as time-series charts. These charts should include trend lines to summarize visually the student’s rate of academic growth, as well as a ‘goal line’ indicating the intermediate or final performance goal toward which the student is working.



Tier 3 Intervention: Jared: Intervention Phase 1: Weeks 1-6

1 131 17

1 201 24

1 271 31

2 32 7

2 102 14

2 242 28

3 33 7

3 103 14

3 173 21

3 243 28

3 314 4

4 74 11

4 144 18

XX

W 1/2271 CRW

W 1/2977 CRW

M 2/375 CRW

Th 2/1375 CRW

Th 2/2779 CRW

F 3/782 CRW



Confidence in E

ligibility D

ecision

Ensure Tier 1 (Classroom) Capacity to Carry Out Quality Interventions.

Collect Benchmarking/Universal Screening Data for Each Grade Level.

Hold Data Meetings to Make Tier 2 Group Placements for Each Grade Level.

Establish the Minimum Number of Intervention Trials Required Prior to a Special Education Referral.

Determine the Minimum Timespan for Each Tier 2 or Tier 3 Intervention Trial.

Define the Level of Student Academic Delay That Will Qualify as a Significant Skill Discrepancy.

Define the Rate of Student Progress That Will Qualify as a Significant Discrepancy in Rate of Learning.



• As a group, review recommendations for creating special education decision rules for LD eligibility.

• What are concerns or questions that your team has as you move forward?

• How will your RTI Team communicate with special education to develop your own decision rules?

Small-Group Activity: Special Education Decision Rules



Assessing Intervention Integrity




Why Assess Intervention Integrity?

When a struggling student fails to respond adequately to a series of evidence-based interventions, that student is likely to face significant and potentially negative consequences, such as failing grades, long-term suspension from school, or even placement in special education.

It is crucial, then, that the school monitor the integrity with which educators implement each intervention plan so that it can confidently rule out poor or limited intervention implementation of the intervention as a possible explanation for any student’s ‘non-response’.



Intervention Integrity Check: Direct Observation

Intervention integrity is best assessed through direct observation (Roach & Elliott, 2008). – The key steps of the intervention are defined and

formatted as an observational checklist. – An observer watches as the intervention is

conducted and checks off on the checklist those steps that were correctly carried out. The observer then computes the percentage of steps correctly carried out.



Limitations of Direct Observation as an Intervention Integrity Check

• Direct observations are time-consuming to conduct. • Teachers who serve as interventionists may at least

initially regard observations of their intervention implementation as evaluations of their job performance, rather than as a child-focused RTI “quality check”.

• An intervention-implementation checklist typically does not distinguish between--or differentially weight--those intervention steps that are more important from those that are less so. If two teachers implement the same 10-step intervention plan, for example, with one instructor omitting a critical step and the other omitting a fairly trivial step, both can still attain the same implementation score of steps correctly completed.

Source: Gansle, K. A., & Noell, G. H. (2007). The fundamental role of intervention implementation in assessing response to intervention. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Response to intervention: The science and practice of assessment and intervention (pp. 244-251).



Intervention Script Builder

‘Yes/No’ Step-by-Step

Intervention Check

Each Step Marked

‘Negotiable or ‘Non-

Negotiable’



Supplemental Methods to Collect Data About Intervention Integrity

Teacher Self-Ratings: As a form of self-monitoring, directing interventionists to rate the integrity of their own interventions may prompt higher rates of compliance (e.g., Kazdin, 1989). However, because teacher self-ratings tend to be ‘upwardly biased (Gansle & Noell, 2007, p. 247), they should not be relied upon as the sole rating of intervention integrity. One suggestion for collecting regular teacher reports on intervention implementation in a convenient manner is to use Daily Behavior Reports (DBRs; Chafouleas, Riley-Tillman,, & Sugai, 2007).

Sources: Chafouleas, S., Riley-Tillman, T.C., & Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York: Guilford Press.Gansle, K. A., & Noell, G. H. (2007). The fundamental role of intervention implementation in assessing response to intervention. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Response to intervention: The science and practice of assessment and intervention (pp. 244-251).Kazdin, A. E. (1989). Behavior modification in applied settings (4th ed.). Pacific Gove, CA: Brooks/Cole..



Intervention Contact Log

Teacher Intervention Integrity

Self-Rating



Supplemental Methods to Collect Data About Intervention Integrity

Intervention Permanent Products: If an intervention plan naturally yields permanent products (e.g., completed scoring sheets, lists of spelling words mastered, behavioral sticker charts), these products can be periodically collected and evaluated as another indicator of intervention integrity (Gansle & Noell, 2007).

Source:Gansle, K. A., & Noell, G. H. (2007). The fundamental role of intervention implementation in assessing response to intervention. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Response to intervention: The science and practice of assessment and intervention (pp. 244-251).



Intervention Integrity: Verify Through a Mix of Information Sources

Schools should consider monitoring intervention integrity through a mix of direct and indirect means, including direct observation and permanent products (Gansle & Noell, 2007), as well as interventionist self-ratings (Roach & Elliott, 2008).

Source: Gansle, K. A., & Noell, G. H. (2007). The fundamental role of intervention implementation in assessing response to intervention. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Response to intervention: The science and practice of assessment and intervention (pp. 244-251).Roach, A. T., & Elliott, S. N. (2008). Best practices in facilitating and evaluating intervention integrity. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp.195-208).



‘Selecting Methods to Track Intervention Integrity’…

56




57




58




59



Team Activity: Measuring ‘Intervention Follow-Through’

At your table:

• Brainstorm ways that your RTI Team will use to measure intervention integrity for interventions.

• What preparations are necessary to introduce these methods for measuring ‘intervention follow-through’ to your faculty?



Monitoring Student Academic or General Behaviors:

Daily Behavior Report Cards



Daily Behavior Report Cards (DBRCs) Are…

brief forms containing student behavior-rating items. The teacher typically rates the student daily (or even more frequently) on the DBRC. The results can be graphed to document student response to an intervention.



http://www.directbehaviorratings.com/



Daily Behavior Report Cards Can Monitor…

• Hyperactivity• On-Task Behavior (Attention)• Work Completion• Organization Skills• Compliance With Adult Requests• Ability to Interact Appropriately With Peers



Daily Behavior

Report Card: Daily

Version

Jim Blalock May 5Mrs. Williams Rm 108



Daily Behavior Report

Card: Weekly Version

40 0 60 60 50

Jim BlalockMrs. Williams Rm 108

05 05 07 05 06 07 05 07 07 05 08 07 05 09 07



Daily Behavior Report Card: Chart



Screening & Monitoring Student Progress




RTI Literacy: Assessment & Progress-Monitoring To measure student ‘response to instruction/intervention’ effectively, the RTI model measures students’ academic performance and progress on schedules matched to each student’s risk profile and intervention Tier membership.

• Benchmarking/Universal Screening. All children in a grade level are assessed at least 3 times per year on a common collection of academic assessments.

• Strategic Monitoring. Students placed in Tier 2 (supplemental) reading groups are assessed 1-2 times per month to gauge their progress with this intervention.

• Intensive Monitoring. Students who participate in an intensive, individualized Tier 3 intervention are assessed at least once per week.

Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge.



Universal Screening at Secondary Schools: Using Existing Data Proactively to Flag ‘Signs of Disengagement’

“Across interventions…, a key component to promoting school completion is the systematic monitoring of all students for signs of disengagement, such as attendance and behavior problems, failing courses, off track in terms of credits earned toward graduation, problematic or few close relationships with peers and/or teachers, and then following up with those who are at risk.”

Source: Jimerson, S., Reschly, A.L., & Hess, R. (2008). Best practices in increasing the likelihood of school completion. In A. Thomas & J. Grimes (Eds). Best Practices in School Psychology - 5th Ed (pp. 1085-1097). Bethesda, MD: National Association of School Psychologists.. p.1090



Mining Archival Data: What Are the ‘Early Warning Flags’ of Student Drop-Out?

A sample of 13,000 students in Philadelphia were tracked for 8 years. These early warning indicators were found to predict student drop-out in the sixth-grade year:

• Failure in English• Failure in math• Missing at least 20% of school days• Receiving an ‘unsatisfactory’ behavior rating from at least one

teacher

Source: Balfanz, R., Herzog, L., MacIver, D. J. (2007). Preventing student disengagement and keeping students on the graduation path in urban middle grades schools: Early identification and effective interventions. Educational Psychologist,42, 223–235. .



What is the Predictive Power of These Early Warning Flags?

Source: Balfanz, R., Herzog, L., MacIver, D. J. (2007). Preventing student disengagement and keeping students on the graduation path in urban middle grades schools: Early identification and effective interventions. Educational Psychologist,42, 223–235. .

Number of ‘Early Warning Flags’ in Student Record

Probability That Student Would Graduate

None 56%

1 36%

2 21%

3 13%

4 7%



Curriculum-Based Measurement: Advantages as a Set of Tools to Monitor RTI/Academic Cases

• Aligns with curriculum-goals and materials• Is reliable and valid (has ‘technical adequacy’) • Is criterion-referenced: sets specific performance levels for specific

tasks• Uses standard procedures to prepare materials, administer, and

score• Samples student performance to give objective, observable ‘low-

inference’ information about student performance • Has decision rules to help educators to interpret student data and

make appropriate instructional decisions• Is efficient to implement in schools (e.g., training can be done quickly;

the measures are brief and feasible for classrooms, etc.)• Provides data that can be converted into visual displays for ease of

communicationSource: Hosp, M.K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM. New York: Guilford.



Team Activity: Creating a Screening Plan for Your School

• Review the measures just discussed for screening students in various academic domains.

• Have a discussion about what measures you might use in a screening program for your school. Who would be involved in developing such a screening plan? When would it start?

84



Defining Academic Problems: Get It Right and Interventions Are More Likely to Be Effective




Defining Academic Problems: Recommended Steps

1. Be knowledgeable of the school academic curriculum and key student academic skills that are taught. The teacher should have a good survey-level knowledge of the key academic skills outlined in the school’s curriculum—for the grade level of their classroom as well as earlier grade levels. If the curriculum alone is not adequate for describing a student’s academic deficit, the instructor can make use of research-based definitions or complete a task analysis to further define the academic problem area. Here are guidelines for consulting curriculum and research-based definitions and for conducting a task analysis for more global skills.

86




Curriculum. The teacher can review the school’s curriculum and related documents (e.g., score-and-sequence charts; curriculum maps) to select specific academic skill or performance goals. First, determine the approximate grade or level in the curriculum that matches the student’s skills. Then, review the curriculum at that alternate grade level to find appropriate descriptions of the student‘s relevant academic deficit.

For example, a second-grade student had limited phonemic awareness. The student was not able accurately to deconstruct a spoken word into its component sound-units, or phonemes. In the school’s curriculum, children were expected to attain proficiency in phonemic awareness by the close of grade 1. The teacher went ‘off level’ to review the grade 1 curriculum and found a specific description of phonemic awareness that she could use as a starting point in defining the student’s skill deficit.

87




Research-Based Skill Definitions. Even when a school’s curriculum identifies key skills, schools may find it useful to corroborate or elaborate those skill definitions by reviewing alternative definitions published in research journals or other trusted sources.

For example, a student had delays in solving quadratic equations. The math instructor found that the school’s math curriculum did not provide a detailed description of the skills required to successfully complete quadratic equations. So the teacher reviewed the National Mathematics Advisory Panel report (Fennell et al., 2008) and found a detailed description of component skills for solving quadratic equations. By combining the skill definitions from the school curriculum with the more detailed descriptions taken from the research-based document, the teacher could better pinpoint the student’s academic deficit in specific terms.

88




Task Analysis. Students may possess deficits in more global ‘academic enabling’ skills that are essential for academic success. Teachers can complete an task analysis of the relevant skill by breaking it down into a checklist of constituent subskills. An instructor can use the resulting checklist to verify that the student can or cannot perform each of the subskills that make up the global ‘academic enabling’ skill.

For example, teachers at a middle school noted that many of their students seemed to have poor ‘organization’ skills. Those instructors conducted a task analysis and determined that--in their classrooms--the essential subskills of ‘student organization’ included (a) arriving to class on time; (b) bringing work materials to class; (c) following teacher directions in a timely manner; (d) knowing how to request teacher assistance when needed; and (e) having an uncluttered desk with only essential work materials.

89




2. Describe the academic problem in specific, skill-based terms (Batsche et al., 2008; Upah, 2008). Write a clear, brief description of the academic skill or performance deficit that focuses on a specific skill or performance area. Here are sample problem-identification statements:

– John reads aloud from grade-appropriate text much more slowly than his classmates.

– Ann lacks proficiency with multiplication math problems (double-digit times double-digit with no regrouping).

– Tye does not turn in homework assignments.– Angela produces limited text on in-class writing assignments.

90



Defining Academic Problems: Recommended Steps3. Develop a fuller description of the academic problem to provide a

meaningful instructional context. When the teacher has described the student’s academic problem, the next step is to expand the problem definition to put it into a meaningful context. This expanded definition includes information about the conditions under which the academic problem is observed and typical or expected level of performance.

– Conditions. Describe the environmental conditions or task demands in place when the academic problem is observed.

– Problem Description. Describe the actual observable academic behavior in which the student is engaged. Include rate, accuracy, or other quantitative information of student performance.

– Typical or Expected Level of Performance. Provide a typical or expected performance criterion for this skill or behavior. Typical or expected academic performance can be calculated using a variety of sources,

91



Defining Academic Problems: Recommended Steps4. Develop a hypothesis statement to explain the academic skill or

performance problem. The hypothesis states the assumed reason(s) or cause(s) for the student’s academic problems. Once it has been developed, the hypothesis statement acts as a compass needle, pointing toward interventions that most logically address the student academic problems.

93

Response to Intervention RTI: Student Assessment & Progress-Monitoring Jim Wright .

Documents

Transcript of Response to Intervention RTI: Student Assessment & Progress-Monitoring Jim Wright .