PROGRESS MONITORING

Glenda SederstromCenter for Special Education Services

NorthEast Washington ESD 101

Spokane, Washington

Cheney Presentation, August 27, 2015

SPECIAL EDUCATION WORKSHOP

Based upon

National Center on Response to Intervention

RTI Implementer SeriesModule 2: RTI Progress Monitoring

SISEP Project: State Implementation and Scaling-up of Evidence-based Practices

UPON COMPLETION OF THIS TRAINING, PARTICIPANTS WILL BE ABLE TO:

1. Understand the “Big” System and the importance of progress monitoring within that system

2. Use progress monitoring to improve student outcomes

3. Use progress monitoring data for making decisions about instruction and interventions

4. Develop guidance for using progress monitoring data

5. Understand the relationship between progress monitoring and appropriately formulated IEP’s

This workshop addresses the Teacher Evaluation Criteria #1, #2, #3, #6, #8 and the Principal Evaluation Criteria #1, #3, #5, #8

3

GROUP NORMS Allow ourselves and others to be seen as learners.

Monitor own airtime and sidebar conversations.

Allow for opportunities for equitable sharing.

Presume positive intentions.

Be respectful when giving and receiving opinions, ideas and approaches.

Honor the time schedule.

SISEP: (The State Implementation & Scaling Up of Evidence Based Practices)

is based at the FPG Child Development Institute at the University of North Carolina at Chapel Hill.

THE SISEP CENTER PROVIDES TECHNICAL ASSISTANCE TO:

increase knowledge of evidence-based implementation supports for evidence-based practices in States, Districts, and OSEP funded Technical Assistance Centers;

establish implementation infrastructures in State Education Agencies and Local Education Agencies in support of full and effective use of evidence-based approaches to education; and

establish implementation capacity in Technical Assistance Centers and projects funded by the U.S. Department of Education’s Office of Special Education Programs.

SISEP IN WASHINGTON STATE. . . . .

Multi-Tiered System of Supports (MTSS)

A framework to help all students graduate from high school ready for career, college, and life.

MTSS: DEFINED A whole-school, data-driven, prevention-based framework for improving

learning outcomes for EVERY student through a layered continuum of evidence-based practices and systems

Integrates academic, social/behavioral, and dropout prevention interventions

Dropout Early Warning Systems

Positive Behavior Supports (PBIS)

Response to Intervention

Student Assistance Program

COMPONENTS OF MTSS Tiered Supports/Evidence-Based Interventions

Data System

Universal Screening & Progress Monitoring

Continuous Improvement Processes

TIERED SUPPORTS

MTSS rely on evidence-based practices that are appropriate to every students need through a tiered approach to intervention

Tier 1: PreventionALL students benefit from school-wide Tier I services and supports (such as core academic instruction and teaching behavioral expectations and social emotional skills) to be prepared for career, college, and life.

Tier 2: Strategic InterventionSOME students benefit from additional Tier II services and supports (such as a reading or math intervention or behavioral check-in). These students are identified as “at-risk” for academic, behavioral, and/or mental health challenges, and require specific supports in addition to Tier I services.

Tier 3: Intensive InterventionFEW students benefit from additional Tier III services and supports (such as those provided through community partnerships to address more profound academic, behavioral, or mental health needs). These students need case management and other support services inaddition to Tier I services.

ESSENTIAL COMPONENTS OF RTI

12

The Relationship between MTSS and Response to Intervention

TURN AND TALK. . . How does what you have learned so far relate to the Teacher Evaluation Process?

TEACHER EVALUATION CRITERIA (RCW 28A.405.100 (2)(B))

1.Centering instruction on high expectations for student achievement.

2.Demonstrating effective teaching practices.

3.Recognizing individual student learning needs and developing strategies to address those needs.

4.Providing clear and intentional focus on subject matter content and curriculum.

5.Fostering and managing a safe, positive learning environment.

6.Using multiple student data elements to modify instruction and improve student learning.

7.Communicating and collaborating with parents and the school community

8.Exhibiting collaborative and collegial practices focused on improving instructional practice and student learning.

MOVING FORWARD: PROGRESS MONITORING DEFINED

Standardized type of formative assessment

Allows you evaluate progress over time to determine: Student response to instruction/intervention Instructional effectiveness for groups & individuals SLD eligibility (in accordance with law)

15

SCREENING V. PROGRESS MONITORING

“Close Cousins” Often the same measures used In some publications, you may see screening described as a type of progress monitoring.

Within RTI it is important to differentiate: Universal Screening, which is for all students from Progress Monitoring, which is for some students who have been

identified as at-risk for poor academic or behavioral outcomes.

Progress monitoring research has been conducted over the past 30 years

Research has demonstrated that when teachers use progress monitoring for instructional decision making:• Students learn more• Teacher decision making improves• Students are more aware of their

performance

WHY PROGRESS MONITORING?

17

18

PROGRESS MONITORING PURPOSE: monitor students’ response to primary, secondary, or tertiary instruction in order to estimate rates of improvement, identify students who are not demonstrating adequate progress, and compare the efficacy of different forms of instruction

FOCUS: students identified through screening as at risk for poor learning outcomes

TOOLS: brief assessments that are valid, reliable, and evidence based

TIMEFRAME: students are assessed at regular intervals (e.g., weekly, biweekly, or monthly)

PURPOSE OF PROGRESS MONITORINGAllows practitioners to… Estimate rates of improvement Identify students who are not demonstrating adequate progress

Compare the efficacy of different forms of instruction in order to design more effective, individualized instruction

19

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Primary Prevention

Wor

ds R

ead

Corre

ctly

ESTIMATE RATES OF IMPROVEMENT

20

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Primary Prevention

Wor

ds R

ead

Corre

ctly

6 WRC

.3 WRC

IDENTIFY STUDENTS NOT MAKING ADEQUATE PROGRESS

Increasing Scores:

21

X

Goal line

trend line

X

Goal line

trend line

Flat Scores:

X

X

X X

THUS, PROGRESS MONITORING TOOLS SHOULD…

Be valid and reliable for both: Level (i.e., that performance at a specific time point is stable and predicts end-

end-of year achievement) AND

Growth (i.e., that rate of improvement is also stable and predictive of end-of-year achievement)

Use standardized administration & scoring procedures Have alternate forms of comparable difficulty

22

WHEN APPROPRIATE MEASURES ARE USED, PROGRESS MONITORING CAN HELP DETERMINE…

Are students making progress at an acceptable rate?

Are students meeting short- and long-term performance goals?

Does the instruction or intervention need to be adjusted or changed?

24

25

COMMON FORMATIVE ASSESSMENTS USED FOR PROGRESS MONITORING

Mastery Measurement

General Outcome Measuresvs.

10

8

6

4

2

0

WEEKS

Multidigit Addition

Multidigit Subtraction

2 4 6 8 10 12 14

Multiplication Facts

Num

ber o

f pro

blem

s co

rrec

t in

5 m

inut

es

0

10

20

30

40

50

60

70

Wor

ds C

orre

ct P

er M

inut

e

Sample Progress Monitoring Chart

Words Correct

Aim Line

Linear (Words Correct)

26

Random numerals within problems (considering specifications of problem types)

Random placement of problem types on page

CBM Math Example

http://www.interventioncentral.org/

Text from Fourth Grade Level Science materials

Copy and Paste passage here

SHOULD MY ASSESSMENT TOOL BE USED FOR PROGRESS

MONITORING?Although a many assessments provide useful information and may be part of your broad approach to formative assessment, consider the following when deciding whether a tool should be used for progress monitoring within your RTI system...

• Are there standardized administration & scoring instructions?

• Are parallel/alternate forms available to allow for repeated assessment?

• Is there evidence of reliability & validity of performance level?

• Is there evidence or reliability & validity of the slope (i.e., growth rate)?

The Progress Monitoring Tools Chart can help you answer these questions!

38

39

NCRTI PROGRESS MONITORING TOOLS CHART

HTTP://WWW.RTI4SUCCESS.ORG/

PROGRESS MONITORING GRADE LEVEL

When possible, assess students at their chronological grade level

The goal should be set where you expect the student to perform at the end of the intervention period

Off grade-level assessment may be used with students performing below grade level.

Many PM tools have specific procedures for appropriately placing students.

Screening data should still be collected at grade level, however.

41

SET GOALS BASED ON LOGICAL & RESEARCH-

BASED PRACTICES

Stakeholders should know…

Why and how the goal was set

How long the student has to achieve the goal

What the student is expected to do when the goal is met

42

GOAL SETTING APPROACHESThree options for setting goals:

1. End-of-year benchmarking2. National norms for weekly rate

of improvement (slope)3. Intra-individual framework

(Tertiary)

43

OPTION 2: SETTING GOALS WITH NATIONAL NORMS FOR WEEKLY IMPROVEMENT (SLOPE)

44

Standard Formula for Calculating Goal Using Rate of Improvement (ROI):

( (ROI) x (# Weeks) ) + Baseline Score = GOAL

Example Using national norms for weekly rate of improvement (slope) Find baseline (e.g., average of first three data points) = 14 Identify ROI norm for fourth-grade computation = 0.70 Multiply norm by number of weeks left in instructional period

16 0.70 = 11.2 Add to baseline

11.2 + 14 = 25.2 End-of-year goal is: 25.2 (or 25)

45

Option 2: Setting Goals With National Norms for Weekly Improvement (slope)

TURN AND TALK. . . You have a fourth grade student who currently reads aloud at the rate of 60 correct words per minute. (Ending First Grade rate)

Typical fourth graders read aloud at a rate of 90-124 correct words per minute (Based on DIBELs Next National Norms)

PROBLEM: In order to bring the student to standard, how many corrects words read aloud must the student increase? If you provide intensive instruction over 18 weeks, how many words per weeks should the student increase each week of instruction?

124 – 60 = 64 WORDS OVER 18 WEEKS64 divided by 18 = 3.55 words per week

Is it possible to increase a students output by 3.5 words per week?

What would the instruction look like and for how long a period of time for each session?

90 – 60 = 30 WORDS OVER 18 WEEKS60 words divided by 18 weeks = 1.67 words per week.

As a teacher, can I grow a student 1-2 words per week? What would the instruction look like and for how long a period of time for each session?

RATES OF WEEKLY IMPROVEMENT

Three things to keep in mind when using ROI for goal setting:

1. What research says are “realistic” and “ambitious” growth rates

2. What norms indicate about “good” growth rates

3. Local versus national norms49

PLANNING FOR PROGRESS MONITORING

50

Timeframe Throughout instruction at regular intervals

(e.g., weekly, bi-weekly, monthly)

Teachers use student data to quantify short- and long-term goals that will meet end-of-year goals

FREQUENCY OF PROGRESS MONITORING Should occur at least monthly.

Ideal: 2x per month at secondary level Ideal: 1-2 x per week at tertiary level

As the number of data points increases, the effects of measurement error on the trend line decreases.

Christ & Silberglitt (2007) recommended six to nine data points.

51

ESTABLISHING THE BASELINE SCORE

To begin progress monitoring you need to know the student’s initial knowledge level or baseline knowledge

Having a stable baseline is important for goal setting

To establish the baseline use the median scores of three probes. (You may choose to use screening data for this, if progress monitoring occurs at the student’s chronological grade level.)

52

CONSIDERATIONS FOR USING THE INTRA-INDIVIDUAL FRAMEWORK Typically used for setting IEP goals and is not very appropriate for students performing at or near grade level.

Since the student’s performance is being compared to his/her previous performance (not a national or local norm) we need to have enough data to demonstrate the existing performance level or rate, which is why at least 8 data points are needed.

Recommended data collection 2x per week to obtain sufficient data points when this option is used.

53

GRAPHING PROGRESS MONITORING DATA

Graphed data allows teachers to quantify rate of student improvement:

• Increasing scores indicate student is making progress and responding to the curriculum.

• Flat or decreasing scores indicate non-response.• Student is not benefiting from instruction and requires a change in the instructional program.

54

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Instruction

Pro

ble

ms C

orr

ect

in 3

M

inu

tes

GRAPHING CBM SCORES

55

The vertical axis is labeled with the range of student scores.

The horizontal axis is labeled with the number of instructional weeks.

TREND LINE, SLOPE, AND ROITrend Line – a line through the scores that visually represents the performance trend

Rate of Improvement (ROI) - specifies the improvement, or average weekly increases, based on a line of best fit through the student’s scores.

Slope – quantification of the trend line, or the rate of improvement (ROI)

57

COLLECTING DATA IS GREAT…

But using data to make instructional decisions is the MOST important.

Select a decision making rule and stick with it.

58

WHY USE PM DATA? Identify students who aren’t making progress and need additional assessment and instruction

Confirm or disconfirm screening dataEvaluate effectiveness of interventions and instruction

Allocate resourcesEvaluate effectiveness of instruction programs for target

groups (e.g., ELL, Title 1)

59

DECISION RULES BASED ON FOUR-POINT METHOD If three weeks of instruction have occurred AND at least six data points have been collected, examine the four most recent data points.• If all four are above goal line, increase goal.• If all four are below goal line, make an instructional change.• If the four data points are both above and below the goal line, keep collecting data until trend line rule or four- point rule can be applied.

60

DECISION RULES BASED ON THE TREND LINE: If the student’s trend line is steeper than the goal line, the

student’s end-of-year performance goal needs to be increased.

If the student’s trend line is flatter than the goal line, the teacher needs to revise the instructional program.

If the student’s trend line and goal line are the same, no changes need to be made.

61

DECISION RULES SUMMARY

Four-point rule—easy to implement, but not as sensitive

The trend line rule—more sensitive to changes, but requires calculation to obtain

63

CONTINUE PROGRESS MONITORING

Follow a set data collection scheduleCommunicating purpose of data collection AND results regularly•Share with parents, teachers, and students

Dissemination with discussion is preferred•Encourage all school teams to talk about results, patterns, possible interpretations, and likely next steps.

64

1 2 3 4 5 6 7 8 9 10 11 12 130

10

20

30

40

50

60

70

80

90

100

110

120

Elizabeth's probe results

probe number

nu

mb

er o

f co

rrec

t d

igit

s in

7 m

in.

Date

D D G K G S N K AJ J E B K B K M D E M DW

Harmon

y1.23.08 18 32 34 44 51 44 61  64 55 60 28 49 67 78 43 75 48 49 67 92 532.11.08 66 71 52 83 60 70 65 58 67 60 75 29 57 91 93 94 74  68 67 84 692.19.08 66 87 59 89 82 47 75 55 64 66 87 59 70  86 78 76 89 71 84 86 742.25.08 59 48 60 73    70 55 76 64 90 53 55 82 103 66 77 74 64 68 87 703.3.08 72 81 63 82 102 86 77 89  85 107 61 90 81 95 102 95 85 33 100 95 843.10.08 80 83 60 98 101 91 100 83 86 87 106 61 75 100 104 87 87 97 60 91 94 873.18.08 48 89 63 92 75 59 78 81 89 109 103  81 104 77 87 102 87 47  105 833.24.08 61 93  77 90 52 93 72 90 77 112 52 64 99 99 70 128 86  77 111 844.7.08 75 91 52 80 89 59 90 72 91 93 116 39 71 113 100 86 108 65 59 107 114 844.14.08 79 103 78 108 99 85 95 98 103 99 120 72 93 117 104 89 113 115 46 108 100 965.5.08 75 98 93 112 100 75 88 105 78 98 100 78 90 80 96 107 81 107 80 98 94 925.20.08 81  29 96 73 56 75 80 99 112 109  66 103 113 74 87 85 76 114 110 866.2.08 93  79 109    106 88 91 107 125 32 88 123 116 89 107 103 79 123 107 98                                            #DIV/0!                                            #DIV/0!                                            #DIV/0!                                            #DIV/0!                                            #DIV/0!5.20.08 50  27 58 35 34 42 48 66 57 59  38 63 70 41 72 43 47 67 66 526.2.08 58  49 68    66 65 63 66 75 22 57 75 75 58 67 66 50 70 63 62                                            #DIV/0!                                            #DIV/0!

QUESTIONS??

We are now going to reconfigure our working partnerships for the upcoming Measurable Annual Goal Work

PASEONumber off by 2’sOne’s will form an outer circle facing in and Two’s will form a circle inside of the One’s and face outward.

Activity

Pack and stack to a new space with your new partner for after lunch activities.