Graphing, Calculating, and Interpreting Rate of Improvement Caitlin S. Flinn, Ed.S., N.C.S.P. Andrew...

Graphing, Calculating, and Graphing, Calculating, and Interpreting Rate of Interpreting Rate of

ImprovementImprovement

Caitlin S. Flinn, Ed.S., N.C.S.P.Caitlin S. Flinn, Ed.S., N.C.S.P.

Andrew E. McCrea, M.S., N.C.S.P.Andrew E. McCrea, M.S., N.C.S.P.

PaTTAN RtII InstitutePaTTAN RtII Institute

June 14, 2010June 14, 2010

ObjectivesObjectives

There needs to be a standardized There needs to be a standardized procedure for calculating RoI procedure for calculating RoI

We’re proposing a method using Simple We’re proposing a method using Simple Linear RegressionLinear Regression

OverviewOverview

Importance of RoIImportance of RoI RoI FoundationsRoI Foundations A Need for A Need for

ConsistencyConsistency Calculating RoICalculating RoI

Individual Student Individual Student GraphsGraphs

Programming ExcelProgramming Excel

Decision MakingDecision Making Grounding the DataGrounding the Data Interpreting GrowthInterpreting Growth

Individual StudentIndividual Student Student GroupsStudent Groups

ConsiderationsConsiderations ResourcesResources

Importance of GraphsImportance of Graphs

Vogel, Dickson, & Lehman, 1990Vogel, Dickson, & Lehman, 1990Speeches that included visuals, especially in Speeches that included visuals, especially in

color, improved:color, improved:Immediate recall by 8.5%Immediate recall by 8.5%Delayed recall (3 days) by 10.1%Delayed recall (3 days) by 10.1%

Importance of GraphsImportance of Graphs

““Seeing is believing.”Seeing is believing.”Useful for communicating large amounts Useful for communicating large amounts

of information quicklyof information quickly ““A picture is worth a thousand words.”A picture is worth a thousand words.”Transcends language barriers (Karwowski, Transcends language barriers (Karwowski,

2006)2006)Responsibility for accurate graphical Responsibility for accurate graphical

representations of datarepresentations of data

Skills Typically GraphedSkills Typically Graphed ReadingReading

Oral Reading Fluency (ORF)Oral Reading Fluency (ORF) Word Use Fluency (WUF)Word Use Fluency (WUF) Reading ComprehensionReading Comprehension

MAZEMAZE Retell FluencyRetell Fluency

Early Literacy SkillsEarly Literacy Skills Initial Sound Fluency (ISF)Initial Sound Fluency (ISF) Letter Naming Fluency (LNF)Letter Naming Fluency (LNF) Letter Sound Fluency (LSF)Letter Sound Fluency (LSF) Phoneme Segmentation Fluency Phoneme Segmentation Fluency

(PSF)(PSF) Nonsense Word Fluency (NWF)Nonsense Word Fluency (NWF)

SpellingSpelling Written Expression Written Expression BehaviorBehavior

MathMath Math ComputationMath Computation Math FactsMath Facts Early NumeracyEarly Numeracy

Oral CountingOral Counting Missing NumberMissing Number Number IdentificationNumber Identification Quantity Quantity

DiscriminationDiscrimination

Importance of RoIImportance of RoI

Multi-tiered modelMulti-tiered modelProgress monitoringProgress monitoringData for decision-makingData for decision-makingGoal setting (Shapiro, 2008)Goal setting (Shapiro, 2008)

Importance of RoIImportance of RoI

Visual inspection of slopeVisual inspection of slopeMultiple interpretationsMultiple interpretations Instructional servicesInstructional servicesNeed for explicit guidelinesNeed for explicit guidelines

RoI FoundationsRoI Foundations

Deno, 1985Deno, 1985Curriculum-based measurementCurriculum-based measurement

General outcome measuresGeneral outcome measuresShortShortStandardizedStandardizedRepeatableRepeatableSensitive to changeSensitive to change


Fuchs & Fuchs, 1998Fuchs & Fuchs, 1998Hallmark components of Response to Hallmark components of Response to

InterventionInterventionOngoing formative assessmentOngoing formative assessmentIdentifying non-responding studentsIdentifying non-responding studentsTreatment fidelity of instructionTreatment fidelity of instruction

Dual discrepancy modelDual discrepancy modelOne standard deviation from typically performing One standard deviation from typically performing

peers in level and peers in level and raterate


Ardoin & Christ, 2008Ardoin & Christ, 2008Slope for benchmarks (3x per year)Slope for benchmarks (3x per year)More growth from fall to winter than winter to More growth from fall to winter than winter to

springspringMight be helpful to use RoI for fall to winterMight be helpful to use RoI for fall to winterAnd a separate RoI for winter to springAnd a separate RoI for winter to spring


Fuchs, Fuchs, Walz, & Germann, 1993Fuchs, Fuchs, Walz, & Germann, 1993Typical weekly growth ratesTypical weekly growth ratesNeeded growthNeeded growth

1.5 to 2.0 times typical slope to close gap1.5 to 2.0 times typical slope to close gap

ExampleExampleBob is below benchmark on ORFBob is below benchmark on ORFTypical slope is 1 wcpm per week growthTypical slope is 1 wcpm per week growthBob would need slope of 1.5 to 2 to close gap Bob would need slope of 1.5 to 2 to close gap

in a reasonable amount of timein a reasonable amount of time


Deno, Fuchs, Marston, & Shin, 2001Deno, Fuchs, Marston, & Shin, 2001Slope of frequently non-responsive children Slope of frequently non-responsive children

approximated slope of children already approximated slope of children already identified as having a specific learning identified as having a specific learning disabilitydisability

RoI DefinitionRoI Definition

Algebraic term: Slope of a lineAlgebraic term: Slope of a lineVertical change over the horizontal changeVertical change over the horizontal changeRise over runRise over runm = (ym = (y2 2 - y- y11) / (x) / (x2 2 - x- x11))Describes the steepness of a line (Gall & Gall, Describes the steepness of a line (Gall & Gall,

2007)2007)

RoI DefinitionRoI Definition

Finding a student’s RoI = finding the slope Finding a student’s RoI = finding the slope of a lineof a lineUsing two data points on that lineUsing two data points on that line

Finding the line itselfFinding the line itselfLinear regressionLinear regressionOrdinary Least SquaresOrdinary Least Squares

RoI & StatisticsRoI & Statistics

Gall & Gall, 2007Gall & Gall, 200710 data points are a minimum requirement for 10 data points are a minimum requirement for

a reliable trendlinea reliable trendlineHow does that affect the frequency of How does that affect the frequency of

administering progress monitoring probes?administering progress monitoring probes?

Ongoing ResearchOngoing Research

Using RoI for instructional decisions is not Using RoI for instructional decisions is not a perfect processa perfect process

Research is currently looking to address Research is currently looking to address sources of error:sources of error:Christ, 2006 – standard error of measurement Christ, 2006 – standard error of measurement

for slopefor slopeArdoin & Christ, 2009 – passage difficulty and Ardoin & Christ, 2009 – passage difficulty and

variabilityvariabilityJenkin, Graff, & Miglioretti, 2009 – frequency Jenkin, Graff, & Miglioretti, 2009 – frequency

of progress monitoringof progress monitoring

Future ConsiderationsFuture Considerations

Questions yet to be empirically answeredQuestions yet to be empirically answeredWhat parameters of RoI indicate a lack of RtI?What parameters of RoI indicate a lack of RtI?How does standard error of measurement How does standard error of measurement

play into using RoI for instructional decision play into using RoI for instructional decision making?making?

How does RoI vary between standard protocol How does RoI vary between standard protocol interventions?interventions?

How does this apply to non-English speaking How does this apply to non-English speaking populations?populations?

How is RoI Calculated? How is RoI Calculated? Which way is best?Which way is best?

Multiple Methods for Multiple Methods for Calculating GrowthCalculating Growth

““Eye ball” ApproachEye ball” Approach Last point minus First point ApproachLast point minus First point Approach Split Middle ApproachSplit Middle Approach Linear Regression ApproachLinear Regression Approach

Eye Ball

8

10

7

17

14

11

19

14

0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8

Last minus First

8

10

7

17

14

11

19

14

0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8

14 - 8 = 6; 6/ 8 weeks = 0.75 words per week

Split Middle

10.5

14.5

0

2

4

6

8

10

12

14

16

18

1 2 3 4 5 6 7 8 9 10

14.5-10.5 = 4; 4/8 weeks = 0.5 words per week

Linear Regression

8

10

7

17

14

11

19

14

y = 1.1429x + 7.3571

0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8

1.1 Words Per Week

RoI Consistency?RoI Consistency?

Eye BallEye Ball ??????

Last minus FirstLast minus First 0.750.75

Split Middle*Split Middle* 0.500.50

Linear Linear RegressionRegression

1.101.10

RoI Consistency?RoI Consistency?

If we are not all using the same model to If we are not all using the same model to compute RoI, we continue to have the same compute RoI, we continue to have the same problems as past models, where under one problems as past models, where under one approach a student meets SLD criteria, but approach a student meets SLD criteria, but under a different approach, the student does under a different approach, the student does not. not.

Hypothetically, if the RoI cut-off was 0.65 or Hypothetically, if the RoI cut-off was 0.65 or 0.95, different approaches would come to 0.95, different approaches would come to different conclusions on the same student. different conclusions on the same student.

Technical AdequacyTechnical Adequacy

Without a consensus on how to compute Without a consensus on how to compute RoI, we risk falling short of having RoI, we risk falling short of having technical adequacy within our model. technical adequacy within our model.

So, Which RoI Method is Best?So, Which RoI Method is Best?

Literature shows that Linear Literature shows that Linear Regression is Best PracticeRegression is Best Practice

Student’s daily test scores…were entered into a Student’s daily test scores…were entered into a computer program…The data analysis program computer program…The data analysis program generated generated slopes of improvementslopes of improvement for each level for each level using and using and Ordinary-Least SquaresOrdinary-Least Squares procedure procedure (Hayes, 1973) and the line of best fit. (Hayes, 1973) and the line of best fit.

This This procedure has been demonstrated to procedure has been demonstrated to represent CBM achievement data validly within represent CBM achievement data validly within individual treatment phasesindividual treatment phases (Marston, 1988; (Marston, 1988; Shinn, Good, & Stein, in press; Stein, 1987).Shinn, Good, & Stein, in press; Stein, 1987).

Shinn, Gleason, & Tindal, 1989Shinn, Gleason, & Tindal, 1989

Growth (RoI) Research Growth (RoI) Research using Linear Regressionusing Linear Regression

Christ, T. J. (2006). Short-term estimates of growth using Christ, T. J. (2006). Short-term estimates of growth using curriculum based measurement of oral reading fluency: curriculum based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence Estimating standard error of the slope to construct confidence intervals. intervals. School Psychology ReviewSchool Psychology Review, 35, 128-133., 35, 128-133.

Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum based measurement to establish growth standards curriculum based measurement to establish growth standards for students with learning disabilities. for students with learning disabilities. School Psychology School Psychology ReviewReview, 30, 507-524., 30, 507-524.

Good, R. H. (1990). Forecasting accuracy of slope estimates for Good, R. H. (1990). Forecasting accuracy of slope estimates for reading curriculum based measurement: Empirical evidence. reading curriculum based measurement: Empirical evidence. Behavioral AssessmentBehavioral Assessment, 12, 179-193., 12, 179-193.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L. & Germann, G. Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L. & Germann, G. (1993). Formative evaluation of academic progress: How much (1993). Formative evaluation of academic progress: How much growth can we expect? growth can we expect? School Psychology ReviewSchool Psychology Review, 22, 27-48., 22, 27-48.

Growth (RoI) ResearchGrowth (RoI) Researchusing Linear Regressionusing Linear Regression

Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. (2009). Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. (2009). Estimating reading growth using intermittent CBM Estimating reading growth using intermittent CBM progress monitoring. progress monitoring. Exceptional ChildrenExceptional Children, 75, 151-163., 75, 151-163.

Shinn, M. R., Gleason, M. M., & Tindal, G. (1989). Shinn, M. R., Gleason, M. M., & Tindal, G. (1989). Varying the difficulty of testing materials: Implications for Varying the difficulty of testing materials: Implications for curriculum-based measurement. curriculum-based measurement. The Journal of Special The Journal of Special EducationEducation, 23, 223-233., 23, 223-233.

Shinn, M. R., Good, R. H., & Stein, S. (1989). Shinn, M. R., Good, R. H., & Stein, S. (1989). Summarizing trend in student achievement: A Summarizing trend in student achievement: A comparison of methods. comparison of methods. School Psychology ReviewSchool Psychology Review, 18, , 18, 356-370.356-370.

So, Why Are There So Many So, Why Are There So Many Other RoI Models?Other RoI Models?

Ease of application Ease of application How many of us want to calculate OLS How many of us want to calculate OLS

Linear Regression formulas (or even Linear Regression formulas (or even remember how)?remember how)?

An Easy and An Easy and Applicable SolutionApplicable Solution

Get Out Your Laptops!Get Out Your Laptops!

Open Microsoft ExcelOpen Microsoft Excel

I loveROI

Graphing RoIGraphing RoIFor Individual StudentsFor Individual Students

Programming Microsoft Excel to Programming Microsoft Excel to Graph Rate of Improvement: Graph Rate of Improvement:

Fall to WinterFall to Winter

Setting Up Your SpreadsheetSetting Up Your Spreadsheet

In cell A1, type In cell A1, type 3rd Grade ORF3rd Grade ORF In cell A2, type In cell A2, type First SemesterFirst Semester In cell A3, type In cell A3, type School WeekSchool Week In cell A4, type In cell A4, type BenchmarkBenchmark In cell A5, type the Student’s Name In cell A5, type the Student’s Name

(Swiper Example)(Swiper Example)

Labeling School WeeksLabeling School Weeks

Starting with cell B3, type numbers Starting with cell B3, type numbers 11 through through 1818 going across row 3 going across row 3 (horizontal).(horizontal).

Numbers 1 through 18 represent the Numbers 1 through 18 represent the number of the school week.number of the school week.

You will end with week 18 in cell S3.You will end with week 18 in cell S3.

Labeling DatesLabeling Dates

NoteNote: You may choose to enter the date of : You may choose to enter the date of that school week across row 2 to easily that school week across row 2 to easily identify the school week.identify the school week.

Entering BenchmarksEntering Benchmarks(3rd Grade ORF)(3rd Grade ORF)

In cell B4, type In cell B4, type 7777. . This is your fall This is your fall benchmark.benchmark.

In cell S4, type In cell S4, type 9292. . This is your winter This is your winter benchmark.benchmark.

Entering Student Data (Sample)Entering Student Data (Sample)

Enter the following Enter the following numbers, going numbers, going across row 5, under across row 5, under corresponding week corresponding week numbers.numbers.

Week 1 – 41Week 1 – 41 Week 8 – 62Week 8 – 62 Week 9 – 63Week 9 – 63 Week 10 – 75Week 10 – 75 Week 11 – 64Week 11 – 64

Week 12 – 80Week 12 – 80 Week 13 – 83Week 13 – 83 Week 14 – 83Week 14 – 83 Week 15 – 56Week 15 – 56 Week 17 – 104Week 17 – 104 Week 18 – 74Week 18 – 74

*CAUTION**CAUTION*

If a student was not assessed during a If a student was not assessed during a certain week, leave that cell blankcertain week, leave that cell blank

Do not enter a score of Zero (0) it will be Do not enter a score of Zero (0) it will be calculated into the trendline and calculated into the trendline and interpreted as the student having read interpreted as the student having read zero words correct per minute during that zero words correct per minute during that week. week.

Graphing the DataGraphing the Data

Highlight cells A4 and A5 through S4 and Highlight cells A4 and A5 through S4 and S5S5

Follow Excel 2003 or Excel 2007 Follow Excel 2003 or Excel 2007 directions from heredirections from here


Excel 2003Excel 2003 Across the top of your Across the top of your

worksheet, click on worksheet, click on “Insert”“Insert”

In that drop-down In that drop-down menu, click on “Chart”menu, click on “Chart”

Excel 2007Excel 2007 Click Click InsertInsert Find the icon for Find the icon for LineLine Click the arrow below Click the arrow below

LineLine


Excel 2003Excel 2003 A Chart Wizard A Chart Wizard

window will appearwindow will appear

Excel 2007Excel 2007 6 graphics appear6 graphics appear


Excel 2003Excel 2003 Choose “Line”Choose “Line” Choose “Line with Choose “Line with

markers…”markers…”

Excel 2007Excel 2007 Choose “Line with Choose “Line with

markers”markers”


Excel 2003Excel 2003 ““Data Range” tabData Range” tab ““Columns”Columns”

Excel 2007Excel 2007 Your graph appearsYour graph appears


Excel 2003Excel 2003 ““Chart Title”Chart Title” ““School Week” X AxisSchool Week” X Axis ““WPM’ Y AxisWPM’ Y Axis

Excel 2007Excel 2007 Change your labels by Change your labels by

right clicking on the right clicking on the graphgraph


Excel 2003Excel 2003 Choose where you Choose where you

want your graphwant your graph

Excel 2007Excel 2007 Your graph was Your graph was

automatically put into automatically put into your data spreadsheetyour data spreadsheet

Graphing the TrendlineGraphing the Trendline

Excel 2003Excel 2003Right click on any of the student data pointsRight click on any of the student data points

Excel 2007Excel 2007


Excel 2003Excel 2003Choose “Linear”Choose “Linear”



Excel 2003Excel 2003Choose “Custom” and check box next to Choose “Custom” and check box next to

“Display equation on chart”“Display equation on chart”



Clicking on the equation highlights a box Clicking on the equation highlights a box around itaround it

Clicking on the box allows you to move it Clicking on the box allows you to move it to a place where you can see it betterto a place where you can see it better


You can repeat the same procedure to You can repeat the same procedure to have a trendline for the benchmark data have a trendline for the benchmark data pointspoints

Suggestion: label the trendline Suggestion: label the trendline Expected Expected ROIROI

Move this equation under the firstMove this equation under the first

Individual Student GraphIndividual Student Graphy = 2.5138x + 42.113

y = 0.8824x + 76.118

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Benchmark

Student: Sw iper

RoI

Linear (Benchmark)

Individual Student GraphIndividual Student Graph

The equation indicates the slope, or rate of The equation indicates the slope, or rate of improvement. improvement.

The number, or coefficient,The number, or coefficient, beforebefore "x" is "x" is the average improvement, which in this the average improvement, which in this case is the average number of words per case is the average number of words per minute per week gained by the student. minute per week gained by the student.


The rate of improvement, or trendline, is The rate of improvement, or trendline, is calculated using a linear regression, a calculated using a linear regression, a simple equation of least squares. simple equation of least squares.

To add additional progress To add additional progress monitoring/benchmark scores once you’ve monitoring/benchmark scores once you’ve already created a graph, enter additional already created a graph, enter additional scores in Row 5 in the corresponding scores in Row 5 in the corresponding school week.school week.


Remember to leave cells blank for the Remember to leave cells blank for the weeks that no score was obtained. weeks that no score was obtained. Otherwise, the graph will incorporate that Otherwise, the graph will incorporate that score into the set of data points and into score into the set of data points and into the trendline.the trendline.


The slope can change depending on The slope can change depending on which week (where) you put the which week (where) you put the benchmark scores on your chart. benchmark scores on your chart.

Enter benchmark scores based on when Enter benchmark scores based on when your school administers their benchmark your school administers their benchmark assessments for the most accurate assessments for the most accurate depiction of expected student progress.depiction of expected student progress.

Options for the GraphOptions for the Graph

ResizingResizingColoringColoringData LabelsData Labels

Programming ExcelProgramming ExcelFirst SemesterFirst Semester

Calculating Needed RoICalculating Needed RoI

Calculating Actual RoI – BenchmarkCalculating Actual RoI – Benchmark

Calculating Actual RoI - StudentCalculating Actual RoI - Student

Calculating Needed RoICalculating Needed RoI In cell T3, type In cell T3, type Needed RoINeeded RoI Click on cell T5Click on cell T5 In the In the fxfx line (at top of sheet) type this formula line (at top of sheet) type this formula

=((S4-B5)/18)=((S4-B5)/18) Then hit enterThen hit enter Your result should read: 2.83333...Your result should read: 2.83333... This formula simply subtracts the student’s This formula simply subtracts the student’s

actual beginning of year (BOY) benchmark from actual beginning of year (BOY) benchmark from the expected middle of year (MOY) benchmark, the expected middle of year (MOY) benchmark, then dividing by 18 for the first 18 weeks (1st then dividing by 18 for the first 18 weeks (1st semester).semester).

Calculating Actual RoI - BenchmarkCalculating Actual RoI - Benchmark

In cell U3, type In cell U3, type Actual RoIActual RoIClick on cell U4Click on cell U4 In the In the fxfx line (at top of sheet) type this line (at top of sheet) type this

formula =SLOPE(B4:S4,B3:S3)formula =SLOPE(B4:S4,B3:S3)Then hit enterThen hit enterYour result should read: 0.8825...Your result should read: 0.8825...This formula considers 18 weeks of This formula considers 18 weeks of

benchmark data and provides an average benchmark data and provides an average growth or change per week.growth or change per week.


Click on cell U5Click on cell U5 In the In the fxfx line (at top of sheet) type this line (at top of sheet) type this

formula =SLOPE(B5:S5,B3:S3)formula =SLOPE(B5:S5,B3:S3)Then hit enterThen hit enterYour result should read: 2.5137...Your result should read: 2.5137...This formula considers 18 weeks of This formula considers 18 weeks of

student data and provides an average student data and provides an average growth or change per week.growth or change per week.

Graphing RoIGraphing RoIFor Individual StudentsFor Individual Students

Programming Microsoft Excel to Programming Microsoft Excel to Graph Rate of Improvement: Graph Rate of Improvement:

Winter to SpringWinter to Spring

Setting Up Your SpreadsheetSetting Up Your Spreadsheet

In cell A1, type In cell A1, type 3rd Grade ORF3rd Grade ORF In cell A2, type In cell A2, type Second SemesterSecond Semester In cell A3, type In cell A3, type School WeekSchool Week In cell A4, type In cell A4, type BenchmarkBenchmark In cell A5, type the Student’s Name In cell A5, type the Student’s Name

(Swiper Example)(Swiper Example)

Labeling School WeeksLabeling School Weeks

Starting with cell B3, type numbers Starting with cell B3, type numbers 11 through through 1818 going across row 3 going across row 3 (horizontal).(horizontal).

Numbers 1 through 18 represent the Numbers 1 through 18 represent the number of the school week.number of the school week.

You will end with week 18 in cell S3.You will end with week 18 in cell S3.

Labeling DatesLabeling Dates

NoteNote: You may choose to enter the date of : You may choose to enter the date of that school week across row 2 to easily that school week across row 2 to easily identify the school week.identify the school week.

Entering BenchmarksEntering Benchmarks(3rd Grade ORF)(3rd Grade ORF)

In cell B4, type In cell B4, type 9292. . This is your fall This is your fall benchmark.benchmark.

In cell S4, type In cell S4, type 110110. . This is your winter This is your winter benchmark.benchmark.

Entering Student Data (Sample)Entering Student Data (Sample)

Enter the following Enter the following numbers, going numbers, going across row 5, under across row 5, under corresponding week corresponding week numbers.numbers.

Week 1 – Week 1 – 7474 Week 3 – Week 3 – 8585 Week 4 – Week 4 – 8989 Week 5 – Week 5 – 6969 Week 6 – Week 6 – 8585

Week 7 – Week 7 – 9696 Week 8 – Week 8 – 9090 Week 9 – Week 9 – 8484 Week 10 – Week 10 – 106106 Week 11 – Week 11 – 9494 Week 15 – Week 15 – 100100

*CAUTION**CAUTION*

If a student was not assessed during a If a student was not assessed during a certain week, leave that cell blankcertain week, leave that cell blank

Do not enter a score of Zero (0) it will be Do not enter a score of Zero (0) it will be calculated into the trendline and calculated into the trendline and interpreted as the student having read interpreted as the student having read zero words correct per minute during that zero words correct per minute during that week. week.


Highlight cells A4 and A5 through S4 and Highlight cells A4 and A5 through S4 and S5S5

Follow Excel 2003 or Excel 2007 Follow Excel 2003 or Excel 2007 directions from heredirections from here


Excel 2003Excel 2003 Across the top of your Across the top of your

worksheet, click on worksheet, click on “Insert”“Insert”

In that drop-down In that drop-down menu, click on “Chart”menu, click on “Chart”

Excel 2007Excel 2007 Click Click InsertInsert Find the icon for Find the icon for LineLine Click the arrow below Click the arrow below

LineLine


Excel 2003Excel 2003 A Chart Wizard A Chart Wizard

window will appearwindow will appear

Excel 2007Excel 2007 6 graphics appear6 graphics appear


Excel 2003Excel 2003 Choose “Line”Choose “Line” Choose “Line with Choose “Line with

markers…”markers…”

Excel 2007Excel 2007 Choose “Line with Choose “Line with

markers”markers”


Excel 2003Excel 2003 ““Data Range” tabData Range” tab ““Columns”Columns”

Excel 2007Excel 2007 Your graph appearsYour graph appears


Excel 2003Excel 2003 ““Chart Title”Chart Title” ““School Week” X AxisSchool Week” X Axis ““WPM’ Y AxisWPM’ Y Axis

Excel 2007Excel 2007 Change your labels by Change your labels by

right clicking on the right clicking on the graphgraph


Excel 2003Excel 2003 Choose where you Choose where you

want your graphwant your graph

Excel 2007Excel 2007 Your graph was Your graph was

automatically put into automatically put into your data spreadsheetyour data spreadsheet


Excel 2003Excel 2003Right click on any of the student data pointsRight click on any of the student data points



Excel 2003Excel 2003Choose “Linear”Choose “Linear”



Excel 2003Excel 2003Choose “Custom” and check box next to Choose “Custom” and check box next to

“Display equation on chart”“Display equation on chart”



Clicking on the equation highlights a box Clicking on the equation highlights a box around itaround it

Clicking on the box allows you to move it Clicking on the box allows you to move it to a place where you can see it betterto a place where you can see it better


You can repeat the same procedure to You can repeat the same procedure to have a trendline for the benchmark data have a trendline for the benchmark data pointspoints

Suggestion: label the trendline Suggestion: label the trendline Expected Expected ROIROI

Move this equation under the firstMove this equation under the first

Individual Student GraphIndividual Student Graphy = 1.8872x + 74.81

y = 1.0588x + 90.941

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Benchmark

Student: Sw iper

Rate of Improvement

Expected RoI


The equation indicates the slope, or rate of The equation indicates the slope, or rate of improvement. improvement.

The number, or coefficient,The number, or coefficient, beforebefore "x" is "x" is the average improvement, which in this the average improvement, which in this case is the average number of words per case is the average number of words per minute per week gained by the student. minute per week gained by the student.


The rate of improvement, or trendline, is The rate of improvement, or trendline, is calculated using a linear regression, a calculated using a linear regression, a simple equation of least squares. simple equation of least squares.

To add additional progress To add additional progress monitoring/benchmark scores once you’ve monitoring/benchmark scores once you’ve already created a graph, enter additional already created a graph, enter additional scores in Row 5 in the corresponding scores in Row 5 in the corresponding school week.school week.


Remember to leave cells blank for the Remember to leave cells blank for the weeks that no score was obtained. weeks that no score was obtained. Otherwise, the graph will incorporate that Otherwise, the graph will incorporate that score into the set of data points and into score into the set of data points and into the trendline.the trendline.


The slope can change depending on The slope can change depending on which week (where) you put the which week (where) you put the benchmark scores on your chart. benchmark scores on your chart.

Enter benchmark scores based on when Enter benchmark scores based on when your school administers their benchmark your school administers their benchmark assessments for the most accurate assessments for the most accurate depiction of expected student progress.depiction of expected student progress.

Options for the GraphOptions for the Graph

ResizingResizingColoringColoringData LabelsData Labels

Programming ExcelProgramming ExcelSecond SemesterSecond Semester

Calculating Needed RoICalculating Needed RoI

Calculating Actual RoI – BenchmarkCalculating Actual RoI – Benchmark


Calculating Needed RoICalculating Needed RoI In cell T3, type In cell T3, type Needed RoINeeded RoI Click on cell T5Click on cell T5 In the In the fxfx line (at top of sheet) type this formula line (at top of sheet) type this formula

=((S4-B5)/18)=((S4-B5)/18) Then hit enterThen hit enter Your result should read: 2Your result should read: 2 This formula simply subtracts the student’s This formula simply subtracts the student’s

actual middle of year (MOY) benchmark from the actual middle of year (MOY) benchmark from the expected end of year (EOY) benchmark, then expected end of year (EOY) benchmark, then dividing by 18 for the first 18 weeks (1st dividing by 18 for the first 18 weeks (1st semester).semester).

Calculating Actual RoI - BenchmarkCalculating Actual RoI - Benchmark

In cell U3, type In cell U3, type Actual RoIActual RoIClick on cell U4Click on cell U4 In the In the fxfx line (at top of sheet) type this line (at top of sheet) type this

formula =SLOPE(B4:S4,B3:S3)formula =SLOPE(B4:S4,B3:S3)Then hit enterThen hit enterYour result should read: 1.06Your result should read: 1.06This formula considers 18 weeks of This formula considers 18 weeks of

benchmark data and provides an average benchmark data and provides an average growth or change per week.growth or change per week.


Click on cell U5Click on cell U5 In the In the fxfx line (at top of sheet) type this line (at top of sheet) type this

formula =SLOPE(B5:S5,B3:S3)formula =SLOPE(B5:S5,B3:S3)Then hit enterThen hit enterYour result should read: 1.89Your result should read: 1.89This formula considers 18 weeks of This formula considers 18 weeks of

student data and provides an average student data and provides an average growth or change per week.growth or change per week.

ROI as a Decision ToolROI as a Decision Tool

within a Problem-Solving Modelwithin a Problem-Solving Model

StepsSteps

1.1. Gather the dataGather the data

2.2. Ground the data & set goalsGround the data & set goals

3.3. Interpret the dataInterpret the data

4.4. Figure out how to fit Best Practice into Figure out how to fit Best Practice into Public EducationPublic Education

Step 1: Gather DataStep 1: Gather Data

Universal Screening Universal Screening

Progress MonitoringProgress Monitoring

Common Screenings in PACommon Screenings in PA

DIBELSDIBELSAIMSwebAIMSwebMBSPMBSP4Sight4SightPSSAPSSA

Validated Progress Validated Progress Monitoring ToolsMonitoring Tools

DIBELSDIBELSAIMSwebAIMSwebMBSPMBSPwww.studentprogress.org www.studentprogress.org

Step 2: Ground the DataStep 2: Ground the Data

1) To what will we compare our 1) To what will we compare our student growth data?student growth data?

2) How will we set goals?2) How will we set goals?

Multiple Ways to Multiple Ways to Look at GrowthLook at Growth

Needed Growth Needed Growth Expected Growth & Percent of Expected GrowthExpected Growth & Percent of Expected Growth Fuchs et. al. (1993) Table of Realistic and Fuchs et. al. (1993) Table of Realistic and

Ambitious GrowthAmbitious Growth Growth Toward Individual Goal*Growth Toward Individual Goal*

*Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008)*Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008)

Needed GrowthNeeded Growth

Difference between student’s BOY (or Difference between student’s BOY (or MOY) score and benchmark score at MOY MOY) score and benchmark score at MOY (or EOY).(or EOY).

Example: MOY ORF = 10, EOY Example: MOY ORF = 10, EOY benchmark is 40, 18 weeks of instruction benchmark is 40, 18 weeks of instruction (40-10/18=1.67). Student must gain 1.67 (40-10/18=1.67). Student must gain 1.67 wcpm per week to make EOY benchmark.wcpm per week to make EOY benchmark.

Expected GrowthExpected Growth

Difference between two benchmarks.Difference between two benchmarks.Example: MOY benchmark is 20, EOY Example: MOY benchmark is 20, EOY

benchmark is 40, expected growth (40-benchmark is 40, expected growth (40-20)/18 weeks of instruction = 1.11 wcpm 20)/18 weeks of instruction = 1.11 wcpm per week.per week.

Tigard-Tualatin School District (www.ttsd.k12.or.us)Tigard-Tualatin School District (www.ttsd.k12.or.us)

Looking at Percent of Looking at Percent of Expected GrowthExpected Growth

Tier I Tier II Tier III

Greater than 150%

Between 110% & 150%

Possible LD

Between 95% & 110%

Likely LD

Between 80% & 95%

May Need More

May Need More

Likely LD

Below 80% Needs More Needs More Likely LD

Fuchs, Fuchs, Hamlett, Walz, & Germann (1993)Fuchs, Fuchs, Hamlett, Walz, & Germann (1993)

Oral Reading Fluency Adequate Oral Reading Fluency Adequate Response TableResponse Table

Realistic Growth

Ambitious Growth

1st 2.0 3.0

2nd 1.5 2.0

3rd 1.0 1.5

4th 0.9 1.1

5th 0.5 0.8

Fuchs, Fuchs, Hamlett, Walz, & Germann (1993)Fuchs, Fuchs, Hamlett, Walz, & Germann (1993)

Digit Fluency Adequate Digit Fluency Adequate Response TableResponse Table

Realistic Growth

Ambitious Growth

1st 0.3 0.5

2nd 0.3 0.5

3rd 0.3 0.5

4th 0.75 1.2

5th 0.75 1.2

From Where Should From Where Should Benchmarks/Criteria Come?Benchmarks/Criteria Come?

Appears to be a theoretical convergence Appears to be a theoretical convergence on use of local criteria (what scores do our on use of local criteria (what scores do our students need to have a high probability of students need to have a high probability of proficiency?) when possible. proficiency?) when possible.

Steps to Develop Local CriteriaSteps to Develop Local Criteria

Not enough time today!Not enough time today!See us in State College in the fall, orSee us in State College in the fall, orCheck out our website later this summer.Check out our website later this summer.

If Local Criteria are Not an If Local Criteria are Not an OptionOption

Use norms that accompany the measure Use norms that accompany the measure (DIBELS, AIMSweb, etc.).(DIBELS, AIMSweb, etc.).

Use national norms.Use national norms.

Making Decisions: Best Practice Making Decisions: Best Practice

Research has yet to establish a blue print Research has yet to establish a blue print for ‘grounding’ student RoI data. for ‘grounding’ student RoI data.

At this point, teams should consider At this point, teams should consider multiple comparisons when planning and multiple comparisons when planning and making decisions. making decisions.

Making Decisions: Lessons Making Decisions: Lessons From the FieldFrom the Field

When tracking on grade level, consider an When tracking on grade level, consider an RoI that is 100% of RoI that is 100% of expectedexpected growth as a growth as a minimum requirement, consider an RoI minimum requirement, consider an RoI that is at or above the that is at or above the neededneeded as optimal. as optimal.

So, 100% of expected and on par with So, 100% of expected and on par with needed become the limits of the range needed become the limits of the range within a student should be achieving.within a student should be achieving.

Oral Reading Fluency01/15/09 01/22/09 01/29/09 02/05/09 02/12/09 02/19/09 02/26/09 03/05/09 03/12/09 03/19/09 03/26/09 04/02/09 04/09/09 04/16/09 04/23/09 04/30/09 05/07/09 05/14/09

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Benchmark 68 90 1.29

Aiden 61 40 52 60 71 95 1.61 2.17 167%

Ava 49 43 49 77 57 54 87 92 2.28 2.76 213%

Noah 49 48 45 69 61 54 84 2.28 2.01 156%

Olivia 65 49 57 70 79 83 1.39 1.50 116%

Liam 55 53 36 54 70 83 1.94 1.58 122%

Hannah 59 54 64 69 52 60 82 1.72 1.20 93%

Gavin 64 40 67 68 84 79 1.44 1.66 129%

Grace 53 48 46 60 74 79 2.06 1.76 136%

Oliver 50 44 46 68 51 51 57 78 2.22 1.45 112%

Peyton 63 50 47 58 75 77 1.50 1.12 87%

Josh 49 38 49 55 48 36 67 77 2.28 1.62 125%

Riley 42 49 54 69 67 50 76 2.67 1.76 136%

Mason 53 53 50 64 60 74 2.06 1.17 91%

Zoe 34 38 42 68 55 51 58 3.11 1.44 111%

Ian 41 31 45 49 47 30 46 2.72 0.24 19%

Faith 29 36 35 36 36 29 45 44 3.39 0.75 58%

David 30 23 44 52 43 19 63 38 3.33 0.79 61%

Alexa 18 19 25 33 33 23 28 37 4.00 0.94 73%

Hunter 23 23 24 48 38 32 34 3.72 0.75 58%

Caroline 28 20 28 40 37 19 25 30 3.44 0.02 2%

** Actual RoI based on linear regression of all data points

Benchmarks based on DIBELS Goals

Expected RoI at Benchmark Level

(Fuchs, Fuchs, Hamlett, Walz, & Germann 1993)

% of Expected RoI

Needed RoI* Actual RoI**

Realistic Grow thAmbitious Grow th

Oral Reading Fluency Adequate Response Table

1st Grade

2nd Grade

* Needed RoI based on difference betw een w eek 1 score and Benchmark score for w eek 18 divided by 18 w eeks

0.5

3rd Grade

4th Grade

5th Grade

0.9

0.8

2.0

1.5

1.0

3.0

2.0

1.5

1.1

Research Support Research Support

RoI by half year vs. whole year RoI by half year vs. whole year (Curvilinear Growth).(Curvilinear Growth).

Expected growth as mediated by initial Expected growth as mediated by initial level.level.

Example of Curvilinear Growth

0

10

20

30

40

50

60

70

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

Weeks

WC

PM

BOY to MOY = 1.60

MOY to EOY = 1.19

BOY to MOY = 1.60

MOY to BOY = 1.19

*BOY to EOY = 1.35

Ardoin & Christ, 2008Ardoin & Christ, 2008

Slope for benchmarks (3x per year)Slope for benchmarks (3x per year)More growth from fall to winter than winter More growth from fall to winter than winter

to springto spring

Christ, Yeo, & Silberglitt, in Christ, Yeo, & Silberglitt, in presspress

Growth across benchmarks (3X per year)Growth across benchmarks (3X per year)More growth from fall to winter than winter More growth from fall to winter than winter

to springto springDisaggregated special education Disaggregated special education

populationpopulation

Graney, Missall, & Martinez, Graney, Missall, & Martinez, 20092009

Growth across benchmarks (3X per year)Growth across benchmarks (3X per year)More growth from winter to spring than fall More growth from winter to spring than fall

to winter with R-CBM.to winter with R-CBM.

Fien, Park, Smith, & Baker, Fien, Park, Smith, & Baker, 2010 2010

Investigated relationship b/w NWF gains Investigated relationship b/w NWF gains and ORF/Comprehensionand ORF/Comprehension

Found greater NWF gains in fall than in Found greater NWF gains in fall than in spring.spring.

DIBELS ORF Change in CriteriaDIBELS ORF Change in Criteria

Fall to Winter Winter to Spring

2nd 24 22

3rd 15 18

4th 13 13

5th 11 9

6th 11 5

AIMSweb NormsAIMSweb Norms

Based on 50th Percentile

Fall to Winter Winter to Spring

1st 18 31

2nd 25 17

3rd 22 15

4th 16 13

5th 17 15

6th 13 12

Speculation as to why Differences Speculation as to why Differences in RoI within the Yearin RoI within the Year

Relax instruction after high stakes testing in Relax instruction after high stakes testing in March/April; March/April; a PSSA effecta PSSA effect. .

Depressed BOY benchmark scores due to Depressed BOY benchmark scores due to summer break; summer break; a rebound effecta rebound effect (Clemens). (Clemens).

Instructional variablesInstructional variables could explain differences could explain differences in Graney (2009) and Ardoin (2008) & Christ (in in Graney (2009) and Ardoin (2008) & Christ (in press) results (press) results (Silberglitt).Silberglitt).

VariabilityVariability within progress monitoring probes within progress monitoring probes ((Ardoin & Christ, 2008) Ardoin & Christ, 2008) (Lent).(Lent).

RoI ResearchRoI Research

Growth Mediated by LevelGrowth Mediated by Level

Fien, Park, Smith, & Baker, 2010Fien, Park, Smith, & Baker, 2010

Nonsense Word FluencyNonsense Word FluencyDifferent growth rates depending on Different growth rates depending on

beginning levelbeginning level

Clemens, 2010Clemens, 2010

Investigated NWF and WIFInvestigated NWF and WIF ““NWF slope validity increased as initial skills NWF slope validity increased as initial skills

were lower, but relationships with outcomes were lower, but relationships with outcomes similar to WIF”similar to WIF”

Silberglitt & Hintze, 2007Silberglitt & Hintze, 2007

R-CBMR-CBMDifferences in growth rates depending on Differences in growth rates depending on

levellevelLowest and highest deciles had least Lowest and highest deciles had least

amount of growthamount of growth

Good et. al., 2010Good et. al., 2010Growth Rate as Function of Level at BOY (2nd Grade)

20th 40th 60th 80th

Intensive 0 to 5 0.11 0.33 0.56 0.98

6 to 15 0.40 0.70 1.05 1.53

16 to 25 0.95 1.43 1.78 2.20

Strategic 26 to 34 1.30 1.73 2.06 2.43

35 to 43 1.50 1.83 2.11 2.50

Conclusions…Conclusions…

Appear to be different RoI within the Appear to be different RoI within the school year.school year.

Compute RoI goals by half-year (Fall to Compute RoI goals by half-year (Fall to Winter, Winter to Spring).Winter, Winter to Spring).

Actual or Expected RoI appears to differ Actual or Expected RoI appears to differ depending on the level at which a student depending on the level at which a student originally scores, which could have goal originally scores, which could have goal setting ramifications.setting ramifications.

Step 3: Interpreting GrowthStep 3: Interpreting Growth

What do we do when we do not What do we do when we do not get the growth we want?get the growth we want?

When to make a change in instruction and When to make a change in instruction and intervention?intervention?

When to consider SLD?When to consider SLD?

When to make a change in When to make a change in instruction and intervention?instruction and intervention?

Enough data points (6 to 10)?Enough data points (6 to 10)?Less than 100% of expected growth.Less than 100% of expected growth.Not on track to make benchmark (needed Not on track to make benchmark (needed

growth).growth).Not on track to reach individual goal.Not on track to reach individual goal.

When to consider SLD?When to consider SLD?

Continued inadequate response despite: Continued inadequate response despite: Fidelity with Tier I instruction and Tier Fidelity with Tier I instruction and Tier

II/III intervention.II/III intervention. Multiple attempts at intervention.Multiple attempts at intervention. Individualized Problem-Solving Individualized Problem-Solving

approach. approach.

Three Levels of ExamplesThree Levels of Examples

Whole ClassWhole ClassSmall GroupSmall Group Individual Student Individual Student

- Academic Data- Academic Data

- Behavior Data- Behavior Data

Whole Class ExampleWhole Class ExampleComputation

01/15/10 01/22/10 01/29/10 02/05/10 02/12/10 02/19/10 02/26/10 03/05/10 03/12/10 03/19/10 03/26/10 04/02/10 04/09/10 04/16/10 04/23/10 04/30/10 05/07/10 05/14/10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

50th P ercentile 25 31 0.35

25th P ercentile 19 23 0.24

Student 6.5 9 8 8.5 5.5 11 13 1.72 0.61 173%

Student 6 7.5 8.5 5 11 11.5 1.72 0.57 161%

Student 4.5 5.5 6.5 9.5 10.5 1.72 1.06 300%

Student 13 8 9.3 8 5.6 9.6 9.6 1.72 -0.23 -66%

Student 8 10.5 10.5 5.6 9.3 9 1.72 -0.03 -7%

Student 8.5 5.5 9 8 4 8 9 1.72 0.07 21%

Student 6.5 5.5 6 10.5 9 1.72 0.43 122%

Student 6.5 9 4.5 6 8 1.72 0.07 20%

Student 8 10.5 4.5 6.5 4 7 1.72 -0.25 -71%

Student 9 10 5.6 6.6 5 4.6 6.6 1.72 -0.42 -119%

Student 8 8 8.5 4 8 6.6 1.72 -0.18 -51%

Student 9 4.5 4.5 4 3.5 3.5 6.5 1.72 -0.24 -67%

Student 6.5 5 6.5 9 7.5 6.5 1.72 0.09 26%

Student 5.5 3 8 4 6.5 6.3 1.72 0.19 55%

Student 7.5 10 6.6 3.3 3 6.3 1.72 -0.46 -130%

Student 5 5.5 6.5 6 5 6 1.72 0.04 11%

Student 5 4 8 8.5 10 8 6 1.72 0.25 71%

Student 4.5 6 5 2.5 3.5 5.5 1.72 -0.03 -8%

Student 5.5 4.5 10.5 11 5 5.3 1.72 -0.14 -40%



Percentiles based on AIMSw eb Grow th Tables

Expected RoI at 50th PercentileExpected RoI at 25th Percentile

Needed RoI* Actual RoI** % of Expected RoI

Digit Fluency Adequate Response Table


1st Grade 0.3 0.5

2nd Grade 0.3 0.5

3rd Grade 0.3 0.5


4th Grade 0.75 1.2

5th Grade 0.75 1.2

33rdrd Grade Math Whole Class Grade Math Whole Class

Who’s responding? Who’s responding? Effective math Effective math

instruction? instruction? Who needs more?Who needs more?

N=19N=194 > 100% growth 4 > 100% growth 15 < 100% growth15 < 100% growth9 w/ negative 9 w/ negative

growthgrowth

Small Group ExampleSmall Group ExampleOral Reading Fluency

09/11/09 09/18/09 09/25/09 10/02/09 10/09/09 10/16/09 10/23/09 10/30/09 11/06/09 11/13/09 11/20/09 11/27/09 12/04/09 12/11/09 12/18/09 01/01/10 01/08/10 01/15/10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Benchmark 44 68 1.41

Student 35 39 41 45 42 45 52 57 62 1.83 1.49 106%

Student 28 38 42 40 50 55 64 72 74 2.22 2.77 196%

Student 26 28 32 31 27 29 35 34 38 2.33 0.57 41%

Student 31 35 39 45 42 47 53 58 65 2.06 1.90 135%

Student 40 44 38 48 52 64 72 74 78 1.56 2.62 186%


Benchmarks based on DIBELS Goals

Expected RoI at Benchmark Level


5th Grade


% of Expected RoI

2nd Grade

0.9 1.1

3rd Grade

4th Grade

0.5 0.8

1.5 2.0

1.0 1.5

Needed RoI* Actual RoI**

2.0 3.0

Oral Reading Fluency Adequte Response Table

1st Grade


Intervention GroupIntervention Group

Intervention working for how many?Intervention working for how many?Can we assume fidelity of intervention Can we assume fidelity of intervention

based on results?based on results?Who needs more?Who needs more?

Individual Kid ExampleIndividual Kid Example 2nd Grade Reading Progress

44

68

90

31

56

45

53 53

4846

60

74

79

y = 1.5333x + 42.8

y = 0.9903x + 36.873

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

09/12/08 09/19/0809/26/0810/03/08 10/10/08 10/17/08 10/24/08 10/31/08 11/07/08 11/14/08 11/21/08 11/28/08 12/05/08 12/12/08 12/19/08 01/16/09 01/23/0901/30/0902/06/0902/13/0902/20/0902/27/0903/06/0903/13/0903/20/0903/27/0904/03/0904/10/0904/17/0904/24/0905/01/09

Wo

rds

Re

ad

Co

rre

ct

Pe

r M

inu

te

Benchmark Linear (Benchmark) Linear

Individual KidIndividual Kid

Making growth?Making growth?How much (65% of expected growth).How much (65% of expected growth).Atypical growth across the year (last 3 Atypical growth across the year (last 3

data points). data points). Continue? Make a change? Need more Continue? Make a change? Need more

data?data?

RoI and Behavior?RoI and Behavior?

Percent of Time Engaged in Appropriate Behavior

y = 2x + 22

y = 3.9x + 19.8

y = 7.2143x - 1.5

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Per

cen

t

Baseline Condition 1 Condition 2 Linear (Baseline) Linear (Condition 1) Linear (Condition 2) Linear (Condition 2)

Step 4: Figure out how to fit Step 4: Figure out how to fit Best Practice into Public Best Practice into Public

EducationEducation

Things to ConsiderThings to Consider

Who is At-Risk and needs progress Who is At-Risk and needs progress monitoring?monitoring?

Who will collect, score, enter the data?Who will collect, score, enter the data?Who will monitor student growth, when, Who will monitor student growth, when,

and how often?and how often?What changes should be made to What changes should be made to

instruction & intervention? instruction & intervention? What about monitoring off of grade level?What about monitoring off of grade level?

Who is At-Risk and needs Who is At-Risk and needs progress monitoring?progress monitoring?

Below level on universal screeningBelow level on universal screening

Entering 4th Grade Example

DORF (110)

ISIP TRWM

(55)

4Sight (1235)

PSSA (1235)

Student A 115 58 1255 1232

Student B 85 48 1216 1126

Student C 72 35 1056 1048

Who will collect, score, and Who will collect, score, and enter the data?enter the data?

Using MBSP for math, teachers can Using MBSP for math, teachers can administer probes to whole class.administer probes to whole class.

DORF probes must be administered one-DORF probes must be administered one-on-one, and creativity pays off (train and on-one, and creativity pays off (train and use art, music, library, etc. specialists).use art, music, library, etc. specialists).

Schedule for progress monitoring math Schedule for progress monitoring math and reading every-other week.and reading every-other week.

Week 1 Week 2

Reading Math Reading Math

1st X X

2nd X X

3rd X X

4th X X

5th X X

Who will monitor student Who will monitor student growth, when, and how often?growth, when, and how often?

Best Practices in Data-Analysis Teaming Best Practices in Data-Analysis Teaming (Kovaleski & Pedersen, 2008)(Kovaleski & Pedersen, 2008)

Chambersburg Area School District Elementary Chambersburg Area School District Elementary Response to Intervention Manual (McCrea et. Response to Intervention Manual (McCrea et. al., 2008)al., 2008)

Derry Township School District Response to Derry Township School District Response to Intervention Model Intervention Model (http://www.hershey.k12.pa.us/56039310111408/lib/56039310111408/_files/Microsoft_Word_-(http://www.hershey.k12.pa.us/56039310111408/lib/56039310111408/_files/Microsoft_Word_-_Response_to_Intervention_Overview_of_Hershey_Elementary_Model.pdf)_Response_to_Intervention_Overview_of_Hershey_Elementary_Model.pdf)

What changes should be made What changes should be made to instruction & intervention?to instruction & intervention?

Ensure treatment fidelity!!!!!!!!Ensure treatment fidelity!!!!!!!! Increase instructional time (active and Increase instructional time (active and

engaged)engaged)Decrease group sizeDecrease group sizeGather additional, diagnostic, informationGather additional, diagnostic, informationChange the intervention Change the intervention

When Instructional Level is Not When Instructional Level is Not the Same as Grade Levelthe Same as Grade Level

Understand needed and expected RoI Understand needed and expected RoI within broader context: within broader context:

Needed growthNeeded growth will only get student will only get student to nextto next level by next benchmark (as opposed to level by next benchmark (as opposed to on on levellevel).).

100% of expected growth100% of expected growth may not be an may not be an acceptable minimum (not enough growth b/c acceptable minimum (not enough growth b/c level is so low).level is so low).

Grounding RoI When Monitoring Grounding RoI When Monitoring Off of Grade Level: Three OptionsOff of Grade Level: Three Options

Best Practices in Setting Progress Best Practices in Setting Progress Monitoring Goals for Academic Skill Monitoring Goals for Academic Skill Improvement (Shapiro, 2008).Improvement (Shapiro, 2008).

Shinn approach as detailed in AIMSweb Shinn approach as detailed in AIMSweb training workshop on Progress Monitoring.training workshop on Progress Monitoring.

Tigard-Tualatin SD Chart: 150% instead of Tigard-Tualatin SD Chart: 150% instead of 100% as minimum RoI requirement???100% as minimum RoI requirement???

Questions? & Comments!Questions? & Comments!

ResourcesResources

www.interventioncentral.comwww.interventioncentral.com

www.aimsweb.comwww.aimsweb.com

http://dibels.uoregon.eduhttp://dibels.uoregon.edu

www.nasponline.orgwww.nasponline.org

ResourcesResources

www.fcrr.orgwww.fcrr.org

Florida Center for Reading ResearchFlorida Center for Reading Researchhttp://ies.ed.gov/ncee/wwc//http://ies.ed.gov/ncee/wwc//

What Works ClearinghouseWhat Works Clearinghousehttp://www.rti4success.orghttp://www.rti4success.org

National Center on RtINational Center on RtI

Flinn & McCrea’s RoI Web SiteFlinn & McCrea’s RoI Web Site

http://sites.google.com/site/http://sites.google.com/site/rateofimprovement/rateofimprovement/Download powerpoints, handouts, Excel Download powerpoints, handouts, Excel

graphs, charts, articles, etc.graphs, charts, articles, etc.Caitlin FlinnCaitlin Flinn

[email protected] [email protected] Andrew McCreaAndrew McCrea

[email protected] [email protected]

ReferencesReferences

Ardoin, S. P., & Christ, T. J. (2009). Curriculum-Ardoin, S. P., & Christ, T. J. (2009). Curriculum-based measurement of oral reading: Standard based measurement of oral reading: Standard errors associated with progress monitoring errors associated with progress monitoring outcomes from DIBELS, AIMSweb, and an outcomes from DIBELS, AIMSweb, and an experimental passage set. experimental passage set. School Psychology School Psychology Review, 38(2),Review, 38(2), 266-283. 266-283.

Ardoin, S. P. & Christ, T. J. (2008). Evaluating Ardoin, S. P. & Christ, T. J. (2008). Evaluating curriculum-based measurement slope estimates curriculum-based measurement slope estimates using triannual universal screenings. using triannual universal screenings. School School Psychology Review, 37Psychology Review, 37(1), 109-125. (1), 109-125.


Christ, T. J. (2006). Short-term estimates of Christ, T. J. (2006). Short-term estimates of growth using curriculum-based measurement growth using curriculum-based measurement of oral reading fluency: Estimating standard of oral reading fluency: Estimating standard error of the slope to construct confidence error of the slope to construct confidence intervals. intervals. School Psychology Review, 35School Psychology Review, 35(1), (1), 128-133.128-133.

Deno, S. L. (1985). Curriculum-based Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. measurement: The emerging alternative. Exceptional Children, 52,Exceptional Children, 52, 219-232. 219-232.


Deno, S. L., Fuchs, L.S., Marston, D., & Deno, S. L., Fuchs, L.S., Marston, D., & Shin, J. (2001). Using curriculum-based Shin, J. (2001). Using curriculum-based measurement to establish growth measurement to establish growth standards for students with learning standards for students with learning disabilities. disabilities. School Psychology Review, School Psychology Review, 3030, 507-524., 507-524.

Flinn, C. S. (2008). Graphing rate of Flinn, C. S. (2008). Graphing rate of improvement for individual students. improvement for individual students. InSight, 28(3InSight, 28(3), 10-12.), 10-12.


Fuchs, L. S., & Fuchs, D. (1998). Treatment Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing validity: A unifying concept for reconceptualizing the identification of learning disabilities. the identification of learning disabilities. Learning Learning Disabilities Research and Practice, 13Disabilities Research and Practice, 13, 204-219., 204-219.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993). Formative evaluation of Germann, G. (1993). Formative evaluation of academic progress: How much growth can we academic progress: How much growth can we expect? expect? School Psychology Review, 22School Psychology Review, 22, 27-48., 27-48.


Gall, M.D., & Gall, J.P. (2007). Gall, M.D., & Gall, J.P. (2007). Educational Educational research: An introductionresearch: An introduction (8th ed.). New (8th ed.). New York: Pearson.York: Pearson.

Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. (2009). (2009). Estimating reading growth using Estimating reading growth using intermittent CBM progress monitoring. intermittent CBM progress monitoring. Exceptional Children, 75Exceptional Children, 75, 151-163., 151-163.


Karwowski, W. (2006). Karwowski, W. (2006). International International encyclopedia of ergonomics and human encyclopedia of ergonomics and human factorsfactors. Boca Raton, FL: Taylor & Francis . Boca Raton, FL: Taylor & Francis Group, LLC.Group, LLC.

Shapiro, E. S. (2008). Best practices in setting Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill progress monitoring goals for academic skill improvement. In A. Thomas and J. Grimes improvement. In A. Thomas and J. Grimes (Eds.), (Eds.), Best practices in school psychology V Best practices in school psychology V (Vol. 2, pp. 141-157). Bethesda, MD: National (Vol. 2, pp. 141-157). Bethesda, MD: National Association of School Psychologists.Association of School Psychologists.


Vogel, D. R., Dickson, G. W., & Lehman, J. Vogel, D. R., Dickson, G. W., & Lehman, J. A. (1990). Persuasion and the role of A. (1990). Persuasion and the role of visual presentation support. The UM/3M visual presentation support. The UM/3M study. In M. Antonoff (Ed.), study. In M. Antonoff (Ed.), Presentations Presentations that persuade. Personal Computing, 14that persuade. Personal Computing, 14..

Graphing, Calculating, and Interpreting Rate of Improvement Caitlin S. Flinn, Ed.S., N.C.S.P. Andrew...

Documents

Transcript of Graphing, Calculating, and Interpreting Rate of Improvement Caitlin S. Flinn, Ed.S., N.C.S.P. Andrew...