RTI and District Assessments

Jack B. Monpas-Huber, Ph.D.Director of Assessment and Student Information

Anzara MillerRTI Coordinator / Professional Development Specialist

RTI Integrity Rubric: Screening

This kind of evidence exists in abundance for DIBELS and easyCBM and was in fact a requirement in our selection of easyCBM as our math screener

Following is a series of slides documenting work the Assessment office has done with our own local data

What does a strong correlation look like?

Districtassessment

MSP

Perfect prediction No relationship

How well do our reading screeners predict performance on the MSP?

Grade 3Zero order correlations among district and state reading measures, 2009-10Measure Mean SD N MSP ORF1 ORF2 ORF3 Lexile1 Lexile2 Lexile3MSP 419.8 30.5 626 1 0.669 0.676 0.654 0.726 0.773 0.791ORF1 102.8 41.9 601 1 0.928 0.901 0.761 0.766 0.769ORF2 113.9 40.9 554 1 0.909 0.742 0.760 0.775ORF3 122.1 37.0 517 1 0.699 0.727 0.742Lexile1 503.8 314.3 620 1 0.922 0.888Lexile2 570.0 308.7 610 1 0.949Lexile3 612.5 306.1 545 1Note . All correlations are statistically significant and based on at least 412 cases.

Grade 4Zero order correlations among district and state reading measures, 2009-10Measure Mean SD N MSP ORF1 ORF2 Lexile1 Lexile2 Lexile3MSP 414.9 22.4 649 1 0.639 0.619 0.735 0.760 0.764ORF1 116.0 37.6 628 1 0.901 0.785 0.783 0.774ORF2 120.0 32.7 448 1 0.723 0.724 0.704Lexile1 723.4 305.3 643 1 0.965 0.929Lexile2 759.8 293.0 613 1 0.959Lexile3 820.1 289.7 577 1Note . All correlations are statistically significant and based on at least 500 cases.

How well do our reading screeners predict performance on the MSP?

Grade 5Zero order correlations among district and state reading measures, 2009-10Measure Mean SD N MSP ORF1 ORF2 ORF3 Lexile1 Lexile2 Lexile3MSP 419.6 24.7 614 1 0.581 0.512 0.603 0.696 0.723 0.734ORF1 134.6 37.1 592 1 0.903 0.914 0.732 0.732 0.738ORF2 126.7 32.4 322 1 0.898 0.655 0.652 0.649ORF3 140.8 32.0 374 1 0.696 0.708 0.714Lexile1 877.4 278.7 603 1 0.963 0.938Lexile2 920.1 276.9 600 1 0.969Lexile3 964.6 267.3 565 1Note . All correlations are statistically significant and based on at least 318 cases.

Grade 6Zero order correlations among district and state reading measures, 2009-10Measure Mean SD N MSP ORF1 ORF2 ORF3 Lexile1 Lexile2 Lexile3MSP 411.2 22.3 644 1 0.587 0.587 0.625 0.753 0.770 0.769ORF1 139.2 33.2 607 1 0.882 0.887 0.692 0.689 0.690ORF2 137.6 34.1 409 1 0.915 0.636 0.638 0.654ORF3 141.4 36.1 438 1 0.684 0.686 0.695Lexile1 966.6 287.6 629 1 0.962 0.937Lexile2 996.9 285.7 621 1 0.965Lexile3 1036.0 274.4 595 1Note . All correlations are statistically significant and based on at least 404 cases.

Benchmark total raw score N r N r N r N r N r N r

Fall 562 0.730 599 0.751 615 0.730 584 0.787 595 0.793 537 0.818

Winter 562 0.737 599 0.779 615 0.705 584 0.826 595 0.799 537 0.806

Spring 562 0.687 599 0.780 615 0.634 584 0.755 595 0.777 537 0.787

Note . All correlations are statistically significant at p < .001

Grade 7 Grade 8Grade 3 Grade 4 Grade 5 Grade 6

easyCBM and the MSPPearson correlations

These are correlation coefficients between easyCBM fall, winter, and spring benchmark scores and the MSP scale scores. All correlations are moderate to strong (ranging from .634 to .818) and statistically significant with large samples of the same students who took all three benchmark tests.

easyCBM and the MSPPearson correlations

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

Grade 3

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46

easyCBM Benchmark Total Raw Score

% met MSP Math standard

Fall Winter Spring

easyCBM and the MSPGrade 3

InterpretationThis chart illustrates the correlation between easyCBM and MSP. We can use this to establish benchmarks for screening students for intervention.

60% of the students who scored a 30 on the fall benchmark test met the MSP standard

Bubbles are sized according to number of students

Grade 4

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46

easyCBM Benchmark Total Raw Score

% met MSP Math standard

Fall Winter Spring

easyCBM and the MSPGrade 4

Bubbles are sized according to number of students

Grade 5

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46

easyCBM Benchmark Total Raw Score

% met MSP Math standard

Fall Winter Spring

easyCBM and the MSPGrade 5

Bubbles are sized according to number of students

Notice also the growth between benchmarks (the spread between waves of bubbles), and where along the achievement scale that growth occurs:

Grade 6

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46

easyCBM Benchmark Total Raw Score

% met MSP Math standard

Fall Winter Spring

easyCBM and the MSPGrade 6

Bubbles are sized according to number of students

Grade 7

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46

easyCBM Benchmark Total Raw Score

% met MSP Math standard

Fall Winter Spring

easyCBM and the MSPGrade 7

Bubbles are sized according to number of students

Grade 8

-10%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

110%

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46

easyCBM Benchmark Total Raw Score

% met MSP Math standard

Fall Winter Spring

easyCBM and the MSPGrade 8

Bubbles are sized according to number of students

0

4

8

12

16

20

24

28

32

36

40

44

48

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

K 1 2 3 4 5 6 7 8

1-25th percentile 26-50th percentile 51-75th percentile 76-99th percentile

Shoreline easyCBM growth norms, 2010-11Gains in mean raw points

0

10

20

30

40

50

60

70

80

90

100

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

Fall

WinterSpring

K 1 2 3 4 5 6 7 8

1-25th percentile 26-50th percentile 51-75th percentile 76-99th percentile

Shoreline easyCBM growth norms, 2010-11Gains in easyCBM percentiles

RTI Integrity Rubric: Screening

Not a 5 because our DIBELS screening is not universal for all three benchmark windows

RTI Integrity Rubric: Progress monitoring

School K 1 2 3 4 5 6 7 8 Total

Briarcrest Elementary School 1 1

Brookside Elementary School 53 2 2 3 60

Echo Lake Elementary School 73 85 79 237

Highland Terrace Elementary 1 57 64 53 175

Lake Forest Park Elementary 4 50 27 81

Meridian Park Elementary 1 26 28 1 5 61

Parkwood Elementary School 58 50 108

Ridgecrest Elementary School 21 55 2 3 60 141

Room Nine Community School 1 1

Syre Elementary School 33 59 92

Einstein Middle School 36 38 74

Total 4 22 131 84 136 308 272 36 38 1031

Who monitored progress?

easyCBM progress monitoring data, 2010-11Counts of students taking at least one progress monitoring test

How much progress monitoring?

easyCBM progress monitoring data, 2010-11Total number of progress monitoring scores

School K 1 2 3 4 5 6 7 8 Total

Briarcrest Elementary School 6 6

Brookside Elementary School 573 7 2 3 585

Echo Lake Elementary School 780 1,384 2,182 4,346

Einstein Middle School 285 286 571

Highland Terrace Elementary 1 1,067 980 615 2,663

Lake Forest Park Elementary 9 197 160 366

Meridian Park Elementary 3 26 28 1 9 67

Parkwood Elementary School 576 655 1,231

Ridgecrest Elementary School 60 420 9 11 1,551 2,051

Room Nine Community School 9 9

Syre Elementary School 279 487 766

Total 9 63 643 611 1,866 4,787 4,111 285 286 12,661

Which students received progress monitoring?

easyCBM progress monitoring data, 2010-11Percent of students progress monitored, by grade level and fall quartile

Grade N% progress monitored

N% progress monitored

N% progress monitored

N% progress monitored

N% progress monitored

K 63 0.0% 65 1.5% 106 0.9% 275 0.7% 509 0.8%

1 103 2.9% 124 0.8% 145 4.8% 206 4.9% 578 3.6%

2 112 26.8% 100 23.0% 147 15.6% 179 24.6% 538 22.3%

3 76 11.8% 132 12.1% 142 14.1% 212 15.6% 562 13.9%

4 121 27.3% 100 18.0% 165 13.9% 215 23.3% 601 20.6%

5 119 49.6% 76 47.4% 126 52.4% 297 42.1% 618 46.3%

6 139 43.9% 129 49.6% 113 52.2% 205 34.6% 586 43.5%

7 125 24.8% 125 0.8% 164 0.0% 183 0.0% 597 5.4%

8 179 18.4% 129 1.6% 102 0.0% 130 0.0% 540 6.5%

All1st - 25th percentile 26th - 50th percentile 51st - 75th percentile 76th - 99th percentile

EFFECT OF PROGRESS MONITORING, 2010-11Gains on Benchmark Scores (easyCBM percentiles)

0

10

20

30

40

50

60

70

80

90

100

FallWinterSpring

FallWinterSpring

FallWinterSpring

FallWinterSpring

FallWinterSpring

FallWinterSpring

FallWinterSpring

FallWinterSpring

FallWinterSpring

K 1 2 3 4 5 6 7 8

PM N = 4 PM N = 22 PM N = 131 PM N = 84 PM N = 136 PM N = 308 PM N = 272 PM N = 36 PM N =38

Q1 NoPM Q1 Any PM Q2 NoPM Q2 Any PM Q3 NoPM Q3 Any PMQ4 NoPM Q4 Any PM

The red lines are students receiving any progress monitoring. In many cases their rate of growth between benchmarks exceeds that of students not progress monitored. This is consistent with research that progress monitoring without intervention still increases achievement by sizable effects (Bangert-Drowns, Kulik & Kulik, 1991).

Bangert-Drowns, R. L., Kulik, J. A., & Kulik, C.-L. C. (1991). Effects of frequent classroom testing. Journal of Educational Research, 85, 89-99.

Reference