Gibbs 2011 1

Dr. Denise P. Gibbs, DirectorAlabama Scottish Rite Foundation Learning

Centersgibbsdenise@aol.com

PSTRtISTUDENTSUCCESS

RTI Implementation: Assessment Components

Gibbs 2011 2

This presentation is provided at no cost to Alabama schools by the Alabama

Scottish Rite Foundation. The philanthropy of the Alabama Scottish

Rite Foundation began in the 1950’s in Alabama and continues today. The

mission of the Alabama Scottish Rite Foundation is to provide help to

Alabama Schools as they work with students who struggle in reading -

particularly those students with dyslexia.

Gibbs 2011 3

Anticipation Guidethen…Turn and Talk

As soon as you are seated, please As soon as you are seated, please complete the Anticipation Guide included complete the Anticipation Guide included

in your handout.in your handout.

Then…turn to your neighbor and talk Then…turn to your neighbor and talk about your initial answersabout your initial answers

Bell ringer activityBell ringer activity

Gibbs 2011 4

Gains From High Impact Instructional Strategies: Research Findings (Marzano, 2001)

Type of Strategy Effect size

%ile gain

# of studies

1. Identifying similarities and differences

1.61 45 31

2. Summarizing and note taking 1.00 34 179

3. Reinforcing effort and providing recognition

.80 29 21

4. Homework and practice .77 28 134

5. Nonlinguistic representations .75 27 246

Gibbs 2011 5

Gains From High Impact Instructional Strategies: Research Findings (Marzano, 2001)

Type of Strategy Effectsize

%ilegain

# of studies

6. Cooperative learning .73 27 122

7. Setting objectives and providing feedback

.61 23 408

8. Generating and testing hypotheses

.61 23 63

9. Questions, cues, advance organizers

.59 22 1251

Gibbs 2011 6

Today’s High Impact Strategies 9. Questions, cues, and advance organizers –

Anticipation Guide, session outcomes, RTI Vocabulary, Double Bubble

6. Cooperative learning – Turn and Talks 2. Summarizing and note taking – Power Point

slides, your notes, 3-2-1, 1. Similarities and differences – CBM vs CAT -

Double Bubble 5. Nonlinguistic representations - VVWAs 4. Homework and practice – utilize at least one

of these strategies in your work

Gibbs 2011 7

Session outcomes…. Today, you will:

Practice high impact strategiesIdentify uses of high stakes tests such as

the ARMT or AHSGE in the RTI screening process.

Analyze essential criteria to guide selection or development of RTI screening and progress monitoring tools.

Use the National Center for Response to Intervention's (NCRTI) tools chart to compare some commercially available screening and progress monitoring tools.

Gibbs 2011 8

RTI Essential Vocabulary: Assessment

Screening process vs tool Progress monitoring process vs tool Classification accuracy

False positives? False negatives? Reliability and predictive validity of the

performance level and slope of improvement

Reliability and predictive validity of the slope of improvement

Sensitivity to student improvement Efficiency!!!!!!

Gibbs 2011 9

Screening Process and Examples of Some Tools

Gibbs 2011 10

Designing your screening process Commercially available screening tools

Curriculum-Based Measures (CBM) Computer Adapted Testing (CAT) Standards-Connected Assessments

High stakes tests ARMT Alabama High School Graduation Exam End of course tests

Other variables Grades Behavior Attendance

Gibbs 2011 11

Using High Stakes Tests in the Screening Process

A starting point in screening older students could be to use the results of the ARMT or the AHSGE. Students who score a I or II on the ARMT in

reading or in math could be considered as at risk and could proceed to the next step in the district’s screening process

AHSGE results could be analyzed to determine the relative severity of the student’s deficiencies

Determine if the student would benefit from remediation classes or is in need of a more basic skill-level intervention class

Gibbs 2011 12

Double Bubble Time! As an advanced organizer, compare and contrast

CBM and CAT

Gibbs 2011 13

Three types of screening measures Curriculum Based Measures

Probes which look at skills which predict that a student will not be successful in reading or math.

Computer Adapted TestingAnswers determine subsequent questions.

Can learn more about skill levels with fewer test items.

Standards Connected AssessmentsUsually a fixed set of questions that mirror

standards-based state assessments and provide practice and performance prediction.

Gibbs 2011 14

Independent Review of Screening Tools National Center for Response to

Intervention has charts summarizing their review of various screening tools.

Some of the tools reviewed as of Spring 2011 are:AIMSweb - CBMDIBELS - CBMSTEEP - CBMSTAR - CATDiscovery Education (Think Link) –

Standards-connected

Gibbs 2011 15

Curriculum-Based Measures (CBM)

Was initially synonymous with RTI. If concerned about reading, have

student complete various reading screening probes

If concerned about math, have student complete math computation and math reasoning/problem solving screening probes.

AIMSweb, DIBELS, and STEEP are all based on CBM

Gibbs 2011 16

A look at some reading CBM probes

Early Literacy MeasuresLetter Naming FluencyLetter Sound FluencyPhonemic Segmentation FluencyNonsense Word Fluency

R-CBM (8th grade example)

Grades 1-12 Mazes (7th grade example)

Grades 2-12

Gibbs 2011 17

A look at some math CBM probes Early Numeracy Measures

Quantity discriminationMissing NumbersOral CountingNumber Identification

Computation (M-CBM) (1st – 6th grade example)Grades 1-12

Concepts and Applications (M-CAP) (7th grade example)Grades 2-8

Focal points (STEEP)

Gibbs 2011 18

Computer Adapted Testing (CAT) The best computer adapted testing is based

on Item Response Theory (IRT) IRT is a statistical framework in which

examinees can be described by a set of one or more ability scores that are predictive, through mathematical models, linking actual performance on test items, item statistics, and examinee abilities.

Test items are determined by responses to questions

Score is reported as a scaled score. STAR is an example of CAT based on IRT

Gibbs 2011 19

Computer Adapted Testing (CAT) STAR Early Literacy

PreK-Grade 3 41 skills in 7 domains 10 minutes

STAR Reading Grades 1-12 10 minutes

STAR Math Grades 1-12 10-15 minutes

Gibbs 2011 20

Standards Connected Assessments

Test includes a fixed number of grade-specific questions reflecting common core or state standards.

Can serve as a screening tool for students who might not be likely to perform satisfactorily on state assessments.

Can serve as a practice test for state assessments.

Generally, will not be able to show growth over time and monitor progress.

Gibbs 2011 21

Reports from Screening Tools

Some of the most helpful screening tools provide reports which reflectStudent’s percentile scoresStudent’s expected performance on high

stakes tests (ARMT!)Longitudinal data for a group of studentsLevel of proficiency on key common core

standards Instructional grouping/planning

suggestions

Gibbs 2011 22

Turn and Talk:Where are we in terms of

screening process?

In the bags, quick fixes, long-term growth

opportunities!

Gibbs 2011 23

Progress Monitoring Process and Examples of

Some Tools

Gibbs 2011 24

What progress to monitor?

Growth of skills in areas recognized as deficientFoundational academic skills in reading or

math Growth of appropriate social and

behavior skills Growth of content knowledge and

information

Gibbs 2011 25

Progress Monitoring Process

Use of a formal commercially available progress monitoring tool or

Use of a formal teacher/school developed progress monitoring tool

Should also consider additional information which can reflect progress Use of work samples reflecting baseline

and current workObservation of classroom behavior and

participation

Gibbs 2011 26

NCRTI Progress Monitoring Tools Chart

Provides administration information Compares tools according to key

reliability, validity, slope, and improvement criteria.

Many of the screening tools have also been favorably reviewed as progress monitoring tools.

But some screening tools were NOT found to have favorably reviewed progress monitoring probes.

Gibbs 2011 27

Key progress monitoring criteria

Reliability of the performance level scorethe extent to which the score (or

average/median of 2-3 scores) is accurate and consistent.

Validity of the performance level scorethe extent to which the score (or

average/median of 2-3 scores) represents the underlying construct

Gibbs 2011 28

Key progress monitoring criteria

Reliability of the slope of improvement an indicator of how well individual differences in

growth trajectories can be detected using a particular measure

Predictive validity of the slope of improvement the extent to which the slope of improvement

corresponds to end-level performance on highly valued outcomes.

Sensitivity to student improvement the extent to which a measure reveals

improvement over time, when improvement actually occurs.

Gibbs 2011 29

Progress Monitoring Tools

Need to be efficient. Need to be able to provide a reliable

baseline. Need to be able to reflect growth over

time. Need multiple, equivalent probes. Results need to be easily graphed.

Gibbs 2011 30

Progress Monitoring Decisions

Skills to be progress monitored? Do you have the probes you need? Frequency of progress monitoring

Weekly? Bi-weekly? Monthly? Who will conduct progress monitoring Who will score? Who will enter data in

computer? Print reports?

Gibbs 2011 31

Some reading skills to be targeted…

Word-level needs General benchmark should be 95%

accuracy for grade-level texts To reach desired accuracy, may need

to target both regular (decodable) words and irregular (learned) words

Rate needs to be that designated for student’s grade or 90-110 words correct per minute (whichever is less)

Prosody/inflection patterns!!!!

Gibbs 2011 32

Some reading skills to be targeted…

Comprehension needs Vocabulary Active reading strategies

Before, During, After Magnificent Seven!

Metacognitive strategies Fix-up strategies

Gibbs 2011 33

Math skills to be targeted… Computation automaticity

I am learning the facts I can think of the answer I can not avoid seeing the answer

when I see the problem. Grade-specific problem solving and

applications Concrete-representational-abstract

continuum Subject specific skills

Gibbs 2011 34

Selection of Progress Monitoring Tools Tools should be able to be administered weekly,

should be efficient and should be sensitive to growth If improved reading comprehension is the goal, then

you may elect to use mazes or computer adapted testing

If improved reading accuracy is the goal, then you may elect to use oral reading fluency passages and graph percentage of accuracy

If improved math problem solving skills is the goal, then you may elect to use timed problem solving probes or computer adapted testing

If improved math calculation skills is the goal, then you may elect to use timed calculation probes

If improved behavior is the goal, then you may elect to use behavior report card points earned .

Gibbs 2011 35

Verbal and Visual Word Association

SensitivityEfficiency

Gibbs 2011 36

Developing Progress Monitoring Probes Interventioncentral.com equips you to

generateMaze passages – from your own textsR-CBM passages – from your own textsWordlist fluency probes Math fact fluency probesBehavior report cards!

Establish baseline and set goal

Gibbs 2011 37

Developing Progress Monitoring Probes To monitor progress in learning

content, you can develop a set of vocabulary matching probes (Espin, et al., 2005).Comprehensive list of terms and

definitions for the course/subjectRandomly generated probes with 25 or

more terms given as frequently as desired.

Graph number or percentage correct

Gibbs 2011 38

About norms for locally developed PM measures

You could initially use norms included in ABCs of CBM and other resources to give you a “ballpark idea” of student’s performance level

But remember, since you are measuring the student’s progress from baseline, you can determine growth without much attention to norms!

Goal setting will require some creativity!

Gibbs 2011 39

Goals setting with locally developed PM measures

You could determine the performance levels on your probes of your most proficient students and use those levels in developing goals for students in interventions.

You could use “norms” included in ABCs of CBM and other sources to set goals

Over time, you could determine scores which correspond to successful performance on ARMT or AHSGE and set those as goals.

Gibbs 2011 40

Graphing progress monitoring data Commercially available progress

monitoring tools are equipped with graphing features.

Results of teacher or school developed progress monitoring probes can be graphed with a number of graphing tools including the Data Management Tool available on the ALSDE website and at this Dropbox URL

http://dl.dropbox.com/u/24788238/GraphingtoolGibbs.xls

Gibbs 2011 41

Some valuable resources

Gibbs 2011 42

The ABCs of CBM: A practical guide to curriculum-based measurement

(Hosp, et al., 2007)

Instructions regarding creating progress monitoring probes

R-CBM, Mazes, Math Computation, Math Problem Solving, Written Expression, Spelling, Early Literacy, Early Numeracy

Expected levels of performance for grades 1-8

Gibbs 2011 43

Assessing Reading Multiple Measures for All Educators Working to Improve Reading

Achievement (2nd ed, 2008)

Phonics Screener Phoneme Deletion Test Phonological Segmentation Test Phoneme Segmentation Test Graded High Frequency Word Survey San Diego Quick Assessment of Reading

Ability Maze passages Oral reading fluency passages

Gibbs 2011 44

3-Minute Reading Assessments: Word Recognition, Fluency, and Comprehension

(Rasinski and Padak, 2005)

Includes a 16 point multidimensional fluency scale!!Expression and volumePhrasing and intonationSmoothnessPace

Gibbs 2011 45

Mathematics RTI: A Problem-Solving Approach to Creating An Effective Model

(Allsopp, et al., 2010)

Includes specific recommendations for analyzing mathematics curriculum as it relates to struggling students.

Includes discussion of responsive mathematics teaching practices.

Gibbs 2011 46

RTI in Mathematics (Riccomini and Witzell, 2010)

Excellent tools to facilitate assessment, instruction, and intervention forNumber senseWhole numbersFractions and decimalsProblem solvingMathematical vocabulary!

Gibbs 2011 47

Anticipation Guide Revisited

Gibbs 2011 48

THANK YOU!RTI for Early Readers: Implementing

Common Core Standards in Your K-5 RTI Model (LRP, 2011)

RTI for Middle and High Schools: Strategies and Structures for Literacy Success

(LRP,2008)

Leading the Dyslexia Challenge: An Action Plan for Schoolwide Identification and

Intervention (LRP, 2004)

gibbsdenise@aol.com