Data Based Decision Making

Reading Review

Stanovich, 2010 Fuchs & Fuchs -- Progress Monitoring

"It ain't so much the things we don't know that get us into trouble. It's the things we know that just ain't so."

-Josh BillingsPerhaps the second most famous humor writer and lecturer in the United States in the second half of the 19th century after Mark Twain

We Never Know for sure…

Even practices with the best research base… may not work for some students.

So… if you are using a research based intervention – implement & COLLECT DATA!

And… if you are struggling to identify a research-based intervention – implement & COLLECT DATA!

Critical Concept:Data Based Decision Making

Continuous, purposeful process of collecting, interpreting, presenting and using data to inform actions that support positive educational outcomes.

Data based decision making considers the learner’s progress within the contexts of instruction, curriculum and environment.

Necessary components of Assessment When a student is experiencing difficulty, several

related & complementary types of assessment should be performed

1) Assessment of the Learner (Student)

2) Assessment of Instruction (or Intervention) Curriculum and Environment

LearnerInstruction/ InterventionCurriculumEnvironment

Measuring -ICEInstruction, Curriculum, Environment

What questions might you have about the instruction/intervention or curriculum?

Are the instructional/interventions methods research based? Implementation fidelity?

Is the classroom environment suitable to learning Time on task Instructional time Academic engaged time Opportunities to Respond & % Correct Responses Positive to Negative Ratio Student problem behavior

Models for Data Based Decision

Making

Problem Solving Models & Outcomes Driven Models

SYST

EMSPRACTICES

DATASupportingStaff Behavior

SupportingStudent Behavior

OUTCOMES

Supporting Social Competence &Academic Achievement

SupportingDecisionMaking

Outcomes Driven Model

In an Outcomes Driven Model, the bottom line is achievement of essential educational or social outcomes

What are the desired outcomes?

Are students attaining the necessary skills to be successful? If not, what changes can we make? Are the changes increasing student progress?

Research Based Frameworks Needed How do we know what to measure & when?

Reading RTI & Big 5 Ideas of Reading

Math RTI

Behavior PBIS, Function of Behavior & ABA

VocabularyReading Comprehension

Phonemic Awareness

Phonics(Alphabetic Principle)

Oral ReadingFluency & Accuracy

Big 5 Ideas of Reading

Acquisition

Fluency

Maintenance &Generalization

3. Accurately identify those who are on track and those who will need more support

Good, Simmons, & Smith (1998)

We must identify struggling students, BEFORE they fall too far behind

1-5% 1-5%

5-10% 5-10%

80-90% 80-90%

Intensive, Individual Interventions•Individual Students•Assessment-based

•High Intensity

Intensive, Individual Interventions•Individual Students•Assessment-based

•Intense, durable procedures

Targeted Group Interventions•Some students (at-risk)

•High efficiency•Rapid response

Targeted Group Interventions•Some students (at-risk)

•High efficiency•Rapid response

Universal Interventions•All students

•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

Response to Intervention

Academic Systems Behavioral Systems

Circa 1996

CollectCollect and Useand Use

DataData

Develop Hypothesis

Discuss andSelect

Solutions

Develop andImplementAction Plan

Evaluate andRevise

Action Plan

Problem Solving Meeting Foundations

Team Initiated Problem Solving (TIPS) Model

Identify

Problems

15

Purposes of Assessment

1. Screening• “Which students need more support?”

2. Progress Monitoring• “Is the student making adequate progress?”

3. Diagnostic• “What and how do we need to teach this student?”

4. Outcome• “Has our instruction been successful?”

Outcomes Driven Model

ScreeningScreening

ScreeningScreeningDiagnosticDiagnostic

DiagnosticDiagnostic

Progress MonitoringProgress Monitoring

OutcomeOutcome

Effective Data Collection

Use the right tools for the right job

Screening Progress Monitoring Diagnostic Assessment Outcomes

Use Good ToolsTechnically Adequate

Reliability = Consistency

The extent that an assessment will be consistent in finding the same results across conditions (across different administrators, across time, etc.)

If same measure is given several times to the same person, their scores would remain stable & not randomly fluctuate

Use Good ToolsTechnically Adequate

Validity = extent that an assessment measures what it is supposed to measure First we need to know what we should be measuring!

Research Based Frameworks for Measurement Students who do well on valid reading tests are proficient

readers Valid = assessing reading by having the student read a passage

aloud and monitoring errors and rate Not Valid = assessing reading by having a student match printed

letters on a page (this is an assessment matching visual figures)

Draw a line to Match the letters: A f U p w

w E A f I U v B p

Use Good Tools A Concern for self-developed assessments

Technical Adequacy can be a problem with self-developed measures

Challenge with Professional Learning Team modelWhich often rely on teacher-developed

assessments to measure important student outcomes & guide decision making

Low Inference

Students are tested using materials that are directly related to important instructional outcomesLow inference

Making judgments on a child’s reading skills based on listening to them read out loud.

High inference Making judgments on a child’s emotional state based

on pictures they’ve drawn

Use the tools correctly Standardized Administration

Administered, scored, and interpreted in the same wayDirections given to students are consistentStudent responses are scored in the same wayEvery student has the exact same opportunity

on the assessment

Efficiency Time is precious in classrooms, efficiency is an

important consideration

When evaluating efficiency of an assessment tool, we must consider: Time & personnel required to design, administer and

score assessment tools

Design Administration & Scoring

PNRT’s Already designed Time intensive (1-2 hours/child)

CBA Some already designed, Some teacher-created

Quick and Easy (1-10 min/child)

CBM Already designed Quick and Easy (1-10 min/child)

Screening

1. Compare ALL students to the same grade-level standard

ALL students are assessed against the grade level-standard, regardless of instructional

level

"If you don't know where you are going, you will wind up somewhere else.“

~ Yogi Berra

2. Be efficient, standardized, reliable, and valid

Robust indicator of academic health Brief and easy to administer

Can be administered frequently Must have multiple, equivalent forms

(If the metric isn’t the same, the data are meaningless)

Must be sensitive to growth

3. Accurately identify those who are on track and those who will need more support

Good, Simmons, & Smith (1998)

We must identify struggling students, BEFORE they fall too far behind

4. Evaluate the quality of your schoolwide instructional system

• Are 80% of your students proficient? • Are 80% of students reaching

benchmarks and “on track” for next goal?

If not, then the core curriculum needs to be addressed

Screening Tools DIBELS Oral Reading Fluency Maze EasyCBM CBM Math Computation CBM Writing – Story

Starters CBM Algebra CBM Early Numeracy

Not Screening Tools Quick Phonics Screener QRI-IV DRA2 Running Records Report cards Meeting OAKS standards Core curriculum weekly

tests on skills that are learned

What are Screening Tools?

32

Column A Column B

Name _______________________________ Date ________________________ Test 4 Page 1

Applications 4

•N

•M

(B)LK••

•Z (A)

(C)

(D)

point

ray

line segment

line

Write the letter in each blank.

(1)

Look at this numbers.:

356.17

Which number is in the hundredths place?

(2)

(3)

Jeff wheels his wheelchair for 33 hoursa week at school and for 28 hours a weekin his neighborhood. About how manyhours does Jeff spend each week wheelinghis wheelchair?

Solve the problem by estimating the sum ordifference to the nearest ten.

(4)

Write the number in each blank.

3 ten thousands, 6 hundreds, 8 ones

2 thousands, 8 hundreds, 4 tens, 6 ones

(5)

Write a number in the blank.

1 week = _____ days

(6) Vacation Plans for SummitSchool Students

SummerSchool

Camp

Travel

Stay home

0 2010 30 40 50 60 70 80 90 100

Number of Students

The P.T.A. will buy a Summit SchoolT-Shirt for each student who goesto summer school. Each shirt costs$4.00. How much money will theP.T.A. spend on these T shirts?

How many students are planning totravel during the summer?

How many fewer students are planningto go to summer school than planningto stay home?

Use the bar graph to answer the questions.

(A) meters

(B) centimeters

(C) kilometers

To measure the distance of the busride from school to your house youwould use

(7)

$ .00

D

Column A Column B

Name _______________________________ Date ________________________ Test 4 Page 1

Applications 4

•N

•M

(B)LK••

•Z (A)

(C)

(D)

point

ray

line segment

line

Write the letter in each blank.

(1)

Look at this numbers.:

356.17

Which number is in the hundredths place?

(2)

(3)

Jeff wheels his wheelchair for 33 hoursa week at school and for 28 hours a weekin his neighborhood. About how manyhours does Jeff spend each week wheelinghis wheelchair?

Solve the problem by estimating the sum ordifference to the nearest ten.

(4)

Write the number in each blank.

3 ten thousands, 6 hundreds, 8 ones

2 thousands, 8 hundreds, 4 tens, 6 ones

(5)

Write a number in the blank.

1 week = _____ days

(6) Vacation Plans for SummitSchool Students

SummerSchool

Camp

Travel

Stay home

0 2010 30 40 50 60 70 80 90 100

Number of Students

The P.T.A. will buy a Summit SchoolT-Shirt for each student who goesto summer school. Each shirt costs$4.00. How much money will theP.T.A. spend on these T shirts?

How many students are planning totravel during the summer?

How many fewer students are planningto go to summer school than planningto stay home?

Use the bar graph to answer the questions.

(A) meters

(B) centimeters

(C) kilometers

To measure the distance of the busride from school to your house youwould use

(7)

$ .00

D

Column A Column B

Name _______________________________ Date ________________________ Test 4 Page 1

Applications 4

•N

•M

(B)LK••

•Z (A)

(C)

(D)

point

ray

line segment

line

Write the letter in each blank.

•N

•M

(B)LK••

•Z (A)

(C)

(D)

point

ray

line segment

line

•N

•M

•N

•M

(B)LK••

LK••

•Z•Z (A)

(C)

(D)

point

ray

line segment

line

Write the letter in each blank.

(1)

Look at this numbers.:

356.17

Which number is in the hundredths place?

(2)

(3)

Jeff wheels his wheelchair for 33 hoursa week at school and for 28 hours a weekin his neighborhood. About how manyhours does Jeff spend each week wheelinghis wheelchair?

Solve the problem by estimating the sum ordifference to the nearest ten.

(4)

Write the number in each blank.

3 ten thousands, 6 hundreds, 8 ones

2 thousands, 8 hundreds, 4 tens, 6 ones

Write the number in each blank.

3 ten thousands, 6 hundreds, 8 ones

2 thousands, 8 hundreds, 4 tens, 6 ones

(5)

Write a number in the blank.

1 week = _____ days

Write a number in the blank.

1 week = _____ days

(6) Vacation Plans for SummitSchool Students

SummerSchool

Camp

Travel

Stay home

0 2010 30 40 50 60 70 80 90 100

Number of Students

SummerSchool

Camp

Travel

Stay home

0 2010 30 40 50 60 70 80 90 100

Number of Students

The P.T.A. will buy a Summit SchoolT-Shirt for each student who goesto summer school. Each shirt costs$4.00. How much money will theP.T.A. spend on these T shirts?

How many students are planning totravel during the summer?

How many fewer students are planningto go to summer school than planningto stay home?

Use the bar graph to answer the questions.

(A) meters

(B) centimeters

(C) kilometers

To measure the distance of the busride from school to your house youwould use

(7)

$ .00

D

One Page of a 3-Page CBM in Math Concepts and Applications (24 Total Blanks)

Previous Years Discipline data

Who had FBA/BSP’s last year?

Which students moved on? Which are returning this year?

Who needs to be on our radar from Day 1?

Decision Rule

Can we get data for our incoming class & new students?

Progress Monitoring

Progress Monitoring Tools

Brief & Easy

Sensitive to growth

FrequentEquivalent forms

Where are we?What is our goal?What course should we follow?

How are we doing?

ActualCourse

DesiredCourse

Our Goal

We are Here

Progress Monitoring: The GPS for Educators!

Purpose of Progress MonitoringAnswers the question(s):

Are the children learning? How can we tell?

Are they making enough progress?

Can we remove some of our supports?

Do we need to change or intensify our supports?

How often do you progress monitor students?

Determined by district decision rules and level of need

Best practice recommendations: Intensive: 1-2 x per weekStrategic: 1x or 2x per month

Cor

rect

wor

ds p

er M

inut

e

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

M ayScores

JuneScores

60

50

How do we know if a student is making adequate progress?

Decision Rules

Questions to Consider How many data points below the line

before you make a change in instruction/intervention?

What do you change?Group size?Time?Curriculum?Other factors?

Progress Monitoring

2731

3530

2532

3438

Phonics for Reading

We do not use progress monitoring data to…

…select specific short-term instructional goals

…take a lot of time away from instruction

…diagnose educational problems

…assign grades to students

…evaluate teachers

Progress Monitoring Tools

DIBELS Oral Reading Fluency Maze EasyCBM CBM Math Computation CBM Writing – Story

Starters CBM Algebra CBM Early Numeracy

Not Progress Monitoring Tools Quick Phonics Screener QRI-IV DRA2 Running Records Report cards Meeting OAKS standards Core curriculum weekly

tests on skills that are learned

What are Progress Monitoring Tools?

Progress Monitoring data tell us WHEN a change is needed

Progress Monitoring data does not always

tell us WHAT change is needed

Point Card

Look at Individual Student graph for Targeted Student(s)

Diagnostic Assessment

Answer the question….

Why?WARNING!

Critical Thinking

Skills may be

Required

•The major purpose for administering diagnostic tests is to provide information that is useful in planning more effective instruction.

Collecting Diagnostic Data

• Diagnostic tests should only be given when there is a clear expectation that they will provide new information about a child’s difficulties learning to read that can be used to provide more focused, or more powerful instruction.

Diagnostic Assessment Questions

“Why is the student not performing at the expected level?”

(Defining the Problem)

“What is the student’s instructional need?”

(Designing an Intervention)

Digging Deeper

In order to be “diagnostic”:We need to know the sequence of skill

developmentContent knowledge may need further

development

Enabling Skills

Enabling skills are skills that could be considered prerequisite skills for the demonstration of proficient performances on larger assessments measures

They represent the sub-skills of higher order performance demonstration

Deficiencies in enabling skills will often result in lower performance on assessments

Phonemic Awareness Developmental Continuum

Easy

Hard

IF DIFFICULTY

DETECTED

HERE..

THEN checkhere!

• Phoneme deletion and manipulation• Blending and segmenting individual

phonemes• Onset-rime blending and segmentation• Syllable segmentation and blending• Sentence segmentation• Rhyming• Word comparison

Vital for

Diagnostic

Process!

Reading: Diagnostic assessments may include: In curriculum assessments:

Quick Phonics ScreenerWeekly assessment data Unit and Benchmark assessment data

Survey Level Assessments Error Analysis or Running Records Any formal or informal assessment that answers the

question:Why is the student having a problem?

Survey Level Assessment

Start at expected level and move backward until specific skill deficits are identified

Match interventions to address specific skill deficits

Example 2nd Grade Math Assignment – Double Digit Math

FACTS sheet (+,-,x,/) -- student cannot do Progress backward in assessment to see where

student can be successful Cannot do basic facts division multiplication or

double digit subtraction or addition Can do single digit addition to +5 successfully

Error Analysis

1. Select a 250 word passage on which you estimate that the student will be 80-85% accurate.

2. Record the student’s errors on your copy of the reading probe.

3. Use at least 25 errors for students in grade 1 to conduct an error analysis and at least 50 errors for students in second grade and above.

4. Use an error analysis sheet to conduct error analysis.

Error Analysis

We do not use diagnostic data…

…for all students

…to monitor progress towards a long-term goal

…to compare students to each other

Outcome

Was the goal reached?Often times, the same assessment as your

screenerCan be CBM, State-testing (OAKS), other

high stakes assessments.Should be linked to district standards and

benchmarks