Designing and using assessment systems to prevent reading difficulties in young children Dr. Joseph...

Designing and using assessment Designing and using assessment systems to prevent reading systems to prevent reading difficulties in young childrendifficulties in young children

Dr. Joseph TorgesenFlorida State University and

Florida Center for Reading Research

Western North Carolina LD/ADD Symposium, November, 2005

1. Increase the quality, consistency, and 1. Increase the quality, consistency, and reachreach of instruction in every K-3 classroomof instruction in every K-3 classroom

2. Conduct timely and valid assessments of 2. Conduct timely and valid assessments of reading growth to identify struggling readersreading growth to identify struggling readers

3. Provide more intensive interventions to 3. Provide more intensive interventions to “catch up” the struggling readers“catch up” the struggling readers

The prevention of reading difficulties is a The prevention of reading difficulties is a school-level challengeschool-level challenge

A model for preventing reading failure in A model for preventing reading failure in grades K-3: The big Ideasgrades K-3: The big Ideas

A model for preventing reading failure in A model for preventing reading failure in grades K-3: The big Ideasgrades K-3: The big Ideas

1. Increase the quality, consistency, and 1. Increase the quality, consistency, and reachreach of instruction in every K-3 classroomof instruction in every K-3 classroom

2. Conduct timely and valid assessments of 2. Conduct timely and valid assessments of reading growth to identify struggling readersreading growth to identify struggling readers

3. Provide more intensive interventions to 3. Provide more intensive interventions to “catch up” the struggling readers“catch up” the struggling readers

Systematic assessments of reading growth: Big Systematic assessments of reading growth: Big IdeasIdeas

Progress monitoringProgress monitoring in growth of critical reading in growth of critical reading skills for all children during the year to help plan skills for all children during the year to help plan instructioninstruction

End of year outcome assessmentsEnd of year outcome assessments in the critical in the critical elements of reading growth –is the child on track to elements of reading growth –is the child on track to read at grade level by third grade-how effective is read at grade level by third grade-how effective is our program?our program?A data management system that supports the use A data management system that supports the use of assessment information in planning instructionof assessment information in planning instruction

Screening assessmentsScreening assessments that identify children who that identify children who are lagging behind in growth of critical skillsare lagging behind in growth of critical skills

Diagnostic assessments Diagnostic assessments to help provide additional to help provide additional information useful for focusing and planning information useful for focusing and planning instructioninstruction


Screening assessmentsScreening assessments that identify children who that identify children who are lagging behind in growth of critical skillsare lagging behind in growth of critical skills

What should we screen for?What should we screen for?

Kindergarten – letter knowledge, phonemic awareness, Kindergarten – letter knowledge, phonemic awareness, vocabularyvocabulary

11stst Grade–, phonemic awareness, phonics, vocabulary Grade–, phonemic awareness, phonics, vocabulary

22ndnd Grade– phonics, reading fluency, vocabulary Grade– phonics, reading fluency, vocabulary

33rdrd Grade– reading fluency, vocabulary Grade– reading fluency, vocabulary

More or less comprehensive screeners – DIBELS, TPRI, More or less comprehensive screeners – DIBELS, TPRI, PALS, PPVT, have information about reliability and validityPALS, PPVT, have information about reliability and validity


Progress monitoringProgress monitoring in growth of critical reading in growth of critical reading skills for all children during the year to help plan skills for all children during the year to help plan instructioninstructionWhat can we progress monitor at presentWhat can we progress monitor at present

Kindergarten – letter knowledge, phonemic awareness, Kindergarten – letter knowledge, phonemic awareness, phonicsphonics

11stst Grade–, phonemic awareness, phonics, fluency Grade–, phonemic awareness, phonics, fluency22ndnd Grade– phonics, reading fluency Grade– phonics, reading fluency33rdrd Grade– reading fluency Grade– reading fluency

Different frequencies for different groupsDifferent frequencies for different groupsWhole class – three, four times a yearWhole class – three, four times a yearStudents receiving interventions – more frequentlyStudents receiving interventions – more frequently

What can teachers learn from PM assessments?What can teachers learn from PM assessments?

Information from progress monitoringInformation from progress monitoring

Are the children actually learning what I am Are the children actually learning what I am teaching?teaching?

Is my intervention strong enough to place the Is my intervention strong enough to place the children on a growth trajectory that ends in grade children on a growth trajectory that ends in grade level performance by the end of the year?level performance by the end of the year?

Are the children ready to move forward in the Are the children ready to move forward in the curriculum?curriculum?

Sept Dec Feb May

2nd Grade Growth in Oral Reading Fluency

16

32

64

48

80

96

Co

rre

ct w

ord

s p

er m

inu

te

What can teachers learn from these What can teachers learn from these assessments?assessments?Information from progress monitoringInformation from progress monitoring


Is my intervention strong enough to place the Is my intervention strong enough to place the children on a growth trajectory that ends in grade children on a growth trajectory that ends in grade level performance by the end of the year?level performance by the end of the year?

Decisions to be madeDecisions to be made

Should I reteach the last unit to some of my children?Should I reteach the last unit to some of my children?

Should I move the child to a smaller group, or program Should I move the child to a smaller group, or program more instructional time?more instructional time?


Should I seek help to implement a more powerful Should I seek help to implement a more powerful instructional strategy?instructional strategy?

In order to monitor progress adequately, we need In order to monitor progress adequately, we need two different kinds of information about progresstwo different kinds of information about progress

Information from curriculum embedded tests or teacher Information from curriculum embedded tests or teacher obs.obs.


Information from “index” tests like the DIBELS or Information from “index” tests like the DIBELS or TPRITPRI


Is my instruction powerful enough to place the child Is my instruction powerful enough to place the child on a trajectory for grade level achievement by the end on a trajectory for grade level achievement by the end of the year?of the year?


End of year outcome assessmentsEnd of year outcome assessments in the critical in the critical elements of reading growth –is the child on track to elements of reading growth –is the child on track to read at grade level by third grade-how effective is read at grade level by third grade-how effective is our program?our program?

Kindergarten – phonemic awareness, letter knowledge, Kindergarten – phonemic awareness, letter knowledge, phonics, vocabularyphonics, vocabulary

11stst Grade– phonics, text fluency, reading comprehension, Grade– phonics, text fluency, reading comprehension, vocabularyvocabulary

22ndnd Grade– text fluency, reading comprehension, vocabulary Grade– text fluency, reading comprehension, vocabulary

33rdrd Grade– text fluency, reading comprehension, vocabulary Grade– text fluency, reading comprehension, vocabulary

Diagnostic TestingDiagnostic Testing

Purpose of diagnostic assessmentPurpose of diagnostic assessment-- -- to provide to provide information that is useful in planning more information that is useful in planning more effective inseffective instructiontruction

Diagnostic tests should be given when there is Diagnostic tests should be given when there is a a clear expectationclear expectation that they will provide new, that they will provide new, or more reliable, information about a child’s or more reliable, information about a child’s reading difficulties that can be used reading difficulties that can be used to provide to provide more powerful instructionmore powerful instruction. .

Diagnostic testing within the rich Diagnostic testing within the rich assessment context of screening, PM, assessment context of screening, PM, and outcome testsand outcome testsIf schools are implementing If schools are implementing screeningscreening, , progress progress monitoringmonitoring, and , and outcome assessmentsoutcome assessments in a reliable in a reliable and valid way, this should substantially reduce the and valid way, this should substantially reduce the need for additional testing using formal diagnostic need for additional testing using formal diagnostic instruments. instruments.

Will the diagnostic assessment actually provide Will the diagnostic assessment actually provide moremore information, or more reliable information, about the child’s information, or more reliable information, about the child’s strengths and weaknesses in the components of reading strengths and weaknesses in the components of reading than they already possess. If it will provide additional than they already possess. If it will provide additional information, will this new information be useful in planning information, will this new information be useful in planning additional instruction for the child?additional instruction for the child?

Diagnostic testing within the rich Diagnostic testing within the rich assessment context of the Reading First assessment context of the Reading First assessment planassessment planShould students identified as “at risk” at the Should students identified as “at risk” at the beginning of the year automatically be beginning of the year automatically be administered a diagnostic test to provide additional administered a diagnostic test to provide additional information to help plan instruction?information to help plan instruction?If you know a child is low in phonemic awareness, If you know a child is low in phonemic awareness, letter knowledge, and vocabulary at the beginning letter knowledge, and vocabulary at the beginning of K, what more can we reliably measure of K, what more can we reliably measure to guide to guide instructioninstruction??If you know a child is low in phonemic awareness, If you know a child is low in phonemic awareness, and knowledge of the alphabetic principal, and and knowledge of the alphabetic principal, and vocabulary at the beginning of 1vocabulary at the beginning of 1stst grade, what grade, what more can we reliably measure more can we reliably measure to guide instructionto guide instruction??

Diagnostic testing within the rich Diagnostic testing within the rich assessment context of the Reading First assessment context of the Reading First assessment planassessment planOn the basis of the On the basis of the diagnostic informationdiagnostic information available available from the screening measures, interventions from the screening measures, interventions focused on critical components of reading could focused on critical components of reading could begin immediatelybegin immediately, rather than waiting for , rather than waiting for additional diagnostic information generated by additional diagnostic information generated by diagnostic tests. The major argument for not doing diagnostic tests. The major argument for not doing additional diagnostic testing in this case is that it additional diagnostic testing in this case is that it would be would be unlikely to add any additional information unlikely to add any additional information critical to planning effective interventionscritical to planning effective interventions, and it , and it might delay the process of beginning the needed might delay the process of beginning the needed interventions for these “at risk” students.interventions for these “at risk” students.


A data management system that supports the use A data management system that supports the use of assessment information in planning instructionof assessment information in planning instruction

The Florida Progress Monitoring and Reporting Network The Florida Progress Monitoring and Reporting Network (PMRN). Web-based data management and reporting (PMRN). Web-based data management and reporting networknetwork

Immediate reports for teachers, principals, district Immediate reports for teachers, principals, district staffstaff

School level entry of screening, progress School level entry of screening, progress monitoring, and some outcome scoresmonitoring, and some outcome scores

Tied to the Florida Student Information System, so Tied to the Florida Student Information System, so that student progress is tracked longitudinally, that student progress is tracked longitudinally, and from school to schooland from school to school

Class Status Report for teacher or principal

This is an example of a first grade classroom report for December.

The teacher can immediately see which of her students are at most risk for not being at grade level at the end of the year, and can use this report to group students for targeted instruction and support

Student Progress Report for teacher or principal

This report shows the progress of a third grade student in becoming a fluent and accurate reader in third grade. At the beginning of the year, the student was classified as high risk on this measure, but the student received excellent intervention, and by the end of the year, he was performing in the low risk range.

Scores of 60% of the class

median

5th percentile for class

Class Tracking Tool

Kindergarten

LNF

Full Year Students only

Sorted by Overall

School Distribution Report

Interval 4

KG-LNF

District Status Report

This report allows district level staff to quickly examine the performance of each of the schools in the district on critical measures of early reading skill.

Using Screening and Using Screening and outcome data to outcome data to

evaluate the evaluate the strength of strength of

Instruction at state, Instruction at state, classroom, and classroom, and

school levelschool level

0

10

20

30

40

50

60

70

80

K 1 2 3

2004-2005

% of students at “grade level” level at Assessment 1 % of students at “grade level” level at Assessment 1 using combined index from DIBELS measuresusing combined index from DIBELS measures

3939

7575

6565

3838

0

10

20

30

40

50

60

70

80

K 1 2 3

2004-2005

% of students at “grade level” level at Assessment 4% of students at “grade level” level at Assessment 4

6767

52524848

3939

Notice the drop in % of students Notice the drop in % of students at grade level from beginning to at grade level from beginning to end of yearend of year

0

10

20

30

40

50

60

70

80

K 1 2 3

2004-2005

Percent of students at High Risk level at Assess 1Percent of students at High Risk level at Assess 1

2323

991414

3333

0

10

20

30

40

50

60

70

80

K 1 2 3

2004-2005

Percent of students at High Risk level at Assess 4Percent of students at High Risk level at Assess 4

1818 2020

2828

2222

Notice the increase in % of Notice the increase in % of students at high risk from students at high risk from beginning to end of yearbeginning to end of year

What are the areas in most need of immediate What are the areas in most need of immediate improvement?improvement?

Stronger support for the growth of text Stronger support for the growth of text reading fluency in second gradereading fluency in second grade

37th percentile

53rd percentile

:

Instructional Emphasis for Second Grade

3.0%

5.1%

49.0% 21.2%

5.5%

3.2%

12.9%

Other

Writing/language art

Comprehension Text reading

Fluency

Spelling

Word study/phonics

Comprehension42%

Non-instructional6%

Writing or lang. arts 3%

Fluency4%

Spelling2%

Word study/phonics21%

Text reading22%

2004 2005

About half our second graders began second grade not having met the February 1st grade benchmark in NWF

Slightly more than 20% still hadn’t met the 1st grade benchmark at the end of second grade

One problem that arises from so many students One problem that arises from so many students coming into 2coming into 2ndnd grade still weak in effective, grade still weak in effective, accurate word reading strategiesaccurate word reading strategies

Growth in fluency requires Growth in fluency requires accurateaccurate practice practice

A A majormajor factor underlying growth in fluency factor underlying growth in fluency for struggling readers is how fast the number for struggling readers is how fast the number of words they can recognize “by sight” of words they can recognize “by sight” increasesincreasesChildren must read unfamiliar words with Children must read unfamiliar words with perfect perfect accuracy on multiple occasions before accuracy on multiple occasions before they can become sight wordsthey can become sight words

Sight vocabulary must grow Sight vocabulary must grow very rapidlyvery rapidly in in second grade to keep pace with normative second grade to keep pace with normative developmentdevelopment

Our third graders are “holding their Our third graders are “holding their own” in fluency development, but own” in fluency development, but they come into third grade too far they come into third grade too far behind in the development of fluent behind in the development of fluent reading skillsreading skills

31rd percentile

32nd percentile



More powerful instruction toward mastery of More powerful instruction toward mastery of the alphabetic principle early in first gradethe alphabetic principle early in first grade

47th percentile

62nd percentile

Over ½ of our students did not make the benchmark on time



More powerful instruction toward mastery of More powerful instruction toward mastery of the alphabetic principle early in first gradethe alphabetic principle early in first grade

Creative work to develop and support stronger Creative work to develop and support stronger interventions for struggling students in grades interventions for struggling students in grades 1,2, and 3 – school level must become 1,2, and 3 – school level must become involvedinvolvedContinued efforts to expand vocabulary Continued efforts to expand vocabulary instruction-linked to comprehension- instruction-linked to comprehension- accessing words in multiple contexts accessing words in multiple contexts

Another way to think about improvements and Another way to think about improvements and performance – two indices of instructional performance – two indices of instructional effectivenesseffectiveness

Effectiveness of Core Instruction (ECI)Effectiveness of Core Instruction (ECI) – what – what percentage of students who began the year at percentage of students who began the year at grade level on the screening measures finish the grade level on the screening measures finish the year at grade level?year at grade level?

Effectiveness of Interventions(EI)Effectiveness of Interventions(EI) – what percentage – what percentage of students who began the year at some level of of students who began the year at some level of risk on the screening measures finish the year at risk on the screening measures finish the year at grade level?grade level?

The ECI and EIThe ECI and EI

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 19 20 21

screening

One academic year

9 of 13 students who were at grade level continue at grade level

ECI = .69ECI = .69

2 of 7 students who began the year “at risk” finished the year at grade level

EI = .29EI = .29

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 19 20 21

outcome

Core and Intervention effectiveness in year 1 Core and Intervention effectiveness in year 1 and 2and 2

Core ProgramCore Program InterventionIntervention

K year 1K year 1 year 2year 2

82%82%87%87%

45%45%55%55%

11stst year 1 year 1 year 2year 2

57%57%65%65%

14%14%16%16%

22ndnd year 1 year 1 year 2year 2

70%70%71%71%

6%6%5%5%

33rdrd year 1 year 1 year 2year 2

84%84%81%81%

15%15%14%14%

District EI %tile %Min %FRL ECI % Grade

Orange 39 99 84 89 82 A

Collier 36 97 98 99 79 D

Orange 34 95 83 80 95 A

Collier 33 93 94 93 93 B

Palm Bea. 33 91 78 75 84 C

Sarasota 32 89 80 85 89 B

Orange 32 89 93 93 67 B

Lee 32 89 67 73 84 B

Palm Bea. 31 89 95 87 66 C

Broward 31 89 99 66 67 B

Schools with High Average Effectiveness of Iterventions

District EI %tile %Min %FRL ECI % Grade

Hillsboro. 8 2 88 93 4 C

Polk 8 3 55 74 10 C

St. Lucie 10 4 58 71 14 C

Polk 10 5 66 89 3 C

Pinellas 10 5 60 74 23 B

Volusia 11 6 68 89 1 C

Brevard 12 7 79 97 32 D

Marion 12 7 47 81 52 A

Hillsboro. 12 7 84 84 45 B

Hillsboro. 12 8 63 58 66 A

Schools with Low Average Effectiveness of Interventions

A final concluding thought….A final concluding thought….

There is no question but that “leaving no There is no question but that “leaving no child behind in reading” is going to be a child behind in reading” is going to be a significant challenge…significant challenge…

It will involve professional development for It will involve professional development for teachers, school reorganization, teachers, school reorganization, careful careful assessmentsassessments, and a relentless focus on the , and a relentless focus on the individual needs of every child…individual needs of every child…

But, its not the most difficult thing we could But, its not the most difficult thing we could be faced with…be faced with…

Consider this task for example…Consider this task for example…

Questions/Questions/

DiscussionDiscussion

Designing and using assessment systems to prevent reading difficulties in young children Dr. Joseph...

Documents

Transcript of Designing and using assessment systems to prevent reading difficulties in young children Dr. Joseph...