The Delaware Statewide Academic Growth Assessment Pilot

Presented to

The Brandywine School District Legislative Dessert

February 22, 2006

How Did the Pilot Come About?

2003: BSD task force developed “Dashboard Metrics”; 2003: BSD task force developed “Dashboard Metrics”; requested a measure of individual growthrequested a measure of individual growth

2004: BSD learned “value-added” analysis couldn’t be done on 2004: BSD learned “value-added” analysis couldn’t be done on DSTPDSTP

2004: BSD invited NWEA and Christina to meet2004: BSD invited NWEA and Christina to meet

2004-2005: BSD and Christina pilot in limited classrooms2004-2005: BSD and Christina pilot in limited classrooms

2005: Urban League requests donations to support broader pilot2005: Urban League requests donations to support broader pilot

2005-2006: 26,000 students in five districts and four charter 2005-2006: 26,000 students in five districts and four charter schools enter pilotschools enter pilot

Quick Facts About NWEA in BSD

WhWhich students in BSD take the NWEA tests?ich students in BSD take the NWEA tests? grades 6-10grades 6-10: : mathmath ggrades 7-10: readingrades 7-10: reading

Will we expand use of it?Will we expand use of it? 2006-2007: add grades 4-62006-2007: add grades 4-6 2007-2008: add grades 2-32007-2008: add grades 2-3

When is it taken? When is it taken? September, January, April/May, end of summer school September, January, April/May, end of summer school

How much time does it take to complete?How much time does it take to complete? Approximately 45-60 minutes per testApproximately 45-60 minutes per test

The Statewide Growth Pilot: Goals of the Pilot

1)1) Improve Teaching and Learning:Improve Teaching and Learning:

Provide teachers with highly useful information about the Provide teachers with highly useful information about the

instructional needs and rate of progress of each student and instructional needs and rate of progress of each student and

each subgroup of students throughout the year so they can each subgroup of students throughout the year so they can

better target and adjust instruction and impbetter target and adjust instruction and improve learningrove learning..

2)2) Support Smart School/System Decisions:Support Smart School/System Decisions:

Use data on student growth from September to May to identify Use data on student growth from September to May to identify

how to get the greatest growth for the investmenthow to get the greatest growth for the investment, and inform , and inform

improvement plansimprovement plans (district, school, grade level, content area, (district, school, grade level, content area,

individual).individual).

3)3) Improve the Accountability System:Improve the Accountability System:

Fairness: Fairness: Judgments based on growthJudgments based on growth

Accuracy: Accuracy: More accurate individual resultsMore accurate individual results

Puts Learning First: Puts Learning First: Fast, helpful feedback several Fast, helpful feedback several times each yeartimes each year

Empowers Parents:Empowers Parents: More detailed information to More detailed information to parentsparents

How Is This Test Different?

Conventional Tests Contain only 45-60 fixed items; Contain only 45-60 fixed items;

designed to measure designed to measure on-grade on-grade levellevel performance only performance only

Very inaccurate scores for Very inaccurate scores for highest and lowest performers – highest and lowest performers – too few questions at extremes too few questions at extremes and student boredom/frustrationand student boredom/frustration

Provide generic, minimally Provide generic, minimally useful feedback due to fixed useful feedback due to fixed items on test and long delay in items on test and long delay in getting resultsgetting results

Beginning Math

High School Math

5th Grad

e

NWEA’s MAP Test “Computer adaptive” -

dynamically developed for each student to “drill down” to their understanding of each concept

Approx. 50 items selected for student from item bank of 15,000, ~ aligned to DE standards

Accurate results across the scale, from lowest to highest performers

Avoids student boredom, frustration

Immediate results to student, next day to teacher

5th Grade

Beginning Math

High School Math

X Daniel

-

-+

-

--

+ +

-++

181899

RIT Scale

Pilot Goal #1: Improve Teaching and Improve Teaching and

LearningLearning Focuses teaching on each student’s instructional needs,

increasing engagement and accelerating growth

Allows teachers to quickly adjust student groupings and instructional materials several times each year based on needs of the students

Quickly highlights effective practices, using common yardstick

Provides usable information to parents: e.g. Lexile score, specific skill needs

Teacher/Class OverviewTeacher Landing Page

Drilling Down to Actionable DataDrilling Down to Actionable Data

Class Reports group students by subjectClass Reports group students by subjectand current performance level.and current performance level.

Goal Reports group students by Delaware skill or concept they need to Goal Reports group students by Delaware skill or concept they need to work on next within each subject,work on next within each subject, and reveal gaps and strengths. and reveal gaps and strengths.

The Continuum of Learning specifies the skills, concepts and The Continuum of Learning specifies the skills, concepts and vocabulary the student needs to work on next – the vocabulary the student needs to work on next – the

instructional level that will challenge but not frustrate.instructional level that will challenge but not frustrate.

“The Learning Continuum is incredibly helpful – this has been a “missing

piece” in Delaware for a long time.” Delaware teacher

DSTPDSTP Math RIT Score Math RIT ScoreLevelLevel 88thth Grade Grade 10 10thth Grade Grade

Distinguished Distinguished 254 and above 257 and above 254 and above 257 and aboveExceeds Exceeds 248 – 253 252 - 256 248 – 253 252 - 256 Meets Meets 233 – 247 240 - 251 233 – 247 240 - 251BelowBelow 217 – 232 226 - 239 217 – 232 226 - 239Well BelowWell Below below 217 below 226 below 217 below 226

We can currently have this information for grades 8 and 10 in reading and math. We will have even more accurate predictions and have them for all

grade levels, 2 – 10, by summer 2006.

Predicting DSTP Performance

Observations from Delaware Teachers after their first NWEA

testing session: Much less misbehavior due to test frustration or boredom. Much less misbehavior due to test frustration or boredom.

Special Education students not stressed, upset or non-Special Education students not stressed, upset or non-compliant. compliant.

Results are more accurate.Results are more accurate.

Results are detailed and immediately useful.Results are detailed and immediately useful.

Able to use results to motivate and focus students, and reduce Able to use results to motivate and focus students, and reduce test anxiety.test anxiety.

Inform and Empower Parents

Joseph W.: Mathematics

www.lexile.com

Advanced Book Search Advanced Book Search (enter some or all)(enter some or all)

Title: Title: Author: Author: Lexile Range: Min: Lexile Range: Min: 330330 Max: Max: 500500Book Type: Book Type: Fiction FictionReading Series:Reading Series:Book Awards:Book Awards:Development Rating: Development Rating: ElementaryElementaryKeyword(s): Keyword(s): HorsesHorsesSort by:Sort by:

Pilot Goal #2: Pilot Goal #2: Support Smart Support Smart

Management DecisionsManagement Decisions

School Growth and Projected Proficiency Principal’s Landing Page

17%

24%

37%

22%

Just a Few Examples of Powerful Uses

of Growth and Performance Reports

Motivate Students: Motivate Students: Teachers and several BSD prTeachers and several BSD priincipals have ncipals have begun meeting with students to discuss their begun meeting with students to discuss their NWEA reports – their NWEA reports – their strengths and areas to target for improvementstrengths and areas to target for improvement

Accelerating Students:Accelerating Students: Identify individuals and groups of students Identify individuals and groups of students ready for more rigorous courseworkready for more rigorous coursework

Coaching:Coaching: Identify teachers who are particularly effective at Identify teachers who are particularly effective at teaching tough concepts and can coach other teachersteaching tough concepts and can coach other teachers

Examples, continued

Curriculum/Instruction Gaps:Curriculum/Instruction Gaps: Identify concepts that students are dentify concepts that students are struggling to learn and determine needed changes in curriculum, struggling to learn and determine needed changes in curriculum, materials, supports or professional development materials, supports or professional development

Cost/Benefit Program Evaluation:Cost/Benefit Program Evaluation: Measure and compare the Measure and compare the effectiveness of professional training programs, tutoring programs, effectiveness of professional training programs, tutoring programs, etc. using NWEA’s virtual control groupsetc. using NWEA’s virtual control groups

National Benchmarking:National Benchmarking: Allows district-to-district benchmarking Allows district-to-district benchmarking across statesacross states

Pilot Goal #3:

Improve the Improve the Accountability SystemAccountability System

Weaknesses of Current Weaknesses of Current NCLB ImplementationNCLB Implementation

Misjudges many schools by looking exclusively at Misjudges many schools by looking exclusively at percent proficient rather than a combination of percent proficient rather than a combination of achievement and growthachievement and growth

no reward or recognition for teachers making great gains no reward or recognition for teachers making great gains with our most challenging students with our most challenging students

misjudgments harms public and educator confidence in the misjudgments harms public and educator confidence in the systemsystem

Weaknesses of Proposed New “Growth” Model for

Delaware Uses “cohort” Uses “cohort”

rather than rather than individual growthindividual growth

Uses DSTP, with Uses DSTP, with low accuracy for low accuracy for highest and lowest highest and lowest kidskids

Pays no attention Pays no attention to those above to those above standard (NCLB)standard (NCLB)

Year 1 Level

Year 2 Level

Level

1A

Level

1B

Level

2A

Level

2B

Level

3

Level

4

Level5

Level 1A 25 150 225 250 300 300 300

Level 1B 25 75 175 225 300 300 300

Level 2A 0 25 125 200 300 300 300

Level 2B 0 0 50 125 300 300 300

Level 3 0 0 25 100 300 300 300

Level 4 0 0 0 50 300 300 300

Level 5 0 0 0 0 300 300 300

Building A Better Future:

Individual Growth Targetsfor All Students

0100200300400500600700800900

3rd 4th 5th 6th 7th 8th

Grade

Sca

le S

core

A

B

C

Individual Growth-to-Standard

Prof.

Below

• A must maintain growth rate• B must accelerate and Meet Stds• C must get on track to Meet Stds

The NWEA Pilot’s Position:

The Ideal Next-Generation DSTP Should…

Provide immediate, highly detailed and useful information to teachers, several times each year, to improve student learning

Provide accurate longitudinal student growth and performance data to schools, districts and the state, on a nationally used scale, for strategic management and resource allocation decisions

Provide the basis for a highly trusted accountability system in which schools are rated based on student growth-to-standard

Be cost-effective

Could NWEA Replace the DSTP?

New state tests require about 2 years of work to bring online, from New state tests require about 2 years of work to bring online, from RFP date to use, to meet psychometric and legal requirementsRFP date to use, to meet psychometric and legal requirements

US DOE would require several improvements to NWEAUS DOE would require several improvements to NWEA – e.g. – e.g. stronger alignment to our Standards – or we would lose all federal stronger alignment to our Standards – or we would lose all federal fundingfunding

NCLB law requires “grade level testing” – either change law or NCLB law requires “grade level testing” – either change law or develop two-phase testdevelop two-phase test

Need to determine if we Need to determine if we want or need student-constructed itemswant or need student-constructed items

• Essay would have to be handled separatelyEssay would have to be handled separately

Conclusion: Begin work soon Conclusion: Begin work soon ifif we want to use this in 2008-2009 we want to use this in 2008-2009

Contact Information:Contact Information:

Delaware Statewide Academic Growth Assessment PilotDelaware Statewide Academic Growth Assessment Pilot

Participating Partners:Participating Partners:

Districts: Brandywine, Christina, Red Clay, Caesar Rodney and MilfordDistricts: Brandywine, Christina, Red Clay, Caesar Rodney and Milford Charter Schools: MOT, Marion T., Kuumba and East SideCharter Schools: MOT, Marion T., Kuumba and East Side Urban League, Business Roundtable, DOE, Rodel Foundation, Charter Urban League, Business Roundtable, DOE, Rodel Foundation, Charter

Schools Network, Delaware Foundation for Science & Math EducationSchools Network, Delaware Foundation for Science & Math Education

Pilot Coordination: Metropolitan Wilmington Urban LeaguePilot Coordination: Metropolitan Wilmington Urban League Nancy Doorey, Pilot Coordinator, Nancy Doorey, Pilot Coordinator, ndoorey@comcast.netndoorey@comcast.net, 302-764-9030, 302-764-9030

Background on NWEA Non-profit, originated 30 years ago by educators; computer-adaptive

since 1998

Used by more than 1,700 school districts in 46 states (~ 11% of US schools)

Vertical scale has been stable for more than 20 years – only one of its kind

Item bank of more than 15,000 items; currently being expanded

Allows national “virtual control groups” by race, SES, achvmt, gender

Developing partnerships with quality educational software vendors for optional supplemental links