Instruction, Teacher Evaluation and Value-Added Student Learning

Minneapolis Public SchoolsNovember, 2002

www.mpls.k12.mn.us/REA/

Caution! Do not use the Federal Framework

for designating Adequate Yearly Progress to evaluate the effectiveness of Instruction: It won’t work!

Clinical studies have shown that AYP can be injurious to school, teacher and student long-term health.

Minneapolis Public SchoolsMeasuring Up Against High Standards

MPS developed standards-based reading and math tests - Northwest Achievement Levels Tests (NALT).

All students participate Sensitive to growth Predictive of high stakes outcomes

Developed indicators in concert with stakeholders who are internal and external to the district.

Indicators of school performance are based on multiple indicators. 33 for elementary and middle schools 24 for high schools

Linking Assessments to High Standards:

Choose items for the assessment bank which directly link to the standards. Use your teacher experts, curriculum specialists, and assessment personnel to create the bank

“All Means ALL” - create levels of the assessment which allow for 98% of students being assessed

Find out where the State benchmarks locate on the scale (linking studies)

Measure continuous progress towards the Standard

Elementary/Middle QPA 33 Indicators

37%

33%

6%

6%

6%

6%6%

Reading

Math

Writing

Behavior

Climate

Attendance

Programs

Criteria for Indicators Meyer (1996)

Indicators must measure things that matter or are valued by society (MCA, MBST).

Indicators must be able to avoid being “corrupted.”

inclusion v. exclusion criteria Indicators must be able to measure the

unique contribution schools “add” for each measured outcome.

Linkage with State Standards

NALT Scale Score Which Predicts Passage of theMinnesota Basic Standards Test (MBST) in 8th Grade

Grade Reading Scale Score Reading Scale Score

3 190 1994 197 2065 204 2146 209 2187 213 2258 218 232

True gain model: Growth Curves

Class of 2005 Longitudinal Reading Gains

190

200

210

220

230

Grade 3

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8

Grade 9

Grade 10

Grade 11

Grade 12

NA

LT

Sc

ale

Sc

ore

(R

as

ch

Mo

de

l)

High Standard

Basic Standard

Class of 2005 (n=2006)

Improving Student Performance One Student at a Time

180

190

200

210

220

230

240

250

Grade 2

Grade 4

Grade 6

Grade 8

Grade 10

Grade 12

Josh (AI)

Fahim (AA)

High Standard

Basic Standard

Figure 22. Longitudinal NALT Reading GainsClass of 2007

Figure 22. Longitudinal NALT Reading GainsClass of 2007

180

190

200

210

220

230

240

Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8

Me

an

Sca

le S

core

High Standard

Basic Standard

Class of 2007(n = 2058)

Correlation of Poverty and Math NALT ScoresR2 = 0.7901

30

40

50

60

70

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Percent free or reduced price lunch

NA

LT M

ath

Ave

rage

NC

E

School P = 44.4 nce

Distribution of school average mathematics gain scores on the 1999 NALT by free or reduced price

lunch percentages.

R2 = 0.0074

-3-2-10123456789

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Percent free or reduced price lunch

NA

LT

Mat

h N

CE

Gai

n

School P = 6.3 nce gain

Percent of Students Making About One Year Growth or More on NALT

Math by Racial/Ethnic Category

51% 48%53% 53%

60%58% 56%

66% 63% 64%

0%

20%

40%

60%

80%

100%

AmericanIndian

AfricanAmerican

AsianAmerican

HispanicAmerican

WhiteAmerican

Per

cent

age

of S

tude

nts

2000

2001National Norm = 57%

Characteristics of “Needs Improvement Schools” in

MPS One school in this category two

years in a row: Student Stability 78% Staff Stability 33% (14/21 new staff) Three principals in three years Low parent choice Higher Poverty (84%) Large ELL Population (32%)

Characteristics of “Distinguished School” in

MPS One school in this category two

years in a row: Student Stability 94% Staff Stability 87% (3/23 new staff) One principals in three years High parent choice (waiting list) Lower Poverty (40%) Moderate ELL Population (20%)

Value-added Model

Primary question is this, “How can an equitable accountability system be put in place when there is a large degree of variability in the way student characteristics are distributed among our schools?”

English language learners Special education programs Race/ethnicity Poverty

MPS relies on an empirical- and literature-based model to make predictions, not set expectations!

Schools are acknowledged/rewarded for “Beating the Odds.”

Value-added Teacher Effects

Post-test reading score= Pretest reading score + Free or reduced price lunch + Racial/ethnic code + Neighborhood poverty concentration + Lives with both parents code + Limited English Proficiency status + Special Education status + Teacher effects

Teachers who “beat the odds” in second grade reading reported:

more use of small group instruction more development of word attack

skills more individual student oral reading more guidance during initial practice more explicit and direct phonics

instruction more use of systematic motivation

strategies

Measurement Recommendations Measure students on standards-based

assessments as soon as they enter school Continuously monitor student progress

towards the standards Produce individual student growth curves Use data to monitor the effectiveness of

interventions Identify schools and teachers who make

exceptional progress with “at risk” students

Intervention Recommendations

Involve exceptional teachers in staff development (modeling,video tape, etc.)

Give incentives for exceptional teachers to work in the highest need schools

Place staff development in the schools with “hands on” modeling and support

Identify teaching practices that correlate with success and replicate them

Reward groups of teachers (e.g. schools) who “beat the odds”

Number of Words Read Correctly per minute End of Kindergarten 2002

0

5

10

15

Percent of Students

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

Number of Words Correct per Minute

Note: 53% of all Kindergarten students read10 or more words correctly in Spring 2002

Letter Sounds Gain from Fall to Spring for Half Day vs. Full Day Kindergarten

05

10152025303540

Number of Letter Sounds

Gain

AfricanAmerican

AmericanIndian

Asian Hispanic White Total

Half Day Full Day

Kindergarten Letter Sounds Gains from Fall to Spring by Length of Day and Intervention Group

05

1015202530354045

Number of Letter Sounds

Gain

AfricanAmerican

AmericanIndian

Asian Hispanic White Total

Half Day - Partial All Day - Full Intervention

Reading Fluency Value-Added

0

0.2

0.4

0.6

0.8

1

Effect Size (in standard

deviation units)

Full DayKindergarten

Full Intervention Full Day + FullIntervention

All Students African American White Hispanic Asian