Teacher Quality, Teacher Evaluation, and “Value-Added”

38
Teacher Quality, Teacher Evaluation, and “Value-Added” Sean P. Corcoran New York University Education Policy Breakfast April 27, 2012

description

Teacher Quality, Teacher Evaluation, and “Value-Added”. Sean P. Corcoran New York University Education Policy Breakfast April 27, 2012. How did we get here?. Research finds teachers are the most important school influence on student achievement - PowerPoint PPT Presentation

Transcript of Teacher Quality, Teacher Evaluation, and “Value-Added”

Page 1: Teacher Quality, Teacher Evaluation, and “Value-Added”

Teacher Quality, Teacher Evaluation, and “Value-Added”

Sean P. Corcoran New York University

Education Policy BreakfastApril 27, 2012

Page 2: Teacher Quality, Teacher Evaluation, and “Value-Added”

How did we get here? Research finds teachers are the most important

school influence on student achievement

Teachers appear to vary widely in effectiveness, as measured by student gains on standardized tests

Teachers can have long-run measurable effects on life outcomes (e.g., Chetty et al., 2012)

2

Page 3: Teacher Quality, Teacher Evaluation, and “Value-Added”

How did we get here? By many measures, teacher quality is inequitably

distributed across students and schools

There is some evidence that teacher quality has declined over the long-run (Corcoran et al., 2004)

3

Page 4: Teacher Quality, Teacher Evaluation, and “Value-Added”

How did we get here? If teachers are so important, what are we doing to

ensure high-quality teachers can be found in every classroom, particularly for those students who need them the most?

The generally accepted answer among policymakers: not much, or at least current efforts are not working very well (e.g., see The Widget Effect)

4

Page 5: Teacher Quality, Teacher Evaluation, and “Value-Added”

Two key issues What is teacher quality and

how do we measure it?

What policies are most effective in improving the level and distribution of teacher quality?

5

Page 6: Teacher Quality, Teacher Evaluation, and “Value-Added”

Potential Teachers• Self-selection• Recruitment

Preparation• Teacher training• Alternative pathways

Screening• Certification• Testing• Hiring / Placement

Retention• Tenure decisions• Involuntary exits• Turnover/attrition

6

EvaluationProfessional Development

The Teacher Quality Pipeline

Page 7: Teacher Quality, Teacher Evaluation, and “Value-Added”

What is teacher quality? The easy (non-)answer: skills, practices, personal

characteristics that positively impact desired student outcomes

Not a very helpful definition … but does make clear that it is ultimately outcomes that indicate quality

7

Page 8: Teacher Quality, Teacher Evaluation, and “Value-Added”

The “old view” Research and policy emphasized qualifications

and experience as presumed indicators of quality

8

Certification

Certification test scores

Educational attainment (e.g. MA)

Subject matter preparation

College selectivity

Own academic abilities (e.g. SAT)

In-service professional development

Page 9: Teacher Quality, Teacher Evaluation, and “Value-Added”

The “old view” – why? Convenience – these measures are readily

available and easily observable; a lack of data on outcomes themselves

Face validity – on their face, they seem sensible

Reward structure – traditional salary structure rewards these qualifications (e.g. MA, experience)

9

Page 10: Teacher Quality, Teacher Evaluation, and “Value-Added”

The “old view” NCLB’s Highly Qualified Teacher provision: all

teachers of core academic subjects must: Have a BA or better in the subject matter taught Have full state certification Demonstrate subject matter knowledge

10

Page 11: Teacher Quality, Teacher Evaluation, and “Value-Added”

Potential Teachers• Self-selection• Recruitment

Preparation• Teacher training• Alternative pathways

Screening• Certification• Testing• Hiring / Placement

Retention• Tenure decisions• Involuntary exits• Turnover/attrition

11

EvaluationProfessional Development

The Teacher Quality Pipeline

ProfessionalDevelopment

Policies that set high professional standards and barriers to entry

Page 12: Teacher Quality, Teacher Evaluation, and “Value-Added”

The “old view” Surprisingly (or not)

research has not found qualifications to be highly predictive of student outcomes (i.e. test scores), although some do better than others

12

Page 13: Teacher Quality, Teacher Evaluation, and “Value-Added”

The “new view” “Teaching should be open to anyone with a pulse

and a college degree—and teachers should be judged after they have started their jobs, not before” Malcolm Gladwell, The New Yorker, 12/15/2008

13

Page 14: Teacher Quality, Teacher Evaluation, and “Value-Added”

The “new view” “Success should be measured by results…That’s

why any state that makes it unlawful to link student progress to teacher evaluation will have to change its ways.” President Barack Obama, July 24, 2009

14

Page 15: Teacher Quality, Teacher Evaluation, and “Value-Added”

The “new view” In other words, let outcomes be the arbiter of quality

Great in theory, but which outcomes do we measure, and how does one measure teachers’ contribution to them?

How does one incorporate this information into personnel policies in ways that have desired effects?

15

Page 16: Teacher Quality, Teacher Evaluation, and “Value-Added”

Potential Teachers• Self-selection• Recruitment

Preparation• Teacher training• Alternative pathways

Screening• Certification• Testing• Hiring / Placement

Retention• Tenure decisions• Involuntary exits• Turnover/attrition

16

EvaluationProfessional Development

The Teacher Quality Pipeline

Evaluation

Policies that focus on measurement and incentives

Page 17: Teacher Quality, Teacher Evaluation, and “Value-Added”

Measurement: outcomes Outcomes: to date, whatever we have on hand

Typically, student growth on standardized tests in reading and math, grades 3-8 (though not for long)

Necessarily a subset of expected skills/outcomes Necessarily a short-run outcome

Is our evaluation measure properly aligned with the goals we have for our educational system?

17

Page 18: Teacher Quality, Teacher Evaluation, and “Value-Added”

Measurement: value-added Value-added:

The theoretical construct: a teacher’s unique impact on student learning

In practice, a statistic used to estimate this impact

18

Page 19: Teacher Quality, Teacher Evaluation, and “Value-Added”

Measurement: value-added “Unique impact” implies causality – i.e. ruling other

possible explanations for student learning

Several possible sources of error: Systematic error (bias): attributing “value-added” to

the teacher when it is really due to some other factor Random error (noise): getting a “noisy signal” of the

teacher’s contribution to learning

19

Page 20: Teacher Quality, Teacher Evaluation, and “Value-Added”

20

Page 21: Teacher Quality, Teacher Evaluation, and “Value-Added”

21

Page 22: Teacher Quality, Teacher Evaluation, and “Value-Added”

Measurement: value-added So how can we attribute

causality to a teacher?

If teachers were randomly assigned, this would be easy: systematic differences would almost surely be due to the teacher

22

Page 23: Teacher Quality, Teacher Evaluation, and “Value-Added”

Measurement: value-added In the absence of this, we can instead devise a

statistical model to account for other factors that explain differences in achievement

23

Page 24: Teacher Quality, Teacher Evaluation, and “Value-Added”

Measurement: value-added Value-added is then defined as student achievement

relative to predicted—in other words, there will always be a distribution of value-added

24

0

+-

Page 25: Teacher Quality, Teacher Evaluation, and “Value-Added”

Value-added: bias How confident are we that value-added measures

isolate the unique contribution of individual teachers? Classroom vs. teacher effects (esp. after 1 year) Teacher vs. school effects Mobile students Tracking (e.g. Rothstein falsification test)

25

Page 26: Teacher Quality, Teacher Evaluation, and “Value-Added”

Value-added: bias Does attributing outcomes to individual teachers

even make sense? Middle and high school settings Team teaching Evidence that teacher peers matter

The higher the stakes places on value-added measures, the more these questions matter

26

Page 27: Teacher Quality, Teacher Evaluation, and “Value-Added”

Value-added: noise Even if value-added measures are not biased, they

are still noisy—i.e. they are estimates with a high “margin of error”

More years of test results helps, although this may be “too late” to provide actionable information

27

Page 28: Teacher Quality, Teacher Evaluation, and “Value-Added”

28

Page 29: Teacher Quality, Teacher Evaluation, and “Value-Added”

29

Page 30: Teacher Quality, Teacher Evaluation, and “Value-Added”

30

Page 31: Teacher Quality, Teacher Evaluation, and “Value-Added”

31

Page 32: Teacher Quality, Teacher Evaluation, and “Value-Added”

Implications for policy The promise of personnel decisions driven by

outcomes has led to sweeping reforms of Performance evaluations Tenure and promotion, dismissal Compensation Principal evaluation Evaluation of teacher training programs

32

Page 33: Teacher Quality, Teacher Evaluation, and “Value-Added”

Implications for policy Race to the Top led numerous states to propose

50% or more of performance evaluations to be the “teacher’s impact on student achievement” E.g. CO, FL, TN, NJ Indiana: “negative” value-added teachers may not

receive an effective rating, and tenure requires 3 years of effective ratings in a row

NY’s APPR: a somewhat more balanced approach

33

Page 34: Teacher Quality, Teacher Evaluation, and “Value-Added”

Potential Teachers• Self-selection• Recruitment

Preparation• Teacher training• Alternative pathways

Screening• Certification• Testing• Hiring / Placement

Retention• Tenure decisions• Involuntary exits• Turnover/attrition

34

EvaluationProfessional Development

The Teacher Quality Pipeline

Page 35: Teacher Quality, Teacher Evaluation, and “Value-Added”

Implications for policy What can we realistically expect from value-added

based policies? Not as much timely, actionable information as we

might like – though perhaps useful as an early warning indicator

Crude differentiation of teachers at best, but more than current practice

35

Page 36: Teacher Quality, Teacher Evaluation, and “Value-Added”

Implications for policy What are the risks and implications of a system

based on high-stakes use of imprecise measures? Mechanical applications are dangerous Risk of improper attribution and “Type I errors” Public reporting has minimal benefits and may do

harm Unnecessary diversion of resources Unclear effects on entry into teaching profession

36

Page 37: Teacher Quality, Teacher Evaluation, and “Value-Added”

Implications for policy Little is know about how value-added measures

will be used in practice

37

Page 38: Teacher Quality, Teacher Evaluation, and “Value-Added”

References Excellent and (mostly) non-technical resources:

Corcoran (2010) report for Annenberg http://www.annenberginstitute.org/products/Corcoran.php

Harris (2010) Value Added Measures in Education Koretz (2008) in American Educator Braun (2005) primer for ETS “Merit Pay for Florida Teachers: Design and

Implementation Issues” (RAND 2007) Rivkin (2007) CALDER policy brief Harris (2009) and Hill (2009) point/counterpoint in the

Journal of Policy Analysis and Management

38