Introduction to NYC Teacher Data Initiative

Training for Schools Fall 2008

22

Session Objectives for Principals

>Become familiar with the new Teacher Data Reports

>Consider ways to incorporate this new tool into school-wide instructional improvement activities

>Plan for sharing Teacher Data Reports with teachers

>Locate support resources

3

NYC Teacher Data Initiative Objectives

> Develop statistical model to isolate the impact that individual teachers have on student achievement on state tests from factors outside of teachers’ control

> Provide this teacher data to principals and teachers in an accessible form

> Support schools and teachers to use data: One of many tools for instructional improvement NOT for teacher evaluation

4

Why Consider Teacher Level Data? Student achievement matters to life outcomes

8TH GRADE PROFICIENCY RATINGS ARE PREDICTIVE OF HIGH SCHOOL REGENTS DIPLOMAS

5

Why Consider Teacher Level Data?Teachers matter to student achievement

NYC “Value-Added” teacher data shows what research consistently says: differences in teacher effectiveness

have a significant impact on student performance.

Range of NYC Teacher Value-Added Scores 2006-075th grade

-0.50 -0.25 0.00 0.25 0.50

Math

ELA-.31

-.45 .41

.29

Portion of a Proficiency Level above or below Predicted Gain

66

Teacher Data Initiative Provides a New Lens on Teacher Effectiveness

Purpose: To contribute another lens through which to look at teacher contributions to student learning

Rationale: Teachers make a big difference, and Value-Added data provides a lens to focus on what teachers bring to students rather than what students bring to the classroom

Framing Question: How might the Teacher Data Reports fit into existing school plans for instructional improvement and professional development?

Teacher Data should not be viewed as a silver bullet, big initiative, or accountability metric.

Rather it is a new tool available to principals and teachers to incorporate into their larger instructional and professional development plans.

77

We Use An Array of Instruments to Determine Teacher And School-wide Professional Development Needs

• Classroom observations

• Lesson plans

• Participation in professional development

• Quality of student work products

• Student performance on state assessments

No one measure gives us the full story, but the various pieces come together to create a more reliable picture.

Areas of convergence and dissonance in our observations are equally useful.

88

• Teacher Data Initiative draws on 10 years of city-wide data (test scores, student, teacher, and school characteristics) to predict individual student gains.

• The predicted gains are compared to the actual gains for each student to determine the teacher’s contributions or the “Value-Added.”

• The teacher’s contribution for each student is averaged, and then compared to other 4-8 ELA and Math teachers by rank ordering top 20%, middle 60%, and bottom 20%.

How Teacher Data Works

Predicted Score

Mathematically isolates factors beyond teacher control e.g. prior year

test scores, ELL status, class size, etc

Predicted Score

Mathematically isolates factors beyond teacher control e.g. prior year

test scores, ELL status, class size, etc

Teacher Contribution

Factors within teacher control e.g. quality of

instruction & high expectations

Teacher Contribution

Factors within teacher control e.g. quality of

instruction & high expectations

Actual Test Score

Student scores on ELA & Math tests

Actual Test Score

Student scores on ELA & Math tests

9

9

How Teacher Data Works: The Model

Value-Added for one student

Pro

fici

ency

rat

ing

3rd Grade 4th Grade

3 -

-

2-

PredictedPredicted

Gain

Actual Value Added

Baseline (Previous Year’s) Score

Teacher A

Teacher B

Teacher E

Teacher D

Teacher C

Least Gain

Most Gain

• The “Value-Added” is the difference between the predicted and actual scores

• Value-Added is averaged for all students in a class

• The Value-Added is measured in proficiency ratings

• Teacher Data Reports order teachers from least to most gain to determine a percentile rank

1010

The Model Mathematically Factors In Measurable Characteristics To Calculate Predicted Gain

Student characteristics Classroom characteristics School characteristics

Prior year reading

Prior year math

Free or reduced price lunch

Special education status

English Language Learner status

Number of suspensions and absences (prior-year)

Student retained in grade

Attended summer school

New to school

Race

Gender

Prior year teacher

Average prior year reading and math

Percent free or reduced price lunch

Percent special education status

Percent English Language Learner status

Average number of suspensions and absences (prior)

Percent of students retained in grade

Percent attended summer school

Class size

Percent by race

Percent by gender

Average classroom characteristics

Average class size

Total tested by grade/subject

Year starting and ending school

Teacher Characteristics

(used when comparing teachers to peer teachers)

Years of experience Years teaching in the same grade and subject

11

Interpreting Teacher Data Results:

Is this Result Good?

Teacher Data Reports put a teacher’s results in context of other teachers:

1. Citywide• All teachers in same grade/subject across the city

2. Peer teachers• Also same grade/subject from across the city• Adjusted for teacher experience level overall and in

grade/subject• ONLY 20% of classrooms with similar

student/classroom and school characteristics» Using classroom predicted gain, each classroom is

assigned to a quintile (20%) for peer comparisons

12

Interpreting Teacher Data Results:What Does the Range Indicate?

• The most likely result for any teacher is the result presented on each line of the report and highlighted on graphs with an arrow.

• However, there is some uncertainty with any statistical measurement.

• Teacher Data reports also provide a range within which we can be 95% confidant of the result.

> Ranges are larger for smaller sample sizes or fewer years of data

0% 25% 50% 75%

2007-08 55% 85%

Last 3 years 58% 78%

100%

My percentile (0%-100%)

68%

Range*

70%

My percentile

13

Analyze Sample Reports to Look for Trends and Consider Key Questions

Key Questions> What is being taught? > How is it taught? > Are the students

learning? > How are teachers

learning? > How are resources

invested?

Potential Trends> Clumps of teachers scoring

low with a particular subgroup> Individual teachers

consistently low/high across many groups

> Sizeable difference between math and ELA

> Similar scores among all teachers on a team or in a grade

14

Example: Think Through a Specific Trend

School summary report reveals much higher scores on math than ELA

What is being taught?

Is our math curriculum stronger than ELA?

Consult with schools using similar curriculum

How are resources invested?

Are more push-in resources allocated for math?

Teacher report reveals high scores on everything except for ELL students

What is being taught?

Might ELL students require additional instruction?

Analyze test items for trends in ELL responses

How is it taught?

Is teacher differentiating instruction?

Analyze quantity and quality of math PD

Request peer observations

Consult with others who have high ELL score

Does this teacher receive adequate ELL Support?

Pair this teacher with ELL coach

1515

Teacher Data: Uses and Cautions

Potential Uses> Look for strengths, areas for development,

surprises and wonderings> Emphasize instructional improvement> Triangulate with other insight

Consider factors you know about the teachers or the classrooms that may not be measurable

Help teachers connect these results with insights from their periodic assessments, student work, and item analysis

> Consider professional development approaches for individual teachers or groups

> Consider implications for student class assignment

> Consider implications for curriculum or instructional programs

> Consider implications for staffing needs

> Prioritizing principal observation and coaching

> Inform principal and teacher goal setting

Cautions> Information is not to be used for

teacher evaluation

> Avoid replacing principal judgment and other forms of information

> Not all negative value-added results are bad and all positive results are good

Use the performance ranges to see how strong a positive result is or how weak a negative result is.

> Remember to consider context that is not easily measured and not in the model for example:

Push-in/pull-out teachers AIS services Life events for teachers, students School context

> Gain the teacher’s permission before sharing the report with other teachers

> Consider individual teacher information confidential and thus not sharable with parents

16

Sharing Reports With Teachers: TipsPrincipals are expected to share reports with teachers

> First, take time to get comfortable with the key concepts and the information on the reports.

Consider holding a group session to explain key concepts and review a sample report

> Use materials provided or adapt

Before each individual meeting to share a report with a teacher, consider these questions:

> What do you know about the teacher?> What outcome do you want from the meeting?> What are the key points you want discussed?> Do you want to use the teacher reflection tools provided?> Should you give the teacher time before the meeting to review the report on their own?> What is the best setting, time, and other logistics for the meeting?

During individual meetings:> Ask teacher what strengths, areas of improvement, surprises, wonderings s/he sees> Validate where you agree and continue to question where there is dissonance> Triangulate with other insight from periodic assessments, observations, student work

etc> Discuss how this fits in or modifies existing development goals for teacher> Be clear what your expectations are and how you will help support the teacher

17

Teacher Conversations: Practice

18

Reviewing Own Schools’ Reports

1919

Next Steps to Consider

• How will you use Teacher Data Reports to improve instruction?

• How will you involve others within the school?

• How will you introduce Teacher Data to your staff?

• How will you share individual reports with teachers?

• What additional supports might you need?

20

Support Resources

• Web Resources:> Principal Portal:

http://intranet.nycboe.net/DOEPortal/Principals/default.htm> Teacher Page: http://schools.nyc.gov/Teachers/default.htm

• Help Desk:> TeacherDataReports@schools.nyc.gov> (212) 374-6646

• Leadership Academy coaches, CSA instructors and key UFT Teacher Center staff are knowledgeable

• Talent Office Support:> Sandra Tacina: stacina@schools.nyc.gov> Amy McIntosh: amcintosh2@schools.nyc.gov