July 19-21, 2011Stonewall Resort, Roanoke, West Virginia

Educator Evaluation Pilot

Overview

• The “Big Picture”–Where are we headed?–Where have we been?–How will we get there?

• Two Day Agenda• Next Steps

VISION:West Virginia will have a

comprehensive and equitable evaluation system that clearly

articulates, measures, rewards, and develops educator

effectiveness

Where Have We Been?

Historical Perspective

2009 Standards Adopted

2011 Evaluation Task Force Worked

2012 Revised System Piloted

Importance of Leadership

• July 20-21 – Stonewall Resort• ALL Pilot schools leadership team

members– Principals and all assistant principals – Title I Directors and/or Assistant Superintendents– Teacher Leaders– Central office representative for non SIG pilot

schools

Clear Expectations

• Embrace the Opportunities:• Be “early adopter”• Engage in professional development• Work as part of a collaborative team• Provide input to policy makers

• Participate in the research study• Communicate, partner and problem-solve

Day 1 Agenda

• Expectations for Training• History Evaluation Creation • Conceptual Framework Overview• Self-Assessment • Observation• Student “Growth”

Day 2 Agenda

• SMART Goals and Collaboration• Student Learning Goals• Professional Conduct• Research• Performance Assessment

What are the Next Steps?

Teacher Evaluation Training Dates

• August 6-12 – Regional Training• Participants: all teachers, administrators

and transformation specialists• Fall and Spring Training• On site visits• Ongoing Technical Assistance

Historical Perspective

2009 Standards Adopted

2011 Evaluation Task Force Worked

2012 Revised System Piloted

2013 Begin Statewide Scale Up

Thank You!

Expectations for Professional Development

Expectations for Professional Development

• Learning Target– Understand how the new evaluation

system works to be able to implement the processes yourself and to be able to assist others in school

• Strategies to accomplish the goal– Hands-on activities– Multiple opportunities to master content– Collaboration with tablemates

Expectations for Professional Development

• Expectations for us– Well prepared– Clarity– Consideration of the audience

• Expectations for you– Active participation– Regular feedback– Focused attention

Teacher Evaluation Creation

Task Force Members’ Perspective

Evaluation Perceptions & Experiences

Table Tasks

• Describe most recent evaluation experience– What was the purpose? What feedback

was provided? What was your role in the process?

• What are the challenges of teacher evaluation (not including time)?

• What will you need to learn to make this PD beneficial?

A New Evaluation System:

The Rationale

A New Evaluation System:

The Rationale• Learning Target: Know

– the three drivers behind a revised evaluation system for principals and teachers

Converging Forces

New Teaching Standards

Weak Evaluation System

Federal Encouragement

Conceptual Framework Overview

Conceptual Framework Overview

• Learning Targets: Understand– the nexus between the WVPTS and the

Critical Standard Elements; – how levels of teacher performance are

defined with rubrics; – the tiered evaluation system with

progressions based on experience

Levels of PerformanceDistinguished Accomplished Emerging Unsatisfactory

Table Tasks

• Share one activity with distinguished performance, one activity with unsatisfactory performance

Levels of PerformanceDistinguished Accomplished Emerging Unsatisfactory

Distinguished performance describes professional teaching that engages students to be highly responsible for their own learning. Performing at this level involves contributing to the professional learning of others through teacher leadership.

Accomplished performance describes professional teaching that exhibits mastery of the work of teaching while improving practice and serving the professional community.

Emerging performance represents teaching that demonstrates knowledge and skills to implement essential elements albeit not always successfully at times.

Unsatisfactory performance describes teaching that does not convey sufficient understanding of concepts or the successful implementation of essential elements.

Self Assessment – Advance Progression

Self Assessment – Advance Progression

• Learning Targets:– Identify individual performance within

the established rubrics

Table Tasks

• Share an insight from doing this activity about individual performance

• Share reaction to critical standard elements

• What implications does this have for individual professional development

Table Tasks

• Discuss perception of how comfortable teachers with 6+ years of experience will be completing self assessment

• Discuss how this process could be used in relation to the PD plan for of the school

• Discuss how this process could be used in collaborative teams.

Observation

Observation

• Learning Targets:– Understand that observation is formative

and offers a window into instructional performance;

– Understand that observation, though limited, is informative about certain key aspects of instruction

Some specifics about observation

• Not the evaluation• Initial Progress-4, Intermediate-2,

Advanced if requested• Class period or minimum of 30 mins• One piece of a two part conversation• Will be supported by evidence and

conversation• Elements that contribute to the

research

Table Tasks

• What does observation tell you about the critical standard elements?

• What elements do you still need to know about to make a fair and accurate assessment about performance?

Evidence

Evidence

• Learning Targets:– Understand that the educator plays an

active role in demonstrating performance level by providing evidence;

– Identify and classify evidence

Table Tasks• Share the kinds of evidence that were

generated.• Were any common among the table?• Discuss whether you agree that

evidence “presented” was acceptable.• Even though none are required, which

ones would be considered essential?• Which ones should be brought to

conference with principal?

Evidence For Self Assessment

Table Tasks• Share the kinds of evidence that were

generated.• Were any common among the table?• Does the evidence convincingly support

the rating?• Discuss whether you agree that

evidence “presented” was acceptable.• Even though none are required, which

ones would be considered essential?

Student Growth 101

Juan D’BrotExecutive Director

Office of Assessment and Accountability

Student Growth 101

• Learning Targets:– Understand how the school-wide growth

measure is developed

The WV Growth Model:Changing Conversations

about Education

Juan D’BrotExecutive Director

Office of Assessment and Accountability

WV Growth Model

1. What questions?2. Purpose3. Status vs. growth4. What student growth data looks like5. What school growth data looks like

First…

A challenge!

Stop me for questions…

Asking the Right Questions…

• Until you’ve defined the question, you cannot examine the appropriate data.

• Why?– Different data answer different

questions – Different questions lead to different

conversations

Fact #1: …increased use of student assessment data as a “significant

factor” in (insert favorite accountability topic here…)

• But what student data? That leads us to start asking questions…

• But what questions?

• Whose questions?

Whose Questions?• Researchers

• Administrators

• Teachers

• Parents

• Policymakers

• Business Leaders

• Community Members

• Guardians

WHY?

To answer questions about the student

How?By not declaring a verdict. Instead, by changing the

conversation…

Components of the System should Align with the Purpose of the System• The revised teacher evaluation system is

intended to do what?– Change conversations– Provide support and direction– Identify areas of best practice and need

• The revised consideration of student learning (growth) is intended to do what?– Change conversations– Provide direction – Reframe evidence of student learning

Bigger Question:

How do we align the revision of both systems?

…Fact #2

Fact #2: Assessment Systems try to be Everything to Everyone

• Danger: answering too many questions with the same data– Differentiated Instruction– Student Proficiency– Class Performance– AYP

• Possible Solution: Can we come up with a common question across stakeholders?

The WV Growth Model: Our Purpose

• To provide an answer to a few common questions: 1. “How much did my student grow this

year?” 2. “What does this growth mean

compared to everyone else?”3. “Is it enough growth?”

What do these three questions have in common?

Information about students

Status vs. Growth

• Before growth, let’s discuss status

– Status: A snapshot measure of a single point in time – “Proficiency”

– Growth: Multiple snapshots across many points in time

Status vs. Growth A Balanced Approach

Low Growth High Growth

Quadrant 2 Is believed not to require

improvement because declining student growth is not recognized

under status model

Quadrant 3Does not require improvement

and is a potential site of interest for best practices

Quadrant 1Legitimately requires

improvement

Quadrant 4Is believed to require

improvement because high rates of growth are not recognized

under the current status model. May be a potential site of interest

for best practices.

High Performing

(Status)

Low Performing

(Status)

Questions - Revisited

1. “How much did my student grow this year?” (Time 2 – Time 1) – We can see this today

2. “What does this growth mean compared to everyone else?” (Normative Component)

3. “Is it enough growth?” (Criterion Component - CSOs)

How much did my student grow this year?

–1st Question » How much academic growth do individual students in

WV exhibit?» Scale Scores – Tell very different stories» Time 2 (2010 Scale Scores – Mastery = 550) – Time 1

(2009 Scale Scores – Mastery = 500)

2009 Scale Score 2010 Scale Score Net “Growth”

525 575 50

550 575 25

425 575 125

375 400 25

What does this growth mean compared to everyone else?–Pre-Growth Model

»This year and last year (only 2 years worth of comparison)

–Post-Growth Model»Contextual consideration of growth»Yields “Student Growth Percentiles” »Think height

What is a Student Growth Percentile?• Given (3 things):

– A student’s prior scale scores – Academically similar students – Current scale score

• A student’s current scale score represents a percentile of growth – a Student Growth Percentile

• Think of it as the probability of a student’s current achievement based on their past achievement:– Pr(Current Achievement|Past Achievement)

• Growth percentiles describe the probability/rarity of a student’s current achievement based upon their prior achievement.

Vertical Axis: Number of Students at a Particular Scale Score

Vertical Axis: Number of Students at a Particular Scale Score

Vertical Axis: Number of Students at a Particular Scale Score

Vertical Axis: Number of Students at a Particular Scale Score

What does this student’s growth mean compared to everyone else?

–2nd Question » What does the academic growth of an individual

student in WV really mean? » Scale Score Growth – Tells very different stories» Time 2 – Time 1: Based on other students with similar

academic histories. 2009 Scale

Score2010 Scale

ScoreNet “Growth” Growth

Percentile525 575 50 40th

550 575 25 20th

425 575 125 99th

375 400 25 50th

But is it enough growth?

–3rd Question » Is the academic growth of an individual student in WV

enough to make it to Mastery? What about staying at Mastery?

» This year’s Growth Percentile isn’t enough

» Where would a student be next year if s/he exhibited:• High Growth• Typical Growth• Low Growth

Our Growth Options• How much growth? – Starts with the student.

Can be aggregated to drive school conversations– Very Low Growth - Unsatisfactory– Lower Growth - Emerging– Typical Growth - Accomplished– Higher Growth – Distinguished

• Is it enough growth? – Can drive student learning conversations– Catching Up– Keeping Up– Falling Behind

How much growth?

Is it enough growth?

Novice

Partial Mastery

Mastery

Above Mastery

Novice Partial Mastery

Partial Mastery

Mastery

Novice

Partial Mastery

Mastery

Above Mastery

Novice

Partial Mastery

Mastery

Above Mastery

Novice Partial Mastery Partial Mastery Mastery

Partial Mastery

Partial Mastery Partial Mastery Partial Mastery

Novice

Partial Mastery

Mastery

Above Mastery

Novice

Partial Mastery

Mastery

Above Mastery

Novice Partial Mastery Partial Mastery Mastery

Partial Mastery

Partial Mastery Partial Mastery Partial Mastery

From Student to School Growth–For whom can we calculate growth?» Only students in grades 4 – 11, in tested subjects» Students must have at least 2 consecutive scores» We examine RLA and Math

–How do we aggregate growth to the school?

» Examine students by grade and by content for the state» Identify all of the students in the school» Take the median (middle) growth percentile of all students in

a school for each grade and content.

Let’s start with 5th gradeStudent Mathematics

Johnny 25th Suzy 35th

Kenny 75th Lori 85th

Juan 15th

Lisa 40th

Amelia 27th

Robert 40th

Lee 45th

Jan 62nd

Frank 51st

Mary 12th

5th Grade MEDIAN Growth 40th

How can you use these data? 1.Rank

ordering?2.Grouping?3.In conjunction

with other data?

Let’s start with 5th gradeStudent Mathematics Proficiency

Johnny 25th NoviceSuzy 35th Novice

Kenny 75th NoviceLori 85th Novice

Juan 15th Novice

Lisa 40th Novice

Amelia 27th Distinguished

Robert 40th Distinguished

Lee 45th Distinguished

Jan 62nd Distinguished

Frank 51st Distinguished

Mary 12th Distinguished

5th Grade MEDIAN Growth 40th

Distribution of Student Growth Percentiles by Scale ScoreGrade 5 Mathematics

Demographics

Aggregated Growth at the School Level

• Remember the 4 quadrants– High vs. Low Achievement– Based on a Scale Score– High vs. Typical vs. Low Growth– Based on Student Growth Percentiles

• An example of a school level report

Moving on to school

• Look at each grade distribution of SGPs• Take the median of a median

– 4th grade– 5th grade– 6th grade

Grade Percentile

4th 35th

5th 40th

6th 37th

School Growth for MATH 37th

How it fits in with the Evaluation System

• 5% of the total evaluation weight at the school level

• School level growth? – Lowest Growth (1-24th percentile)– Lower Growth (25th – 34th percentile)– Typical Growth (35th – 65th percentile)– Higher Growth (66th – 99th percentile)

Percentile Bands within the Evaluation System

Lowest Growth

Low Growth

Typical Growth

High Growth

Unsatisfactory

Emerging

Accomplished

Distinguished

Why the ranges?

• Empirical consideration of current data

• Proposed percentile bands for first pilot year

• Open for revision• Aligns with 4 point rubrics

Questions?

Thank You

Juan D’Brot(jdbrot@access.k12.wv.us)

Executive Director Office of Assessment and Accountability

Student Learning in Context

Student Learning in Context

• Learning Targets:– Know the definition of student learning;– Identify examples and non-examples; – Evaluate the quality of potential

measures

Wrap Up & Feedback

A New Landscape

Some Parting Instructions

• Feedback Forms• Principals and Assistant Principals

– Part Two• Dine well and rest well!• See you tomorrow at 8:00 a.m.