Western Suffolk BOCES Boot Camp Emma Klimek [email protected] Eastern Suffolk BOCES 2012.

Eastern Suffolk BOCES 2012

Western Suffolk BOCES Boot Camp

Emma [email protected]

College and Career Ready

Students

Highly EffectiveSchool Leaders

Highly Effective Teachers

Animating the Reform AgendaInvesting in human capital, supporting with critical tools

4

Teacher Evaluation Components

Measures of Growth (20%)

Locally-selected measuresof student achieve-ment(20%)

Other Measures(60%)

Grades 4-8 ELAMath

All Other

Focus of this WebinarStudent Learning

Objectives

Emma Klimek 2012

What do we mean by “Growth”?

Growth is change from point A to point BGrowth is an expectation of learningGrowth is a relative measure compared to like students in like conditions

The What, Why, and How of Growth Models and Measures

By the End of This Section….

You should be able to: Explain why the state is measuring

student growth and not achievement Describe how the state is measuring

growth compared to similar students Define a student growth percentile and

mean growth percentile

0

200

400

600

800

Student AStudent BStudent CStudent DStudent E

Ms. Smith

Prior Performance

0

200

400

600

800


Ms. Jones

Prior Performance

Prior Year Performance for Students in Two Teachers’ Classrooms

─ Proficiency

0

200

400

600

800


Ms. Smith

Prior Performance Current Performance

0

200

400

600

800


Ms. Jones

Prior Performance Current Performance

Current Year Performance of Same Students

─ Proficiency

Prior and Current Year Performance for Ms. Smith’s Students

Ms. Smith’s Class

Prior Score Current Score

Student A 450 510

Student B 470 500

Student C 480 525

Student D 500 550

Student E 600 650

www.engageNY.org10

EL

A S

cale

Sco

re

2011 2012

Student A450

High SGPs

Low SGPs

Student A’s Current Year Performance Compared to “Similar” Students

If we compare student A’s current score to other students who had the same prior score (450), we can measure her growth

relative to other students. We describe

her growth as a “student growth

percentile (SGP”). Student A’s SGP is 45,

meaning she performed better in the

current year than 45 percent of similar

students.

Comparing Performance of “Similar” Students

Prior Year Score

Curr

ent Y

ear

Scor

e

Given any prior score, we see a range of current

year scores, which give us

SGPs of 1 to 99.

SGPs for Ms. Smith’s StudentsMs. Smith’s Class

Prior Score

Current Score

SGP

Student A 450 510 45

Student B 470 500 40

Student C 480 525 70

Student D 500 550 60

Student E 600 650 40

Which Students Count in a Teacher or Principal’s MGP for 2011-12?

Student has valid test scores for at least 2011-12 and

2010-11

Student has valid test scores for at least 2011-12 and

2010-11

Student scores do not count for

2011-12

Student scores do not count for

2011-12

Yes

Student meets continuous enrollment

standard for 2011-12

Student meets continuous enrollment

standard for 2011-12

No

Student growth is attributed to the teacher and the

school

Student growth is attributed to the teacher and the

school

Yes

No

Expected for 2012-13: students weighted by duration of

instructional linkage

From Student Growth to Teachers and Principals

In order for an educator to receive a growth score, he or she must have a minimum sample size of 16 student scores in ELA or mathematics across all grades he or she teaches.Examples: A teacher has a self-contained classroom with 8 students who

take the 4th grade ELA and math assessments; this teacher would then have 16 student scores contributing to his or her growth score.

A teacher has a class with 12 students who are in varied grades (4th, 5th, 6th) who take the ELA and math assessments for their respective enrolled grade level; this teacher would then have 24 student scores contributing to his or her growth score.

If an educator does not have 16 student scores, they will not receive a growth score from the State and will not receive information in the reporting system at the educator level. Educators likely to have fewer than 16 scores should use SLOs.

From Student Growth to Teachers and Principals (continued)

Ms. Smith’s Class

SGP

Student A

45

Student B

40

Student C

70

Student D

60

Student E

40

To measure teacher performance, we find the mean growth percentile (MGP) for her students. To find an educator’s mean growth percentile, take the average of SGPs in the classroom. In this case:

Step 1: 45+40+70+60+40=255

Step 2. 255/5=51

Ms. Smith’s mean growth percentile (MGP) is 51, meaning on average her students performed better than 51 percent of similar students.

A principal’s performance is measured by finding the mean growth percentile for all students in the school.

Expanding the Definition of “Similar” Students

So far we have been talking about “similar” students as those with the same prior year assessment score

We will now add two additional features to the conversation:

Two additional years of prior assessment scores• Remember—a student MUST have current year and prior year

assessment score to be included

Student-level factors• Economic disadvantage• Students with disabilities (SWDs)• English language learners (ELLs)

Adjustments for Three Student-Level Factors in Measuring Student Growth

Student performance

Teacher Instruction

Other factors(12-13)

Economic disadvantage

Language proficiency

Disability

EL

A S

cale

Sco

re

2011 2012

Student A450

High SGPs

Low SGPs

Going Back to Student A’s Current Year Performance Compared to “Similar” Students

If we compare student A’s current score to other students who had the same prior score (450), we can measure her growth

relative to other students. We describe

her growth as a “student growth

percentile (SGP”). Student A’s SGP is 45,

meaning she performed better in the

current year than 45 percent of similar

students.

EL

A S

cale

Sco

re

2011 2012

Student A450

High SGPs

Low SGPs

Expanding the Definition of “Similar” Students to Include Economically Disadvantaged—An Example

Now if student A is economically

disadvantaged, we compare student A’s

current score to other students who had the same prior score (450)

AND who are also economically

disadvantaged. In this new comparison group, we see that student A now has an SGP of 48.

Further Information on Including Student Characteristics in the Growth Model

The following slides were developed using sample data from 2010-2011. The “combined” MGPs on the charts have

been calculated at the educator level (combining all grades and subjects).

Not all districts provided data linked to teachers for grades 4-8 ELA/Math in 2010-11.

Teacher MGPs after Accounting for Economic Disadvantage

Taking student-level characteristics into account helps ensure educators with many students with those characteristics have a fair chance to achieve high or low MGPs. For example, note that for teachers with any percent of economically disadvantaged students, teacher MGPs range from 1 to 99.

NOTE: Beta results using available 2010-2011 data.

Teacher MGPs after Accounting for SWD


Teacher MGPs after Accounting for ELL

Percent of ELL Students in Class


“Similar” Students: A Summary“Similar” Student Characteristics

Unadjusted Mean Growth Percentiles

Adjusted Mean Growth Percentiles

Up to Three Years of Prior Achievement

Up to Three Years of Prior Achievement

English Language Learner (ELL) Status

Students with Disabilities (SWD) Status

Economic Disadvantage

Reported to Educators

Reported to EducatorsUsed for

Evaluation

One Last Feature of the Growth Model….

All tests contain measurement error, with

greater uncertainty for highest and lowest-achieving students

The New York growth model accounts for measurement error in computing student growth percentiles.

State Growth Model Summary

Regulations allow

Prior years of student test results

Three student-level variables: SWD, ELL, Econ Disadvantage

Measurement error correction

Growth model for 2011-12 only for grades 4-8 ELA/Math for teachers and principals

Using Growth Measures for Educator Evaluation

MGPs and Statistical Confidence

87

Confidence Range

Upper Limit

Lower Limit

MGP

• NYSED will report a 95 percent confidence range, meaning we can be 95 percent confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be reported.

• An educator’s confidence range depends on a number of factors, including: number of student scores included in their MGP and the variability of student performance in their classroom.

Illustrating Possible Growth Ratings

MGP 1

MGP 99

Well Below

Average

Below Average

AverageWell

Above Average

MGP 50MGP

MGP

MGP

MGP

MGP

MGP

MGP

From MGPs to Growth Ratings: TeachersRules on last slide result in these HEDI criteria for 2011-12

Yes

No

Is your MGP ≥ 69?

Is your Lower Limit > Mean of

52?

Highly Effective: Results are well

above state average for

similar students

Is your MGP ≤ 35?

Is your Upper Limit

< 44?

Ineffective: Results are well

below state average for

similar students

Developing: Results are below state average for similar students

No

Effective: Results equal

state average for similar students

Mean Growth Percentile Confidence Range HEDI Rating

Is your MGP 42-

68?

Any Confidence

Range

Yes

No

Is your MGP 36-

41?

Is your Upper Limit < Mean of

52?

Yes

Yes

Yes

Yes

Yes

Yes

No

No

No


MGP 1

MGP 99

Well Below

Average

Below Average

AverageWell

Above Average

MGP 50

MGP

MGP

MGP

MGP

MGP

MGP

MGP


MGP 1

MGP 99

Well Below

Average

Below Average

AverageWell

Above Average

MGP 50MGP

MGP

MGP

MGPIneffective

Developing

Highly Effective

Effective


MGP 1

MGP 99

Well Below

Average

Below Average

AverageWell

Above Average

MGP 50

MGP

MGP

MGPDeveloping

Effective

Effective

Assignment of Points with HEDI Category

HEDI Points

Min MGP

Max MGP

N of Teacher

s

0 3 28 660

1 29 32 651

2 33 35 693

3 29 35 241

4 36 37 826

5 38 38 495

6 39 39 535

7 40 40 561

8 41 41 683

9 36 44 2661

10 45 46 2001

11 47 49 3376

12 50 51 2432

13 52 54 3648

14 55 56 2415

15 57 59 3144

16 60 62 2624

17 63 68 3277

18 69 70 662

19 71 73 666

20 74 96 878

HEDI Points

Min MGP

Max MGP

N of School

s

0 16 36.5 71

1 37 39 75

2 39.5 41 97

3 34.5 41 22

4 41.5 42 65

5 42.5 42.5 40

6 43 43 37

7 43.5 43.5 41

8 44 44 64

9 41.5 46 270

10 46.5 48 350

11 48.5 49 209

12 49.5 50.5 328

13 51 52 313

14 52.5 53.5 324

15 54 55 316

16 55.5 57 353

17 57.5 63.5 358

18 61 61.5 65

19 62 63 70

20 63.5 74 88

Point value of 3 includes educators with MGPs in the Ineffective category but CRs above 44 (for teachers) and above 46 (for principals)

Point value of 3 includes educators with MGPs in the Ineffective category but CRs above 44 (for teachers) and above 46 (for principals)

Point value of 9 includes educators with MGPs in the Developing category but CRs above state average

Point value of 9 includes educators with MGPs in the Developing category but CRs above state average

Point value of 17 Includes educators with MGPs in the Highly Effective category but CRs below state average

Point value of 17 Includes educators with MGPs in the Highly Effective category but CRs below state average

Teachers Principals

DefinitionsSGP (student growth percentile): measure of a student’s growth relative to similar studentsSimilar students: students with the same prior test scores, ELL, SWD, and economic disadvantage statusELLs: English language learnersSWD: students with disabilitiesEconomic disadvantage: a student who participates in, or whose family participates in, economic assistance programs such as the Free- or Reduced-price Lunch Programs (FRPL), Social Security Insurance (SSI), Food Stamps, Foster Care and othersHigh-achieving, low-achieving: defined by the performance of students based on prior year State assessment scores (i.e., Level 1 = low-achieving, Level 4 = high-achieving)

DefinitionsMGP (mean growth percentile): the average of the student growth percentiles attributed to a given educator “Unadjusted” MGP: an MGP based on SGPs for which ELL, SWD, and economic disadvantage status have NOT been accounted“Adjusted” MGP: an MGP based on SGPs for which ELL, SWD, and economic disadvantage status have been accountedGrowth rating: HEDI rating based on growthGrowth score: growth subcomponent points from 0-20

DefinitionsMeasurement error: uncertainty in test scores due to sampling of content and other factorsStandard error: a measure of the statistical uncertainty surrounding a scoreUpper/lower limit: highest and lowest possible MGP taking statistical confidence into accountConfidence range: range of MGPs within which we have a given level of statistical confidence that the true MGP falls (95 percent statistical confidence level used for state growth measure)

STUDENT LEARNING OBJECTIVES:

The State Language

“For teachers where there is no State-provided measure of student growth, “comparable measures” are the state-determined District-wide growth goal-setting process. Student Learning Objectives (SLOs) are the State determined process.”

Emma Klimek 2012

Student Learning Objective

Do a close reading of the state’s paragraph on Student Learning Objectives

Highlight or underline 5 key words (only 5) with a shoulder partner

Use all 5 of the words, if possible, in a “sound bite” or graphic

Emma Klimek 2012

State Message Regarding Student Learning Objectives

SLOs name what students need to know and be able to do at the end of the year.

SLOs place student learning at the center of the conversation.

SLOs are a critical part of all great educator’s practice.

SLOs are an opportunity to document the impact educators make with students.

Emma Klimek 2012

Key Messages for SLOs continued…

SLOs provide principals with critical information that can be used to manage performance, differentiate and target professional development, and focus support for teachers.

The SLO process encourages collaboration within school buildings.

School leaders are accountable for ensuring all teachers have SLOs that will support their District and school goals.

Emma Klimek 2012

Who has SLOs and how will SLOs be set?

WHO NEEDS AN SLO?

www.engageNY.org47

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Teacher % in These Assignments

Teacher Coverage

Other 36%

(CTE; Arts, Foreign Language, Music, Theatre, Dance, Humanities; Phsyical

Education/Health; Library; Pre-K; Reading 4-12; Combined Courses;

Literacy & Math K - 3 15%

Social Studies 6-8, Regents - 4% Science 6-8. Regents - 4%

ELA 9 - 11 2%

Special Education, ESL, Bilingual 21%

Math & ELA 4 - 8

Regents Math 2%

Who Will Have SLOs in 2012-13?

6

SGP/VA as Data Allow;

otherwise SLOs

SGP/VA

SLOs

Emma Klimek 2012

Emma Klimek 2012

Teachers by Grade and Subject:

Growth is State-provided SGP/VA

Growth is SLO

K-2 Teachers Future Possibly - 1 SLO for ELA (literacy and writing)- 1 SLO for Math- (unless teacher focuses on single subject area)

3 Teachers Future Possibly - 1 SLO for ELA (literacy and writing)- 1 SLO for Math- (unless teacher focuses on single subject area)- (must use State assessment as evidence)

4-8 Common Branch/ ELA/Math Teachers

YES N/A

4-8 Science and Social Studies Subject Teachers

Future Likely - 1 SLO for each subject/assessment – (SLOs must cover classes with largest numbers of students until a majority of students are covered)

- Grades 6-7 Science and 6-8 Social Studies must use a State-approved 3 rd party assessment as evidence; Grade 4 and Grade 8 Science must use a State assessment as evidence.

4-8 Other Subject Teachers NO

9-12 Core Subjects, Regents Subjects andRegents Equivalents

As available • 1 SLO for each subject/assessment - (SLOs must cover classes with largest numbers of students until a majority of students are covered) - Core Subjects: High school ELA, Math, Science and Social Studies courses associated in 2010-11 with Regents or, in the future, with other State assessments must use a State assessment as evidence if one exists (or Regents equivalents). If not, SLO must use assessment from list of State-approved 3rd party assessments and Regents equivalents.

9-12 Other Subject Teachers • 1 SLO for each subject/assessment - (SLOs must cover classes with largest numbers of students until a majority of students are covered

Teachers with a Mix of Sections/Courses With/ Without State-Provided Growth Measures

Yes if ≥50%of sections/students are covered by SGP/VA

• If <50% covered by SGP/VA, then SLOs will be used. - First, SLOs must use SGP/VA where available; then create SLOs for largest sections without SGP/VA until majority of students are covered.

Emma Klimek 2012

General Rules

If 50% or more of a teacher’s students receive a State Growth Percentile (SGP) then the teacher does not need an SLOCommon Branch teachers, who do not get an SGP, must have 2 SLOs, one in ELA and one in mathA common branch teacher will not have an SLO in other subjectsIf a teacher has 16 or more State test scores in grades 4-8, ELA or math, but that is less than 50% of his/her students, he/she must use this as the first SLO

Emma Klimek 2012

IDENTIFYING WHO NEEDS AN SLO AND WHICH ONE

Emma Klimek 2012

Determining SLOs

Locate the handout With a partner or in a triad, complete the tableBe prepared to share out one scenario

STATE RULES REGARDING SLOS

Emma Klimek 2012

Assessment Rules

Emma Klimek 2012

Assessment Rules

Teacher developed assessments are not permittedThe same assessment may be used for the 2nd 20% but it must be used in a different way

Emma Klimek 2012

Assessment Rules

Use common assessments across grade levels or courses, if at all possibleUse high quality assessments, if possible

Emma Klimek 2012

Third Party Assessments

There is no variance process in place to use a non-Approved 3rd party assessment for the purposes of APPR.

Emma Klimek 2012

Attendance Rule

You may not exclude students due to attendance for the computation of the SLO

Emma Klimek 2012

Format Rules

Must include all the elements of the SLO as outlined by the state Population Learning Content Instructional Interval Target Baseline Evidence HEDI Rationale

Emma Klimek 2012

Other Criteria Rules

50% rule for students is set on BEDS dayIndividual student growth must be determined, but the final outcome is on the aggregate groupNo teacher may score an assessment in which they have a vested interestLead evaluators must be trained

Emma Klimek 2012

Target Setting

Emma Klimek 2012

District determines:

Who, specifically in the district needs and SLO and in what academic area,based on state rulesSystem for scoring and teacher ratingsDistrict wide process for setting, reviewing and assessing SLOs

Emma Klimek 2012

Determine District-wide Priorities and Academic Needs

Assess and identify District-wide priorities and academic needs. Start with commitments and focus areas in District strategic plans.

Decide how prescriptive the District will be and where decisions will be made by principals, or principals with teachers.

SLO EXCEPTIONAL CASES

Emma Klimek 2012

Emma Klimek 2012

Case 1: Less than 50% of students have a SGP measurement

18-20 9-17 3-8 0-2

Emma Klimek 2012

Case 2: Multiple Sections w/wo SGP

Emma Klimek 2012

Case 3: ESL/Bilingual/SWD

Emma Klimek 2012

Case 4: Co-Teachers

Emma Klimek 2012

Case 5: Push-in, pull-out

Case 6: NYSESLAT Students

Emma Klimek 2012

Case 7: NYSAA

Emma Klimek 2012

Case 8: Special Cases

Emma Klimek 2012


Emma Klimek 2012

Emma Klimek 2012


18-20 9-17 3-8 0-2

Emma Klimek 2012


Emma Klimek 2012

Case 4: Co-Teachers

Emma Klimek 2012


Case 7: NYSAA

Emma Klimek 2012


Emma Klimek 2012

Emma Klimek 2012

Individual Teacher’s Student Option

For all teachers an SLO can be based solely on the students they specifically teach For example: a second grade teacher’s

class room of students determine the “first 20%” of the teacher’s APPR

Emma Klimek 2012

Team, school, district, BOCES wide measures

If a group measure is chosen then the following apply: This cannot be used for 6-8 science and

social studies teachers and any teacher whose SLO course ends in a Regents

A state assessment must be used

Emma Klimek 2012

An example:

Emma Klimek 2012

An Example:


Emma Klimek 2012

Emma Klimek 2012


18-20 9-17 3-8 0-2

Emma Klimek 2012


Emma Klimek 2012

Case 4: Co-Teachers

Emma Klimek 2012


Case 7: NYSAA

Emma Klimek 2012


Emma Klimek 2012

Emma Klimek 2012

Individual Teacher’s Student Option

For all teachers an SLO can be based solely on the students they specifically teach For example: a second grade teacher’s

class room of students determine the “first 20%” of the teacher’s APPR

Emma Klimek 2012

Team, school, district, BOCES wide measures

If a group measure is chosen then the following apply: This cannot be used for 6-8 science and

social studies teachers and any teacher whose SLO course ends in a Regents

A state assessment must be used

Emma Klimek 2012

An example:

Emma Klimek 2012

An Example:

Emma Klimek 2012

REVIEWING EXAMPLES

Emma Klimek 2012

Reviewing Examples

Locate handout of examplesEach team choose one example to review Identify which method was used: Target for mastery of standards Target for score gain Individual student growth gain

Identify which assessment was chosen or requiredDoes the assessment exist?If not, how will it be developed? Is the assessment measure an equal interval unit assessment? Will the metric work for all students in all conditions? Explain the scoring method

Is the SLO as presented “executable”?Does the SLO meet state requirements? Is the SLO fair and reasonable? Rate it on the SLO rubric

Thank you

[email protected]

Western Suffolk BOCES Boot Camp Emma Klimek [email protected] Eastern Suffolk BOCES 2012.

Documents

Transcript of Western Suffolk BOCES Boot Camp Emma Klimek [email protected] Eastern Suffolk BOCES 2012.