Post on 31-Dec-2015
description
Eastern Suffolk BOCES 2012
Western Suffolk BOCES Boot Camp
Emma Klimekeklimek33@gmail.com
College and Career Ready
Students
Highly EffectiveSchool Leaders
Highly Effective Teachers
Animating the Reform AgendaInvesting in human capital, supporting with critical tools
4
Teacher Evaluation Components
Measures of Growth (20%)
Locally-selected measuresof student achieve-ment(20%)
Other Measures(60%)
Grades 4-8 ELAMath
All Other
Focus of this WebinarStudent Learning
Objectives
Emma Klimek 2012
What do we mean by “Growth”?
Growth is change from point A to point BGrowth is an expectation of learningGrowth is a relative measure compared to like students in like conditions
The What, Why, and How of Growth Models and Measures
By the End of This Section….
You should be able to: Explain why the state is measuring
student growth and not achievement Describe how the state is measuring
growth compared to similar students Define a student growth percentile and
mean growth percentile
0
200
400
600
800
Student AStudent BStudent CStudent DStudent E
Ms. Smith
Prior Performance
0
200
400
600
800
Student AStudent BStudent CStudent DStudent E
Ms. Jones
Prior Performance
Prior Year Performance for Students in Two Teachers’ Classrooms
─ Proficiency
0
200
400
600
800
Student AStudent BStudent CStudent DStudent E
Ms. Smith
Prior Performance Current Performance
0
200
400
600
800
Student AStudent BStudent CStudent DStudent E
Ms. Jones
Prior Performance Current Performance
Current Year Performance of Same Students
─ Proficiency
Prior and Current Year Performance for Ms. Smith’s Students
Ms. Smith’s Class
Prior Score Current Score
Student A 450 510
Student B 470 500
Student C 480 525
Student D 500 550
Student E 600 650
Prior and Current Year Performance for Ms. Smith’s Students
Ms. Smith’s Class
Prior Score Current Score
Student A 450 510
Student B 470 500
Student C 480 525
Student D 500 550
Student E 600 650
www.engageNY.org10
EL
A S
cale
Sco
re
2011 2012
Student A450
High SGPs
Low SGPs
Student A’s Current Year Performance Compared to “Similar” Students
If we compare student A’s current score to other students who had the same prior score (450), we can measure her growth
relative to other students. We describe
her growth as a “student growth
percentile (SGP”). Student A’s SGP is 45,
meaning she performed better in the
current year than 45 percent of similar
students.
Comparing Performance of “Similar” Students
Prior Year Score
Curr
ent Y
ear
Scor
e
Given any prior score, we see a range of current
year scores, which give us
SGPs of 1 to 99.
SGPs for Ms. Smith’s StudentsMs. Smith’s Class
Prior Score
Current Score
SGP
Student A 450 510 45
Student B 470 500 40
Student C 480 525 70
Student D 500 550 60
Student E 600 650 40
Which Students Count in a Teacher or Principal’s MGP for 2011-12?
Student has valid test scores for at least 2011-12 and
2010-11
Student has valid test scores for at least 2011-12 and
2010-11
Student scores do not count for
2011-12
Student scores do not count for
2011-12
Yes
Student meets continuous enrollment
standard for 2011-12
Student meets continuous enrollment
standard for 2011-12
No
Student growth is attributed to the teacher and the
school
Student growth is attributed to the teacher and the
school
Yes
No
Expected for 2012-13: students weighted by duration of
instructional linkage
From Student Growth to Teachers and Principals
In order for an educator to receive a growth score, he or she must have a minimum sample size of 16 student scores in ELA or mathematics across all grades he or she teaches.Examples: A teacher has a self-contained classroom with 8 students who
take the 4th grade ELA and math assessments; this teacher would then have 16 student scores contributing to his or her growth score.
A teacher has a class with 12 students who are in varied grades (4th, 5th, 6th) who take the ELA and math assessments for their respective enrolled grade level; this teacher would then have 24 student scores contributing to his or her growth score.
If an educator does not have 16 student scores, they will not receive a growth score from the State and will not receive information in the reporting system at the educator level. Educators likely to have fewer than 16 scores should use SLOs.
From Student Growth to Teachers and Principals (continued)
Ms. Smith’s Class
SGP
Student A
45
Student B
40
Student C
70
Student D
60
Student E
40
To measure teacher performance, we find the mean growth percentile (MGP) for her students. To find an educator’s mean growth percentile, take the average of SGPs in the classroom. In this case:
Step 1: 45+40+70+60+40=255
Step 2. 255/5=51
Ms. Smith’s mean growth percentile (MGP) is 51, meaning on average her students performed better than 51 percent of similar students.
A principal’s performance is measured by finding the mean growth percentile for all students in the school.
Expanding the Definition of “Similar” Students
So far we have been talking about “similar” students as those with the same prior year assessment score
We will now add two additional features to the conversation:
Two additional years of prior assessment scores• Remember—a student MUST have current year and prior year
assessment score to be included
Student-level factors• Economic disadvantage• Students with disabilities (SWDs)• English language learners (ELLs)
Adjustments for Three Student-Level Factors in Measuring Student Growth
Student performance
Teacher Instruction
Other factors(12-13)
Economic disadvantage
Language proficiency
Disability
EL
A S
cale
Sco
re
2011 2012
Student A450
High SGPs
Low SGPs
Going Back to Student A’s Current Year Performance Compared to “Similar” Students
If we compare student A’s current score to other students who had the same prior score (450), we can measure her growth
relative to other students. We describe
her growth as a “student growth
percentile (SGP”). Student A’s SGP is 45,
meaning she performed better in the
current year than 45 percent of similar
students.
EL
A S
cale
Sco
re
2011 2012
Student A450
High SGPs
Low SGPs
Expanding the Definition of “Similar” Students to Include Economically Disadvantaged—An Example
Now if student A is economically
disadvantaged, we compare student A’s
current score to other students who had the same prior score (450)
AND who are also economically
disadvantaged. In this new comparison group, we see that student A now has an SGP of 48.
Further Information on Including Student Characteristics in the Growth Model
The following slides were developed using sample data from 2010-2011. The “combined” MGPs on the charts have
been calculated at the educator level (combining all grades and subjects).
Not all districts provided data linked to teachers for grades 4-8 ELA/Math in 2010-11.
Teacher MGPs after Accounting for Economic Disadvantage
Taking student-level characteristics into account helps ensure educators with many students with those characteristics have a fair chance to achieve high or low MGPs. For example, note that for teachers with any percent of economically disadvantaged students, teacher MGPs range from 1 to 99.
NOTE: Beta results using available 2010-2011 data.
Teacher MGPs after Accounting for SWD
NOTE: Beta results using available 2010-2011 data.
Teacher MGPs after Accounting for ELL
Percent of ELL Students in Class
NOTE: Beta results using available 2010-2011 data.
“Similar” Students: A Summary“Similar” Student Characteristics
Unadjusted Mean Growth Percentiles
Adjusted Mean Growth Percentiles
Up to Three Years of Prior Achievement
Up to Three Years of Prior Achievement
English Language Learner (ELL) Status
Students with Disabilities (SWD) Status
Economic Disadvantage
Reported to Educators
Reported to EducatorsUsed for
Evaluation
One Last Feature of the Growth Model….
All tests contain measurement error, with
greater uncertainty for highest and lowest-achieving students
The New York growth model accounts for measurement error in computing student growth percentiles.
State Growth Model Summary
Regulations allow
Prior years of student test results
Three student-level variables: SWD, ELL, Econ Disadvantage
Measurement error correction
Growth model for 2011-12 only for grades 4-8 ELA/Math for teachers and principals
Using Growth Measures for Educator Evaluation
MGPs and Statistical Confidence
87
Confidence Range
Upper Limit
Lower Limit
MGP
• NYSED will report a 95 percent confidence range, meaning we can be 95 percent confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be reported.
• An educator’s confidence range depends on a number of factors, including: number of student scores included in their MGP and the variability of student performance in their classroom.
Illustrating Possible Growth Ratings
MGP 1
MGP 99
Well Below
Average
Below Average
AverageWell
Above Average
MGP 50MGP
MGP
MGP
MGP
MGP
MGP
MGP
From MGPs to Growth Ratings: TeachersRules on last slide result in these HEDI criteria for 2011-12
Yes
No
Is your MGP ≥ 69?
Is your Lower Limit > Mean of
52?
Highly Effective: Results are well
above state average for
similar students
Is your MGP ≤ 35?
Is your Upper Limit
< 44?
Ineffective: Results are well
below state average for
similar students
Developing: Results are below state average for similar students
No
Effective: Results equal
state average for similar students
Mean Growth Percentile Confidence Range HEDI Rating
Is your MGP 42-
68?
Any Confidence
Range
Yes
No
Is your MGP 36-
41?
Is your Upper Limit < Mean of
52?
Yes
Yes
Yes
Yes
Yes
Yes
No
No
No
Illustrating Possible Growth Ratings
MGP 1
MGP 99
Well Below
Average
Below Average
AverageWell
Above Average
MGP 50
MGP
MGP
MGP
MGP
MGP
MGP
MGP
Illustrating Possible Growth Ratings
MGP 1
MGP 99
Well Below
Average
Below Average
AverageWell
Above Average
MGP 50MGP
MGP
MGP
MGPIneffective
Developing
Highly Effective
Effective
Illustrating Possible Growth Ratings
MGP 1
MGP 99
Well Below
Average
Below Average
AverageWell
Above Average
MGP 50
MGP
MGP
MGPDeveloping
Effective
Effective
Assignment of Points with HEDI Category
HEDI Points
Min MGP
Max MGP
N of Teacher
s
0 3 28 660
1 29 32 651
2 33 35 693
3 29 35 241
4 36 37 826
5 38 38 495
6 39 39 535
7 40 40 561
8 41 41 683
9 36 44 2661
10 45 46 2001
11 47 49 3376
12 50 51 2432
13 52 54 3648
14 55 56 2415
15 57 59 3144
16 60 62 2624
17 63 68 3277
18 69 70 662
19 71 73 666
20 74 96 878
HEDI Points
Min MGP
Max MGP
N of School
s
0 16 36.5 71
1 37 39 75
2 39.5 41 97
3 34.5 41 22
4 41.5 42 65
5 42.5 42.5 40
6 43 43 37
7 43.5 43.5 41
8 44 44 64
9 41.5 46 270
10 46.5 48 350
11 48.5 49 209
12 49.5 50.5 328
13 51 52 313
14 52.5 53.5 324
15 54 55 316
16 55.5 57 353
17 57.5 63.5 358
18 61 61.5 65
19 62 63 70
20 63.5 74 88
Point value of 3 includes educators with MGPs in the Ineffective category but CRs above 44 (for teachers) and above 46 (for principals)
Point value of 3 includes educators with MGPs in the Ineffective category but CRs above 44 (for teachers) and above 46 (for principals)
Point value of 9 includes educators with MGPs in the Developing category but CRs above state average
Point value of 9 includes educators with MGPs in the Developing category but CRs above state average
Point value of 17 Includes educators with MGPs in the Highly Effective category but CRs below state average
Point value of 17 Includes educators with MGPs in the Highly Effective category but CRs below state average
Teachers Principals
DefinitionsSGP (student growth percentile): measure of a student’s growth relative to similar studentsSimilar students: students with the same prior test scores, ELL, SWD, and economic disadvantage statusELLs: English language learnersSWD: students with disabilitiesEconomic disadvantage: a student who participates in, or whose family participates in, economic assistance programs such as the Free- or Reduced-price Lunch Programs (FRPL), Social Security Insurance (SSI), Food Stamps, Foster Care and othersHigh-achieving, low-achieving: defined by the performance of students based on prior year State assessment scores (i.e., Level 1 = low-achieving, Level 4 = high-achieving)
DefinitionsMGP (mean growth percentile): the average of the student growth percentiles attributed to a given educator “Unadjusted” MGP: an MGP based on SGPs for which ELL, SWD, and economic disadvantage status have NOT been accounted“Adjusted” MGP: an MGP based on SGPs for which ELL, SWD, and economic disadvantage status have been accountedGrowth rating: HEDI rating based on growthGrowth score: growth subcomponent points from 0-20
DefinitionsMeasurement error: uncertainty in test scores due to sampling of content and other factorsStandard error: a measure of the statistical uncertainty surrounding a scoreUpper/lower limit: highest and lowest possible MGP taking statistical confidence into accountConfidence range: range of MGPs within which we have a given level of statistical confidence that the true MGP falls (95 percent statistical confidence level used for state growth measure)
Break
STUDENT LEARNING OBJECTIVES:
The State Language
“For teachers where there is no State-provided measure of student growth, “comparable measures” are the state-determined District-wide growth goal-setting process. Student Learning Objectives (SLOs) are the State determined process.”
Emma Klimek 2012
Student Learning Objective
Do a close reading of the state’s paragraph on Student Learning Objectives
Highlight or underline 5 key words (only 5) with a shoulder partner
Use all 5 of the words, if possible, in a “sound bite” or graphic
Emma Klimek 2012
State Message Regarding Student Learning Objectives
SLOs name what students need to know and be able to do at the end of the year.
SLOs place student learning at the center of the conversation.
SLOs are a critical part of all great educator’s practice.
SLOs are an opportunity to document the impact educators make with students.
Emma Klimek 2012
Key Messages for SLOs continued…
SLOs provide principals with critical information that can be used to manage performance, differentiate and target professional development, and focus support for teachers.
The SLO process encourages collaboration within school buildings.
School leaders are accountable for ensuring all teachers have SLOs that will support their District and school goals.
Emma Klimek 2012
Who has SLOs and how will SLOs be set?
WHO NEEDS AN SLO?
www.engageNY.org47
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Teacher % in These Assignments
Teacher Coverage
Other 36%
(CTE; Arts, Foreign Language, Music, Theatre, Dance, Humanities; Phsyical
Education/Health; Library; Pre-K; Reading 4-12; Combined Courses;
Literacy & Math K - 3 15%
Social Studies 6-8, Regents - 4% Science 6-8. Regents - 4%
ELA 9 - 11 2%
Special Education, ESL, Bilingual 21%
Math & ELA 4 - 8
Regents Math 2%
Who Will Have SLOs in 2012-13?
6
SGP/VA as Data Allow;
otherwise SLOs
SGP/VA
SLOs
Emma Klimek 2012
Emma Klimek 2012
Teachers by Grade and Subject:
Growth is State-provided SGP/VA
Growth is SLO
K-2 Teachers Future Possibly - 1 SLO for ELA (literacy and writing)- 1 SLO for Math- (unless teacher focuses on single subject area)
3 Teachers Future Possibly - 1 SLO for ELA (literacy and writing)- 1 SLO for Math- (unless teacher focuses on single subject area)- (must use State assessment as evidence)
4-8 Common Branch/ ELA/Math Teachers
YES N/A
4-8 Science and Social Studies Subject Teachers
Future Likely - 1 SLO for each subject/assessment – (SLOs must cover classes with largest numbers of students until a majority of students are covered)
- Grades 6-7 Science and 6-8 Social Studies must use a State-approved 3 rd party assessment as evidence; Grade 4 and Grade 8 Science must use a State assessment as evidence.
4-8 Other Subject Teachers NO
9-12 Core Subjects, Regents Subjects andRegents Equivalents
As available • 1 SLO for each subject/assessment - (SLOs must cover classes with largest numbers of students until a majority of students are covered) - Core Subjects: High school ELA, Math, Science and Social Studies courses associated in 2010-11 with Regents or, in the future, with other State assessments must use a State assessment as evidence if one exists (or Regents equivalents). If not, SLO must use assessment from list of State-approved 3rd party assessments and Regents equivalents.
9-12 Other Subject Teachers • 1 SLO for each subject/assessment - (SLOs must cover classes with largest numbers of students until a majority of students are covered
Teachers with a Mix of Sections/Courses With/ Without State-Provided Growth Measures
Yes if ≥50%of sections/students are covered by SGP/VA
• If <50% covered by SGP/VA, then SLOs will be used. - First, SLOs must use SGP/VA where available; then create SLOs for largest sections without SGP/VA until majority of students are covered.
Emma Klimek 2012
General Rules
If 50% or more of a teacher’s students receive a State Growth Percentile (SGP) then the teacher does not need an SLOCommon Branch teachers, who do not get an SGP, must have 2 SLOs, one in ELA and one in mathA common branch teacher will not have an SLO in other subjectsIf a teacher has 16 or more State test scores in grades 4-8, ELA or math, but that is less than 50% of his/her students, he/she must use this as the first SLO
Emma Klimek 2012
IDENTIFYING WHO NEEDS AN SLO AND WHICH ONE
Emma Klimek 2012
Determining SLOs
Locate the handout With a partner or in a triad, complete the tableBe prepared to share out one scenario
Lunch
STATE RULES REGARDING SLOS
Emma Klimek 2012
Assessment Rules
Emma Klimek 2012
Assessment Rules
Teacher developed assessments are not permittedThe same assessment may be used for the 2nd 20% but it must be used in a different way
Emma Klimek 2012
Assessment Rules
Use common assessments across grade levels or courses, if at all possibleUse high quality assessments, if possible
Emma Klimek 2012
Third Party Assessments
There is no variance process in place to use a non-Approved 3rd party assessment for the purposes of APPR.
Emma Klimek 2012
Attendance Rule
You may not exclude students due to attendance for the computation of the SLO
Emma Klimek 2012
Format Rules
Must include all the elements of the SLO as outlined by the state Population Learning Content Instructional Interval Target Baseline Evidence HEDI Rationale
Emma Klimek 2012
Other Criteria Rules
50% rule for students is set on BEDS dayIndividual student growth must be determined, but the final outcome is on the aggregate groupNo teacher may score an assessment in which they have a vested interestLead evaluators must be trained
Emma Klimek 2012
Target Setting
Emma Klimek 2012
District determines:
Who, specifically in the district needs and SLO and in what academic area,based on state rulesSystem for scoring and teacher ratingsDistrict wide process for setting, reviewing and assessing SLOs
Emma Klimek 2012
Determine District-wide Priorities and Academic Needs
Assess and identify District-wide priorities and academic needs. Start with commitments and focus areas in District strategic plans.
Decide how prescriptive the District will be and where decisions will be made by principals, or principals with teachers.
SLO EXCEPTIONAL CASES
Emma Klimek 2012
Emma Klimek 2012
Case 1: Less than 50% of students have a SGP measurement
18-20 9-17 3-8 0-2
Emma Klimek 2012
Case 2: Multiple Sections w/wo SGP
Emma Klimek 2012
Case 3: ESL/Bilingual/SWD
Emma Klimek 2012
Case 4: Co-Teachers
Emma Klimek 2012
Case 5: Push-in, pull-out
Case 6: NYSESLAT Students
Emma Klimek 2012
Case 7: NYSAA
Emma Klimek 2012
Case 8: Special Cases
Emma Klimek 2012
SLO EXCEPTIONAL CASES
Emma Klimek 2012
Emma Klimek 2012
Case 1: Less than 50% of students have a SGP measurement
18-20 9-17 3-8 0-2
Emma Klimek 2012
Case 2: Multiple Sections w/wo SGP
Emma Klimek 2012
Case 3: ESL/Bilingual/SWD
Emma Klimek 2012
Case 4: Co-Teachers
Emma Klimek 2012
Case 5: Push-in, pull-out
Case 6: NYSESLAT Students
Emma Klimek 2012
Case 7: NYSAA
Emma Klimek 2012
Case 8: Special Cases
Emma Klimek 2012
Emma Klimek 2012
Individual Teacher’s Student Option
For all teachers an SLO can be based solely on the students they specifically teach For example: a second grade teacher’s
class room of students determine the “first 20%” of the teacher’s APPR
Emma Klimek 2012
Team, school, district, BOCES wide measures
If a group measure is chosen then the following apply: This cannot be used for 6-8 science and
social studies teachers and any teacher whose SLO course ends in a Regents
A state assessment must be used
Emma Klimek 2012
An example:
Emma Klimek 2012
An Example:
SLO EXCEPTIONAL CASES
Emma Klimek 2012
Emma Klimek 2012
Case 1: Less than 50% of students have a SGP measurement
18-20 9-17 3-8 0-2
Emma Klimek 2012
Case 2: Multiple Sections w/wo SGP
Emma Klimek 2012
Case 3: ESL/Bilingual/SWD
Emma Klimek 2012
Case 4: Co-Teachers
Emma Klimek 2012
Case 5: Push-in, pull-out
Case 6: NYSESLAT Students
Emma Klimek 2012
Case 7: NYSAA
Emma Klimek 2012
Case 8: Special Cases
Emma Klimek 2012
Emma Klimek 2012
Individual Teacher’s Student Option
For all teachers an SLO can be based solely on the students they specifically teach For example: a second grade teacher’s
class room of students determine the “first 20%” of the teacher’s APPR
Emma Klimek 2012
Team, school, district, BOCES wide measures
If a group measure is chosen then the following apply: This cannot be used for 6-8 science and
social studies teachers and any teacher whose SLO course ends in a Regents
A state assessment must be used
Emma Klimek 2012
An example:
Emma Klimek 2012
An Example:
Emma Klimek 2012
REVIEWING EXAMPLES
Emma Klimek 2012
Reviewing Examples
Locate handout of examplesEach team choose one example to review Identify which method was used: Target for mastery of standards Target for score gain Individual student growth gain
Identify which assessment was chosen or requiredDoes the assessment exist?If not, how will it be developed? Is the assessment measure an equal interval unit assessment? Will the metric work for all students in all conditions? Explain the scoring method
Is the SLO as presented “executable”?Does the SLO meet state requirements? Is the SLO fair and reasonable? Rate it on the SLO rubric
Thank you
eklimek@optonline.net