An Alternative Method to Rate Teacher Performance
28
An Alternative Method to Rate School and Teacher Performance By Patricio A. Rojas, Ph.D. Associate Faculty at University of Phoenix Albuquerque, NM Director of Research, Data, and Assessment Los Lunas Schools, NM June 2012
description
An Alternative Method to Rate Teacher Performance Patricio A. Rojas, PH.D. Director of Research, Data & Assessment, Los Lunas, NM Fusion 2012, the NWEA summer conference in Portland, Oregon This session will provide participants the opportunity to experience an alternative method of rating teachers, under new regulations of New Mexico. This is an updated version of the work presented last year in FUSION 2011. The alternative method is needed because we do not have growth points in the year 2010-2011 in New Mexico. Learning outcome: - Learn easy graphs to analyze growth and how to rate teacher performance without using grown points. Los Lunas is located 35 miles south from Albuquerque, the district has 9,000 students; 17 schools (3 high schools, 2 middle schools, and 12 elementary schools). The district is one of the few nationally accredited districts in the nation. We have been using MAP as short cycle assessment for the last six years. MAP scores are an important piece of data used to rate both schools and teachers. Audience: - Experienced data user - District leadership - Curriculum and Instruction
Transcript of An Alternative Method to Rate Teacher Performance
An Alternative Method to Rate School and Teacher Performance
By Patricio A. Rojas, Ph.D.Associate Faculty at University of Phoenix
Albuquerque, NM Director of Research, Data, and Assessment
Los Lunas Schools, NMJune 2012
2
BACKGROUND• Director of Research, Data, and Assessment, Department of Curriculum
and Instruction, Los Lunas Schools, Los Lunas – NM.
• Member of AAAC (Accountability and Assessment Advisory Committee), nominated by the office of the Education Secretary of State.
• New Mexico State Coordinator for the Math EAG (Enhancement
Assessment Grant) Project, nominated by the Education Secretary of State.
• Associate Professor at University of Phoenix, Albuquerque – NM.
• Faculty consulting for:– San Jose State University, San Jose, CA– Universidad de Don Bosco, El Salvador.– Universidad Catolica del Norte, Antofagasta, Chile.
3
PRELIMINARIES
The legislation for No Child Left behind Act was proposed by President George W. Bush on January 23, 2001. It was coauthored by Representatives Boehner (R-OH), Miller (D-CA), and Senator Gregg (R-NH). The United States House of Representatives passed the bill on May 23, 2001 (voting 384–45), and the United States Senate passed it on June 14, 2001 (voting 91–8). President Bush signed it into law on January 8, 2002.
4
No Child Left BehindNo Child Left Behind requires all government-run schools receiving federal funding to administer a state-wide standardized test (all students take the same test under the same conditions) annually to all students. The students' scores are used to determine whether the school/teacher has taught the students well. Schools which receive Title I funding through the Elementary and Secondary Education Act of 1965 must make Adequate Yearly Progress (AYP) in test scores (e.g. each year, its fifth graders must do better on standardized tests than the previous year's fifth graders).
5
RESEARCH• Research shows that every year fewer schools are
making AYP using SBA-Proficiency model. In 2006, 29% of schools in the nation did not make AYP; last year 2011, 35% did not make AYP, and the forecast for 2012, indicates that the percentage of schools not making AYP will be 45%. Source: CEP (Center on Education Policy, Dec 2011)
• In 2006, 54% of districts in New Mexico did not make AYP; last year 2011, 80% did not make AYP, and the forecast for 2012, indicates that the percentage of schools not making AYP will be 85%. Source: CEP (Center on Education Policy, Dec 2011)
6
NCLB WAIVERS• Obviously, the “attainment proficiency model“ or
SBA-Proficiency model is not working at national/state/ district level neither is the growth model, using either SBA or MAP scores.
• With the addition of eight new states, so far, we have 19 states on waivers, while another 17 states and the District of Columbia are under review.
• States granted waivers are exempt from the laws’ requirement that all students pass achievement tests by 2014 and make progress toward that goal each year.
7
Consequences•Last year we proposed an alternative growth model based on the concept of students “on track to graduation (OTTG)” if the student is either proficient or makes expected growth.•The state of New Mexico changed the scale score making each grade ranging X00-X80 with the proficiency level at X40, where X represents the grade, for example scale scores for grade 5 ranges 500-580 with proficiency level set at 540.•This change eliminated the OTTG model we presented last year.
8
More changes• In 2011, the state released a
preliminary report named the School Grading Report 2010-2011, giving schools ratings of A-F using a statistical tool not very well explained to the stakeholders. The state is planning to make this report official for the year 2011-2012, while promising a guide for stakeholders to understand and replicate the scores generated by the statistical tool they are using.
9
NEW PROPOSAL
Because of this dramatic change in data and regulations, this year we are proposing a combination of the attainment proficiency and growth models based on trend analysis to rate schools and teachers performance.
10
NEW MODEL• The new model is based on 6 trend analysis
for the following parameters:– SBA Math Proficiency (SMP)– SBA Reading Proficiency (SRP)– MAP Math Growth (MMG)– MAP Reading Growth (MRG)– MAP Math Proficiency (MPP)– MAP Reading Proficiency (MRP)
• Next year we could add two more parameters:– SBA Math Growth (SMG)– SBA Reading Growth (SRG)
11
REGRESSION ANALYSIS
• We collected information for the six parameters as shown below
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
MATH 42.0 43.2 43.2 30.7 36.8 41.1 33.5 34.5READING 65.6 58.6 63.7 57.2 59.0 46.2 43.3 38.6
MATH 28.0 47.6 55.2 57.4 68.9 79.6 76.1 72.1READING 57.0 60.0 65.5 70.3 74.0 79.1 75.1 71.3
MATH 40.2 42.1 43.9 41.3 43.3 39.9 37.8 39.0READING 32.0 27.3 26.4 20.9 29.3 36.9 28.8 24.0
SCHOOL A SBA PROFICIENCYSCHOOL A MAP PROFICIENCYSCHOOL A MAP GROWTH
PERCENTAGE OF PROFICIENT STUDENTS
• Next we performed a regression analysis in each of these parameters to find the regression line
equation. We used the slope of the lines to create a rank by using the average of the slopes computed for each parameter.
122004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
65.6
58.6
63.7
57.259.0
46.2
43.3
38.6
f(x) = − 3.7675 x + 70.985
SCHOOL APERCENTAGE OF STUDENTS
SBA READ PROFICIENCY
READING Linear (READING)
13
2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
42.043.2
43.2
30.7
36.8
41.1
33.5
34.5
f(x) = − 1.20869047619048 x + 43.5703571428571
SCHOOL APERCENTAGE OF STUDENTS
SBA MATH PROFICIENCY
MATH Linear (MATH)
142004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
40.2
42.1
43.9
41.3
43.3
39.9
37.8
39.0
f(x) = − 0.477380952380953 x + 43.0907142857143
SCHOOL APERCENTAGE OF STUDENTS
MAP MATH GROWTH
MATH Linear (MATH)
152004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
28.0
47.6
55.257.4
68.9
79.676.1
72.1
f(x) = 6.37857142857143 x + 31.9114285714286
SCHOOL APERCENTAGE OFSTUDENTSMAP MATH PROFICIENCY
MATH Linear (MATH)
162004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
57.0
60.0
65.5
70.3
74.0
79.1
75.1
71.3
f(x) = 2.62071428571428 x + 57.2392857142857
SCHOOL APERCENTAGE OF STUDENTS
MAP READ PROFICIENCY
READING Linear (READING)
172004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012
32.0
27.3 26.4
20.9
29.3
36.9
28.8
24.0
f(x) = − 0.0994047619047618 x + 28.6410714285714
SCHOOL APERCENTAGE OF STUDENTS
MAP READ GROWTH
READING Linear (READING)
18
SCHOOL A
65.6
58.6
63.757.2 59.0
46.2 43.338.6
y = -3.77x + 70.99SCHOOL A
PERCENT OF STUDENTSSBA READ PROFICIENCY
READING Linear (READING)
42.0 43.243.2
30.7
36.8
41.1
33.5 34.5
y = -1.21x + 43.57SCHOOL APERCENT OF STUDENTS
SBA MATH PROFICIENCY
MATH Linear (MATH)
28.0
47.655.2 57.4
68.979.676.1 72.1
y = 6.38x + 31.91SCHOOL APERCENT OFSTUDENTS
MAP MATH PROFICIENCY
MATH Linear (MATH)
40.242.1
43.9
41.3
43.3
39.937.8
39.0
y = -0.48x + 43.09SCHOOL APERCENT OF STUDENTSMAP MATH GROWTH
MATH Linear (MATH)
57.0 60.0
65.5 70.374.0
79.175.1
71.3
y = 2.62x + 57.24SCHOOL APERCENT OF STUDENTS
MAP READ PROFICIENCY
READING Linear (READING)
32.027.3 26.4
20.9
29.3
36.9
28.824.0
y = -0.10x + 28.64SCHOOL APERCENT OF STUDENTS
MAP READ GROWTH
READING Linear (READING)
This is a view of all six graphs at once. It is hard to make a decision if this school is performing as desired. We need to find a measure to rate the school, we will use the slopes of the regression lines as explained in the next slide
19
MEASURE TO RATE SCHOOLS• The following are the six regression lines– SRP– SMP– MMG– MMP– MRP– MRG
• We compute the average of the slopes and we called the rank, in this case RANK = 1.41
• Now we need to decide a scale to assign a grade.
99.7077.3 xy
57.4321.1 xy09.4348.0 xy91.3138.6 xy24.5762.2 xy
64.2810.0 xy
20
MEASURE TO RATE SCHOOLS• This is a tentative scale that requires some
more research
• The school in the example with rank = 1.41 will receive a grade C.
• Another example is shown on the next slide
RANK GRADE
4+ A
2.00 – 3.99 B
0.00 – 1.99 C
-1.99 - -0.01 D
-2.00 - F
21
SCHOOL B
RANK = - 1.45 GRADE = D
67.6
60.6 53.7 47.2 45.2 46.243.338.6
y = -3.7381x + 67.121SCHOOL BPERCENT OF STUDENTSSBA READ PROFICIENCY
READING Linear (READING)
42.0 43.2
50.1
32.5
43.5 42.1 43.544.1
y = 0.0342x + 42.478SCHOOL BPERCENT OF STUDENTS
SBA MATH PROFICIENCY
MATH Linear (MATH)
52.3 52.6 54.2
30.4
39.542.1 42.5
35.2
y = -2.35x + 54.175SCHOOL BPERCENT OFSTUDENTS
MAP MATH PROFICIENCY
MATH Linear (MATH)
42.045.3 43.2
35.636.8
41.140.3
42.0
y = -0.3617x + 42.42SCHOOL BPERCENT OF STUDENTSMAP MATH GROWTH
MATH Linear (MATH)
56.9 58.6
63.7
57.2
62.558.1
42.345.2
y = -2.0845x + 64.948SCHOOL BPERCENT OF STUDENTS
MAP READ PROFICIENCY
READING Linear (READING)
42.045.3 43.2
35.6 36.841.1 40.3
42.0
y = -0.3617x + 42.42SCHOOL BPERCENT OF STUDENTS
MAP READ GROWTH
MATH Linear (MATH)
22
RATING TEACHERS
• We could use the same methodology and scale to rate teachers, here are a couple of examples.
• Teacher A RANK = 2.81 GRADE = B• Teacher B RANK = 4.19 GRADE = A• Teacher C RANK = - 3.52 GRADE = F
TEACHER A
34.735.0
45.048.5
2009 2010 2011 2012
TEACHER ASBA/READ/PROF
53.9
45.0
47.4
42.3
2009 2010 2011 2012
TEACHER ASBA/MATH/PROF
36.2
65.0 68.468.4
84.5
2009 2010 2011 2012 2013
TEACHER AMAP/READ/GROWTH
28.6
37.9
33.3
26.3
35.032.6
2008 2009 2010 2011 2012 2013
TEACHER AMAP/MATH/GROWTH
46.2
25.3
65.0
52.6 52.660.4
2008 2009 2010 2011 2012 2013
TEACHER AMAP/MATH/PROF
42.7
38.1
42.1
45.044.7
2009 2010 2011 2012 2013
TEACHER AMAP/READ/PROF
RANK = 2.81 – GRADE = B
24
TEACHER B
35.6 36.1
42.144.4
2009 2010 2011 2012
TEACHER BSBA/READ/PROF
42.4
47.5 47.4
50.8
2009 2010 2011 2012
TEACHER BSBA/MATH/PROF
45.0
88.8 86.9
31.2
52.1
2009 2010 2011 2012 2013
TEACHER BMAP/READ/GROWTH
10.0
27.8
56.5
25.0
48.3
2009 2010 2011 2012 2013
TEACHER BMAP/MATH/GROWTH
30.0
88.8
52.1
75.0
86.0
2009 2010 2011 2012 2013
TEACHER BMAP/READ/PROF
10.0
61.1
43.537.5
54.3
2009 2010 2011 2012 2013
TEACHER BMAP/MATH/PROF
RANK = 4.19 - GRADE = A
25
TEACHER C
80.0
45.9
91.7
80.8
86.6
TEACHER CSBA/READ/PROF
76.0 77.6 79.2
57.7 59.3
TEACHER CSBA/MATH/PROF
38.8
52.7
66.6
45.4
38.4
46.0
TEACHER CMAP/READ/GROWTH
64.0
76.0
88.0
74.1 72.2
79.2
TEACHER CMAP/MATH/GROWTH
64.070.0
76.0
55.6
28.533.2
TEACHER CMAP/READ/PROF
86.962.9
38.8 38.0 38.4
16.4
TEACHER CMAP/MATH/PROF
RANK = -3.52 - GRADE = F
26
EXTRA VALUE OF THE MODEL• Using forecasting methodologies we could
design an expected growth/proficiency for both teachers and schools.
• For School A the forecasted values are: PERCENTAGE OF PROFICIENT
STUDENTS
SCHOOL A SBA PROFICIENCY
MATH
READING
SCHOOL A MAP PROFICIENCY
MATH
READING
SCHOOL A MAP GROWTH
MATH
READING
2011-2012 FORECAST
44.1 42.838.6 33.535.2 33.045.2 46.242.0 39.235.2 35.9
GOAL44.138.635.246.242.035.9
27
CONCLUSIONS1. The new model presented seems to be
1. Fair2. Easy to use and replicate3. Easy to explain to families
2. We could easily introduce the concept of “added value” for any other parameter: College readiness, extracurricular activities, sports, etc.
3. Any new parameter should be measured by percentage of students, parents, or teachers participating.
4. We tested six schools and ten teachers. We requested input from principals and administrators, over 90% agreed with the grades assigned by this model.
