Hb 153 evaluation presentation january 2012
-
Upload
thespis3955 -
Category
Education
-
view
716 -
download
2
description
Transcript of Hb 153 evaluation presentation january 2012
2
SUPPORTING DOCUMENTS Presentation slides Ohio Teacher Evaluation System
Framework Current draft Ohio Teacher Evaluation
System Model (OTES) Battelle for Kids Value-Added Talking
Points Battelle for Kids Value-Added Resources Race to the Top Work Flow Chart District Assessment Mapping District Assessment Mapping Sample
Template
3
A NEW ERA IN TEACHER EVALUATION A national push for teacher evaluation reform
from policy makers Recognition through research that current teacher
evaluation practices are not effective in helping teachers improve performance and identifying underperforming teachers
A desire to identify levels of teacher performance to reward high performers and remove low performers
RttT mandate to change evaluation practices State-level policies that change evaluation
requirements Student performance as a significant factor
in teacher (and principal) evaluation (adopted in 13 states currently)
4
WHY WE NEED EVALUATION REFORM…
5
HB 153 EVALUATION FRAMEWORK REQUIREMENTS Sec. 3319.112 (A) Not later than December 31, 2011, the state board of education shall develop a
standards-based state framework for the evaluation of teachers. The framework shall establish an evaluation system that does the following:
(1) Provides for multiple evaluation factors, including student academic growth which shall account for fifty per cent of each evaluation;
(2) Is aligned with the standards for teachers adopted under section 3319.61 of the Revised Code;
(3) Requires observation of the teacher being evaluated, including at least two formal observations by the evaluator of at least thirty minutes each and classroom walkthroughs;
(4) Assigns a rating on each evaluation in accordance with division (B) of this section; (5) Requires each teacher to be provided with a written report of the results of the
teacher's evaluation; (6) Identifies measures of student academic growth for grade levels and subjects for
which the value-added progress dimension prescribed by section 3302.021 of the Revised Code does not apply;
(7) Implements a classroom-level, value-added program developed by a nonprofit organization described in division (B) of section 3302.021 of the Revised Code;
(8) Provides for professional development to accelerate and continue teacher growth and provide support to poorly performing teachers;
(9) Provides for the allocation of financial resources to support professional development.
(HB 153 as signed by the Governor)
6
7
HB 153 LOCAL REQUIREMENTS FOR TEACHER EVALUATIONSec. 3319.111 [Effective 9/29/2011] Teacher evaluation
(A) Not later than July 1, 2013, the board of education that of each school district, in consultation with teachers employed by the board, shall adopt a standards-based teacher evaluation policy that conforms with the framework for evaluation of teachers developed under section 3319.112 of the Revised Code. The policy shall become operative at the expiration of any collective bargaining agreement covering teachers employed by the board that is in effect on the effective date of this section and shall be included in any renewal or extension of such an agreement.
(B) When using measures of student academic growth as a component of a teacher's evaluation, those measures shall include the value-added progress dimension prescribed by section 3302.021 of the Revised Code. For teachers of grade levels and subjects for which the value-added progress dimension is not applicable, the board shall administer assessments on the list developed under division (B)(2) of section 3319.112 of the Revised Code.
(C)(1) The board shall conduct an evaluation of each teacher employed by the board at least once each school year, except as provided in divisions (C)(2) and (3) of this section. The evaluation shall be completed by the first day of April and the teacher shall receive a written report of the results of the evaluation by the tenth day of April.
(2) If the board has entered into a limited contract or extended limited contract with the teacher pursuant to section 3319.11 of the Revised Code, the board shall evaluate the teacher at least twice in any school year in which the board may wish to declare its intention not to re-employ the teacher pursuant to division (B), (C)(3), (D), or (E) of that section.
One evaluation shall be conducted and completed not later than the fifteenth day of January and the teacher being evaluated shall receive a written report of the results of this evaluation not later than the twenty-fifth day of January. One evaluation shall be conducted and completed between the tenth day of February and the first day of April and the teacher being evaluated shall receive a written report of the results of this evaluation not later than the tenth day of April.
(3) The board may elect, by adoption of a resolution, to evaluate each teacher who received a rating of accomplished on the teacher's most recent evaluation conducted under this section once every two school years. In that case, the biennial evaluation shall be completed by the first day of April of the applicable school year, and the teacher shall receive a written report of the results of the evaluation by the tenth day of April of that school year.
8
HB 153 OPPORTUNITIES AND CHALLENGES Opportunities
Create evaluation systems that improve instructional practice through formative feedback and educator reflection
Design a complete “system” of evaluation with formative feedback and support and not just a typical observation check list
Work together to identify best practices and scale them up through our locals
Bargain the process for changing the evaluation system as well as the procedures, practices and tools
Work collaboratively with administrators who are subject to the same requirements
9
HB 153 OPPORTUNITIES AND CHALLENGES Challenges
Short timeline to complete the work and operationalize the system (July 1, 2013)
Unfunded mandate for non-RttT locals Changing perceptions (ours and theirs) about the
purpose of evaluation Incorporating student growth in a way that
benefits teachers and doesn’t rank and sort them Limited state support at the present time Non-explicit requirement to create assessment
systems to provide required student growth metric Annual evaluations for all but accomplished
teachers
10
POST HB 153—RTTT AND NON-RTTT DISTRICTS HB 153 leveled the evaluation playing field
RttT districts and non-RttT districts are all required to reconstruct their evaluation systems to align with the adopted state framework based on the Ohio Standards for the Teaching Profession
All districts are now on virtually the same timeline: RttT districts were required to implement their new
evaluation systems by the 2013-2014 school year or sooner depending on their Scope of Work timeline and changes that are bargained collaboratively (MOU)
Non-RttT districts are required to adopt their evaluation systems no later than July 1, 2013 and implement them at the expiration of the current CBA (discrepancy in timeline—can’t implement if not created)
11
POST HB 153—RTTT AND NON-RTTT DISTRICTS HB 153 places an additional burden on ALL
districts to address the requirement of the 50% student growth measure The only measure currently available is value-
added data for teachers in grades 3-8 in reading and math (some districts have extended data through Battelle for Kids initiatives)
ODE is creating a “list of student assessments that measure mastery of course content” which districts can use (may need to purchase)
However, there will be many grade levels and courses with no existing assessments; districts will have to create their own
12
POST HB 153—RTTT AND NON-RTTT DISTRICTS HB 153 creates an advantage for RttT
districtsRttT districts can use their funds to buy the
time and support to re-create their evaluation systems, including the development of an assessment system
RttT districts can use their funds to purchase support for assessment systems (data management, specific testing protocols, testing materials and grading support)
However…these funds will go away. How will the systems be supported financially in the future?
13
EVALUATION REFORM GUIDELINES We must begin with the belief that the main
purpose of teacher evaluation is improved teaching practice and student learning.
Teacher performance is to be measured through multiple sources of evidence, with observation as one source.
Student performance is required to be 50% of the evaluation, BUT student performance is to be measured through multiple sources of data, not just a single standardized test score.
The State Board of Education has adopted a framework; districts must still develop their evaluation system that includes processes, procedures and forms.
14
15
EVALUATION 50% PART 1STUDENT GROWTH MEASURES Student academic growth will be measured
through multiple measures which must include value-added scores on evaluations for teachers where value-added scores are available.
Value-added scores are ONLY available for tested grades and subjects, math and reading in grades 3 – 8. Some extended reports are available in locals who participate in Battelle for Kids projects.
Even if there are value-added scores, there must be additional student growth measures for all teachers.
16
THREE CATEGORIES OF STUDENT GROWTH MEASURES Districts will create a local
student growth measure worth 50% of the evaluation from a combination of the following:Value-Added DataODE-approved Student Assessments
Menu of Options Determined by the District
17
STUDENT GROWTH MEASURES Local boards of education may administer
assessments chosen from the Ohio Department of Education’s assessment list ($$$) for teachers of subjects where value-added scores are not available.
and/or local measures of student growth using state-designed criteria and guidance.
This will require districts to create
local measures of student growth (assessments) in areas where there are no standardized assessments.
18
DIVIDING THE 50% STUDENT PERFORMANCE PIE
Student Achievement, Including Improve-ment of Achievement, in Tested Grades and
Subjects
20% State Achievement Growth Measure
30% District-level Growth Metric
30% School-based Growth Metric
20% Other Locally Determined Measures of Achievement
19
DIVIDING THE 50% STUDENT PERFORMANCE PIE
Student Achievement, Including Improve-ment of Achievement in Non-tested Grades
and Subjects
40% District-level Growth Metric40% School-based Growth Metric20% Other Locally Determined Measures of Achievement
20
OHIO’S VALUE-ADDED SYSTEM
Accountability Measures and
Reports
Technical Assistance
and Support
SAS® Data Processing
Maintained by SAS® Single Limited Access Password Protected Data:
District/LEA and school Student information Analytic tools Teacher-level reports
Limited Use Public Access Includes BFK SAS® EVAAS® reporting Enhanced reporting features
Developed & supported by BFK
Regional System Trained VAL’s support
districts/LEAs through DVALT training
Support to teacher-teams
Focus on school improvement
Toolkits Online courses
Maintained by ODELRC
Valued-Added AYP Growth Measure
ODE Reports-School and District (LEA) Measures
Diagnostic Tools
EVAAS®
ODE-BFK
Partnership
21
VALUE-ADDED MODELING Value-Added Modeling (VAM) has
become the “gold” standard for measuring educator effectiveness.
One year’s growth in one year’s time is the benchmark = effective.Teachers who exceed this growth rate have
a positive value-added rating (+) = highly effective
Teachers who fail to meet this growth rate have a negative value-added rating (-)
22
VALUE-ADDED MODELING BUT…VAM modeling is flawed. The tests used to generate the scores
were never designed to measure teacher effectiveness.
“Student test scores alone are not sufficiently reliable and valid indicators of teacher effectiveness to be used in high-stakes personnel decisions, even when the most sophisticated statistical applications such as value-added modeling are employed.” (EPI Briefing Paper--Problems with the Use of Student Test Scores to Evaluate Teachers)
23
LIMITATIONS OF VAM Given that students
are not randomly assigned to classes, VAM can’t distinguish between teacher effects and the effects based on students’ needs.
VAM do not provide information to help “struggling” teachers.
Lack of properly scaled year-to-year tests makes it difficult to evaluate gains along the continuum.
Mobility of students (especially in high needs schools) impact the data
VAM cannot distinguish among teachers in the middle range of performance.
24
LIMITATIONS OF VAM About 69% of
teachers can’t be accurately assessed with VAMs*
Teachers in subject areas that are not tested
Teachers in grade levels (lower elementary) where no prior test scores are available
Special education & ELL
VAM estimates vary with the tests used
If a teacher is in the bottom quintile based on one test there is a 43% chance she will be in the bottom quintile on a different test, but a 16% chance she will be in the top two quintiles.
If a teacher is in the top quintile based on one test there is a 43% chance she will be in the top quintile on a different test, but a 13% chance she will be in the bottom two quintiles.
25
TEACHER-LEVEL VALUE-ADDED REPORTINGo Rollout Schedule
o 30% of LEAs Link in Year 1 RttT (reports received fall 2011)o 60% of all RttT LEAs in Year 2o 100% of all LEAs in Ohio in Years 3 & 4
o Requirements—Accuracy of Reportingo Must conduct linkage o Minimum number of students and time enrolled
o Access to Reportingo Online via EVAAS® accountso Password protected
o Grades/Subjects Availableo ODE: grades 4-8, math & readingo BFK: grade 3, math & reading; grades 3-8, science & social
studies; high school—algebra I & II, geometry, pre-calculus, biology, chemistry, English 9, 10 & 11
Issue—Public Records Requests
26
27
28
RESOURCES FOR VALUE-ADDED
29
SIGN UP FOR A PORTAL ACCOUNT
30
USING STUDENT PERFORMANCE DATA EFFECTIVELY Used properly, student performance
data DOES have a role in school and district improvement efforts, it CAN positively impact student performance.
Nationally, we have come to believe that the data itself—the “score”—is the end game instead of the starting point.
And…an overreliance on and faith in value-added metrics as accurate measures of TEACHER performance has entirely skewed the way we use student performance data.
31
USING STUDENT PERFORMANCE DATA EFFECTIVELY To be meaningful, student performance
data should be used by educators to Identify achievement gaps, Inform instructional practice, and Direct professional development.
To effectively use the data, teams of educators should Be trained in the analysis and interpretation of
student performance data, Have real-time access to the data, and Meet regularly in teams to analyze the data
and plan intervention, instruction and professional development.
32
USING STUDENT PERFORMANCE DATA EFFECTIVELY
How do we create the conditions for educators to use student performance data effectively?
33
DEVELOPING A STUDENT ASSESSMENT SYSTEM Use the assessments you have first. Determine what assessments you
need to create a rigorous, comparable and inclusive assessment system that is designed to provide student performance data to be used for educator professional growth and also for inclusion in an evaluation system.
Chart a course of action with a timeline, persons responsible and deliverables.
34
DEVELOPING A STUDENT ASSESSMENT SYSTEM Requiring student performance in teacher
evaluations means districts will need to:1. Map current school-based and district-wide
assessments in all grades and subjects2. Determine where assessment “gaps” exist3. Create groups of educators to select/develop
appropriate assessments for “gaps”4. Create an assessment timeline for all grades and
subjects5. Collect, analyze and store student performance
data6. Provide time and training for educators to work
together with student data to improve their own instruction
35
DISTRICT ASSESSMENT MAPPING
36
EVALUATION 50% PART 2TEACHER PERFORMANCE Each evaluation will consist of two formal
observations of the teacher at least thirty minutes each in duration, as well as periodic classroom walkthroughs.
Teacher performance metrics must also use multiple and variable sources of data, such as lesson plans, samples of student work, classroom assessment results, and portfolios, in addition to data from direct observation in classrooms.
37
THE OHIO TEACHER EVALUATION SYSTEM MODEL(OTES)
38
OTES TEACHER PERFORMANCE COMPONENTS (PAGE 6) Goal Setting
Self Assessment against Ohio StandardsAnalysis of student data Identifying 2 professional growth goals
Formative Assessment of Teacher Performance—Formal ObservationPre-observation conferenceObservationPost-observation conference and reflection
Evidence Collaboration and Professionalism (determined locally)
Student Growth
39
SUMMATIVE EVALUATION The overall teacher performance rating
(50%) will be combined with the results of student growth measures (50%) to produce a summative evaluation rating as depicted in the following matrix.
Teachers will be rated in one of four categories:AccomplishedProficientDeveloping Ineffective
40
Teacher Performance
4 3 2 1
Stu
de
nt
Gro
wth
Me
as
ure
s
Ab
ove
Accomplished Accomplished Proficient Developing
Exp
ecte
d
Proficient Proficient Developing Developing
Belo
w Developing Developing Ineffective Ineffective
Evaluation Matrix
41
FOLLOWING THE EVALUATION… Teachers with above expected levels of student
growth will develop a professional growth plan and may choose their credentialed evaluator for the evaluation cycle.
Teachers with expected levels of student growth will develop a professional growth plan collaboratively with the credentialed evaluator and will have input on their credentialed evaluator for the evaluation cycle.
Teachers with below expected levels of student growth will develop an improvement plan with their credentialed evaluator. The administration will assign the credentialed evaluator for the evaluation cycle and approve the improvement plan.
This is entirely unrealistic and does not reflect what actually happens in schools.
42
ADDITIONAL HB 153 REQUIREMENTS At the local level, the board of education will
include in its evaluation policy, procedures for using the evaluation results for retention and promotion decisions and for removal of poorly-performing teachers.
Seniority will not be the basis for teacher retention decisions, except when deciding between teachers who have comparable evaluations.
The local board of education will also provide for the allocation of financial resources to support professional development.
43
BARGAINING CONSIDERATIONS With a July 1, 2013 deadline for system
completion, evaluation work will need to begin ASAP and may not fit into current bargaining cycle
Effective evaluation reform will require collaboration with administration at a very different level in many locals
Future evaluation language in CBAs will need to include all processes, procedures and tools
Stakes are high; we can’t afford to adopt systems that aren’t designed to support teachers
44
BARGAINING CONSIDERATIONS:PROCESS FOR EVALUATION REFORM
Composition and selection of evaluation team members
Timeline for evaluation work Compensation for work outside of the
school day Mandatory training for evaluators for
observation protocols and ratings Training for staff about evaluation
processes, procedures and tools No-fault piloting provision to work out
problems
45
NEXT STEPS FOR TRANSFORMING EVALUATION1. Identify and engage district evaluation team, including
teachers from various levels/areas2. Review and analyze teacher current evaluation polices
and rules 3. Conduct ODE Evaluation GAP Analysis4. Review effective evaluation models including the OTES5. Select/Develop a district evaluation system and tools6. Map and develop student assessments that will provide
student performance data7. Create training for evaluators and teachers8. Construct a pilot timeline9. Have volunteer teachers and evaluators pilot the system10. Review and revise the system based on pilot data11. Train all evaluators and teachers12. Implement the new evaluation system
47
RESOURCES Teacher Evaluation Systems materials and
resources (login required) http://www.ohea.org/teacher-evaluation-systems
www.lauragoe.com Includes various state and local systems and examples of multiple measures for teacher performance and student growth
Teacher Assessment and Evaluation: The NEA's Framework http://www.nea.org/home/41858.htm
Getting Teacher Assessment Right: What Policymakers Can Learn from Research -- the source for Dr. Hinchey’s presentation: http://nepc.colorado.edu/publication/getting-teacher-assessment-right
48
RESOURCES Goe, L., Bell, C., & Little, O. (2008).
Approaches to evaluating teacher effectiveness: A research synthesis. Washington, DC: National Comprehensive Center for Teacher Quality.
Goe, L., Holdheide, L., Miller, T. (2011) A practical guide to designing comprehensive teacher evaluation systems. Washington, DC: National Comprehensive Center for Teacher Quality.
Hinchy, P. (2010). Getting Teacher Assessment Right: What Policymakers Can Learn From Research. Boulder, CO: National Education Policy Center.
Mathers, C., Oliva, M., with Laine, S. W. M. (2008). Improving instruction through effective teacher evaluation: Options for states and districts. Research and Policy Brief. Washington, DC: National Comprehensive Center for Teacher Quality.
National Education Association. (2009). Teacher evaluation systems: The window for opportunity and reform. Washington, D.C.
Stronge, J. H, & Tucker, P. D. (2003). Handbook on teacher evaluation: Assessing and improving performance. Larchmont, NY: Eye on Education.
49
REFERENCES Laura Goe--Webinar for Oregon School
Coaches, April 20, 2011: http://www.lauragoe.com/LauraGoe/Oregon-April%202011.pptx
EPI Briefing Paper--Problems with the Use of Student Test Scores to Evaluate Teachers: http://www.epi.org/publications/entry/bp278
Rand Education—Evaluating Value-Added Models for Teacher Accountability: http://www.rand.org/pubs/monographs/2004/RAND_MG158.pdf