Chile Measuring Teacher Effectiveness Rec
Transcript of Chile Measuring Teacher Effectiveness Rec
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
1/14
1
Formative Teacher Evaluation with Ensea Chile
Final Report
Delivered by Mitch Kochanski and Stephen Zerfas
Outline
1. Executive Summary ................................................................................................................ 2
2. Analysis and Communication of Student Survey Data ............................................................. 4
3. Feedback Tools/Techniques ................................................................................................... 7
4. Evaluating Teaching vs. Analyzing Learning ........................................................................... 9
5. Prior Recommendations ........................................................................................................ 12
6. Next Steps ............................................................................................................................ 14
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
2/14
2
Executive Summary
The purpose of this report is to: 1) Provide recommendations on the three action points
identified after our initial report, and 2) wrap up our engagement by providing a short overview
of prior recommendations as well as suggestions for next steps.
Analysis and Communication of Student Survey Data
We offer five recommendations for the analysis and communication of the student survey data,
as well as for all future data dissemination processes:
1. Ensea Chile should build a one-page feedback sheet. The format should be similar to
that used in the MET and should combine student survey data with classroom
observation scores (and, once available, student learning data). This is a quick, effective
way to disseminate the most important information.
2. Distinct versions of the one-page summary should be given to teachers and tutors:
teachers should receive numerical results, while tutors should receive relative rankings
of the teachers. This would prevent unwanted competition but maximize the feedback
utility of the data.
3. Each tutor should meet with their teachers in a one-to-one setting to hand over the
summary for the first time. The one-to-one setting will allow tutors to emphasize they are
there to help, work alongside teachers in developing goals and milestones to translate
feedback into action, and dig deep to understand what makes exemplary teachers so
extraordinary.
4. Tutors should follow up on the customized questions teachers used in their surveys,
identify the ideas, and incorporate them in the next survey.
5. Ensea Chile should draw attention to outstanding teachers who are passionate about
measures. These champions can serve to generate buy-in across the network.
Specific Feedback Tools/Techniques
A comprehensive list of the feedback tools and techniques we have encountered throughout our
research is compiled in Section 2 for future reference. This section does not contain specific
recommendations; it is simply intended as a feedback toolkit.
Evaluating Teaching vs. Analyzing Learning
Ensea Chile has expressed an interest in reforming their teacher evaluation processes to
better focus upon student learning (as opposed to teacher practice). The analysis of student
learning requires collection of student learning data, therefore the task at hand is to develop
effective data collection processes. Speaking from a broad level, we recommend that Ensea
Chile consider this task as requiring two simultaneous approaches: top-down and bottom-up.
The top-down approach is concerned with developing standardized, organization-wide student
achievement data collection processes. The bottom-up approach is concerned with training
teachers to gather, analyze and interpret student data on their own (on-the-spot assessments,
quizzes, tests, etc.).
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
3/14
3
Prior Recommendations
We believe several of the insights from our initial report are worth repeating, specifically
because their adherence demands the ongoing attention of Ensea Chile leadership:
1. All metrics need to be self-validated in student outcomes.
2. Use student achievement gains, teacher observations, and student surveys together,
especially when evaluating teachers.3. Engage teachers in design and implementation of feedback/evaluation systems.
4. Language is essential (Formative >> Evaluative), and it all comes down to
relationships.
Next Steps
In reflecting on the engagement, we have identified several next steps:
1. Build one-page feedback sheet modeled after MET.
2. Develop robust student achievement data collection processes that, at a minimum, use
standardized tests.
3. Validate teacher observations and student surveys by observing how they correlate with
student achievement gains.
4. Share feedback toolkit with tutors and discuss which opportunities seem most
interesting and useful.
5. Revisit Ensea Chiles existing teacher training practices and consider how well we are
training our teachers to gather, analyze and interpret student learning data on their own.
Investigate how other teacher-training organizations train their teachers in these
methods.
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
4/14
4
Analysis and Communication of Student Survey Data
How can we make this information valuable for teachers?
We suggest Ensea Chile focus on five things in the dissemination process:
1. Develop a one pager to visually represent statistical results
2. Provide absolute scores to teachers but relative ranks of teachers to tutors3. Deliver results for the first time in a one-to-one meeting with teachers
4. Translate data into action for each teacher by setting specific goals and dates for
progress reports
5. Seek out top-performing and enthusiastic teachers as examples for the network
1. Developing a One Pager
We suggest Ensea Chile develop a one-page summary of allevaluation/feedback data
available on individual teachers, not just the student survey, modeled after the MET project.
This comprehensive one pager will allow Ensea Chile to integrate delivery of (1) student
survey results with (2) teacher observation feedback using TAL and, eventually, (3) student
achievement gains once Ensea Chile has obtained the right data.
The sample one pager provided by MET can be found in an attached document, but it roughly
takes the three-part format illustrated below. Each section can use any number of graphics to
represent statistical results:
Overall Score
Student Achievement Gains(Not currently available; Ensea Chile needs to
develope this)
TeacherObservation
Student SurveyResults
The benefits to putting all three measures on the same page are that it (a) succinctly provides
feedback of every type and (b) graphically reinforces the breakdown of the measurement
system. Ensea Chile does not currently have helpful data on student achievement gains, which
typically come in the form of standardized test scores, but needs to secure these as soon as
possible anyway in order to validate their teacher observation and student survey results. Once
obtained, these should be included in the one pager.
2. Provide Absolute Scores to Teachers but Relative Rankings to Tutors: Solving the Feedback-
Competition Tradeoff
Ensea Chile has the option of delivering absolute scores or relative rankings to its teachers
and tutors. Relative rankings are always much more helpful because they give meaning to the
otherwise largely arbitrary numbers. For example, if a teacher were to receive an average of 4
out of 5 on a particular student survey question, she might think she did very well on that aspect
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
5/14
5
of her teaching. If, though, the average for the entire network was a 4.2, she could very well be
one of the lowest performers, indicating she should work to improve. On the other hand, relative
rankings are inherently competitive. Given that Ensea Chile is challenged with creating a
culture of cooperation and candid relationships and soliciting buy-in from anyone, the
organization needs to be sensitive to the pitfalls of competition. The last thing Ensea Chile
wants is to see its efforts to provide feedback to teachers backfire because its teachers now feelthey are isolated and even more inadequate when constantly compared to everyone else.
In solution to this problem, we think Ensea Chile can get the informative benefits of relative
rankings without the dangers of excessive competition by providing relative rankings of each
teacher (easily displayed with boxplots as shown in the attachment) to every tutor and providing
absolute scores in each category to every teacher (easily displayed numerically). This will allow
a teacher to immediately see what he or she can work on according to the current scales (which
we have found elsewhere to be successfully tied to student achievement) without pitting each
teacher against one another in too competitive of an environment. At the same time, each tutor
will can correct for misleading high or low scores where appropriate. Consider a situation in
which a teacher receives an average score in both keeping students busy and keeping them
interested, for example. If the entire network of teachers averaged high scores on keeping
teachers busy but low scores on keeping students interested, a tutor might encourage that
specific teacher to spend more effort on keeping students busy rather than keeping students
interested because she is a relatively low performer in the former but a relatively high performer
in the latter.
Note: we think providing relative performance information to tutors and absolute information to
teachers should be considered for all three types of feedback, not just student surveys.
3. Maximizing a One-to-One MeetingWe recommend Ensea Chile deliver each teachers student survey results in person in a one-
to-one meeting. This could easily be done in the same visit as the monthly teacher observation,
especially since the proposed one pager incorporates both teacher observation and student
survey results. This will allow tutors to deliver the results while pursuing four additional goals:
personally emphasize this is about helping and not evaluating teachers, pay special attention to
which questions teachers added to the survey and why, ask teachers what else they would like
to know or gather data on, and identify Champions and their questions.
1. Tutors can never have enough opportunities to demonstrate that this process is about
helping teachers first. They can and should deliberately stress this in the meeting.
2. Three blank spaces were added for teachers to write in additional questions to allow us
to cast a thousand seeds and allow good ideas we missed to rise to the surface. We
need to finish the job and capitalize on this opportunity. Each tutor should question the
motivation behind each teachers custom questions and dig deep when necessary.
Dialogue among tutors should be encouraged, and truly good ideas need to be
incorporated into the next survey.
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
6/14
6
3. In the same way the student survey asked for teacher input in order to solicit buy-in and
sow a thousand seeds, tutors should ask teachers what else they would like to know in
a one-to-one meeting.
4. Tutors should be on the look out for teachers that scored very highly on the survey (and
TAL rubric) and how they used their blank questions. These outstanding teachers can
serve as Champions (see Incorporating Champions in the Process below).
4. Translate Teacher Feedback into an Action Plan
The attached Diocese of Memphis teacher-input form (Appendix A) is a tool that pulls teachers
into the goal-setting and evaluating process. During the one-on-one meeting, tutors could use
this form to work together with teachers to translate the student survey data into actionable
goals. This is the final step in the feedback process; it is both essential and complex, and for
both of these reasons it is ideal that the teacher have the tutors direct support. This would
create a partnership out of a mentor-mentee relationship and customize the process to each
teacher. This would both a powerful way to encourage teacher buy-in and an effective way to
track success.
We believe the opportunity to transform the mentor-mentee relationship into a partnership using
this tool (or something similar) is so great an opportunity that it should be used in response to all
forms of feedback, not just the student survey.
5. Incorporating Champions in the Process
Exemplary teachers are not just assets for students, they are role models for other teachers in
the Ensea Chile network. Ensea Chile should be on the lookout for outstanding teachers to
(a) ask them what else they would like to know in the measurement process and (b) serve as a
testament to other teachers how the measurement process is helpful. Exemplary teachers who
are excited about measurement are likely to think of things we have not yet considered, makingtheir input especially valuable. Those who serve as role models could also go a long way in
getting other teachers or even tutors excited about the student survey progress. Quotes or
stories about how these teachers used measurement tools to aid in their success should be
shared with everyone associated with Ensea Chile. This could be done via a newsletter, a
webpage, reference in one-to-one meetings, detailed in handouts, or any number of creative
ways.
Resources
Gates, F. (2010). Learning About Teaching: Initial Findings from the Measures of
Effective Teaching Project
Gates, F. (2012). Gathering Feedback for Teaching: Combining High-Quality Teacher
Observations with Student Surveys and Achievement Gains
Gates, F. (2013). Feedback for Better Teaching: Nine Principles for Using Measures of
Effective Teaching
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
7/14
7
Specific Feedback Tools/Techniques
What are interesting options for teach feedback between tutor class observations?
This section presents a simple list of the many different tools/techniques for teacher feedback
that we have encountered during our research. We do not present specific recommendations in
this section; we simply mean to supply a toolbox of ideas (and their sources) that may be ofinterest to Ensea Chile either now or in the future.
Microphone/Video Recordings (Tom Kane): In situations where frequent classroom visits are
unfeasible, teachers might record a lecture to later share with their tutor and review together.
Tom Kane suggests that a teacher can learn some things from watching him/herself teach than
he/she cannot learn from a tutors feedback.
Teacher Mentor (ACE): Every ACE teacher is paired with a teacher mentor" in the school
where they work. The mentor is typically a veteran teacher on the school faculty, and the
purpose of the relationship is to create a candid and accessible resource of teaching wisdom for
the young ACE teacher.
Written Teacher Reflections (ACE): Every two weeks, ACE teachers write and submit a short
self-evaluative reflection in response to a prompt from their tutor. Examples of useful prompts
include: How did you feel about your classroom management this week? or What specific
improvement do you hope to make next week? These reflections create a frequent flow of
conversation between the teacher and tutor, and allow the tutor to respond directly to the
teachers most pressing concerns.
One Minute Essay (Light, 2001):after every lesson students are given one minute (or a
comparable amount of time) to answer the question What was the main idea of the lesson?The essays are anonymous and collected by the teacher. There are two major benefits:
1. Teacher receives immediate feedback on how well students grasp the main idea.
2. Students areforced to think in a synthetic way. Over time, it seems students expecting
to write the one minute essay think differently as the lesson unfolded!
Formal Time Logs (Light, 2001):students are asked to track on a hour-by-hour basis what
they do each day for up to two weeks. An advisor or teacher then sets up a small group or
individual meeting with a student (if individual, no more than 10 min is necessary) to review the
log. Three key things happen in this meeting.
1. Teachers very quickly get to know their students quite well, including their life outside the
classroom. You learn an enormous amount about someone by observing how they
spend their time!
2. Teachers are able to assess students time management skills and see over time how
they have influenced students time management skills.
3. Teachers are able to instruct students on how to improve time management where
necessary and suggest extracurricular involvement (there is lots of research
emphasizing the importance of connecting life inside and outside the classroom).
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
8/14
8
Solicit Student Critique of a Teachers Work (Light, 2001):A student(s) is asked how they
might improve a teachers work. This is most easily done in an advanced class with a teachers
own essay, article, or work in process. This is a powerful shift from usual exercises because it
engages the student in collegial partnership (this can still be done with young grades). We see
three major benefits: (1) when done right, the student often puts enormous effort into the critiquebecause they feel the teacher is counting on them for support, (2) the teacher gets to see
employ the creativity of new minds, and (3) the teacher gets to see which areas the student has
mastered in application.
Testing Teachers on Pedagogical Knowledge (MET): teachers take a written test that
assesses their knowledge about the art of teaching. This appears to have potential in testing
how well a teacher has internalized or is able to articulate critical principles or tactics such as
those that underlie the TAL rubric. However, the MET project deemed success on this test was
not associated with student achievement gains and did not report in its final results.
Real-Time Earpiece Feedback (TFA): Teach for America uses a subtle earpiece to deliver
real-time feedback to teachers, especially for teachers that are struggling with controlling the
classroom and dealing with gross behavior problems. The equivalent of the tutor sits in the back
of the class and speaks into a mouthpiece that is linked to the teachers earpiece. The teacher
can then receive real-time instruction on how to handle the situation.
Learning from the Best (TFA): TFA is experimenting with how to effectively use their very best
teachers (truly remarkable and inspiring) to help instruct other teachers. TFA has used video
material because it is easy to disseminate, but is also experimenting with live teacher
observations (only this time, the observer is learning from the example rather than evaluating).
So far, they have faced two major challenges. First, how can TFA codify these very best teacherpractices? Second, how can you use one truly talented teacher as an example without making a
beginner or less effective teacher feel inadequate?
Resources
Light, R. (2001). Making the Most of College, Harvard University Press.
This resource has great tools for effective teaching at the advanced level. There is less
direct focus on feedback.
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
9/14
9
Evaluating Teaching vs. Analyzing Learning
How can we incorporate student learning into our teacher evaluation practices?
We all agree that an effective system of teacher evaluations should look beyond teacher
practice and focus on student learning as well. Ensea Chile currently focuses more upon the
former than the latter; what steps should be taken to strike a better balance? The analysis ofstudent learning requires the collection of student data, but what data is most helpful and how
should it be gathered? Ensea Chile should equally consider two alternative approaches to
obtaining and utilizing feedback on student learning: top-down and bottom-up. The top-down
approach involves developing the large-scale, long-term processes needed in order to
administer standardized assessments. The bottom-up approach involves equipping teachers
with the tools and techniques they need to autonomously gather feedback on student learning
on a day-to-day basis within the classroom.
Top-Down Approach
Ensea Chile needs to obtain helpful standardized test data as fast as possible. This is a must
for several reasons. First, student achievement outcomes are the ultimate measure of success,
and every other measure has merit only insofar as it can be traced back to improving student
outcomes. Second, measures of success are only useful if they can be compared.
Standardizing these measures is critical if Ensea Chile hopes to compare its results across
teachers and against other organizations. Finally, while some argue non-academic outcomes
are even more important than academic outcomes, there is (a) consensus that academic
outcomes are important and (b) much previous work on standardized academic tests relative to
standardized non-academic measures. Ensea Chile should continue to be a pioneer in looking
into non-academic outcomes, but it needs to move to secure helpful standardized testing data
as fast as possible to validate all of its efforts with teacher observations, student surveys, or any
other metric. Everything will have been for nothing if these measures cannot be traced back tosome form of student success.
Bottom-Up Approach
The most effective feedback mechanisms for analyzing learning and improving teaching are not
large-scale processes that can be administered by Ensea Chile or the tutors; the most effective
mechanisms for analyzing learning and improving teaching are small-scale processes that must
be administered by the teacher inside the classroom. Ensea Chile should be concerned with
training its teachers to analyze student learning themselves. Listed below are recommendations
from Trav Johnson and Kim Marshall for how to help teachers gather feedback on student
learning.
Consider Student Learning in Classroom Observations. Teacher evaluation systems can begin
to incorporate student learning even before standardized tests are in place. Johnson
recommends that classroom observations and evaluations shift their focus to consider student
learning. Johnson recommends the following framework for helping teachers think about how
they achieve student learning:
1. Learning Goals: What are the most important things students are expected to learn?
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
10/14
10
2. Learning Activities: How are the learning activities designed to achieve these goals?
3. Learning Assessments: How do the assessment seek to evaluate the goals?
4. Learning Outcomes: What evidence exists to show achievement of learning goals?
Use On-The-Spot Assessments. Kim Marshall suggests that on-the-spot assessments are
essential to giving teachers crucial feedback for adjusting their teaching methods and lessonplans.Examples of effective on-the-spot assessments include:
Think-Pair-Share: After initial instruction, have students 1) reflect for a moment about a
question, 2) talk to an elbow partner (the teacher circulates and listens in), and 3)
share thoughts with the whole class
Cold-Calling. By randomizing which students are called on as opposed to selecting
from volunteers, you get a more accurate sample of the entire classs level of
understanding.
Quick-Writes/Quizzes.Ask students to write a summary, or to respond to a few key
questions. Should not take more than five minutes, and the data should be assessed
immediately.
Exit Tickets. Gauge student understanding of one or two key points at the end of every
lesson.
Use Interim Assessments.Kim Marshall suggests that short-term on-the-spot assessments
must be complemented by long-term interim assessments. Interim assessments are typically
administered every 4-9 weeks.
Evidence Supporting the Formative Utility of Assessments
Inside the Black Box (1998) by Black and William: Voluminous evidence of the impact
of in-class assessments on student learning
Visible Learning (2008) by John Hattie: Hattie suggests that one of the mostdeterministic factors of student learning (more powerful than parental involvement or
socioeconomic status) was the effective use of formative, in-class assessments
Growth Mindset by Carol Dweck. Regular and frequent assessments send the
message to students that they should be learning something new every single day.
Resources
http://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms
.pdf
Contrasts between teacher-centered and learner-centered paradigms
http://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-
evaluation
Framework for learner-centered teacher evaluation
http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-
goals/22423#.U1Uz_HddVv0
Student-centered teacher evaluations focus on learning goals
http://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://edsource.org/2012/student-centered-teacher-evaluations-focus-on-learning-goals/22423#.U1Uz_HddVv0http://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.teachingprofessor.com/articles/teaching-and-learning/learner-centered-evaluationhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdfhttp://www.assessment.uconn.edu/docs/TeacherCenteredVsLearnerCenteredParadigms.pdf -
8/12/2019 Chile Measuring Teacher Effectiveness Rec
11/14
11
http://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.html
PDSA Framework (for teachers): developing learner-centered assessment
http://www.slideshare.net/chesleyjr/student-centered-assessment-presentation
Various methods for measuring student learning
Gates, F. (2010). Learning About Teaching: Initial Findings from the Measures of
Effective Teaching Project(Tech. Rep.). Gates, F. (2010). Learning About Teaching: Initial Findings from the Measures of
Effective Teaching Project(Policy Brief).
http://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.htmlhttp://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.htmlhttp://www.slideshare.net/chesleyjr/student-centered-assessment-presentationhttp://www.slideshare.net/chesleyjr/student-centered-assessment-presentationhttp://www.slideshare.net/chesleyjr/student-centered-assessment-presentationhttp://cte.udel.edu/sites/cte.udel.edu/files/u7/v17n8.html -
8/12/2019 Chile Measuring Teacher Effectiveness Rec
12/14
12
Repeated Recommendations from Prior Report
The purpose of this section is to briefly echo the most essential recommendations which were
made in the two initial progress reports. Each of the recommendations listed below are were
chosen to be repeated for two reasons: 1) they are crucial to success, and 2) their
implementation requires ongoing attention
1. All metrics need to be self-validated in student outcomes. Success in teaching is
measured by the student outcomes, and any attempt to measure teacher effectiveness via
student surveys or teacher observations is merely an intermediate attempt to measure the
process, not the outcome. The process, in turn, only has value if it is tied to the outcome, so all
Ensea Chiles hard work will have been for nothing if their efforts can not be proven to have
been associated with improvement in student outcomes. Literature from other contexts with their
own unique situations is not good enough. Ensea Chile must self-validate its metrics with its
own student outcomes.
2. Use student achievement gains, teacher observations, and student surveys together,
especially when evaluating teachers.Each ones weaknesses are offset by the others
strengths in terms of feedback potential, predictive power, and reliability. MET emphasizes over
and over again that using these together is most important when evaluating teachers and
making high-stakes decisions. When providing merely formative feedback, however, it is fine if
Ensea Chile spends more effort on strong feedback measures.
3. Engage teachers in design and implementation of evaluation system.Any formative
evaluation is a tool for teachers; our task is not only to design a good tool, but to ensure that
teachers value this tool. No evaluation system will impact student learning unless teachers
respect the system and appreciate its results; as such, it is essential that we constantly seek tofoster teacher buy-in. One approach to fostering teacher buy-in is to engage them as much as
possible in its design and implementation. Several specific actions that might be undertaken to
foster teacher buy-in are listed below:
Invite teachers to design their own student survey questions
Invite teachers to participate in focus groups or conference calls
Regularly gather feedback from teachers - either formally or informally - about the how
well the system is working and how useful the results are.
4. Language is essential. Ensea Chile tutors should be careful and deliberate in their use of
language when communicating with teachers regarding evaluations. In every interaction, tutors
should use language that cultivates an attitude of collaboration, candor andcommitment to
learning.Avoid use of the word evaluation whenever possible, especially in regard to feedback
instruments that are exclusively formative in nature. Additional thoughts/examples of the ideal
use of language are listed below:
Formative vs. Evaluative
Im here to collaborate with you vs. Im here to evaluate you
What do you think? vs. Heres what I think?
-
8/12/2019 Chile Measuring Teacher Effectiveness Rec
13/14
13
How can we improve? vs. How can you improve?
This new system will have a positive impact on teachers everyday lives
How can we learn from whatyoure doing well, and support you in places where youd
like to improve
5. It all comes down to relationships. If teachers have good relationships with theirtutors/principals/mentors, then teachers will be more open to constructive criticism and feedback
where they need it most.
6. Many, unannounced short observations >> few, announced long observations. Most
everyone agrees that six 10-minute observations are more useful for teachers than one 60-
minute observation. Here is the rationale:
Teachers need feedback more frequently than once or twice a semester
Once-a-semester observations produce long lists of feedback, but teachers can work
better with just one or two suggestions at a time
Teachers are nervous about having an entire semesters teaching evaluated in one
session; with many sessions, the stakes are lower and the observation is more
candid/authentic.
Resources
Rethinking Teacher Supervision and Evaluation, by Kim Marshall
Is Supervising the Heck out of Teachers the Answer?
http://www.edweek.org/ew/articles/2008/05/07/36marshall_ep.h27.html
Designing and Implementing the Next Generation of Teacher Evaluation Systems:
http://ccsr.uchicago.edu/publications/designing-and-implementing-next-
generation-teacher-evaluation-systems-lessons-learned
http://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learnedhttp://ccsr.uchicago.edu/publications/designing-and-implementing-next-generation-teacher-evaluation-systems-lessons-learned -
8/12/2019 Chile Measuring Teacher Effectiveness Rec
14/14
14
Next Steps
In reflecting on the engagement, we have identified several next steps:
1. Build one-page feedback sheet modeled after MET. This is detailed in the
recommendations, but needs to be done to move forward with the dissemination process.
2. Develop robust student achievement data collection processes that, at a minimum, use
standardized tests.As mentioned earlier, Ensea Chile needs to self-validate to generate
credibility for all of its feedback measures. Using standardized tests is a major part of that
process. Ensea Chile should take advantage of standardized tests required by the state, but
should not stop until it is content with the quality and amount of data. There are many types of
standardized tests that Ensea Chile could get a hold of and implement with its teachers at the
classroom level, if not the school level (i.e. MET uses the Stanford 9 Assessment). Whether
Ensea Chile uses tests currently used by the state, implements its own, or mixes both, Ensea
Chile should definitely use tests with a vertical scale that transcends grades. This will allow
results that reveal improvement for students that are far behind grade level and would simply fail
tests with narrow focus limited to the current grade.
3. Validate teacher observations and student surveys by observing how they correlate
with student achievement gains.Once tests are implemented, analysis needs to be done to
validate teacher observations and student surveys. Feedback from these tools is a only a
guess, and a very imperfect one at that, until they are validated with student outcomes.
4. Share feedback toolkit with tutors and discuss which opportunities seem most
interesting and useful.This takes the toolkit out of the office and gets it into the hands of
teachers who are in the best position to identify when and where to use a new tool.
5. Revisit Ensea Chiles existing teacher training practices and consider how well we
are training our teachers to gather, analyze and interpret student learning data on their
own.This takes the benefits of measurement and evaluation the Training and Support Team
has deemed so valuable and enables each teacher to reap the benefits of their own micro
measurement and evaluation processes. Ensea Chile should investigate how other teacher-
training organizations train their teachers in these methods. A possible first step would be Kim
Marshalls book, which has much more information regarding on-the-spot and interim
assessments.