NASDAP Conference 2013

Prof. Eileen Piggot-Irvine

Eileen.Piggotirvine@royalroads.ca

APPRAISAL: WE ALWAYS WERE

AHEAD OF THE PLAY AND THE WORLD IS JUST CATCHING UP!

• My own rationale for appraisal!• What’s happening in varied countries?• Barriers to effectiveness• A model for effective appraisal that ensures a ‘productive

approach’• Distilling the most important principles• Collaborative re-design of own school appraisal systems

with a view to incorporating the prioritised principles

SESSION OUTLINE

From Looney (2012 – a superb paper):

“Students from low-income families may benefit the most from learning with very effective teachers (Nye et al., 2004; Sanders & Rivers, 1996) and those with a succession of high-quality teachers may potentially achieve the same levels as their peers from high-income families (Motoko et al., 2007). On the other hand, Sanders and Rivers (1996) found that the residual effects of learning with poor quality teachers were long lasting, and that students assigned to more effective teachers in later grades were unable to compensate for earlier gaps.” (p.440)

THE CASE FOR RECOGNISING EFFECTIVE TEACHING

We need great teachers and we need to know that the best teaching is happening in our schools.

In my opinion, a leader who does not want to know whether great teaching is happening, or not, is potentially

robbing our children of learning.

BEING HONEST ABOUT MY STAND

The US uses the term ‘evaluation’ as the description for overall appraisal activity. Approaches vary for individual

states.

USA

In US overall, worrying trends, standardised testing, firing and moving teachers in poorly performing schools instead of looking at the context. In a low performing Ohio school, all teachers were requested to re-sit their teacher entrance exam.

40% of teacher evaluation evidence is based on test score evidence.

Inadequate evaluation systems miss a prime opportunity to improve practice.

Evaluation (appraisal) systems need more than student achievement scores – need information on how to further improve (feedback).

An example effective system is in Toledo - based on continuous improvement, has professional standards, measures student achievement (but not only achievement).

FRANCINE EDWARDS, AMERICAN FEDERATION OF TEACHERS, 2012 KEYNOTE ADDRESS TO AERA ….

“… a technically valid and reliable assessment framework will not by itself ensure the improvement of teaching… In our experience, the assessment of teacher quality fails more often because of organizational neglect than because of technical deficiencies. In particular, school districts have typically not done a good job of managing the relational and political aspects of the process.” (Donaldson & Donaldson, 2012)

FROM USA RESEARCHERS

Widescale incentives (Teacher Incentive Fund, TIF) to implement performance pay. Extensively evaluated (Humphreys et al., 2012), but:

“… existing knowledge about how to make performance pay a transformative reform is relatively sparse.” (p.61)

General advice and long checklists of

components that must be in place was

not enough to overcome the uneven

implementation from one school to the next,

even within the same TIF grant holder.

THE MOVE TOWARDS PERFORMANCE PAY IN THE US

VAM are designed to evaluate student test score gains from one year to the next.

A cautionary tale from Darling-Hammond et al. (2012):

“Research indicates that value-added measures of student achievement tied to individual teachers should not be used for high-stakes, individual-level decisions, or comparisons across highly dissimilar schools or student populations.” (p. 14)

Extensive research has shown:

1. Value-added models of teacher effectiveness are

inconsistent.

2. Teachers’ value-added performance is affected

by the students assigned to them.

3. Value-added ratings can’t disentangle the many

influences on student progress.

MOVEMENT TOWARDS USE OF VALUE-ADDED MEASURES (VAM)

Lissitz (2012), an excellent summary paper qualified with strong statistical analysis.

Lissitz is skeptical, pessimistic and advising caution about VAM.

AND MORE ON VAM

National Board for Professional Teaching Standards (NBPTS) – created standards that are grounded in student learning.

Interstate New Teacher Assessment and Support Consortium (INTASC) translated NBPTS for beginning teachers - adopted by over 40 states for initial teacher licensing and have become basis for evaluation that is much

more stable than VAM.

The hope is that these will extend to the

evaluation process for all teachers.

BUT THERE IS HOPE AND PROGRESS BEYOND VAM!

The Measures of Effective Teaching (MET) Project - developed a number of tools (observations, video of teachers) supplemented with other artifacts of practice (lesson plans, assignments, etc.) – evidence is scored against standards of effective teaching.

Teacher Advancement Program (TAP), based on NBPTS and INTASC standards as well as the Conneticut standards-based assessment rubrics: evaluate teachers 4-6 times a year using master/mentor teachers or principals certified in a rigorous four-day training. Teachers look at and evaluate videotaped teaching episodes using the rubric, then dialogue with mentor and plan ongoing growth. Schools provide professional development, mentoring, and classroom support to help teachers meet these standards.

HOPEFUL SYSTEMS CONT.

So… a mix of good and not so good exists in the US. It is highly variable, but there is hope in these latest trends.

In the UK ‘performance management’ is the term applied to appraisal

UK

In Britain a national system exists and there are direct links to salary (Bubb & Hoare, 2001).

Forrester (2011) - movement from appraisal (backward looking) to goal setting focused performance management (forward looking) with a performance related pay (PRP) component. The intent of the movement to PRP was to incentivise teachers. Forrester suggests rather that it has been a millstone.

Recent calls in the UK for strengthening the emphasis on student learning outcomes eg Moreland (2011):

 

What we should be doing, in my view, is to base judgements on the responsibility the individual teacher takes for her/his own professional development and the learning and progress made within lessons… In this way, we may begin to move away from the concept of performance management as a once a year, paper-chasing exercise and move towards a process of managing performance aimed at improving the classroom experience for both teachers and students alike. (p.24)

CANADA - BC

“Teacher evaluations a hot-button topic” Geoff Johnson, The Victoria Times Colonist August 28, 2012

“If you want to press a hot button inside or outside the hallowed halls of public education, teacher evaluation would be the one almost

guaranteed to light up the "Emergency!" display on every teachers' union office desk.”

British Columbia Liberal leadership candidate Kevin Falcon proposed the idea of merit pay for

educators. The B.C. Teachers’ Federation – predictably - was not in favour.

The current system gives most responsibility for teacher evaluation to principals, who are guided by language in collective agreements.

But that's not working, George Abbott, Education Minister (2011) said:

"The assessment process, I hear from most parties,

is not a satisfactory one. Most assessment is done

by principals and vice-principals and I believe

that they feel there's much room for improvement."

A TOPIC FILLED WITH POLITICIAN AND UNION CLASH

Teacher evaluation is a formal cyclical review (3 yearly, Christian schools 4 yearly) conducted by a principal or vice principal. The collective agreement includes well-defined procedures for this retrospective model. Evaluation generally is not seen as an opportunity for reflection and growth, but as a potential impingement on teachers’ professional competence.

Teacher assessment is a more collaborative

process that can include self-assessment,

goalsetting and planning as well as

external feedback.

IN BC - A TWIN PROCESS

The guidelines for evaluation include:

1. Evaluations shall take place:

- At the request of the teacher (T)

- When initiated by the Administrator (A)

- At a mutually agreeable time (M)

2. The teacher shall have the opportunity to select the observation times.

3. At least X days prior to evaluation, the teacher and the evaluator will meet and discuss evaluation criteria, timelines, etc.

4. Reports shall focus mainly on the teacher’s area of expertise

5. The teacher will receive a draft copy of the report and have the opportunity to propose changes prior to filing the final report

6. The teacher will have an opportunity to submit written commentary of the final report, which will be filed along with all copies of the final report a report written by the administrator that includes a rating of either satisfactory or unsatisfactory.

TEACHER EVALUATION IS ALMOST TOTALLY SUMMATIVE

A minority of school districts in BC use a professional growth plan model as part of their assessment process.

However, this process and use of a growth plan model, is not supported by the BCTF (the union).

Because growth plans are developed in consultation with a principal and because they may link to school or district goals, they are seen to

impinge on professional autonomy.

You can’t win!!

TEACHER ASSESSMENT IS ALMOST TOTALLY FORMATIVE

Donaldson & Donaldson (2012, p. 77-), suggest the five steps to effective systems include:

Step 1: Include teachers in designing the performance evaluation system.

Step 2: Protect opportunities to learn and grow.

Step 3: Hone principals’ and coaches’ skills at observing and consulting with teachers.

Step 4: Build time for teacher evaluation into principals’ workloads.

Step 5: Make instructional improvement a district priority.

See how this was done in NZ in Piggot-Irvine (2010).

APPRAISAL EFFECTIVENESS GENERALLY

NZ

An activity “… intended to benefit both the individual and organisation by leading to affirmation that performance

expectations are being met, and to the identification of areas for improvement”

(Piggot-Irvine & Cardno, 2005, p. 12)

Appraisal fits within a larger performance management cycle. Performance management (a macro-descriptor) - encompasses all of the micro-processes associated with

personnel.

 

Performance management covers the functions that begin when a staff member enters the organisation through to their

exit. (Piggot-Irvine, 2003)

Performance Management

VacancyRecruitment

selection

Induction

Mentoring

Appraisal

Discipline andCompetence

Development

Resignation

ExitInterview

Barriers to effectiveness overall in appraisal abound. Perillo (2006) sums up many of these when stating:

 

“… employee conflict and competition, the undermining of teamwork, disproportionate employee responsibility for poor performance and perceptions of bureaucratic control … poor recall of development goals, confusion and doubts about the

purpose and value of the process, criticism about the time lost by participating in it …” (p.365)

Just one example of barriers:

Mather and Siefert (2011) think a strong focus on accountability (tighter control) has created an environment where:

 

“… senior managers have employed … close supervision of task content and its execution. As a result the professionals have resisted

collectively and formally through unions, informally in the common rooms and individually through grievance, absenteeism, increased

instrumentalism and dull compliance in the job.” (p.26)

 

Mather and Siefert continue - such activity displaces professional self-regulation, undermines high trust relations and

educational autonomy.

“To hold schools and teachers accountable for differences in attainment of social groups is particularly unfair and

unreasonable” (Ivan Snook, in Hislop, 2012, p.1)

“… but the general policy approach continues to emphasise that it is underperforming teachers and

schools that are at fault” (Hislop, 2012, p.1)

Hislop believes that the Minister’s agenda with improving the appraisal system is to link the accountability

component to pay.

PERFORMANCE PAY IN NZ

Although Forrester (2011) reminds us that:

“… little is known of actual costs let alone the extent to which performance management has contributed to ‘improvement’, ‘efficiency’ and ‘excellence’...” (p.8)

… it is my experience that appraisal can

achieve multiple advances in clarity, focus

and improved performance and relationships

as positive outcomes if a developmental and

accountability balance is maintained.

BUT THERE IS GREAT HOPE!

• Key elements of an effective systems• Strengthening student learning outcomes via ‘deep’ goal setting.• Establishing and maintaining trust.

WHAT IS NEEDED? WHAT IS EFFECTIVE?

Elements of Appraisal Effectiveness (Piggot-Irvine, 2003)

Criteria For

Effective Appraisal

Independent from Disciplinary Aspects

Integrated Development and Accountability

Beyond the Superficial – On-Going and In-depth

Well Resourced with Training and Time

Transparent and Confidential

An Educative Process

Clear Guidelines

Trust Generating

Based on Mutual Respect

Based on Objective, Informative, Data

PLUS ….. making links to enhanced student learning! Sinnema and Robinson (2007) - appraisal role in promoting teacher inquiry focused on teaching (an input) that strengthens the student learning focus (an output). Has been a weak focus on student learning in appraisal due to:- Low priority placed on student learning or achievement nationally in

statements of teaching standards;- Limited examination in appraisal of how teaching is impacting on

students; - A compliance orientation predominating in appraisal with a lack of

specific/challenging goals being set; and - The belief that appraisal is widely seen as an opportunity for celebration

and support may prevent confronting of assumptions and requests for evidence in the process.

The ‘deep’ goal setting that has strongly been promoted previously by Piggot-Irvine (2003) and subsequently

Piggot-Irvine and Cardno (2005), supports the strengthening of the student learning outcomes focus that Sinemma and Robinson have subsequently emphasised.

The goal setting session addresses this issue.

DEEP GOAL SETTING SHOULD CASCADE FROM THE STRATEGIC OBJECTIVES OF THE SCHOOL

Cascading alignment between school wide vision and goals, team and individual goals

In one-third of schools, the principal’s appraisal was linked to targets or strategic goals.

In 21 percent of schools, both the principals’ and the teachers’ appraisals were linked to targets or achievement goals.

Many schools did not recognise that appraisal is a key part of a systematic approach to improving the effectiveness of teaching and outcomes for students. Improvements are more likely to occur when key elements are connected and support each other. Appraisal provides an opportunity for leaders and teachers to inquire together into the impact of teaching on student learning.

2013 ERO REPORT

Many of the schools that linked school and appraisal goals demonstrated consistently high performance in the six dimensions of a successful school. These included a commitment to all students achieving, and a sense of shared responsibility for student learning, as well as a systematic approach including:

• using data to identify student and staff needs

• setting school goals

• linking staff goals to the school goals

• planning and accessing related professional learning

• establishing processes where teachers discuss strategies to accelerate progress

• monitoring the impact on student learning to identify effective practices.

ERO (2013)

My Thinking!

The Appraisal Inquiry Model (AIM)

An ongoing, inclusive (strongly so with the ‘productive’ intent), situationally sensitive, teacher and student learning

oriented Model.

If a national appraisal system is to proceed … it must be about teacher development leading to more effective in-

class performance … must have a basis of peer review in a climate of trust …properly resourced … Brian

McEntee (Listener, 9 June, 2012: p.9)

A FURTHER AND PERHAPS MOST IMPORTANT KEY FEATURE OF

EFFECTIVENESS

HOW DO WE DEVELOP TRUST?

42

- Non-defensive (non avoiding, non-controlling) openness and honesty are the underpinning values needed throughout appraisal (Argyris, 2003; Piggot-Irvine, 2012). These LEAD TO trust development.

- Openness and trust do not result from an emphasis on needing to be nice or liked (this creates avoidance).

- Non-defensive strategies are easy to employ in non-contentious situations but this is not the case when there are difficult messages to be delivered, as is sometimes the case in appraisal.

- In such tricky situations we often unknowingly resort to defensive behaviours including the avoidance of confronting issues.

TWO DOMINANT DEFENSIVE STRATEGIES

AVOIDANCE CONTROL - MIXING MESSAGES - PREDETERMINING OUTCOMES- SOFT-SELLING - LEAPING THE LADDER OF INFERENCE- EASING-IN - MAKING DECISIONS WITHOUT CHECKING- NOT REVEALING THINKING - OFFERING SOLUTIONS- ENGAGING IN SLL - OVER-ADVOCATING OWN POSITION

NON-DEFENSIVE = PRODUCTIVE STANCE

44

Employing:DLL

Surfacing underlying issues and dealing with them

Reflection in Action (rather than just on Action) Reflecting and responding during rather than after

action

Staying low on the Ladder of Inference Withholding judgments until after dialogue

Engaging in ‘Dialogue’ based conversations

A PRODUCTIVE STANCE – DIALOGUE STEPS

45

Advocacy Stating the issues/recommendations, backed by

evidenceInquiry

Inviting challenge of position, views, evidence Checking what views others hold

Receiving others’ views non-defensively Summarizing mutual understanding of issues

Gaining Internal Commitment Seeking bi or multi-lateral (genuine two or multi-

sided) solutionsMonitoring

Taking joint responsibility for planning, implementation and monitoring

Group Task:

From what that you have heard that is happening in in varied countries, collectively brainstorm what would you suggest are the 8 most important principles (eg high ownership by appraisees) that need to be present for an appraisal system to be effective.

Next, prioritise the principles in terms of how much they would influence appraisal effectiveness, with the first in the list being most influential.

Record the prioritised list on the large sheets of paper provided.

DISTILLING RECURRING PRINCIPLES

A LITTLE MORE ABOUT THE TOP PRIORITIES

Now that you have a reasonably clear idea about what is important for effectiveness, its time to think about how to apply/incorporate these principles into your own school appraisal system.

Complete the following activities in your group:

1. Create a model that would fit your own school.

2. Outline the values that underpin the model.

3. Describe the key elements (e.g. 2 sentences per element)

You will be presenting your model to the whole group.

RE-DESIGN OF OWN SCHOOL APPRAISAL SYSTEMS

Bubb, S. & Hoare, P. (2001). Performance management: Monitoring teachers in the primary school. London: David Fulton.

Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation. Phi Delta Kappan, 93(6), 8-15.

Donaldson, M..L., & Donaldson, G.A. (2012). Strengthening Teacher Evaluation: What District Leaders Can Do. To cultivate high-quality teaching, school leaders need to focus on the human side of teacher assessment. Educational Leadership, May, 78-82.

Education Review Office (2013). Board Employment Responsibilities: Linking Charter Targets to Appraisal in Primary Schools (February 2013), http://www.ero.govt.nz/National-Reports/Board-Employment-Responsibilities-Linking-Charter-Targets-to-Appraisal-in-Primary-Schools-February-2013/Overview

Forrester, G. (2011). Performance management in education: milestone or millstone? Management in Education 25(1), 5–9. DOI: 10.1177/0892020610383902Hislop, D. (2012). Is performance pay just over the horizon? Education Today, 2, 1.Humphrey, D.C., Gallagher, H.A., Yee, K.M., Gross, K., Campbell, A.Z., Cassidy, L.J., & Mitchell, N.M. (2012). Teacher Incentive Fund: First Implementation Report. U.S. Department of Education Office of Planning, Evaluation and Policy Development Policy and Program Studies Service.

Lissitz, R.W. (2012). The Evaluation of Teacher and School Effectiveness Using Growth Models and Value Added Modeling:

Hope Versus Reality. AERA Presentation, Vancouver, May.

Looney, J. (2011) Developing High-Quality Teachers: Teacher evaluation for improvement European Journal of Education, 46(4), 440-455.

Mather, K. & Siefert, R. (2011). Teacher, lecturer or labourer? Performance management issues in education. Management in Education 25(1), 26-31. DOI: 10.1177/0892020610383902Moreland, J. (2011). Managing performance for effective classrooms. Management in Education 25(1), 21-25. DOI: 10.1177/0892020610383902Perillo, S. (2006). Practice enhancement: Optimising teaching performance in schools. International Journal of Educational Management, 20(5), 365-379.Piggot-Irvine, E. (2003). Key features of appraisal effectiveness. International Journal of Educational Management, 17(4), 170-178.Piggot-Irvine, E. & Cardno, C. (2005). Appraising performance productively: Integrating accountability and development. Auckland: Eversleigh Publishing.

Piggot-Irvine, E. (2010). One school’s approach to overcoming resistance and improving appraisal: Organizational learning in action. Educational Management Administration & Leadership 38(2) 229–245.

Piggot-Irvine, E. (2010). Confronting evaluation blindness: Evidence of influence of action science based feedback. American Journal of Evaluation, 31(3), 314-325, DOI: 10.1177/1098214010369251.

Piggot-Irvine, E. (2012). Creating authentic collaboration: A central feature of effectiveness. In O. Zuber-Skerritt, Action research for sustainable development in a turbulent world (pp.89-107). Bingley, UK: Emerald.

Sinemma, C.E.L. & Robinson, V.M.J. (2007). The leadership of teaching and learning: Implications for teacher evaluation. Leadership and Policy in Schools, 6, 319-343.

REFERENCES