Peer observation, feedback and reflection for online practice

Mark H. Jones, Faculty of Science & eSTEeM

Overview• Motivation for the project• Models of peer-review• Implementation of a peer-review exercise• Evaluation • Comments on wider scale adoption

Motivation • Level 3 physics and astronomy modules have moved

completely to online tuition – with online rooms replacing face-to-face contact.

• Staff Tutor has a responsibility to:– develop staff in their pedagogic practice– monitor the quality of what is provided

• In an online environment, there are new possibilities to set up peer observation.

• Can peer-observation be useful in developing practice for synchronous online tuition?

Models of peer-review• Peer-observation is used throughout HE in several

distinct contexts. • A useful classification was provided by Gosling (2002):

- Evaluation model- Development model- Collaborative model

• “Observation” is a loaded term.

Peer observation models Characteristic Evaluation model Development model Collaborative model

Who does it, and with whom?

Senior staff, or chosen ‘evaluators’ other staff

Educational developers observe practitioners

Teachers/peers/ colleagues

Purpose Quality assurance (e.g. identify under-performance); appraisal (e.g. promotion cases)

Demonstrate or improve competencies in teaching; assessment.

Improve teaching and learning through dialogue, reflection; stimulate innovation

Outcome Report / judgement Report / action plan Analysis, discussion, wider experience of teaching methods

Status of evidence

Authority Expert diagnosis Peer shared perceptions

Adapted from Gosling (2009)

The collaborative ideal?• Purpose: improve teaching and student learning through

dialogue, self and mutual reflection; stimulate innovation• Outcome: analysis, reflection, discussion, wider

experience, improvement to teaching and learning. • Status of peer review: peer shared understandings and

perceptions. Non-judgmental, constructive facilitated dialogue

• Intended beneficiaries: mutual benefits for both peers, students

• Conditions for success: a culture in which teaching is valued and discussed.

• Risks: confirms existing practice, passive compliance, bureaucratic.

Implementation• Voluntary participation• Reviewee is in control of the process:

- makes initial contact with reviewer- reports when process is complete

• Briefing for participants (online meeting – recorded)• Aimed for review of live sessions (but some review of

recordings did take place)• A cohort of tutors on third level physical science

modules• Tutor group tutorials (rather than module wide

presentations)• Aimed to observe different modules (wasn’t always

possible) SM358, SMT359, S382, S383

Practicalities: three stages1. Pre-observation contact

2. Observation

3. Feedback and reflection meeting

4. (Notification that process is finished)

Pre-observation discussion• The context of the tutorial (reviewers may not know the

module).• Note-taking – are you both happy with the note-taking

protocols (see below)?• Will whiteboards be shared beforehand?• Will notes be shared before or after discussion (see

below)• How will the reviewer be introduced to the students?• Agree when feedback meeting will be held.

From briefing session

The observed session• Turn up in good time.• Tutor to introduce reviewer as agreed.• Reviewer takes notes as agreed.• Probably not a good idea to run on from the observed

session straight into the feedback/reflection meeting.

From briefing session

Feedback / reflection meeting• Aim: to reflect on a teaching session in order to:

- identify good practice- analyse situations which were less successful- feel confident about making changes to practice

• NOT an aim to either identify everything that wasn’t perfect, or to solve every issue in online teaching.

From briefing session

Timeline• Number of participants: 12 (out of possible 20)• Call for volunteers: April 2012• Online briefing: May 2012• Observations took place: June to September 2012• Feedback / reflection sessions after observations

Practical problems• Timetabling (and use of recordings)• Permissions setting on VLE• Timescale for the process (too short)

EvaluationMain tool for analysis is a structured conversation conducted at the end of the presentation (October 2012). Explores:•Practical problems.•Practical considerations.•Themes around peer observation. The extent to which the process meets the ideal of Gosling’s collaborative model (see also Martin & Double 2006)

Evaluation: themes• Purpose: learning and reflection? Other peer-

observation experience.• Apprehension (Bell, 2010) threatening nature (Cosh,

2006)• Nature of the feedback (Bell, 2010). Training observers

to give constructive criticism (Cosh,2006) • Effectiveness (Bell, 2010) and uncritical positive

feedback. • Benefits to observer, creative reflection (Cosh,2006)

and Bell (2010) - 'modelling teaching skills'. • Ongoing professional development (Bell, 2010)• Identifying good online teaching (Swinglehurst, Russell,

Greenhalgh, 2008 )

Evaluation: current status• Evaluative conversations were conducted in October /

November.• Currently being transcribed to text.• Qualitative data to inform understanding of the themes.

Wider adoptionSome comments:•Findings of small scale qualitative survey should be a useful guide to issues for any larger scale implementation.•There is a specific need to develop discussion around practice in teaching using synchronous conferencing. •Important to create a model that is perceived to be collaborative.•Care needed to avoid the process being appropriated for other purposes (e.g. quality assurance).