Transcript of Peer Reviewer Training Workshop Shea Wang & Trudi Ohki CAFÉ Sept. 22, 2014.
- Slide 1
- Peer Reviewer Training Workshop Shea Wang & Trudi Ohki CAF
Sept. 22, 2014
- Slide 2
- Workshop Overview Upon completion of this workshop, you will be
able to: Conduct a pre-observation meeting Review teaching
materials Use the narrative, checklist, rating, and data collection
forms for classroom observation Conduct a post-observation debrief
Write a final post-observation report
- Slide 3
- Activities Overview Practise using the pre-observation meeting
form Practise using the review of teaching materials form Practise
classroom observation skills and techniques Practise using the
narrative, checklist, rating, and data collection forms for
classroom observation Practise post-observation debrief Practice
post-observation report writing
- Slide 4
- Faculty Evaluation Policies: Instructional Faculty C5065
Non-Instructional Faculty C5066
- Slide 5
- Evaluation at MacEwan Policy C5065 guided by the philosophy
that we: promote a culture of teaching and learning foster
professional development and scholarly activity promote fairness
and transparency timely and accurate feedback comply with
collective agreements undertake as a process involving multiple
stakeholders, and a variety of assessment approaches
- Slide 6
- Faculty strive for excellence in teaching, which is assessed
by: teaching (including curriculum development) research,
scholarly, or artistic engagement service (collegiate citizenship)
collegiality (professionalism, respect for students & peers,
and ethical behaviour) professional development Evaluation at
MacEwan
- Slide 7
- C5065 Instructional Faculty Requirements based on appointment
type ContinuingProbationary Sessional/Sessional- extended Teaching
Dossier Peer Review Sessional-extended at renewal (Collective
Agreement 2011-14) Annual Report Student Online Feedback
Performance Review
- Slide 8
- C5065 Probationary Requirements Teaching Dossier - Annually
Peer Review - Two reviews during each of the first two years (4
total) Annual Report - Annually Student Online Feedback - Every
course during appointment Performance Review - Annually
- Slide 9
- Introduction What is Peer Review? Scholarly publications review
process to assess quality of the work Evaluation of
classroom/online teaching and teaching materials ASAC position
review committees Any evaluation process where the evaluators are
peers
- Slide 10
- Peer Review Literature Peer review is appropriate for Course
content, expertise in field of study Course objectives and
materials Appropriate teaching methodology Course organization
Appropriate tests, assignments, evaluation methods Seldin,
1999
- Slide 11
- Peer Review at MacEwan Teaching effectiveness Classroom visits
Teaching materials ASAC position review committees
- Slide 12
- Qualifications of Peer Reviewers Criteria developed by FEC
Completion of an appropriate peer review training course and / or
demonstration of peer review experience Minimum 3 years full-time
teaching experience Pedagogical expertise Possess a continuing
appointment Recommendation by Dean / Associate Dean / Director /
senior administrator
- Slide 13
- Attributes of Peer Reviewers Utilizes effective instructional
practices (demonstrated through dossiers, annual reports, awards,
etc) Demonstrates collegiality Provides constructive, objective
feedback focusing on teaching behaviours and improvement Models
ethical behaviour Appreciates different styles / modes of
teaching
- Slide 14
- Exceptions For all peer reviewers, should there be a deficit in
meeting the recommended qualifications, the Dean, Associate Dean,
Director, or senior administrator will assess and provide final
approval.
- Slide 15
- Choosing Peer Reviewers Chairs will support and advise faculty
members in the selection of two peer reviewers to conduct classroom
observations and review of relevant teaching materials. It is not
necessary that peer reviewers be content- area experts. Peer
reviewers can expect to be asked to perform peer reviews outside of
their home department(s).
- Slide 16
- Defining Effective Teaching Conversations about teaching
(30min) Pair and Share Break into small groups and discuss the
following sentences: I most enjoyed teaching when __________ I knew
I had a problem when __________ Have a member of your group report
on your discussion to the main group *Chickering and Gamson,
1987
- Slide 17
- Measuring Good Teaching Myths Dr. Fox Expressiveness High
Ratings Reflect Lenient Grading Workload/course difficulty Rank and
experience Gender
https://facultycommons.macewan.ca/headlines/professors-
from-sfu-share-mean-student-evaluations
- Slide 18
- Measuring Good Teaching
- Slide 19
- Measuring in context Take 15 min. and complete the following
sentences in your group: I knew my teacher was a good teacher
when/because I knew my teacher was a poor teacher when/because
- Slide 20
- Literature on Good Teaching Eight dimensions of college
instruction : Course organization & planning Faculty
communication Faculty/student interaction Assignments, exams, &
grading Instructional methods & materials Course outcomes
Student effort & involvement Course difficulty, workload, &
pace (Marsh, 1984 & 1987)
- Slide 21
- Literature on Good Teaching (2) Encourages student interaction
Encourages student cooperation Active learning Prompt feedback to
students Emphasizes time on task High expectations Respect
(Chickering and Gamson, 1987)
- Slide 22
- Literature on Good Teaching (3) Good teachers Are good
learners; Display enthusiasm; Know how to modify their teaching
strategies; Encourage learning for understanding; Demonstrate an
ability to transform and extend knowledge, rather than just
transmit it; Set clear goals; Provide high-quality feedback Show
respect for their students (Ramsden, 2003)
- Slide 23
- Overall Good Teaching Parker Palmer capricious chemistry Good
teaching is ultimately irreducible to a simple technical formula
Characteristics are beyond common descriptors or attributes of a
checklist Solution: multifaceted approach Student feedback, peer
review, self-reflection, and administrative evaluation
- Slide 24
- Peer Review Activity In your small group, reflect on your past
experiences with giving and receiving peer feedback. Address the
questions: What did you learn from the experience? In what ways it
was a positive/negative experience? Did it influence your
instruction? How did receiving and giving feedback change the way
you thought about teaching and learning?
- Slide 25
- Peer Observation The two forms of observation: Summative
observation - This observation is an evaluation. It can be done for
administrative or supervisory reports for the purpose of job
retention, promotion, salary increases, etc. Formative observation
- The purpose of a formative peer observation is to help
instructors become better teachers and more knowledgeable
professionals.
- Slide 26
- Peer Review Process at MacEwan Evaluation Policy outlines two
main areas of teaching assessed by peer reviewers: 1.Teaching
materials - lecture notes, course outlines, tests and exams,
assignments, grading rubrics, etc.; and 2.Instructional delivery -
observation of teaching in formal lectures, as well as other
delivery modes such as labs, studio, clinical, and online/distance
formats
- Slide 27
- Peer Review Process at MacEwan Five main elements:
Pre-observation meeting Review of teaching materials Classroom
observation Post-observation debrief Post-observation report
- Slide 28
- Forms Some of the forms available Narrative form Checklist form
Rating form Data Collection (see appendices of workbook)
- Slide 29
- Pre-Observation Meeting Emotionally, it is important to address
distrust of evaluation, violation of personal style and space,
insecurities about time, and fear of bias (Chism, 2007) Essential
that trust be established - Acknowledge the emotional aspect. Clear
objectives of observation should be outlined - Communicate
expectations Stressed that the process is collaborative - Engage in
active listening skills (Review pre-observation forms in
appendices)
- Slide 30
- Course Materials Some examples of materials include: Syllabus
Course guides Course packets and assigned texts PowerPoint
Presentations, overheads, BlackBoard, etc Handouts Videos and other
multimedia supplements Project assignment directions and
handouts
- Slide 31
- Procedure for Evaluation of Course Materials Importantly, we
are not focusing on student achievement measures when reviewing
teaching materials. Rather, we are looking at content, design,
clarity, etc Materials enable reviewers to see an instructor's
philosophy enacted, and can alert reviewers to the areas in which a
given instructor excels Materials can promote discussion about the
overall goals (Review course materials checklist in
appendices)
- Slide 32
- Classroom Observation Observers should use the same observation
form they should take separate minutes/notes At least two observers
should attend each class observation session At least two visits is
preferable No unannounced visits Discuss with the instructor the
format of the review prior to visit: Where will you sit? Will you
be introduced?
- Slide 33
- Classroom Observation During the Classroom Visit Be unobtrusive
(basically, remain quiet) Do not ask questions Do not volunteer
answers Do not correct the instructor Be attentive Note specific
examples of observed behaviours (positive & negative)
- Slide 34
- Classroom Observation Challenges One class may be not
representative Atypical performance (good or bad) Observer may not
have a good understanding of the context Personal biases on what
constitutes good teaching
- Slide 35
- Some Solutions Attend 2+ classes Video tape class? Focus-group
students after class? Use the data collection form, cite examples
Discuss visit with co-reviewer, debrief together Contextualize the
course reflect back on: Pre-observation meeting Examination of
course materials
- Slide 36
- Peer Observation Exercise Pre-observation In your small group,
decide on the form you will be using for review (narrative,
checklist, rating) Review form and highlight areas that each
reviewer will be responsible for focusing on and taking notes about
Observation Choose one of the two scenarios to review Reflect on
the class from the following viewpoints: Viewpoint 1: students
Viewpoint 2: instructor
- Slide 37
- Peer Observation Exercise Compare your list of observer
behaviors with others in your group Be reflective and systematic
through the criteria outlined in your observation form Embed your
observations & conclusions in examples of observed behaviours
Did the instructors behavior appear to be what you might expect in
a formal observation?
- Slide 38
- Peer Observation Exercise Role Play In your small group,
designate an individual to be the instructor, the reviewer, and a
reporter Break into large groups based on role as instructors &
reviewers & reporters Compare the notes your groups took during
the observation and discuss strategies for approaching the debrief
(and how the characters might react) Break back into your small
group and role play the post-observation debrief meeting
- Slide 39
- Post Observation Debrief Meet with the instructors immediately
following the classroom visit Ask the instructor to describe how
they felt the class went in relation to the goals, objectives and
expectations outlined in the pre-observation meeting At this time,
the peer reviewers should bring up any immediate concerns that were
highlighted by the observation session. **This is an important
aspect of the review as it can potentially change the observers
perceptions about what they saw and heard.
- Slide 40
- Post Observation Debrief Procedure Ask the person who taught
how s/he felt the lesson went Reconfirm the observation focus
points for the final report Offer positive comments on what you
think went well Identify any urgent issues Ask questions for
clarification, as needed End on a courteous note Provide an
expected timeline for the final report and final meeting
- Slide 41
- Post-Observation Exercise Prepare yourself to care watch that
you are not overly forgiving or overly critical (put yourself in
their shoes) Use descriptive language rather than evaluative Be
responsive to the goals outlined in the pre- observation meeting
Check to make sure the instructor has heard your message as you
intended Be forward-looking and positive in your discussion, i.e.
what can be improved over what went wrong
- Slide 42
- Peer Review Final Report Policy C5065 states Section 4.4.4,
Peer reviewers shall prepare brief written assessments, grounded in
evidence, and submit a copy of reports to the faculty members and
Chairs.
- Slide 43
- Who Sees the Final Report? Faculty member used to improve
teaching effectiveness, make adjustments as necessary for
application for continuing status Chair and Dean used for
administrative review ASACs used for personnel decisions
- Slide 44
- Characteristics of a Good Report Reference the goals
established during the pre- observation meeting Provide a written
record of what was observed and what was discussed Consider the
audience Use examples from the observation Language is clear,
concise, avoids jargon and abbreviations Tone is supportive and
phrasing is positive Action-oriented and provides specific and
practical suggestions for improvement
- Slide 45
- Characteristics (contd) One, joint report per instructor -
Structure of report should be agreed upon by all reviewers Return
report in a timely manner Review with faculty member within 3 days
of classroom visit Consider faculty member response before
submitting for administrative review Confidential (Review report
template in appendices)
- Slide 46
- Peer Review for Online Courses Online Peer Review Form (4
categories) Student behaviour Faculty-student interaction (multiple
interactive communication strategies) Technology support Learning
environment
- Slide 47
- Challenges - Peer Review for Online Courses A connection
between specific materials and instruction and student action that
manifest learning objectives. Technology and accessibility Are
instructors responsible for these? Yes, no, maybe, more
nowadays?
- Slide 48
- What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction: Clear
objectives as stated in: Syllabus: must be clear. Super important.
Grading Rubrics: explains how they will be graded Contd.
- Slide 49
- What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction Clear
objectives as stated in: Models or Examples of quality work from
successfully submitted and graded student work Explanations of the
relevance of course materials to learning. WHY are we doing this?
(think of students perspective). Contd.
- Slide 50
- What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction Assessment
tools should be used: Throughout the course to establish: Learning
styles Skill/knowledge level Challenge preconceptions Clarify
current learning Contd.
- Slide 51
- What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction
Opportunities for students to construct and experience their own
knowledge: We all learn differently and its important to be clear
and understanding about this. Learning processes and outcomes are
not all the same. Contd.
- Slide 52
- What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction
Opportunities for students to become increasingly responsible for
their own learning: Instructor should provide consistent and timely
input. Encouragement, good examples, and instructor presence Try to
develop intrinsic motivation. Contd.
- Slide 53
- What Is A Good Online Course? Content? Easy use of technology?
Interaction? Learner-centered outcomes-based instruction Options
for demonstrating learning outcomes (if appropriate) Opportunities
for reflection Effective student support services Contd.
- Slide 54
- What Is A Good Online Course? Clear and captivating
instructional design the writing style, the images and plug-ins
selected, and the layout are appropriate to the intellectual
sophistication, technical savvy and needs of the audience a pathway
to guide student learning appealing presentation Contd.
- Slide 55
- What Is A Good Online Course? Varied forms of interactivity
with material, other students and the instructor Regular
Communication with peers and instructor Varied Interaction with
Content Source:
http://www.4faculty.org/includes/digdeeper/online/lc_outcome
s_based_instruc.htm
- Slide 56
- Online Course Material for Review 1.Instructional material and
resources 2.Assignments and student assessment (clear and detailed
instructions and guidelines, purposes, formal writing
standards/academic writing requirement) 3.Discussion forums
(ability for students to give feedback and ask questions during the
class is critical) 4.E-mail messages 5.Course syllabus (the
syllabus is the most important document of the entire course. It is
the road map or game plan for the course) 6.Chat space
- Slide 57
- Where to Go For Resources CAF website: Then click on Faculty
Evaluation Then click on Peer Review Faculty Evaluation
Coordinator