TLC2016 - Peer Review, Peer Assessment, and Peer Feedback methods based on Blackboard Learn Tools

Click here to load reader

download TLC2016 - Peer Review, Peer Assessment, and Peer Feedback methods based on Blackboard Learn Tools

of 41

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of TLC2016 - Peer Review, Peer Assessment, and Peer Feedback methods based on Blackboard Learn Tools

Peer Feedback methods based on Bb Learn Tools

Peer Feedback methodsbased on Bb Learn Tools

Extensions to the Bb-Learn Tool Self and Peer Assessment

Peer Feedback Methodsbased on Blackboard Learn Tools Peer Review with Rubrics Peer Review between Groups Peer Assessment within Groups Peer Feedback via My GradesPresentation by:Hubert Nachtegaele, Educational TechnologistChristine Verclairen, Assistant Academic Staff

Extensions to the Bb-Learn Tool Self and Peer Assessment

We will present some methods of Peer Feedback, Peer Review and Peer Assessment, some of which are well established in our institution.Typical for these methods is that we realise them almost exclusively using Blackboard Learn tools, without need for extra costly investment.From the students point of view: only simple Blackboard Learn tools are used.

Peer Reviewwith Rubrics

Cartoon by Nick D Kim, (please see site for terms of reuse)Extensions to the Bb-Learn Tool Self and Peer Assessment Hubert Nachtegaele, Educational TechnologistAnnick De Vylder, Assistant Academic StaffProf. Dr. Jelle Boeve-de Pauw and prof. Dr. Piet Van den Bossche, Faculty of Social Sciences

The first method I will present is a method of Peer Review in which the students have to evaluate a paper of one of their peers by using an assessment rubric which has to be identical to the assessment rubric used by the the course leaders.

Why? 100 students Master of Instructional and Educational SciencesAssignment: write paper, apply/show Scientific Reporting SkillsAssessed using a Rubric focused on Scientific Reporting Skills 30 good ; 70 need more exercise, need a lot of feedback

too much work?impossible?method?Peer Assessment, using the same Rubric, with Peer Feedback, ...Rubrics is not only a method to evaluate these students, but also a skill to be mastered by them. Problem: In Bb Course Tool Self and Peer Assessment : No Rubrics !


The aim of the teaching staff isTo improve the Scientific Reporting Skills of their studentsAnd let their students become familiar with the structured method of assessment rubrics for evaluation and feedbackI show you the rubric used by the teaching staff...

This teaching staff contacted our Institutional Blackboard Support for tools to facilitate this learning process.They knew the Blackboard Learn Tools Self and Peer Assessment and the tool Rubrics, but it was obvious that these tools werent adequate because of different reasons: In Blackboard Learn, assessment rubrics are a tool for teachers, not for students. In the Self and Peer Assessment tool, its impossible to organise the Criteria in a structure similar to that of an assessment rubric.The Blackboard Tool Self and Peer Assessment assumes that all students in a course participate. This is not realistic for our purpose.

Copy/Paste the Rubric Criteria and Levels of Achievement to spreadsheet:

Simulate Rubric & Feedback by a Bb Test

We choose to get around these restrictions by combining other Bb Learn tools to organise our own Self and Peer Assessment, with as little compromise as possible.For the Evaluation Criteria we simulate the Rubric form with a Bb Learn test. For the comfort of the teaching staff, a template was made to import this Criteria Test in Blackboard.

Some basic Hocus-Pocus in the spreadsheet...

Simulate Rubric & Feedback by a Bb Test

With some basic Excel manipulations ...

Copy & Paste this to a notepad document...

Simulate Rubric & Feedback by a Bb Test

Followed by a copy/paste to notepad and upload into a Blackboard Learn test ...

and upload this notepad document as questions in a Bb-Learn Test:

Criteria (29 in this case)

Levels of Achievement(MC - questions)Feedback-field(SA - questions)Simulate Rubric & Feedback by a Bb Test

this procedure creates the Bb Learn test. For this specific case:29 criteria with Levels of Achievement in Multiple Choice format, each followed by an essay question for Peer Feedback.This was the best approach we found to simulate the Bb Learn Assessment Rubric format with tools available for students.I will call this the Rubric Test.

Students Choose Theme > Write > Submit Paper

Groupset:Choose one of three themes as subject for your paper...

Instructions specificfor chosen themeAdaptive Release,specific by theme

Only one assignment,common for different themes.Adaptive Release,common for all themes

The students choose a theme for their paper by enrolling themselves in a theme group, one group for each theme.This step of the process is organised with the Blackboard Learn Self Enrolment mechanism for Groups.

Students who proved earlier to master the desired skills do not enrol in these groups.This enrolment in groups bypasses the problem we earlier encountered with the standard S&PA which requires that all students participate to the process.

The students write their paper and submit them using the Bb Learn assignment tool.We need for this process only one assignment for all themes together.

Random Distribution of Submitted Papers Assignment File DownloadTheme Group Members exportEXCELRandom Number for each submissionSort by Theme and Random NumberReplace RN with a Submission Code...



Crucial in a S&PA system is the random distribution of the submissions and the return of the feedback to the authors of the papers.To organise the first step of this distribution, we download the assignment files and sort them by theme group and by an associated random number. Some simple Excel functions create batch copy commands to rename and send each submission to the authors of the two foregoing submission within the same theme group.

Excelparsing filenamesrandomisation (see one slide up)constructing anonymous filenamesconstructing batch-commands for renaming and copyingContent CollectionCC map of students...Blackboard DriveLink Content Collection map to Windows Drive letterCommand WindowExecute Rename and Copy batch commands

Method to automate the delivery of a set of dedicated documents to the respective, individual students..Other methods possible, but a native Bb-Learn method is missing.

Tools for this Distribution of Papers

The purpose of these batch commands is: Copy the submitted papers to a Content Collection map which is dedicated to the student who has to review that paper.

We use BlackboardDrive to establish the connection to these Content Collection maps.This makes it possible to open a command window and execute the batch commands in it.

This way, the whole procedure of downloading the submitted papers and distributing them between the peer evaluators takes only a few minutes.

Student in role ofPeer ReviewerPeer Assessment 2 papers > 2 tests 29 criteria: level & feedbackFinally: Self assessment: 3th test

In the role of Peer Reviewer the students evaluate the two assigned papers and give feedback on it.They do this solely by using the Rubric Test and the Criteria and Feedback questions in that test.

Redistributing Peer Feedback

Paste in the spreadsheetVLookup the author of the reviewed paper

Copy to CC map of author using same techniques as for distributing the papers, but in the other direction!

here we need a VBA script

The system for returning these reviews and feedback to the authors of the papers is roughly the same as the foregoing system of distribution of the papers, but in the reverse direction.It starts with a Download Results of the Rubric Test, followed by an automatic reformatting and export of these results to individual web documents. This is the only step in the procedure where programming was needed: for this reformatting and export we have made a VBA script within the Excel spreadsheet.

Overall Process, Student View Choose theme, Write paper, Submit paperReceive 1st paper to review > peer assess it using rubric & feedbackReceive 2d paper to review > peer assess it using rubric & feedbackSelf assess own paper using rubric & feedbackReceive 1st and 2d peer assessment & feedback on own paperRevision and Final Submission of own paper

This slide shows the overall process, as seen by the students.Only standard Blackboard Learn tools are used by the students.For them, there are no technical instructions needed specific for this method of Peer Review.


Bb Learn tools for communicationExcel & VBA & BbDrive for processingWorth to integrate (partly) in Bb ! complex combination of relative simple tools

Students and Teaching Staff are happy with this system.It helps them to reach some of their learning outcomes more efficiently.

Although the used tools are relative simple, the overall process is a complex combination of these simple tools. So, the process needs a manager whos not afraid of this challenge.

In the future, maybe this can be simplified if Blackboard integrates some parts of this procedure in the Bb Learn tools. For example: make Rubrics available for students in Peer Assessment, not only for course leaders.

End of Part 1But what if the paper and the Peer Feedback isnt individual?

This is the end of part one.Course leaders of other faculties heard of it as asked:Is this procedure possible for teamwork?What if the paper and the feedback isnt an individual task?

Peer Reviewbetween groupsHubert Nachtegaele, Educational TechnologistChristine Verclairen, Assistant Academic StaffProf. Dr. ing. Cedric VuyeIng. Tom Verheyden, Teaching AssistantFaculty of Applied Engineering.

Extension to the Bb-Learn Tool Self and Peer Assessment