Peer Feedback methodsbased on Bb Learn Tools

Extensions to the Bb-Learn Tool “Self and Peer Assessment”

Peer Feedback Methodsbased on Blackboard Learn Tools

Peer Review with Rubrics

Peer Review between Groups

Peer Assessment within Groups

Peer Feedback via My Grades

Presentation by:Hubert Nachtegaele, Educational TechnologistChristine Verclairen, Assistant Academic Staff

Extensions to the Bb-Learn Tool “Self and Peer Assessment”

Peer Reviewwith Rubrics

Cartoon by Nick D Kim, strange-matter.net (please see site for terms of reuse)

Extensions to the Bb-Learn Tool “Self and Peer Assessment”

Hubert Nachtegaele, Educational TechnologistAnnick De Vylder, Assistant Academic StaffProf. Dr. Jelle Boeve-de Pauw and prof. Dr. Piet Van den Bossche, Faculty of Social Sciences

Why?

± 100 students “Master of Instructional and Educational Sciences”

Assignment: write paper, apply/show Scientific Reporting Skills

Assessed using a Rubric focused on Scientific Reporting Skills

± 30 good ; ± 70 need more exercise, need a lot of feedback

too much work?impossible?

method?

Peer Assessment, using the same Rubric, with Peer Feedback, ...

Rubrics is not only a method to evaluate these students, but also a skill to be mastered by them.

Problem: In Bb Course Tool Self and Peer Assessment : No Rubrics !

?

Copy/Paste the Rubric “Criteria” and “Levels of Achievement” to spreadsheet:

Simulate Rubric & Feedback by a Bb Test

Some basic Hocus-Pocus in the spreadsheet...

Simulate Rubric & Feedback by a Bb Test

Copy & Paste this to a notepad document...

Simulate Rubric & Feedback by a Bb Test

and upload this notepad document as questions in a Bb-Learn Test:

Criteria (29 in this case)

Levels of Achievement(MC - questions)

Feedback-field(SA - questions)

Simulate Rubric & Feedback by a Bb Test

Students Choose Theme > Write > Submit Paper

Groupset:Choose one of three themes as subject for your paper...

Instructions specificfor chosen theme

Adaptive Release,specific by theme

Only one assignment,common for different themes.

Adaptive Release,common for all themes

Random Distribution of Submitted Papers

Assignment File Download

Theme Group Members export

EXC

EL

Random Number for each submissionSort by Theme and Random NumberReplace RN with a “Submission Code”...

RENAME

COPY TO

Excelparsing filenamesrandomisation (see one slide up)constructing anonymous filenamesconstructing batch-commands for renaming and copying

Content CollectionCC map of students...

Blackboard DriveLink Content Collection map to Windows Drive letter

Command WindowExecute Rename and Copy batch commands

Method to automate the delivery of a set of dedicated documents to the respective, individual students..Other methods possible, but a native Bb-Learn method is missing.

Tools for this Distribution of Papers

Student in role ofPeer Reviewer

Peer Assessment 2 papers > 2 “tests” 29 criteria: level & feedback

Finally: Self assessment: 3th test

Redistributing Peer Feedback

Paste in the spreadsheet

VLookup the author of the reviewed paper

Copy to CC map of author using same techniques as for distributing the papers, but in the other direction!

here we need a VBA script

Overall Process, Student View

Choose theme, Write paper, Submit paper

Receive 1st paper to review > peer assess it using rubric & feedback

Receive 2d paper to review > peer assess it using rubric & feedback

Self assess own paper using rubric & feedback

Receive 1st and 2d peer assessment & feedback on own paper

Revision and Final Submission of own paper

Conclusion

Bb Learn tools for communicationExcel & VBA & BbDrive for processing

Worth to integrate (partly) in Bb !

complex combination of relative simple tools

End of Part 1But what if the paper and the Peer Feedback isn’t individual?

Peer Reviewbetween groups

Hubert Nachtegaele, Educational TechnologistChristine Verclairen, Assistant Academic StaffProf. Dr. ing. Cedric VuyeIng. Tom Verheyden, Teaching AssistantFaculty of Applied Engineering.

Extension to the Bb-Learn Tool “Self and Peer Assessment”

Why?8 group reports (about site visits)

Extend to(>) 15 groups

15 groups(4 students) X

Extend to(>) 8 site visits

How to make students reflect as a group about their writings?

each group

for each site visit

writes:1 report

reviews:2 reports

How?Prepare Groups and Group Assignments

for submission of reports

for submission of review of reports

Prepare Course Content Collection maps for each groupto deliver the reports to review (posted by other groups)

to deliver the reviews of their own reports (reviewed by other groups)

Prepare a structured Excel Tableto randomize the reports for reviewing by other groups

to construct batch commands for submitting posted reports to reviewers

to construct batch commands for submitting reviews of the reports back to the group who was author of the report

Following pictures show more details...

Prepare groups and groupassignments...

1. Design a process for onesite visit, independent ofthe number of groups

2. Repeat that processfor each site visit

site visit 1deadline report: xx/xx/xxxx

site visit 2deadline report: xx/xx/xxxxdeadline reviews visit 1:

xx/xx/xxxx

site visit 3deadline report: xx/xx/xxxxdeadline reviews visit 2:

xx/xx/xxxx

Report of the second activity

Reviews of foregoing reports(reports that were written by two other groups)

Distribution of reports to review...

Not possible to create subfoldersNot possible to automate copying to ...

Alternative:Create Course Content Collection map for each group, set permissions based on group membership,batch copying is possible using BbDrive

Download group submissions

gradebook_report1_......

.zip

Unzip & list report filenames

…and paste them in the prepared Excel table List report filenames…

Copy report to random reviewers group

randomize order of groups

create batch commands to copy reports to random “next” group

Connect to Course Content Collection maps for groups using BbDrive.

BbDrive

Execute the batch commands to distribute the reports to the group that has to review that report.

Return delivery of the reviews

Reviewers download:ToReview1_score=.docx

Reviewers do their review work, using the criteria in the scoring form incorporated in the rport template.

Reviewers incorporate the overall score in the filename of the review:

ToReview1_score=19.docx

Reviewers post their review (assignment “Review X”)

Course Leader initiates the automated return delivery guided by the Excel.

Conclusion

Intention, educational, for these groups:

After repeating this report/review process for 8 site visits, the groups write clear and correct reports.

Intention, educational, for the institution:

A new Learning Tool for enhancing output of teamwork.

What can simplify this method?

Accessibility of the “file exchange” group tool through BbDrive no need to create CC maps for the groups⇒

End of Part 2And the assessment within the group?

Hubert Nachtegaele, Educational TechnologistRaoul Deuss, Domain Coördinator Ecampus

Peer Assessmentwithin Groups

Group Members evaluate their Teamwork Skills

Construction of the PA-test

From the Groups Membership in Blackboard and the list of Teamwork Skills ...

Construction of the PA-test

The Course Leader starts a VBA-script to create a Bb-test...

The result is a Bb-test in which students assess theseTeamwork Skills of all the Group Members, self-assessment included.

The PA-test

Meaning of the Scores

By means of the scores in the PA-test you evaluate yourself and the other members of your group:

You assign scores from 1 to 5. The meaning of these figures is:

5 = For this criterium, the contribution of group member ... was better than the average of the group4 = For this criterium, the contribution of group member ... was roughly the average of the group3 = For this criterium, the contribution of group member ... was less than the average of the group2 = For this criterium, group member ... contributed nothing at all1 = For this criterium, group member ... was a nuisance to the group

Assign a score for all criteria for all members of your group (yourself included).

PA-rapports

After the download of the test results, another VBA-script calculates scores and creates individual reports.These reports are sent automatically to the personal CC map of each student.Only the course leader sees all the details.

End of Part 3

Peer Feedbackvia My Grades

the most appropriate way?

Hubert Nachtegaele, Educational Technologist

Upload Comments to the Grade CenterIn the Bb documentation we found that it is possible to download and upload not only the grades, but also the “Comments”.

We decided to use this to return Peer Feedback to the students.

Feedback experiment

We constructed a test consisting of 3 questions about the oral presentation of each of the 24 students 3 x 24 ⇒questions

After the students gave feedback about each other via this test, we downloaded the results and ...

Using this “test”, each student can give feedback to any other student.

Feedback from Excel to Grade CenterAfter the students gave feedback on each others presentation via this test, we downloaded the results, reordered and combined these pieces of feedback in function of the assessed student

In CSV-form, this is uploaded to the Grade Center column...

Peer Feedback: Student View

The result of this procedure is that the

students see their peer feedback the same way as

they see the feedback from instructors:

through “My Grades”.

End of Part 4

Overall ConclusionThese peer (review, feedback, assessment) methodsgive satisfactory educational results

Managing these processes can be simplified by integrating some elements in future releases of Bb Learn (Ultra)...● make the use of rubrics also available, in some way, to

students for peer assessment● simple tool for submitting dedicated documents to

respective students, eg via My Grades● limit focus of Bb Learn tool “Self & Peer Assessment” to

“group(s) of students” instead of “all students”

End