Trends and developments in online marking of essays
-
Upload
patris-van-boxel -
Category
Education
-
view
486 -
download
0
description
Transcript of Trends and developments in online marking of essays
![Page 1: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/1.jpg)
Online Marking of Essays Trends and Developments
Patris van BoxelVrije Universiteit Amsterdam
30 November 2012
![Page 2: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/2.jpg)
2
Content
• Role of (written) feedback in learning • Challenges in marking of essays• Trends and developments– Growth in marking and feedback tools – Feedback types (annotations, rubrics, voice)– Learning analytics
• Pilots and evaluation in Dutch SCALA project• Conclusions
![Page 3: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/3.jpg)
3
The role of feedback
• Powerful impact on study behaviour & performance
• Characteristics of good feedback– Timely, personalised, information-rich– ‘Feedforward’ function: bridging the gap
between current and desired (end) result(The Power of Feedback, Hattie &
Timperley, 2007)
![Page 4: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/4.jpg)
4
Giving feedback on essays
Typical teacher challenges:• Logistical workflow– Collection, printing, distribution, storage
• Content– May involve a lot of repetitive commentary– Inter-marker inconsistency
• Contextual demands– Large cohorts– Often need for fast turnaround
![Page 5: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/5.jpg)
5
Receiving feedback on essays
• Is feedback– legible?– understandable?– timely? – actionable?– accessible?
Illegible demand for legibility OUR item expressing suspicion that the UK Royal Mail is deliberately smudging the printing on its "write the postcode clearly" messages reminded Howard Greenwood that he once received a piece of homework marked by his physics teacher bearing an unreadable scrawl in the usual red ink.
Howard took his homework book up to the teacher and asked him what his comment said. The teacher looked at the page and exploded, sending Howard back to his desk with grave threats of what would happen if he were to be so impertinent in future. Howard was completely bewildered until, after much deciphering, he and his friends worked out that the scrawl said, "You must write more legibly".
SOURCE: NEW SCIENTIST, March 2012
![Page 6: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/6.jpg)
6
Tools for essay marking
• Generic software solutions• Dedicated marking tools– Annotation tools (e.g. Markin)– Automated feedback (e.g. ETS)– Electronic assignment management (e.g.
GradeMark/Turnitin)
• Integration with learning management systems(e.g. Blackboard-GradeMark or Moodle-Lightwork)
![Page 7: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/7.jpg)
7
Marking with GradeMark in Bb at VU
• Turnitin software suite
• Advantages of integration with LMS• Lowers barrier for
teachers to use• Less system admin
overhead• Integrates with pre-
and post-marking workflows
• Shared marking
![Page 8: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/8.jpg)
GradeMark in Blackboard
![Page 9: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/9.jpg)
9
Feedback types
• Annotations• Rubrics• Voice grading
![Page 10: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/10.jpg)
10
Annotations
• Contextual• Can be stored • Reused• Categorised• Shared amongst
teachers• Basis for learning
analytics
![Page 11: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/11.jpg)
11
Rubrics
• Matrix with multiple assessment criteria• Described in terms of achievement levels• May also be used for grading• Advantages– Provides insight in current and required
performance– Offers growth perspective– Higher-level feedback over lifespan of course and /
or curriculum
![Page 12: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/12.jpg)
Rubrics• Growing number of tools for (digital) rubric development (e.g.
Waypoint Outcomes) • Basic rubric functions emerging in LMS environments• Self-made solutions e.g. in Google docs
![Page 13: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/13.jpg)
13
Voice grading
• Personalised feel• Localised – inline - feedback• Emphasis, clarification, repetition
Examples• Voice comments in GradeMark• Audio feedback via iAnnotate (demo)
![Page 14: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/14.jpg)
14
Grading analytics
• Marking tools’ ability to provide data – Feedback type and
quantity– Individual / group
level• Diagnostic evidence
of recurring problems• More information:
EBEAM project (JISC, UK)
C. Ellis (2011), Univ of Huddersfield
![Page 15: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/15.jpg)
15
‘Green grading’Imagine an instructor has 15 students each turning in two-page journalentries printed single-sided. Let’s say the instructor will spend about eight minutes grading each journal entry with corrections and comments regardless if it is digital or on paper.
→Traditional grading: It would take 1,800 watt-hours (1.8 kWh) of energy to produce 30 sheets of paper to print the assignments.*
→Paperless grading: It would take 316 watt-hours (0.316 kWh) to run an average computer continuously for 2 hours to grade electronically.**
That’s a savings of 1,484 watt-hours (1.484 kWh) by grading on acomputer as opposed to using paper, the equivalent of saving 1.4 lbs (.635 kg) of coal and reducing emissions by 3.9 lbs (1.769 kg) of carbon dioxide.
![Page 16: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/16.jpg)
SCALA Investigating the potential of digital marking • ‘Scaffolding Assessment for Learning’
(2012-2014)• National funding (SURF) to use (digital) methods for
efficient provision of written feedback • Project aims– Reduction in feedback workload
• Storage and re-use of feedback • Beter assignment management procedures
– Improvement in feedback quality and timeliness
![Page 17: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/17.jpg)
17
SCALA project plan
• 30 pilots across four institutions• Variation of marking tools & feedback types• Variation in disciplines, study years, cohort sizes• Deliverables– Good practices– Teacher guidelines on digital marking– Generic feedback components e.g. rubrics for
assessment of writing skills
![Page 18: Trends and developments in online marking of essays](https://reader033.fdocuments.in/reader033/viewer/2022052905/5583fbcfd8b42aa82c8b4ba0/html5/thumbnails/18.jpg)
18
Conclusions
Marking tools • make anytime, anywhere marking increasing
possible• offer potential for efficiency and quality gains• stimulate more consistent approaches
towards marking and use of assessment criteria
• are able to provide timely, rich, and personal feedback