@glynnmark

Rethinking our approach plagiarism

Dr Mark Glynn

Dr Shadi Karazi

@glynnmark #heie

Outline

1. Background2. Criteria3. Text Matching

Evaluation4. Next steps

@glynnmark #heie

Text matching evaluation

What are we trying to achieve?

@glynnmark #heie

Overview of use of TurnItIn

Year Originality reports

Grade Mark

2014 26678 31072015 31003 42162016 31735 58972017 23019 3614

@glynnmark #heie

Criteria

Cost Functionality Matching ability

Turnaround time Support

@glynnmark #heie

Support required

Both systems produce a similarity score

23019 reports generated through submissions made to TurnItIn. In that time we had 52 tickets to our helpdesk related to TurnItIn

In the same period we ran 55000+ submissions through Urkund and we had 4 tickets.

@glynnmark #heie

Turn around time

In consistent and unreliable

Plugin related

@glynnmark #heie

Matching Ability

90% of the documents analysed had a difference in similarity scores of less than 10% (with 80% of the documents having less than 5% difference in similarity score).

The differences in the score can be due to three main factors.

Similarity Score

75 – 100% 50 – 74% 25 – 49% 0 – 24%

% of Reports

1.65% 7.67% 27.5% 67.98%

@glynnmark #heie

FunctionalityFeature TurnItIn Moodle +

Urkund

Annotation*Can the lecturer annotate a student’s submission

x x

Marking GuidesStructured feedback on specified criteria with a marks range for each student

x

RubricsIn Built Rubrics

x x

Group work x

Similarity reports x x

Peer Review x x

Rubric / Marking guides  reports**Can the lecturer get a report providing a breakdown of all of the marks and comments for each criteria within a rubric/marking guide given to the entire class

x

Audio Feedback x

ExtensionsCan the lecturer grant deadline extensions to one or more students easily

x

@glynnmark #heie

Cost

Year Cost2014 £15,0002015 £23,3022016 £25,247

Urkund - £9158

@glynnmark #heie

Next steps

We terminate TurnItIn in June 2017.

We have developed an audio plugin to facilitate audio feedback similar to TurnItIn.

We get more people buying Urkund to build the database and help us negotiate a better price.