Authentic and authenticated learning: Moving forward with a dedicated assignment platform

ProfessorAngelaHill ECU

DrCatherineMoore ECU

ProfessorLizJohnsonDeakinUniversity

AssociateProfessorMarcusO’DonnellDeakinUniversity

MsSheona Thomson QUT

MrHerk Kailis Cadmus.

Shared challenge: Proactively reframing assessment design

In 2008, I argued that a new approach to academic integrity in the 21st century was needed because the dominant approaches had been proven to be relatively ineffective (Bertram Gallant, 2008). This new approach, the teaching and learning approach, challenged educators to situate integrity practices within the goal of improving student learning, in essence shifting the focus from how educators could stop students from cheating to how they could ensure students are learning (Bertram Gallant, 2017, p. 88).

Assessment design

TechnologyLearning support

Authentic Authenticated

cadmus

cadmus 1

Improving

Learning

Students feelsupported

Relevant, real-world tasks

Enhanced learning

experience

Achieve graduate + learning outcomes

Empower teachers wth information

Manage teacher

workload

Maintain quality,standards and accountability

Timely + effective feedback

Evidence of learning

cadmus 2

A. Our current technologies and processes

Q. What’s stopping us improving assessment?

1. Make it hard to understand the effectiveness of our assessment tasks.2. Lack flexibility in how we design and conduct our assessment.3. Add complexity and increase staff and student workload.

cadmus 3

A shared, online environment where teachers create and students complete written assessment. With Cadmus we can:

What is Cadmus?

1. Gain insight into student learning, guiding them towards assessment outcomes.2. Easily design and deliver flexible, authentic assessment tasks.3. Simplify assessment, making it easier to manage and clearer for students.

How does it Work?

Teachers design, release and manage

assessment tasks

Features

Templated assessment structure

Attach additional resources

Directly release to students

cadmus 4

How does it Work?

Students access,complete and

submit these tasks

Features

Submit within Cadmus

Autosave to the cloud

Access instructions directly

Dedicated notes area

cadmus 5

How does it Work?

Inbuilt educative features help students

learn and follow an authentic learning

process

Features

Cadmus Manual

Educative pasting scaffolds

Referencing support

cadmus 6

How does it Work?

Learning analytics help teachers measure

student engagement, allowing them to take

actions during an assignment

Features

Weekly progress emails

Class summary report

Student activity report

cadmus 7

How does it Work?

Teachers can update task instructions

in real time, to create scaffolded or

progressive learning experiences

Features

Drafting

Dynamic assessment updates

cadmus 8

How does it Work?

Integrates with university tools and

services, including the L.M.S. and Turnitin

Features

L.M.S. integration

Turnitin integration

Class list

cadmus 9

1. Learn how to improve our current assessment designs.2. Boost student engagement, by integrating classes, workshops and tutorials.3. Provide more feedback to students, by scaffolding the assessment journey.4. Adopt a formative alternative to examinations.

Using Cadmus

cadmus 10

Learn how to improve our current assessment design

Standard Cadmus Assignment

1. Use learning analytics to better understand how the class worked.2. Make adjustments during an assignment, to guide student outcomes.3. Link students to behavioural triggered educative resources.

cadmus 11

Boost student engagement, by integrating classes, workshops and tutorials

Class Integrated Assessment

1. Cost effective way of engaging and providing feedback to students.2. Target tutorials to provide students with the guidance they really need.3. Data informed, to understand what’s important for students.

cadmus 12

Provide more feedback to students, by scaffolding the assessment journey

Multi-stage Assessment

1. Receive drafts and easily provide feedback, all within Cadmus.2. Regularly update instructions and progressively guide students through a task.3. Provide real-time feedback, without students waiting for the next assignment.

cadmus 13

Adopt a formative alternative to examinations

Time Limited Assessment

1. Conduct summative assessment tasks with authentic learning outcomes.2. Leverage functionality to mimic real world scenarios.3. Assessment design which builds in academic integrity measures.

cadmus 14

Cadmus at QUT

APPROACH ACADEMIC INTEGRITY HOLISTICALLY

Our PolicyThe underpinning ‘legislation’.

Emphasises educativeover punitive.

Promotes shared responsibility for integrity.

Programmatic approachAim to build learners’ integrity skills and understanding over time in a whole of course approach:

� scaffold,

� use frameworks for understanding,

� balance low, medium and high stakes tasks.

Professional PracticeAlign what we do at university with both internal and external expectations of conduct and integrity.

TECHNOLOGY – HOW CAN IT HELP?

Typical practiceTurnitinSimilarity/Originality Check

Emerging practiceCadmus

Underused practiceePortfolio as personalised/personalisablerecord of learners’ capabilities.

Supportive practices and tools

In-person, self-help

The appeal of Cadmus for QUT?The value proposition goes well beyond ‘integrity’.It’s a supported writing environment with assurances of authentic authorship.

CadmusTeach

CadmusManual

CadmusLearn

Learners understand more about their writing approach, benefit from behavioural nudges and in-line/just-in-time resources to introduce/integrate norms of academic writing.

Teachers understand more about learner interactions with task, including actual workload.

Love the facilitation of the ‘shared’ learning environment, with ‘in-place’ supports.

Cadmus Online Assessment EnvironmentContinuous small scale trials

Semester 2, 2016

Semester 2, 2017 Summer 2017 Semester 1, 2018

Semester 2, 2018

Sustainability: the materiality of fashion160 studentsAnnotated

bibliography

600 words

(15%)

Communication 1111 studentsEssay

1500-2000 words

with formative

draft

(35%)

Communication 28 studentsEssay (as

S2/2017)

(35%)

Unspeakable Beauty: a history of fashion109 studentsFormative use

in tutorials -

lecturer

distributes

resources and

tutors provide

feedback,

students take

notes and

receive feedback

Sustainability: the materiality of fashion160 studentsAnnotated

bibliography

600 words

(15%)

CASE- COMMUNICATION 1 RESEARCH ESSAY

ContextInternational College

University Certificate in Tertiary Preparation for Postgraduate Studies

Students develop academic skills to successfully engage in postgraduate studies.

109 enrolled students

TaskResearch essay with draft

Worth 35% of unit

Cadmus’ valueEmphasis on academic writing processProgressive and supported student engagement

Assurance of authenticity of authorship within educative frame

1 2 3 4 5 6 7 8 9 10 11 12 13

Feedback

return

Final

submission

In-class writing activity in

Cadmus

Tutors supporting in-class

writing activity

Draft

submitted

Release task

CASE- COMMUNICATION 1 RESEARCH ESSAY

OutcomeSubmission rate of 98.2% (107 of 109 students).

Unit Coordinator felt that Cadmus facilitated student’s understanding of the academic writing process:

This was reflected in the following results:

� The significant increase in number of drafts submitted compared to previous unit instances.

� Positive student sentiment towards note taking and reference sections in Cadmus (89% of students found them valuable).

� Significant reduction in the number of cases of plagiarism, with no major cases of plagiarism detected (compared to typical range of 5-10 cases).

Issues encountered

Student preference of writing environment may conflict

Data equity, data privacy concerns

For individual work only

To note

Not for every writing task.

Useful environment for introduction to norms of academic writing, with behavioural nudges and in-line/just-in-time resources.

Assessment design is important to take advantage of what the environment offers.

Cadmus at Deakin

Why Cadmus @ Deakin

Authentic & Authenticated• Move from

concerns about academic integrity to holistic view of authenticated and authentic

Scaffolding assessment & feedback processes• Better

assessment design

• Sustainable assessment

• Sustainable feedback

Partnership model• Building

relationships with new vendors

• Building relationships with unit teams

• Next generation learning tools

Cadmus @DeakinDr Andy Wear, Dr Chie Adachi, A/Prof Berni Murphy, A/Prof Marcus O’Donnell, Prof Liz Johnson

Cadmus Online Assessment EnvironmentResearch guided engagement

Trimester 1/2, 2017

Trimester 3, 2017/18

Trimester 1, 2018 Trimester 2, 2018

Research & Development

Engaged with Deakin Learning Futures Digital Learning Innovation team to develop educative aspects of Cadmus

Initial pilot

• 2 units• 252

assignments submitted

Research & Development

Evaluated pilot and formed project team to take systematic approach to assessment redesign• Time is required to

effectively leverage new opportunities

• Strong and consistent messaging is required to introduce and support students and staff in new environments

Principles and Practice of PublicHealth

• 150 students• Ministerial Briefing • 2,500 words• (50%)

Context1st year PG Subject

83% international students

151 enrolled students

TaskAT2: Ministerial Briefing (2,500 words)

Worth 50% of unit

** Note: this task was to be completed during ongoing presentations (AT1)

Cadmus’ valueEmphasis on the process engagement

Weekly Analytics on engagement and activity

Learning Analytics to inform Assessment Design

1 2 3 4 5 6 7 8 9 10 11 12 13

AT2 Submit

AT1 Ongoing

AT2 Released

AT1 Released

CASE - PRINCIPLES & PRACTICE OF PUBLIC HEALTH MINISTERIAL BRIEFING

Outcome - Submission rate of 96.7% (146 of 151 students)

Unit team felt that the students were not focused on assessment task 2 with assessment task 1 being simultaneously released, but engaged positively with Cadmus.

This was reflected in the following results:

� One week before the due date: � 60.1% of students had started the task (approx. 400 of 2,500 words) � 64.4% of students had accessed the resources

� After the submission date� 100% of students had started the task (approx. 3,165 of 2,500 words)� 92.7% of students had accessed the resources

� Student engagement with Cadmus:� 41.1% of students using the notes section� 76.8% accessing University resources in the Cadmus Manual

CASE - PRINCIPLES & PRACTICE OF PUBLIC HEALTH MINISTERIAL BRIEFING

CASEPRINCIPLES & PRACTICE OF PUBLIC HEALTH MINISTERIAL BRIEFING

Lesson Learnt

� Cadmus Learning Analytics can provide us insights into student behavior during the semester� Cadmus Learning Analytics can help us understand if the changes we have put in

place improved the final outcome� Students enjoyed their Cadmus experience and the streamlined environment with 73.4% of

students rating their experience as good to excellent� Students need to be carefully scaffolded through new modes of assessment submission – this

requires a team effort: Cadmus/Deakin Learning Futures/Unit team

NextIntroduction to university studies• 1st year undergraduate unit - learning to learn unit• Scaffold use of Cadmus across 2 assessment tasks – build on related feedback• Leverage affordances of carefully integrated assessment resources

Cadmus at ECU

Cadmus@ECU

OurAssessmentPolicyØ Assessmentframesstudentlearning

Ø Assessmentfostersself-regulatedlearningandacademicintegrity

Ø Assessmentpracticesmustbefairandequitabletoallstudents

AssessmentPractices@ECUAccessible Dynamic Authenticated

TheCadmusenvironmentpromotesauthenticandauthenticatedlearningandfacilitatescontinuousimprovementofassessmentswhichisdatainformed.

Teachers areempoweredtomaketheassessmentenvironmentdynamicandinteractive,encouragingself-regulatedlearningandacademicintegrity.

Students benefitfromanenvironmentfreefromdistractionsandtotallydedicatedtoscaffoldingtheachievementofintendedlearningoutcomesthroughengagementinthedevelopmentaltaskactivities.

Cadmus

Teacher

Student

Task

CadmusOnlineAssessmentEnvironmentProgressivelylargerpilots

Semester1,2017 Semester2,2017Initialpilot• 2units• 512assignments submitted

NewAssessmentPolicyEvaluationofCadmusto seeiffitwithrevisedapproachtoassessmentandassessmenttechnologies

Semester 1,2018 Semester2,2018Expanded pilot• 6units• 2,165assignmentssubmitted

Expanded pilot• 7units• 2,542assignmentssubmitted

GODS,GOSSIPANDGENESTUTORIALSUBMISSIONS

Context

Introductiontohistoryandpoliticsdegree

Studentsdevelopcriticalthinkingandanalysisskills

68enrolledstudents

Task

Tutorialsubmissionswithdraftsubmittedbeforetutorial(200words)

Worth30%ofunit

1 2 3 4 5 6 7

Inclassdiscussions

FinalSubmissionDraftSubmission

Outcomes

Onaveragestudentsspent2hoursoneachtutorialtask.

Averagestudentsubmissionrateof92.5%(exceededaimof65%).

100%ofstudentswhosignedupcompletedworkontheirsubmission.

Cadmus’value

Weeklyanalyticsthatreframetheassessmenttask.

Facilitateanadaptiveanddynamiclearningenvironment.

Emphasisoncontinuousengagementtoenhanceandassurelearning.

GODS,GOSSIPANDGENESTUTORIALSUBMISSIONS

LecturerComments

Iamthecoach,nottheplayer,andthatishowIteach.Myintentionwithreadingsandtutorialswastogetstudentstostepupandtakeownershipfortheirlearning.

Cadmusbringsittogether– especiallyinfirstyear,it’ssettingupahabit.Studentssaythatthesearetheonlyclassestheygotowhereeveryonehasdonethereading.

AgreatthingaboutCadmuswasthenotesfacility,andthedatathatmadeiteasiertotrackstudentsandwhattheyweredoing.NowIknowhowstudentsareengaging,andthequalityofreferencinghasimproved.

LessonsLearnt

Valueofsupportteam:ourECUlecturersworkedwithCadmusMelbourneandwithourCadmusPilotsupportteamconsistingofanAcademicDeveloper,aLearningAdvisorandaLibrarian.

ReviewdesignofassessmentprocesstomaximiseaffordancesofCadmus.LearnfromtheCadmusMelbourneteam’sexperiencesatotheruniversities.

What’snext?

ReviewresultsofpilottodeterminewhatdifferenceCadmusismakingandwhereitcouldbemosteffectivelydeployedtomeetourgoals.