Learning Environment Online

Doing less with more

Dr Stephen DannSenior Lecturer, Research School of Management

Australian National University

Optimisation

About me• PhD from 1998• Been in the business since 1997

• Business School Students• Compulsory first year courses• Compulsory second year courses• Angry engineering students doing a “Soft Bludge Subject In Commerce”• Really angry quantitative students doing “This Stupid Subject With Words”• Very upset and confused students doing this really hard and scary subject that uses

words, and “how do I sentence not-equation answer?” fears

• “Small” classes of 60 to 130 students

Background• ANU Vice Chancellor’s Citation for Outstanding Contribution to Student Learning• ANU Achievement award: Candidate for Citation for Outstanding Contribution to Learning• ANU Commendation for Outstanding Contribution to Student Learning• Pearson ANZMAC Emerging Educator of the Year• ANU College of Business and Economics Award for Teaching Excellence• Senior Fellow, Higher Education Academy (UK)

A journey of assessment, technology, and much

improved outcomes (for me)Because when it’s no fun, it’s no fun for anyone

Where I came from…• Assessment was a dreaded chore• Stories told slightly differently, slightly wrong, • Grades were on the ordinary end of extra ordinary• Marking was postponed, avoided or generally delayed

• It wasn’t a lot of fun for anyone.• Student complaints about feedback, guidance and assessment load

• I was burning time out of my research quota to perform badly in teaching

Assessment (including marking) should be fun (for me), and assessment should be enjoyed, not feared.The Epiphany

People do maths for fun.• Video games are hard• Tough Mudder exists• And charges $165

• Sudoko. Crosswords. IKEA

• People don’t fear challenges when they understand the requirements of the task.

Faster marking for fun and profitWhy do in 20 minutes what you can knock over in 10?

How to make assessment fun to mark?• Connecting what you expect

someone to do with what you teach/train them to do…

• Ask questions that you want to discuss.

• Pursue research in the classroom as a “What if?” and workshop the ideas

• Assessment as exchange of ideas• You get ideas• They get feedback

• You both win.

• Teach from your research, research from your teaching

Research + Teaching? Works nicely as well

Came from when I was teaching Introduction to Marketing to first years

Why should we enjoy marking?• We’re going to be doing a lot of it, for a long career

• Death and Taxes are covering “inevitable not fun” shifts

• Students are a lens to see problems in a new light• What if we ask questions for which we want know the answer?

• Where else do we get this audience?

Workload and MarkingTime is a budget, and you’re about to pay down the temporal credit card

First Decision• Do you want to buy time to mark, or time to research?

• If you are marking answers in the broad domain of your research, you can overspend the marking allocation• Buy marking from your research time by using the student essays as a synthesis of the

literature

• If you are marking faster, you can buy five to ten minutes per student back to your research (10 minutes x n+1 students)• Gain research time and have more fun with teaching

Upfront Investment• Assessment Criteria • Know if it can be marked• Something I was routinely bad at doing, and it cost days at the marking end

• Set expectations well in advance• Reward the expected. • Do not begrudge success

• Assessment Support Before The First Assignment• You’ve been there before, and they haven’t

The Pointy End of AssessmentBudgeting your Marking

1 hour per student per semester• That’s 6 minutes per 10%

• Six minutes is not enough time to have any fun with assessment

• Minimum cut-off: 20% and 12 minutes.• In 12 minutes, you can

• Read the paper (5 minutes) • Select marks from the criteria (1 minute) • Add QuickMarks for feedback (3 minutes)• Leave a 3 minute voice feedback of personalised instruction (3 minutes)

• 40% Assignments – 24 minutes• 3 minutes of voice feedback (21 minutes left) • Read the paper (10 minutes) (11 minutes left)• Select marks from the criteria (1 minute) (10 minutes left)• Add QuickMarks for feedback (5 minutes) (Change on the clock)

Mandatory Business Unit ExamsUnit Exam Weighting Exam Structure Total Subcomponents Minutes per component

1A 50% 2 x 4 questions 17 1 min 42 sec 1B 50% 3 from 4 questions 4-5 6 – 7 mins 30 sec 1C 40% 6 compulsory 24 1 minute1D 50%

A 12.5%B 37.5%

A – 25 mcqB – 3 compulsory

25 + 11 MCQ – 7.5 minutes for processing, calculation, delivery and inputB – 22.5 total = 2 minutes

1E 45% A – 20 mcqB – 3C - 2

20 + 9 MCQ – 6 minutes for processing, calculation, delivery and inputB – 1 min 26 secC – 4 mins

1F 50% 5 questions 10 3 minutes

Assessment Maths: 25 students• 30% Exam. 1 x 20 mark, 2 x 5% from a choice of 3• 75 moving parts

• 40: Exam with 6 questions, choice of 5, marked from 100, 20 marks each, 5 sub components• 625 moving parts

• Also.. 25 x 24 minutes = 10 hours (aka about two days)

Check your scoring pain points20 16 15 12 8 7 6 5 1

HD 85 100 17 13.6 12.75 10.2 6.8 5.95 5.1 4.25 0.85

DI 75 84 15 12 11.25 9 6 5.25 4.5 3.75 0.75

CR 65 74 13 10.4 9.75 7.8 5.2 4.55 3.9 3.25 0.65

PA 50 64 10 8 7.5 6 4 3.5 3 2.5 0.5

FeedbackThe single, universal, consistent point of failure in education (according to student satisfaction surveys)

Feedback• the most powerful single moderator that enhances achievement• https://teaching.unsw.edu.au/assessment-feedback

• It’s also really fun (and sometime emotional)• You got to get a little teared up when they do so well

Doing Feedback Well• Constructive.• Tell the students what they did right, and why it was right

• Timely

• Meaningful.

Why is feedback not done well?• Timely Feedback is hard work• 100 assessment tasks x 10 comments of 5 to 10 words in hand writing = a very long day at the office

• Repetition only happens on the marking side, not the student side• “Didn’t I just correct this?”

• Repetition creates Resentment • Resentment is never fun

• Fatigue

Fatal 4 Marking• Fatigue• Tired markers make mistakes

• Frustration • Anger does not help the process either

• Inconsistency • Is this good? Bad? What even was my question or even my answer?

• Mmrmrmrmrmm• Handwriting approaches catastrophic failure

Turnitin: Feedback Machine

For when feedback needs to be timely, industrial strength, and fasthttp://www.acu.edu.au/staff/our_university/training_and_development/academic_staff/leo_assessments

Turnitin and Textmatch

Textmatch is an opportunity• Establish expectations• Explain that a text match is cliché,

and clichéd writing is tacky

• Encourage the deeper learning of using the idea, not the phrase

• Tag and highlight matches with advice on how to write from the ACU Academic Skills Unit• http://students.acu.edu.au/office_of

_student_success/academic_skills_unit_asu

Turnitin buys time backQuickmarks are quick.

QuickMarks are drag and drop

Quickmarks can give depth

Quickmarks can praise

Quickmarks can inform

Quickmarks are more than you’re ever going to consider writing by hand…

Voice feedback (3 minutes)

1 hour per student per semester• That’s 6 minutes per 10%• Six minutes is not enough time to have any fun with assessment

• Minimum cut-off: 20% and 12 minutes.• In 12 minutes, you can

• Read the paper (5 minutes) • Select marks from the criteria (1 minute) • Add QuickMarks for feedback (3 minutes)• Leave a 3 minute voice feedback of personalised instruction (3 minutes)

• Whilst doing quickmarks and Criteria

But what about?

I like to mark on the couch/beach/park bench/backpacking in the Andes/McDonalds

But I like the physical paper…• Turnitin provides• Reduced liability for lost assignments• Backups of comments• Security and confidentiality• Can enable blind marking• “Dog ate my homework” proof to p=0.01 level• Reduced physical strain from handwriting• Improved portability with increased security for work-from-home • Much lighter to carry than 100 20 page essays

What does it do for you?• Improves student outcomes• Feedback has impact

• Individual personalised voice feedback• Creates cut through• Really reaches the student• Adds nuance to the conversation• Reduces assessment follow-up meetings

• Buys time back from the workload

Why do it?Because it buys time back in your work-life

Questions?Stephen.dann@anu.edu.au