TLC2016 - Data for Students - A student-centred approach to analytics in Learn

Data for StudentsA student-centred approach to analytics in Learn

2

Ross Ward

Introduction

Information ServicesLearning, Teaching & Web Services

@rosswoss

@UoE_LTW

3

LearnIntroduction

5500 Courses per year (and growing)

Over 70% Course Coverage

Primary platform for on-campus courses

4

Learning AnalyticsIntroduction

Rapidly Developing & Evolving

Powerful tools & reports

Support & Advice

Generally not student facing!

5

Big Data | Little Data

Big Data Little Data

• Tutor Facing

• Interventions

• Course Evaluation

• Student Retention

• Student Facing

• Self Monitoring

• Personal Motivation

• Student Attainment

Focus of industry Largely ignored

6

Aim of our project• Awareness of learning analytics developments at national level

– JISC | Code of Practice for learning analytics– Prof Dragan Gasevic

• Student Retention not an aim or outcome

• Student focussed & Student facing

• Increasing Awareness– Existing tools and reports– Support and guidance– Keeping in-touch with academic ideas and practices

• Develop a student-facing building block– What data is suitable?– What do students want?– Would students use it?

7

Tutor Facing Analytics

What are the expectations of our tutors?

User Stories

• Current practice• Awareness of tools• Risks and fears

8


What are tutors allowed to do?

Learning Analytics

Steering Group

• University Policies• Legal Requirements• Ethical Considerations• JISC Code of Practice

9


How can we support our users?

Self-enrolSupport Course

• 92% of approaches were available• “hand-cranked” data analysis

common• Examples of usage• Highlight areas for consideration

10

That’s all good and well, but…

What about the students?

11

Exposing data to students


Some initial research

Worked closely with Student Association

• Not really that bothered• Mixed opinions about data privacy• Grades! Grades! Grades!

12

Requirements for the Building Block


• Simple as possible• Easy to interpret• Anonymous• Manageable by Instructors• Opt-in by course

Clicks &

Grades

13

Click Data

• Heatmap of student activity

• Personal activity only

• “Quantified Self”

• Data presented from data centre – not live

• Course opt-in

14

Grade Data

• Tutor controls which grade columns are included

• Simple to understand

• Cannot be used on small cohort sizes

15

Pilot Courses


UG Physics Large cohort Driven & competitive students Tutor engaged with students about

toolsOnline Distance Prof Development

Very large cohort No grade data Ad-hoc usage of click data

(minimal)PG Foreign Language

Little data to this point

3Pilot

Courses(first semester)

16

Feedback from course


UG Physics Large cohort Driven & competitive students Tutor engaged with students about tools

Click Count

Focus Group&

Survey

89% Easy to understand at first glance

69% Found the information useful

61% Would check information weekly

12% Would never check click count data

17




Click Count

Focus Group&

Survey

allows me to plan my week

see when I am most productive

little relevance to my performancecan track my activity in the course

Visualise how often I am actually on the website

I've used this to build a 'timetable' of when i should do work pretty pointless information

I can't think of a way in which I could utilise the information.

The information is set out in an easy to read way and tabulated well.

The amount of clicks per day does not correlate with how much work you are doing

It shows you how much (or little) you are doing and gives an incentive to maybe visit the learn page more often.

I have not yet found a use for it. I just find it interesting.

18




Grade Averages

Focus Group&

Survey

97% Easy to understand at first glance

100% Found the information useful

78% Would check information weekly

4% Would never check click count data

19




Grade Averages

Focus Group&

Survey

Allows me to see how well I'm doing compared to the rest of the class

Shows you if other people are struggling with the course like you or not

allows to compare own results to the average to give show what results I should be getting

It gives me a good overview of how I'm doing and which topics I struggled with

I find it very useful (and sometimes reassuring) to compare my grades with my peers

It's a motivating tool if I do well or not compared to others.

Column chart is better then a table especially for comparison

Provides me with an estimation of how well I am coping with the course work.

It gives me assurance that i am at the same level as the class average in the assessments and that i am doing ok in the course work.

20




Have you used this information to inform decisions on your study and work patterns?

Focus Group&

Survey

Yes

No

60.3%

39.7%

21




The Tutors Perspective

Focus Group&

Survey

22

Summary

Analytics is not a“one size fits all”

solution

23

Summary

Data must be easy to interpret

Fit in with existing support and guidance offered

to students

24

Summary

Used at student’s own discretion

25

Summary

Provide support andadvice to all

26

Summary

Just one tool of many available

27

What next…

• Courses still piloting the tool

• Not currently developing building block further– Will be making it available on Oscelot over the summer

• Keeping in-touch with other Student Facing developments– JISC App of interest

• Continue to engage with Student association

• Continually shifting attitudes towards analytics

28

Thank you

Thanks.

Questions?

http://www.ed.ac.uk/information-services/learning-technology

TLC2016 - Data for Students - A student-centred approach to analytics in Learn

Education

Transcript of TLC2016 - Data for Students - A student-centred approach to analytics in Learn