OU Analyse: Analysing at-risk students at The Open University

Analysing at-risk students at The Open University

Date: 18th March, 2015

Author: Jakub Kuzilek, Martin Hlosta, Drahomira Herrmannova, Zdenek Zdrahal, Annika Wolff

The Open University

• Largest distance learning university in the UK (200k students, over 500 courses)

• Main campus in Milton Keynes

• Many efforts of the OU aims at improving retention rate of students

• Several university units exist only for providing help to at-risk students

Problem specification

• Given:– Demographic data at the Start (may include information about student’s previous

modules studied at the OU and his/her objectives)

– Assessments (TMAs) as they are available during the module

– Virtual Learning Environment activities between TMAs

– Conditions student must satisfy to pass the module

• Goal: – Identify students at risk of failing the module as early as possible so that OU intervention

is efficient and meaningful.

Genesis of OU Analyse

time

Darkness


time

Darkness OU Analyse prehistory

2011 20132012Project with

MSR Cambridge

Analysis ofVLE data 3 courses

Experimentaldashboard


time


2011 2013

OU Analyse success stories

2014Feb

2 courses

2012Project with

MSR Cambridge

Weekly support for OU courses

4 predictivemodels




time


2011 2013


2014Feb

2 courses

2012Project with

MSR Cambridge


2014Oct

12 courses

4 predictivemodels

DashboardAnalysis ofVLE data 3 courses



time


2011 2013


2014Feb

2 courses

2012Project with

MSR Cambridge


2015Feb

2014Oct

12 courses 18 courses

4 predictivemodels

Dashboard Dashboard& recommender




time


2011 2013


2014Feb

2 courses

OU Analyse future

2012Project with

MSR Cambridge


2015Feb

2014Oct

12 courses 18 courses

4 predictivemodels

Dashboard Dashboard& recommender



Our approach to support at-risk students

• Predictive modeling

Predictive modeling

We are here

History we know Future we can estimate

What is the best time for at-risk student identification?

• Students, who fail first Tutor Marked Assignment

(TMA) in fourth week has high probability of course

failure (>95%)

We need to start predicting before first TMA

Data

• Demographic data

– Static data during the course

– Gender, Age, Highest education, New/Continuing student, Index of multiple deprivation, Number of previous course attempts, Student workload during the course

• Virtual Learning Envinronment (VLE) data

– Data from student interaction with VLE

– One day summary data

Importance of VLE data

• Demographic data

– New student

– Male

– No formal qualification

Sex

Education

N/C

TMA1

Without VLE:Probability of failing at TMA1 = 18.5%

Sex

Education

N/C

VLE

TMA1Clicks Probability Nr of students

0 64% 4

1-20 44% 3

21-100 26% 5

101-800 6.3% 14

With VLE:

Identifying module fingerprints

• Identification of the

most informative VLE

“resources”

• Course specific

• Using historical data

Important VLE activities

• Identified VLE activities: Forum (F), Subpage (S),

Resource (R), OU_content (O), No activity (N)

• Possible activities each week are: F, FS, N, O, OF, OFS,

OR, ORF, ORFS, ORS, OS, R, RF, RFS, RS, S

FSF RFSOFS ORFN O SRFROF OR ORSORFS OS RS

Start






Pass Fail No submit TMA-1time

VLE opens

Start

Activity space



Start






VLE opens

Start

VLE trail: successful

student



Start







VLE opens

Start

VLE trail: student who

did not submit

Module fingerprinttime

TMA1

VLE

start

Predictive models

Recommender

Prediction of at-risk students

TUTOR

List of at-risk students

Module Overview

Student overview

Results

• Four predictive modules

• Important activities identification -> recommendations

• Support of 18 modules

• Weekly predictions

• Dashboard (from scratch to working application in 1 year)

Future work

• Scaling up

• 2nd round of evaluations

• Addressing new challenges: modules without historical

data, model voting, new models, module finger prints,

alignment of assessments

• Evaluation of interventions

Thank you and see you at tech showcase!

(SC 3102-3105, Thu 19th March, 4:30-5:30 PM)

OU Analyse: Analysing at-risk students at The Open University

Education

Transcript of OU Analyse: Analysing at-risk students at The Open University