Getting Started with Learning Analytics

Lori LockyerTim RogersShane Dawson

What about today?

• Introductions and background• From base camp to summit• Data – it seems important• Analytics for teachers• Wrap up• Beer and cookies• Questions, concerns or issues

…is the collection, collation, analysis and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning

Learning Analytics

Ed theory, Ed practice, SNA, Data mining, Machine learning, semantic, data visualisations, sense-making, psychology (social, cognitive, organisational), learning sciences

Learning Analytics

Examine large data sets – trends/ patterns or anomalies.

What do patterns indicate and what do changes in habit indicate?

Higher education: • Lots of isolated work targeting attrition. Very few

large enterprise egs.

• Commercial – IBM, SAS, Hobsons, D2L Insight, BB analytics

Current State

Education Examples

Education Examples

Education Examples

Education - Purdue

Education - UMBC

Education – UniSA

Pass/Fail, RetentionConcept understanding

Current Focus

Kentucky: 1.3% - 80 stds approx 400k

Where next?

Beyond dashboards

Predictive and recommender states

Future

Learner control

NLP – video annotations

Emotions/ face tracking

Future

Confusion Engaged

FrustratedActivity modified

Continuous state of challenge

First steps – the why, what and how

of dataImproving feedback in mass higher education

Data, data, everywhere…

•Where data is accessible it is usually lagged, scattered, indecipherable, requires manipulating, lacks context…

•Yes, there are BI reports, but they are mostly for the converted and don’t flag exceptions

…but not a digit of use•Currently, despite all the data,

•Students often don’t know how they are going

•Academics don’t know if their teaching is effective

•Program/degree owners don’t know how their students navigate their way through

•Management don’t know if the Uni is on track

Outline of data thinking process

•What is the purpose for the data?

•What data is needed (and who ‘owns’ it)

•How to work with the data?

•How to make the data actionable?

Data for what purpose?

•Student level support (success and retention)

•Educator needs – improving teaching and learning

•Program designer/owner needs – curriculum flows

•Management/QA requirements – are courses/subjects meeting standards and improving?

Who owns the data…

•…aka where do you get it? IT, Business Intelligence, Admin?

•And others, e.g. class rolls, library data, orientation attendance, in-class formative and summative assessments etc

Working with data

•All data will need various degrees of extraction and transformation

•All data needs contextualisation, and a decision about how fine-grained that needs to be

•For example, is this a problem?…

42

42 Student

42 Student Test score/100

42 Student Test score/100

10% course mark

42 Student Test score/100

10% course mark

Degree: Chemical Engineering

42 Student Test score/100

10% course mark

Degree: Chemical Engineering

Course: Shakespeare and Society

42 Student Test score/100

10% course mark

Degree: Chemical Engineering

Course: Shakespeare and Society

Class Position: 1/36

Making data actionable

•Visualising the data for summary and exception highlighting

•Trends, key junctures, cumulative risk

•Tools for action, e.g. CRMs, and business processes

Visualising critical metrics

The work of Stephen FewContext is everything

Data and analytics to support learning design and implementation

Learning Analytics:

… measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.

(LAK11 - https://tekri.athabascau.ca/analytics/)

Where and how does learning occur in HE in Australia?

• Within courses/units• which are designed predominately by

teachers (not instructional designers)• who interact with students as they

are learning• who can, may, may not, intervene in

the learning process.

How might a university teacher use data and analytics?

• Analytics to inform design decisions

• Just-in-time analytics to understand learner activity and experience during implementation

• Recommendations for learner action

• Analytics for post-implementation reflection and revision

• Support scholarship of teaching

What can data help us with?

• Moe than…– retention/attrition– “… and they liked it”

• To are they…– doing what you intended?– understanding the task?– on-task/off-task?– motivated, engaged?– actually learning anything?

Learning analytics can only help us answer these questions if they are:

- specific to the learning outcomes of the unit

- related to how we think learning occurs for such outcomes, in the discipline…

- relevant to the learning design we have put in place

In other words…

… the teaching and learning context matters.

Mapping a design

Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

LMS log: Student log in; access case

Network diagram: even pattern of participation

Network diagram: teacher-centred pattern

Network diagram: even pattern of participationDocument sharing logs of contribution

LMS log: access to teacher feedback

LMS log: submission of reflection templateContent analysis: depth of reflection

Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

LMS log: Student log in; access case

Network diagram: even pattern of participation

Network diagram: teacher-centred pattern

Network diagram: even pattern of participationDocument sharing logs of contribution

LMS log: access to teacher feedback

LMS log: submission of reflection templateContent analysis: depth of reflection

Case Based Learning Design adapted from Bennett (2002) available at http://needle.uow.edu.au/ldt/ld/4wpX5Bun.

LMS log: Student log in; access case

Network diagram: even pattern of participation

Network diagram: teacher-centred pattern

Network diagram: even pattern of participationDocument sharing logs of contribution

LMS log: access to teacher feedback

LMS log: submission of reflection templateContent analysis: depth of reflection

Now it is your turn:Sample design or

Your learning design?

• What are the learning outcomes?• What does the design look like? Map it?• What do you want to know?• What data will inform these?• What patterns do you anticipate? • What can you do about it?

Summary

• We are already capturing a lot of data

• There’s a lot of information we are not systematically capturing

• Current or possible answer might answer our questions

• First we have to have relevant questions and know what we are prepared to do with the answers