Building Analytics Capability @open.edu
-
Upload
simon-buckingham-shum -
Category
Education
-
view
1.550 -
download
4
description
Transcript of Building Analytics Capability @open.edu
Simon Buckingham Shum
Professor of Learning Informatics Knowledge Media Institute, The Open University, UK http://simon.buckinghamshum.net @sbskmi
JISC CETIS 2013 Conference: Analytics and Institutional Capabilities
Building Analytics Capability @open.edu
bit.ly/OULAprof
Same outcomes, but higher scores?
Learning Analy=cs as
Evolu&onary Technology. Same training + educa=onal paradigms
• more engaging • beBer assessed • beBer outcomes
• deliverable at scale 3
Learning dynamics we couldn’t assess before?
Learning Analy=cs as
Revolu&onary Technology. A vehicle for paradigm shiF?
• interpersonal learning networks • quality of discourse + wri=ng • lifelong learning disposi=ons • problem solving strategies
• lifewide learning
open.edu BI perspective
5
OU data warehouse (in progress)
Data Warehouse
IT corral key institutional data in the
central warehouse 1 IT provide data dictionary 2
IT provide data marts and cubes for commonly used data sets
3 Business data users propose
action 5
Explore the challenge/issue/problem/opportunity/question using SAS/preferred tool
4 “Data Wranglers”
assist staff in understanding BI OU Analytics
Board
open.edu VLE
perspective 7
VLE Analy;cs @ the OU
Virtual Learning
Environment
Usage sta;s;cs at system, faculty and module level – general paCerns
‘Par;cipa;on Tracking’ func;on to track individual students’ interac;on with specific
online learning ac;vi;es In pilot 2012/13
Data Warehouse
VLE Analy;cs @ the OU
Virtual Learning
Environment
Usage sta;s;cs at system, faculty and module level – general paCerns
‘Par;cipa;on Tracking’ func;on to track individual students’ interac;on with specific
online learning ac;vi;es In pilot 2012/13
Data Warehouse
VLE Analy;cs @ the OU
Virtual Learning
Environment
Usage sta;s;cs at system, faculty and module level – general paCerns
‘Par;cipa;on Tracking’ func;on to track individual students’ interac;on with specific
online learning ac;vi;es In pilot 2012/13
Data Warehouse
open.edu predictive modelling
11
Predictive analytics
Registra=on PaBern
CRM contact
VLE interac=on
Assignment grades
Demo-‐graphics
? How early can we predict likelihood of dropout, formal withdrawal, failure? Now exploring conventional statistics, machine learning and growing datasets New fees regime may well change student behaviour…
Library interac=on
OpenLearn interac=on
Futurelearn interac=on
Social App X interac=on
OU track record
OU Analytics: Predictive modelling
§ Probability models help us to identify patterns of success that vary between: § student groups / areas of
curriculum / study methods § Benefits
§ provide a more robust comparison of module pass rates
§ support the institution in identifying aspects of good performance that can be shared, and aspects where improvement could be realised
13 OU Student Statistics & Surveys Team, Institute of Educational Technology
Best predictors of future success:
previous OU study data – quantity
and results
Improving student retention with predictive analytics
A.L. Wolff and Z. Zdrahal (2012). Improving Retention by Identifying and Supporting “At-risk” Students. EDUCAUSE Review Online, July-August 2012. http://www.educause.edu/ero/article/improving-retention-identifying-and-supporting-risk-students
4 predictive models: final result (pass/fail) final numerical score drop in the next TMA score of the next TMA
Demo- graphics
Previous results
VLE activity
open.edu Library
perspective 15
Learning Analytics – the Library dimension
http://www.flickr.com/photos/davepattern/6928727645/sizes/o/in/photostream/
Library Impact Data Project – Huddersfield University
‘Students who looked at this article also looked at this article’
‘Students on your course are looking at these articles’
Student achievement
Library use
Recommender services
open.edu Research
perspective 17
Visualizing and filtering social ;es in SocialLearn by topic and type
Schreurs, B., Teplovs, C., Ferguson, R., De Laat, M. and Buckingham Shum, S., Visualizing Social Learning Ties by Type and Topic: Ra;onale and Concept Demonstrator. In: Proc. 3rd Interna6onal Conference on Learning Analy6cs & Knowledge (Leuven, BE, 8-‐12 April, 2013). ACM hCps://dl.dropbox.com/u/15264330/papers/Schreurs-‐etal-‐LAK2013.pdf
-60
-40
-20
0
20
40
60
80
9:2
8 9
:32
9:3
6
9:4
0
9:4
1
9:4
6
9:5
0
9:5
3
9:5
6
10:
00
10:
05
10:
07
10:
07
10:
09
10:
13
10:
17
10:
23
10:
27
10:
31
10:
35
10:
40
10:
45
10:
52
10:
55
11:
04
11:
08
11:
11
11:
17
11:
20
11:
24
11:
26
11:
28
11:
31
11:
32
11:
35
11:
36
11:
38
11:
39
11:
41
11:
44
11:
46
11:
48
11:
52
11:
54
12:
00
12:
03
12:
04
12:
05
Average Exploratory
Discourse analytics on webinar textchat
Sheffield, UK not as sunny as yesterday - still warm Greetings from Hong Kong Morning from Wiltshire, sunny here!
See you! bye for now! bye, and thank you Bye all for now
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar, but if we zoom in on a peak…
Discourse analytics on webinar textchat
-100
-50
0
50
100
9:2
8 9
:40
9
:50
1
0:00
1
0:07
1
0:17
1
0:31
1
0:45
1
1:04
1
1:17
1
1:26
1
1:32
1
1:38
1
1:44
1
1:52
1
2:03
Averag
Classified as
“exploratory talk”
(more
substantive for learning)
“non-exploratory
”
Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations?
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
Discourse analytics on webinar textchat Visualizing by individual user. The gradient of the threshold line is
adjusted to every 5 posts in 6 classified as “Exploratory Talk”
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
Analytics for “21st Century Competencies & Learning Dispositions”
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823
Different social network patterns in different contexts
may load onto Learning
Relationships
Questioning and challenging may load onto Critical Curiosity
Sharing relevant resources from other
contexts may load onto Meaning Making
Repeated attempts to pass an online test
may load onto Resilience
open.edu coming soon…
23
On the horizon… MOOCs + Analytics…
Ethics
What Data? Biz Models
‘vs’ Open
Partnerships/Collab
Research
http://people.kmi.open.ac.uk/sbs/2013/01/emerging-mooc-data-analytics-ecosystem
Educ Research at SCALE
On the horizon… Educational Data Scientists