HETL-SOTE Case study #HETL15 #HETLUtah

Case StudyCan activity analytics support understanding engagement a measurable process?

James Ballard – HETL SOTE 2015, Utah Valley University

jameslballard

JamesBallard2

@jameslballard

http://www.slideshare.net/JamesBallard2


https://twitter.com/jameslballard

https://twitter.com/jameslballard

https://www.linkedin.com/in/jameslballard

https://www.linkedin.com/in/jameslballard

Background

DescriptionThe data for this case study comes from the Virtual Learning Environment (Moodle) and e-Portfolio (Mahara) of a Further Education college in the UK. The college curriculum is vocational and employer led supporting around 18,000 students annually with an average age of 29. As a community college the majority of learners live in the same borough while others travel from adjoining boroughs.

Comparatively this is one of the most ethnically diverse and youngest populations in London with census data reporting 47% of the population as being of Black or Minority Ethnicity (BME) and 4 in 5 people under 50;

English is a second language for many students with over 170 mother tongue languages spoken in the area;

Population turnover is high with 20% moving in and out of the borough annually;

It is in the top 10% of deprived areas of the UK.

Descriptive Statistics

MEASURE ALL STUDENTS

Total Students 7,647

Total Activity (logs) 1,466,907

Mean Activity (logs per learner) 192

Standard Deviation 348

Median 65

Mode 2

Min 0

Max 7,263

Data is taken from the 2013/14 Academic Year

Histogram

Summarised frequencies of logs per user

884

491

1014

1020

1168

1439

800

579

230

21

1

0

0 200 400 600 800 1000 1200 1400 1600

5

10

25

50

100

250

500

1000

2500

5000

7500

More

Activity 1 – Instrument FormationWhat might we measure?

Log data – Page 1/5Date Course Tool Action

03/01/2014 10:01 Functional Skills - Construction course view

03/01/2014 10:02 Functional Skills - Construction forum view forum

03/01/2014 10:02 Functional Skills - Construction course view

03/01/2014 10:02 Functional Skills - Construction forum view forum

03/01/2014 10:03 CIH L3 Cert in Housing Practice

Programme

course view


Programme

course view section


Programme

forum view forum


Programme

forum view discussion


Programme

forum add post


Programme



Programme

forum view forum



Programme



Programme

forum view forum


Programme

course view


Programme

course view section


Programme

course view section


Programme

resource view


Programme

folder view


Programme

course view section


Programme

resource view


Programme

user view all


Programme

user view



Programme

user view


Programme

assign view


Programme

course view section


Programme

folder view


Programme

course view


Programme

user view all


Programme

forum view forums


Programme

forum view forum


Programme



Programme

assign view


Programme

assign view submit assignment form

Log Data - Page 4/5Date Course Tool Action


Programme

assign submit


Programme

assign view


Programme

assign view confirm submit assignment form


Programme

assign submit for grading


Programme

assign view


Programme

forum view forum


Programme



Programme

forum add post


Programme



Programme

course view


Programme

course view section

Log Data - Page 5/5

Date Course Tool Action


Programme

course view


Programme

course view section


Programme

course view


Programme

forum view forum


Programme

course view

Worksheet 1/2TASK TYPE TALLY TOTAL

Assimilative: attending and understanding content

Information Handling: gathering and classifying

resource or manipulating data

Adaptive: use of modelling or simulation software

Communicative: dialogic activities (e.g. pair

dialogues or group based discussion)

Productive: construction of an artefact such as a

written essay or a sculpture

Experiential: practicing skills in a particular context

or undertaking an investigation

PIE CHART COLUMN CHART

Worksheet 2/2

0

Activity 2 – Practical ApplicationCreating an Engagement Profile

CIH L3 Cert in Housing Practice Programme

Total Activity Logs: 1,458

Student 1 – Jo March

Activity Over Time

17

236

156

118

222

265

143

101 97 103

Sep-13 Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Summarised count of the number of logs per month

Site Areas

Summary of the areas of the site that were accessed

15

210

139

107

173

218

119

85 84 89

2

31

1

41

12

2

23

16

11

44

43

22

16 13 14

SEP-13 OCT-13 NOV-13 DEC-13 JAN-14 FEB-14 MAR-14 APR-14 MAY-14 JUN-14

Portal Homepage

Learning Centre

Induction

Functional Skills - Construction

Cross College English

Construction Study Skills

CIH L3 Cert in Housing Practice Programme

Module Key

Affordances of the tools as a key for understanding the log summary table next

Module Description Module Description

Assignment Assessed assignment submission Label Inline course text

Book Chaptered web content Gallery Image gallery

Chat Synchronous discussion NLN Content Packaged content module

Course Course page Page Teacher created page

Folder Folder of teacher uploaded documents Resource Teacher uploaded file

Forum Asynchronous threaded discussion Web link URL to external site

IMS CP Packaged content module User User profile

Log Summary

Summarised table showing how the logs were distributed across specific LMS modules.

Ass

ign

men

t

Bo

ok

Ch

at

Co

urs

e

Fold

er

Foru

m

IMS

CP

Lab

el

Gal

lery

NLN

Co

nte

nt

Page

Res

ou

rce

Web

lin

k

Use

r

Tota

l

add discussion 2 2

add post 68 68

submission statement accepted 1 1

submit 6 6

submit for grading 3 3

subscribe 7 7

subscribeall 1 1

talk 6 6

update post 3 3

view 71 1 91 83 1 59 3 3 312

view all 15 14 14 14 14 14 14 14 14 14 12 153

view confirm submit assignment form 3 3

view discussion 292 292

view forum 252 252

view forums 18 18

view section 112 112

view submit assignment form 15 15

Total 114 14 7 203 97 643 14 14 14 14 15 73 17 15 1254

Task Type Analysis – Page 1/3

Summary of the different task types the learner is involved in over time.

0

5

10

15

20

25

30

35

Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Productive

Communicative

Assimilative


Summary of the percentage of different task types the learner is involved in

Communicative

Assimilative

Productive

Information Handling

Adaptive

Experiential


Summary of the distribution of different task types the learner is involved in

0

50

100

150

200

250

300

350

400Communicative

Assimilative

Productive


Adaptive

Experiential

Course Information – Page 1/2

Summary of the different task types per learner in the CIH L3 Cert in Housing Practice Programme.

0

200

400

600

800

1000

1200

1400

1600

29316 32603 31349 25331 34323 36027 34673 36231 JoMarch

13799 35727 35827 33758 33595 35469

Productive

Communicative

Assimilative

SOCIAL NETWORK ANALYSIS KEY

Each node represents a user in the course (blue = learner and red = teacher). A connected arrow indicates a forum post in reply to that user. The weight of the arrow indicates the number of replies.

Teachers:

29316

25331


Summary of the network analysis for the forum.

INVOLVEMENT INTERACTION INTIMACY INFLUENCEActivities – list indicators of positive or negative engagement

Level of intensity – mark on the scale from low to high

Low High Low High Low High Low High

WorksheetStudent Profile 1: Jo March

IT Level 2 Diploma student

Total Activity Logs: 4,958

Student 2 – Oliver Twist

Activity Over Time

Summarised count of the number of logs per month

32 104 12 0 0 89

4713

8 0 0Sep-13 Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Site Areas

Summary of the areas of the site that were accessed

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000


Portal homepage

Mystery Shopping

Level 2 IT

Learning Centre

Get Involved

CG Award for IT Users (ITQ) E-QUALS

Module Key

Affordances of the tools as a key for understanding the log summary table next

Module Description Module Description

Assignment Assessed assignment submission Label Inline course text

Book Chaptered web content Gallery Image gallery

Chat Synchronous discussion NLN Content Packaged content module

Course Course page Page Teacher created page

Folder Folder of teacher uploaded documents Resource Teacher uploaded file

Forum Asynchronous threaded discussion Web link URL to external site

IMS CP Packaged content module User User profile

Log Summary

Summarised table showing how the logs were distributed across specific LMS modules.

Ass

ign

men

t

Co

urs

e

Fold

er

Res

ou

rce

Tota

l

view 1 76 17 3328 3422

view section 1476 1476

view submit

assignment form

1 1

Total 2 1552 17 3328 4899


Summary of the different task types the learner is involved in over time.

0

500

1000

1500

2000

2500

3000

3500


Assimilative


Summary of the percentage of different task types the learner is involved in

Communicative

Assimilative

Productive


Adaptive

Experiential


Summary of the distribution of different task types the learner is involved in

0

500

1000

1500

2000

2500

3000

3500Communicative

Assimilative

Productive


Adaptive

Experiential


Summary of the different task types per learner in the IT Level 2 Diploma student.

0

500

1000

1500

2000

2500

3000

3500

4000

Oliv

er T

wis

t

34

56

8

38

55

25

36

4

11

28

1

20

39

5

33

85

0

35

75

9

13

69

4

37

36

1

35

38

8

72

53

39

54

2

13

20

3

34

28

0

34

21

0

36

06

33

33

5

54

45

39

50

1

13

80

6

32

38

0

60

02

24

32

6

34

81

0

42

74

2

33

68

8

35

82

4

40

23

0

33

96

5

16

89

8

36

36

1

39

09

34

02

3

47

74

14

15

3

41

12

8

26

31

1

Productive

Communicative

Assimilative

SOCIAL NETWORK ANALYSIS KEY

There are no discussions in this course so the Social Network Analysis is not included.


Summary of the network analysis for the forum.

INVOLVEMENT INTERACTION INTIMACY INFLUENCEActivities – list indicators of positive or negative engagement

Level of intensity – mark on the scale from low to high

Low High Low High Low High Low High

WorksheetStudent Profile 2: Oliver Twist

Learner Enhanced Technology

Understanding engagement as a measurable process

Thank You

James Ballard

[email protected]

JamesBallard2

@jameslballard

jameslballard


http://au.linkedin.com/in/jameslballard

HETL-SOTE Case study #HETL15 #HETLUtah

Education

Transcript of HETL-SOTE Case study #HETL15 #HETLUtah