Learning Analytics Dashboards

148
Learning Analytics dashboards Sten Govaerts EPFL, Zwitserland slides partially based on “Information Dashboard” by Katrien Verbert

description

My presentation at the Learning Analytics day at VOV Lerend Netwerk in Brussels on May 22, 2014.

Transcript of Learning Analytics Dashboards

Page 1: Learning Analytics Dashboards

Learning Analytics dashboards

Sten Govaerts EPFL, Zwitserland

slides partially based on “Information Dashboard” by Katrien Verbert

Page 2: Learning Analytics Dashboards

about me

Page 3: Learning Analytics Dashboards

KULeuven team: Human-Computer Interaction !•technology enhanced learning •music •research

EPFL team: REACT - Real-time Coordination & Distributed Interaction Systems Group !•technology enhanced learning •social media •transportation

http://hci.cs.kuleuven.be/

http://react.epfl.ch/

Page 4: Learning Analytics Dashboards

•2008: context-aware music information retrieval

•2009

•2010

•2011

•2012

•2013

•2014

Page 5: Learning Analytics Dashboards

•2008: context-aware music information retrieval

•2009

•2010

•2011

•2012

•2013

•2014

information visualisation}

Page 6: Learning Analytics Dashboards

•2008: context-aware music information retrieval

•2009

•2010

•2011

•2012

•2013

•2014

social recommendations}information visualisation}

Page 7: Learning Analytics Dashboards

•2008: context-aware music information retrieval

•2009

•2010

•2011

•2012

•2013

•2014

social recommendations}information visualisation}

learning analytics}

Page 8: Learning Analytics Dashboards

•2008: context-aware music information retrieval

•2009

•2010

•2011

•2012

•2013

•2014

social recommendations}information visualisation}

learning analytics} inquiry learning with online labs}

Page 9: Learning Analytics Dashboards

PhD

stu

dent

!@

KU

Leuv

en

•2008: context-aware music information retrieval

•2009

•2010

•2011

•2012

•2013

•2014

social recommendations}information visualisation}

learning analytics} inquiry learning with online labs}

Page 10: Learning Analytics Dashboards

PhD

stu

dent

!@

KU

Leuv

enPo

stdo

c

@ E

PFL

•2008: context-aware music information retrieval

•2009

•2010

•2011

•2012

•2013

•2014

social recommendations}information visualisation}

learning analytics} inquiry learning with online labs}

Page 11: Learning Analytics Dashboards

about you…

Page 12: Learning Analytics Dashboards

why are you interested in dashboards?

Page 13: Learning Analytics Dashboards

what I planned…

Page 14: Learning Analytics Dashboards

what I planned…what are dashboards?

Page 15: Learning Analytics Dashboards

what I planned…what are dashboards?

dashboard examples

Page 16: Learning Analytics Dashboards

what I planned…what are dashboards?

what is wrong with most dashboards?

dashboard examples

Page 17: Learning Analytics Dashboards

what I planned…what are dashboards?

what is wrong with most dashboards?

how can you design better dashboards?

dashboard examples

Page 18: Learning Analytics Dashboards

what I planned…what are dashboards?

what is wrong with most dashboards?

how can you design better dashboards?

dashboard examples

learning analytics dashboard examples

Page 19: Learning Analytics Dashboards

what I planned…what are dashboards?

what is wrong with most dashboards?

how can you design better dashboards?

dashboard examples

how can you quickly get started making your own dashboards?

learning analytics dashboard examples

Page 20: Learning Analytics Dashboards

Typical dashboards…

Page 21: Learning Analytics Dashboards

Typical dashboards…

Page 22: Learning Analytics Dashboards

Typical dashboards…

Page 23: Learning Analytics Dashboards

Typical dashboards…

Page 24: Learning Analytics Dashboards

Typical dashboards…

Page 25: Learning Analytics Dashboards

Typical dashboards…

Page 26: Learning Analytics Dashboards

What is an information dashboard?

A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance.

- Stephen Few

Stephen Few, "Dashboard Confusion," Intelligent Enterprise, March 20, 2004.

Page 27: Learning Analytics Dashboards

Learning Analytics!dashboards (& visualisations ;-)

Page 28: Learning Analytics Dashboards

http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html SNAPP

Page 29: Learning Analytics Dashboards

GISMO http://www.noe-kaleidoscope.org/pub/researcher/resources/video/aied/mazza.html

Page 30: Learning Analytics Dashboards

http://www.cs.uu.nl/docs/vakken/uem/Janssen.pdfThe Participation Tool

Page 31: Learning Analytics Dashboards

Backstage

Pohl A, Bry F, Schwarz J, Gottstein M (2012) Sensing the classroom: improving awareness and self-awareness of students with backstage. In: Proceedings of the international conference on interactive and collaborative learning (ICL), Villach, Austria, 26–28 September 2012.

Page 32: Learning Analytics Dashboards

Dollár A, Steif PS (2012) Web-based statics course with learning dashboard for instructors. In: Uskov V (ed) Proceedings of computers and advanced technology in education (CATE 2012), June 25–27, 2012, Napoli, Italy

OLI Dashboard

Page 33: Learning Analytics Dashboards

LARA Learning Analytics Reflection

& Awareness environment

Page 34: Learning Analytics Dashboards
Page 35: Learning Analytics Dashboards
Page 36: Learning Analytics Dashboards

also on tabletop & large displays

Page 37: Learning Analytics Dashboards

J. L. Santos, K. Verbert, S. Govaerts, and E. Duval. Visualizing PLE Usage. Proceedings of EFEPLE11: 1st Workshop on Exploring the Fitness and Evolvability of Personal Learning Environments, CEUR Laclusaz, France, 30-31 March 2011

Page 38: Learning Analytics Dashboards

Student Activity Meter

Govaerts, Sten; Verbert, Katrien, Duval, Erik; Pardo, Abelardo, The Student Activity Meter for Awareness and Self-reflection. http://ariadne.cs.kuleuven.be/monitorwidget-chi

Page 39: Learning Analytics Dashboards

Problem

http://www.ypfp.org/classroom

Page 40: Learning Analytics Dashboards

Problem

http://www.ypfp.org/classroom

Page 41: Learning Analytics Dashboards

Objectives

• self-monitoring for learners

http://free-extras.com/images/obama_hope_progress-2848.htm

Page 42: Learning Analytics Dashboards

Objectives

• self-monitoring for learners

• awareness for teachers

http://free-extras.com/images/obama_hope_progress-2848.htm

Page 43: Learning Analytics Dashboards

Objectives

• self-monitoring for learners

• awareness for teachers

• time tracking

http://free-extras.com/images/obama_hope_progress-2848.htm

Page 44: Learning Analytics Dashboards

Objectives

• self-monitoring for learners

• awareness for teachers

• time tracking

• learning resource use and recommendations

http://free-extras.com/images/obama_hope_progress-2848.htm

Page 45: Learning Analytics Dashboards

STUDENT ACTIVITY METER

Page 46: Learning Analytics Dashboards

STUDENT ACTIVITY METER

Page 47: Learning Analytics Dashboards

STUDENT ACTIVITY METER

Page 48: Learning Analytics Dashboards

STUDENT ACTIVITY METER

Page 49: Learning Analytics Dashboards

Methodology.• design-based research [Obrenović 2011]: design a tool, give

students & teachers and evaluate

• iterative process of design & evaluation

• paper mockups

• usability studies

• surveys on usefulness

http://funnytogo.com/pictures/evolution/timeline.htm

Page 50: Learning Analytics Dashboards

iter. evaluation goal methodology demo-graphics data

I.usability, satisfaction,

preliminary usefulness

task-based interview with think-aloud, SUS

& MSDT12 CS students at KULeuven

timetracking via Twitter

II.assessing teacher

needs, use & usefulness

online survey with Likert items, yes/no and open questions

19 teachers & TA’s with

CGIAR coursesMoodle logs

III.assessing teacher

needs, expert feedback, use &

usefulness

online survey with Likert items, yes/no and open questions

12 participants from LAK MOOC

Moodle logs

IV. use, usefulness & satisfaction

structured face-to-face interview with tasks &

open questions

11 teachers & TA’s at Carlos III

Uni. Madrid

VM with extensive

tool tracking

Page 51: Learning Analytics Dashboards

Conclusion• SAM is perceived useful and contributes to better

awareness.

Page 52: Learning Analytics Dashboards

Conclusion• SAM is perceived useful and contributes to better

awareness.

• SAM enables:

Page 53: Learning Analytics Dashboards

Conclusion• SAM is perceived useful and contributes to better

awareness.

• SAM enables:

• to detect potential students doing well and at risk

Page 54: Learning Analytics Dashboards

Conclusion• SAM is perceived useful and contributes to better

awareness.

• SAM enables:

• to detect potential students doing well and at risk• a better course overview

Page 55: Learning Analytics Dashboards

Conclusion• SAM is perceived useful and contributes to better

awareness.

• SAM enables:

• to detect potential students doing well and at risk• a better course overview• understanding student time spending

Page 56: Learning Analytics Dashboards

Conclusion• SAM is perceived useful and contributes to better

awareness.

• SAM enables:

• to detect potential students doing well and at risk• a better course overview• understanding student time spending• to find a wide variety of new insights

Page 57: Learning Analytics Dashboards

Conclusion• SAM is perceived useful and contributes to better

awareness.

• SAM enables:

• to detect potential students doing well and at risk• a better course overview• understanding student time spending• to find a wide variety of new insights

• almost all want to continue using SAM

Page 58: Learning Analytics Dashboards

Go-Lab Learning Analytics dashboard

Page 59: Learning Analytics Dashboards
Page 60: Learning Analytics Dashboards
Page 61: Learning Analytics Dashboards
Page 62: Learning Analytics Dashboards
Page 63: Learning Analytics Dashboards
Page 64: Learning Analytics Dashboards
Page 65: Learning Analytics Dashboards
Page 66: Learning Analytics Dashboards
Page 67: Learning Analytics Dashboards

STRUCTURE• explain main idea

• science edu -> to labs -> lab fire -> need something safer

• objectives golab

• partner list

• react’s job

• portal

• architecture

Page 68: Learning Analytics Dashboards

Orientation

Discussion

Investigation

Conceptua- lisationConclusion

Page 69: Learning Analytics Dashboards

An overview of where students are

Currently there are 4 students in the orientation phase, 8 in the conceptualization phase, 3 in the investigation phase and one discussing.

Student currently in phase

4 8 3 0 1

Page 70: Learning Analytics Dashboards

A detailed view of where each student is

Tommy is in the discussion phase, Becky in the investigation phase, etc.

Student current phase

Tommy

Becky

Jessi

Alex

Page 71: Learning Analytics Dashboards

An overview of the time spent in each space for all students

There are currently students in the conclusion and the discussion phases, the average time spent in the discussion phase so far is 20 seconds and the average time spent in the investigation phase is 15:30 minutes

Average time per phase

2:50 3:01 15:30 4:20 0:20

Page 72: Learning Analytics Dashboards

The number of time students change phases

Alex has been changing 12 times between phases so far, Jessi only 5 times.

Number of phase changes

Tommy Becky Jessi Alex4 8 5 12

Page 73: Learning Analytics Dashboards

An overview of who handed in their assignment

Tommy, Jessi and Alex have already handed in their assignments.

Student reports

Tommy

Becky

Jessi

Alex

report.pdf

Jessi.pdf

Pat

pat.pdf

Page 74: Learning Analytics Dashboards

Student reports

Tommy

Becky

Jessi

Alex

report.pdf

Jessi.pdf

Pat

pat.pdf

Number of phase changes

Tommy Becky Jessi Alex4 8 5 12

Connected Students

11Connected Students

4Connected Students

6Student current phase

Tommy

Becky

Jessi

Alex

Page 75: Learning Analytics Dashboards

common issues with dashboards

Page 76: Learning Analytics Dashboards

multiple screens.

Page 77: Learning Analytics Dashboards

multiple screens.

Page 78: Learning Analytics Dashboards

multiple screens.

Page 79: Learning Analytics Dashboards

inadequate context for the data.

Page 80: Learning Analytics Dashboards

inadequate context for the data.

Page 81: Learning Analytics Dashboards

too much detail/precision.

Page 82: Learning Analytics Dashboards

choosing a deficient measure.

Page 83: Learning Analytics Dashboards

choosing a deficient measure.

Page 84: Learning Analytics Dashboards

inappropriate display media.

Page 85: Learning Analytics Dashboards

inappropriate display media.

Page 86: Learning Analytics Dashboards

inappropriate display media.

Page 87: Learning Analytics Dashboards

meaningless variety.

Page 88: Learning Analytics Dashboards

poor visualisations.

Page 89: Learning Analytics Dashboards

poor visualisations.

Page 90: Learning Analytics Dashboards

poor visualisations.

Page 91: Learning Analytics Dashboards

poor visualisations.

Page 92: Learning Analytics Dashboards
Page 93: Learning Analytics Dashboards

bad encoding of data

Page 94: Learning Analytics Dashboards

not highlighting important data.

Page 95: Learning Analytics Dashboards

useless decoration.

Page 96: Learning Analytics Dashboards

too much colour.

Page 97: Learning Analytics Dashboards

dashboards require thoughtful design…

Page 98: Learning Analytics Dashboards

how to design better dashboards?

Page 99: Learning Analytics Dashboards

how to design better dashboards?

1. reduce the non-data pixels

Page 100: Learning Analytics Dashboards

how to design better dashboards?

1. reduce the non-data pixels

2. enhance the data pixels

Page 101: Learning Analytics Dashboards

how to design better dashboards?

1. reduce the non-data pixels

2. enhance the data pixels

3. design for usability

Page 102: Learning Analytics Dashboards

how to design better dashboards?

1. reduce the non-data pixels

2. enhance the data pixels

3. design for usability

Page 103: Learning Analytics Dashboards

reduce non-data pixels

Page 104: Learning Analytics Dashboards

reduce non-data pixels

Page 105: Learning Analytics Dashboards

reduce non-data pixels

Page 106: Learning Analytics Dashboards

reduce non-data pixels

Page 107: Learning Analytics Dashboards

reduce non-data pixels

Page 108: Learning Analytics Dashboards

Enhance data pixels

Page 109: Learning Analytics Dashboards

Enhance data pixels

Page 110: Learning Analytics Dashboards

Design for usability.

Page 111: Learning Analytics Dashboards

Design for usability.1. organize information tosupport its meaning & use

Page 112: Learning Analytics Dashboards

Design for usability.1. organize information tosupport its meaning & use

2. make it aestheticallypleasing!

Page 113: Learning Analytics Dashboards

Design for usability.1. organize information tosupport its meaning & use

2. make it aestheticallypleasing!

3. design for a launch pad

Page 114: Learning Analytics Dashboards

Design for usability.1. organize information tosupport its meaning & use

2. make it aestheticallypleasing!

3. design for a launch pad

Page 115: Learning Analytics Dashboards

Design for usability.1. organize information tosupport its meaning & use

2. make it aestheticallypleasing!

3. design for a launch pad

Page 116: Learning Analytics Dashboards

How to evaluate?us

abili

ty

System Usability Scale (SUS)

Page 117: Learning Analytics Dashboards

How to evaluate?us

abili

ty

System Usability Scale (SUS)

Page 118: Learning Analytics Dashboards

How to evaluate?us

abili

ty

interviews:

task list

Page 119: Learning Analytics Dashboards

How to evaluate?us

abili

ty

interviews:

task list think aloud+

Page 120: Learning Analytics Dashboards

How to evaluate?us

abili

ty

interviews:

task list think aloud+ observe+

Page 121: Learning Analytics Dashboards

How to evaluate?us

abili

ty

interviews:

task list think aloud+ observe+

understand what the user does

understand what the issues are+

Page 122: Learning Analytics Dashboards

How to evaluate?us

eful

ness

& im

pact

Page 123: Learning Analytics Dashboards

How to evaluate?us

eful

ness

& im

pact

much HARDER…

Page 124: Learning Analytics Dashboards

How to evaluate?us

eful

ness

& im

pact

interviews

much HARDER…

Page 125: Learning Analytics Dashboards

How to evaluate?us

eful

ness

& im

pact

interviews

much HARDER…

perceived usefulness surveys

Page 126: Learning Analytics Dashboards

How to evaluate?us

eful

ness

& im

pact

interviews

much HARDER…

perceived usefulness surveys

longitudinal studies

Page 127: Learning Analytics Dashboards

How to quickly build your own dashboard?

Page 128: Learning Analytics Dashboards

How to quickly build your own dashboard?

get data

Page 129: Learning Analytics Dashboards

How to quickly build your own dashboard?

get data

(clean data)

Page 130: Learning Analytics Dashboards

How to quickly build your own dashboard?

get data

(clean data)

visualise!

Page 131: Learning Analytics Dashboards

get data

import.io

Page 132: Learning Analytics Dashboards

get data

import.iokimonolabs.com

Page 133: Learning Analytics Dashboards

Organic Chemistry

Organic chemistry

Organical Chemestry

OrganicChemistry

Organ Chemistry

(clean data)

Page 134: Learning Analytics Dashboards

Organic Chemistry

Organic chemistry

Organical Chemestry

OrganicChemistry

Organ Chemistry

(clean data)

Google Refine/OpenRefinehttp://openrefine.org/

Page 135: Learning Analytics Dashboards

Organic Chemistry

Organic chemistry

Organical Chemestry

OrganicChemistry

Organ Chemistry

(clean data)

Google Refine/OpenRefinehttp://openrefine.org/

DataWrangler

http://vis.stanford.edu/wrangler/

Page 136: Learning Analytics Dashboards

visualise!

ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/

Page 137: Learning Analytics Dashboards

visualise!

ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/

RAW

http://app.raw.densitydesign.org/#%2F

Page 138: Learning Analytics Dashboards

visualise!

ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/

RAW

http://app.raw.densitydesign.org/#%2Fvideo: http://www.ted.com/talks/

hans_rosling_shows_the_best_stats_you_ve_ever_seen

http://www.gapminder.org/

Page 139: Learning Analytics Dashboards

visualise!

ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/

RAW

http://app.raw.densitydesign.org/#%2Fvideo: http://www.ted.com/talks/

hans_rosling_shows_the_best_stats_you_ve_ever_seen

http://www.gapminder.org/

Page 140: Learning Analytics Dashboards

visualise!

http://www.simile-widgets.org/exhibit/

Page 141: Learning Analytics Dashboards

visualise!

http://www.simile-widgets.org/exhibit/

http://www.simile-widgets.org/timeline/

Page 142: Learning Analytics Dashboards

visualise!

http://www.simile-widgets.org/exhibit/

http://www.simile-widgets.org/timeline/ Google Charts

https://developers.google.com/chart/

Page 143: Learning Analytics Dashboards

there are also all-in-one solutions…

Page 144: Learning Analytics Dashboards

How many kids are repeating a year in elementary school?

Page 145: Learning Analytics Dashboards

• http://www.wolframalpha.com/input/?i=how+many+kids+are+repeating+primary+school+in+Belgium

• http://www.wolframalpha.com/input/?i=university+with+the+largest+enrollment&lk=3

• http://www.wolframalpha.com/input/?i=university+with+the+largest+enrollment+in+Belgium

• http://www.wolframalpha.com/input/?i=Belgium+vs+USA+education+expenditures+per+high+school+student

Page 146: Learning Analytics Dashboards

further reading…

http://pdf.th7.cn/down/files/1312/information_dashboard_design.pdf

Page 147: Learning Analytics Dashboards

Edward Tufte.

Page 148: Learning Analytics Dashboards

Questions?

slides on: http://www.slideshare.net/stengovaerts