Post on 19-Aug-2014
description
Learning Analytics dashboards
Sten Govaerts EPFL, Zwitserland
slides partially based on “Information Dashboard” by Katrien Verbert
about me
KULeuven team: Human-Computer Interaction !•technology enhanced learning •music •research
EPFL team: REACT - Real-time Coordination & Distributed Interaction Systems Group !•technology enhanced learning •social media •transportation
http://hci.cs.kuleuven.be/
http://react.epfl.ch/
•2008: context-aware music information retrieval
•2009
•2010
•2011
•2012
•2013
•2014
•2008: context-aware music information retrieval
•2009
•2010
•2011
•2012
•2013
•2014
information visualisation}
•2008: context-aware music information retrieval
•2009
•2010
•2011
•2012
•2013
•2014
social recommendations}information visualisation}
•2008: context-aware music information retrieval
•2009
•2010
•2011
•2012
•2013
•2014
social recommendations}information visualisation}
learning analytics}
•2008: context-aware music information retrieval
•2009
•2010
•2011
•2012
•2013
•2014
social recommendations}information visualisation}
learning analytics} inquiry learning with online labs}
PhD
stu
dent
!@
KU
Leuv
en
•2008: context-aware music information retrieval
•2009
•2010
•2011
•2012
•2013
•2014
social recommendations}information visualisation}
learning analytics} inquiry learning with online labs}
PhD
stu
dent
!@
KU
Leuv
enPo
stdo
c
@ E
PFL
•2008: context-aware music information retrieval
•2009
•2010
•2011
•2012
•2013
•2014
social recommendations}information visualisation}
learning analytics} inquiry learning with online labs}
about you…
why are you interested in dashboards?
what I planned…
what I planned…what are dashboards?
what I planned…what are dashboards?
dashboard examples
what I planned…what are dashboards?
what is wrong with most dashboards?
dashboard examples
what I planned…what are dashboards?
what is wrong with most dashboards?
how can you design better dashboards?
dashboard examples
what I planned…what are dashboards?
what is wrong with most dashboards?
how can you design better dashboards?
dashboard examples
learning analytics dashboard examples
what I planned…what are dashboards?
what is wrong with most dashboards?
how can you design better dashboards?
dashboard examples
how can you quickly get started making your own dashboards?
learning analytics dashboard examples
Typical dashboards…
Typical dashboards…
Typical dashboards…
Typical dashboards…
Typical dashboards…
Typical dashboards…
What is an information dashboard?
A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance.
- Stephen Few
Stephen Few, "Dashboard Confusion," Intelligent Enterprise, March 20, 2004.
Learning Analytics!dashboards (& visualisations ;-)
http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html SNAPP
GISMO http://www.noe-kaleidoscope.org/pub/researcher/resources/video/aied/mazza.html
http://www.cs.uu.nl/docs/vakken/uem/Janssen.pdfThe Participation Tool
Backstage
Pohl A, Bry F, Schwarz J, Gottstein M (2012) Sensing the classroom: improving awareness and self-awareness of students with backstage. In: Proceedings of the international conference on interactive and collaborative learning (ICL), Villach, Austria, 26–28 September 2012.
Dollár A, Steif PS (2012) Web-based statics course with learning dashboard for instructors. In: Uskov V (ed) Proceedings of computers and advanced technology in education (CATE 2012), June 25–27, 2012, Napoli, Italy
OLI Dashboard
LARA Learning Analytics Reflection
& Awareness environment
also on tabletop & large displays
J. L. Santos, K. Verbert, S. Govaerts, and E. Duval. Visualizing PLE Usage. Proceedings of EFEPLE11: 1st Workshop on Exploring the Fitness and Evolvability of Personal Learning Environments, CEUR Laclusaz, France, 30-31 March 2011
Student Activity Meter
Govaerts, Sten; Verbert, Katrien, Duval, Erik; Pardo, Abelardo, The Student Activity Meter for Awareness and Self-reflection. http://ariadne.cs.kuleuven.be/monitorwidget-chi
Objectives
• self-monitoring for learners
http://free-extras.com/images/obama_hope_progress-2848.htm
Objectives
• self-monitoring for learners
• awareness for teachers
http://free-extras.com/images/obama_hope_progress-2848.htm
Objectives
• self-monitoring for learners
• awareness for teachers
• time tracking
http://free-extras.com/images/obama_hope_progress-2848.htm
Objectives
• self-monitoring for learners
• awareness for teachers
• time tracking
• learning resource use and recommendations
http://free-extras.com/images/obama_hope_progress-2848.htm
STUDENT ACTIVITY METER
STUDENT ACTIVITY METER
STUDENT ACTIVITY METER
STUDENT ACTIVITY METER
Methodology.• design-based research [Obrenović 2011]: design a tool, give
students & teachers and evaluate
• iterative process of design & evaluation
• paper mockups
• usability studies
• surveys on usefulness
http://funnytogo.com/pictures/evolution/timeline.htm
iter. evaluation goal methodology demo-graphics data
I.usability, satisfaction,
preliminary usefulness
task-based interview with think-aloud, SUS
& MSDT12 CS students at KULeuven
timetracking via Twitter
II.assessing teacher
needs, use & usefulness
online survey with Likert items, yes/no and open questions
19 teachers & TA’s with
CGIAR coursesMoodle logs
III.assessing teacher
needs, expert feedback, use &
usefulness
online survey with Likert items, yes/no and open questions
12 participants from LAK MOOC
Moodle logs
IV. use, usefulness & satisfaction
structured face-to-face interview with tasks &
open questions
11 teachers & TA’s at Carlos III
Uni. Madrid
VM with extensive
tool tracking
Conclusion• SAM is perceived useful and contributes to better
awareness.
Conclusion• SAM is perceived useful and contributes to better
awareness.
• SAM enables:
Conclusion• SAM is perceived useful and contributes to better
awareness.
• SAM enables:
• to detect potential students doing well and at risk
Conclusion• SAM is perceived useful and contributes to better
awareness.
• SAM enables:
• to detect potential students doing well and at risk• a better course overview
Conclusion• SAM is perceived useful and contributes to better
awareness.
• SAM enables:
• to detect potential students doing well and at risk• a better course overview• understanding student time spending
Conclusion• SAM is perceived useful and contributes to better
awareness.
• SAM enables:
• to detect potential students doing well and at risk• a better course overview• understanding student time spending• to find a wide variety of new insights
Conclusion• SAM is perceived useful and contributes to better
awareness.
• SAM enables:
• to detect potential students doing well and at risk• a better course overview• understanding student time spending• to find a wide variety of new insights
• almost all want to continue using SAM
Go-Lab Learning Analytics dashboard
STRUCTURE• explain main idea
• science edu -> to labs -> lab fire -> need something safer
• objectives golab
• partner list
• react’s job
• portal
• architecture
Orientation
Discussion
Investigation
Conceptua- lisationConclusion
An overview of where students are
Currently there are 4 students in the orientation phase, 8 in the conceptualization phase, 3 in the investigation phase and one discussing.
Student currently in phase
4 8 3 0 1
A detailed view of where each student is
Tommy is in the discussion phase, Becky in the investigation phase, etc.
Student current phase
Tommy
Becky
Jessi
Alex
An overview of the time spent in each space for all students
There are currently students in the conclusion and the discussion phases, the average time spent in the discussion phase so far is 20 seconds and the average time spent in the investigation phase is 15:30 minutes
Average time per phase
2:50 3:01 15:30 4:20 0:20
The number of time students change phases
Alex has been changing 12 times between phases so far, Jessi only 5 times.
Number of phase changes
Tommy Becky Jessi Alex4 8 5 12
An overview of who handed in their assignment
Tommy, Jessi and Alex have already handed in their assignments.
Student reports
Tommy
Becky
Jessi
Alex
report.pdf
Jessi.pdf
Pat
pat.pdf
Student reports
Tommy
Becky
Jessi
Alex
report.pdf
Jessi.pdf
Pat
pat.pdf
Number of phase changes
Tommy Becky Jessi Alex4 8 5 12
Connected Students
11Connected Students
4Connected Students
6Student current phase
Tommy
Becky
Jessi
Alex
common issues with dashboards
multiple screens.
multiple screens.
multiple screens.
inadequate context for the data.
inadequate context for the data.
too much detail/precision.
choosing a deficient measure.
choosing a deficient measure.
inappropriate display media.
inappropriate display media.
inappropriate display media.
meaningless variety.
poor visualisations.
poor visualisations.
poor visualisations.
poor visualisations.
bad encoding of data
not highlighting important data.
useless decoration.
too much colour.
dashboards require thoughtful design…
how to design better dashboards?
how to design better dashboards?
1. reduce the non-data pixels
how to design better dashboards?
1. reduce the non-data pixels
2. enhance the data pixels
how to design better dashboards?
1. reduce the non-data pixels
2. enhance the data pixels
3. design for usability
how to design better dashboards?
1. reduce the non-data pixels
2. enhance the data pixels
3. design for usability
reduce non-data pixels
reduce non-data pixels
reduce non-data pixels
reduce non-data pixels
reduce non-data pixels
Enhance data pixels
Enhance data pixels
Design for usability.
Design for usability.1. organize information tosupport its meaning & use
Design for usability.1. organize information tosupport its meaning & use
2. make it aestheticallypleasing!
Design for usability.1. organize information tosupport its meaning & use
2. make it aestheticallypleasing!
3. design for a launch pad
Design for usability.1. organize information tosupport its meaning & use
2. make it aestheticallypleasing!
3. design for a launch pad
Design for usability.1. organize information tosupport its meaning & use
2. make it aestheticallypleasing!
3. design for a launch pad
How to evaluate?us
abili
ty
System Usability Scale (SUS)
How to evaluate?us
abili
ty
System Usability Scale (SUS)
How to evaluate?us
abili
ty
interviews:
task list
How to evaluate?us
abili
ty
interviews:
task list think aloud+
How to evaluate?us
abili
ty
interviews:
task list think aloud+ observe+
How to evaluate?us
abili
ty
interviews:
task list think aloud+ observe+
understand what the user does
understand what the issues are+
How to evaluate?us
eful
ness
& im
pact
How to evaluate?us
eful
ness
& im
pact
much HARDER…
How to evaluate?us
eful
ness
& im
pact
interviews
much HARDER…
How to evaluate?us
eful
ness
& im
pact
interviews
much HARDER…
perceived usefulness surveys
How to evaluate?us
eful
ness
& im
pact
interviews
much HARDER…
perceived usefulness surveys
longitudinal studies
How to quickly build your own dashboard?
How to quickly build your own dashboard?
get data
How to quickly build your own dashboard?
get data
(clean data)
How to quickly build your own dashboard?
get data
(clean data)
visualise!
get data
import.io
get data
import.iokimonolabs.com
Organic Chemistry
Organic chemistry
Organical Chemestry
OrganicChemistry
Organ Chemistry
(clean data)
Organic Chemistry
Organic chemistry
Organical Chemestry
OrganicChemistry
Organ Chemistry
(clean data)
Google Refine/OpenRefinehttp://openrefine.org/
Organic Chemistry
Organic chemistry
Organical Chemestry
OrganicChemistry
Organ Chemistry
(clean data)
Google Refine/OpenRefinehttp://openrefine.org/
DataWrangler
http://vis.stanford.edu/wrangler/
visualise!
ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/
visualise!
ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/
RAW
http://app.raw.densitydesign.org/#%2F
visualise!
ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/
RAW
http://app.raw.densitydesign.org/#%2Fvideo: http://www.ted.com/talks/
hans_rosling_shows_the_best_stats_you_ve_ever_seen
http://www.gapminder.org/
visualise!
ManyEyeshttp://www.manyeyes.com/software/analytics/manyeyes/
RAW
http://app.raw.densitydesign.org/#%2Fvideo: http://www.ted.com/talks/
hans_rosling_shows_the_best_stats_you_ve_ever_seen
http://www.gapminder.org/
visualise!
http://www.simile-widgets.org/exhibit/
visualise!
http://www.simile-widgets.org/exhibit/
http://www.simile-widgets.org/timeline/
visualise!
http://www.simile-widgets.org/exhibit/
http://www.simile-widgets.org/timeline/ Google Charts
https://developers.google.com/chart/
there are also all-in-one solutions…
How many kids are repeating a year in elementary school?
• http://www.wolframalpha.com/input/?i=how+many+kids+are+repeating+primary+school+in+Belgium
• http://www.wolframalpha.com/input/?i=university+with+the+largest+enrollment&lk=3
• http://www.wolframalpha.com/input/?i=university+with+the+largest+enrollment+in+Belgium
• http://www.wolframalpha.com/input/?i=Belgium+vs+USA+education+expenditures+per+high+school+student
further reading…
http://pdf.th7.cn/down/files/1312/information_dashboard_design.pdf
Edward Tufte.