Pushing the awareness envelope: from interactions

to emotionsIsrael GutiérrezDerick Leony

University Carlos III of Madrid

Awareness across spaces with learning analytics

Data collection

Process /Analysis

Inter-ventions

• Subject: Teachers and students

• Tool: Learning analytics with traces across spaces

• Objective:

• Recommend: classON and affective recommender

Guess what?

Company 1

Company 2

0 2,5 5 7,5 10

Number of months to reach 1 million users

Guess what?

Company 1

Company 2

0 2,5 5 7,5 10

Number of months to reach 1 million users

Guess what?

Company 1

Company 2

0 2,5 5 7,5 10

Number of months to reach 1 million users

Online learning and... flipping the classroom

• Lectures at home

• Homework in class

• Let’s support this new types of classes!!!

Flipped session = computer lab session• Observations in lab sessions

• Inefficient interactions

• Problems orchestrating the session

• Definition of metrics (ICALT’12)

• Waiting time

• Order

Proposed solution

Data collection

Process /Analysis

Inter-ventions

Events from web sensors

Attach info to augmented physical space

Recommend interactions

• Awareness tool supporting teachers

• Context

• Lab sessions (f2f) = flipped session

• Regulate interactions

• Empower teacher for improving orchestration

http://www.class-on.org

Student interface

BART SIMPSONLISA SIMPSON

Teacher interface

25

32

BUSY

Teacher interface

BART SIMPSON

LISA SIMPSON

BUSY

Orchestration aspects that could be improved• Based on the 5+3 aspects framework

(ECTEL’12)

• Management

• Intervention

• Assessment

• Design

classON Evaluation

• Quantitative data: not enough improvement

• Teachers perception

• info in a glance

• fair distribution of feedback

• Students perception

• more fair (time/order)

• trust the help is coming

classON Further Steps

• Suggest peer support interactions

• New strategies for feedback dispatching

• Annotation system

• Exploitation of qualitative data in questions & answers

• New experiment(authoring tool): Superpowers to teachers (workshop at Madrid Science Week)

• Visualization + gamification (explored in my stay at KUL, but in the context of Research 2.0)

Ambient displays• Not the main focus of

user attention

• Used for

• Awareness

• Persuade

• Contexts

• Health / good habits

• Design factors

• Appeal (usability + aesthetics)

• Learnability: match user’s expectations

• Awareness: match user’s interests

• Distraction: low

Gamification

• Applying game design and game mechanics to non-game contexts

• Engage users in a new system

• Engage users keep using the system

• Metaphors for ambient displays using game concepts

GambientFication

• KU Leuven group

• TiNYARM

• Use iPads as ambient displays

• Use during working hours

• Use a holder as a photo frame

• 10 last days of the stay

Ambient displays + Gamification (Powered by Analytics)

Evaluation

• Awareness of research activity in the short/medium/long term

• Appeal (usability + aesthetics)

• Distraction

• User perception

• Awareness/reflection/sense-making/behavior change

And now, let’s jump to emotions...

13/09/2012 EEE Meeting 2012 1

Context

● Life-cycle of Learning Analytics process

DataCollection

Process /Analysis

Inter-ventions

Events fromVirtual

Machine

URLsBash commands

FilesCompilations

13/09/2012 EEE Meeting 2012 2

Data collection

● Collect events from:

– Web browser

– Text editor

– Programming tools

– Session

● More information:

– Partial grades

– Forums

13/09/2012 EEE Meeting 2012 3

Problem statement

● Detecting emotions in educational settings

– Affective computing (Picard et al., 1997)

– Use of physical sensors

– Applied mainly in ITS

● Recommender systems in TEL

– SoA Review (Draschler et al., 2011)

– There are no implementations that

consider the affective state of the learner

13/09/2012 EEE Meeting 2012 4

Objectives

● Detect learner emotions from the events generated within a learning environment

● Recommend learning resources based on the detected emotions (and other variables)

● Reflecting affective states into the virtual environment

● Providing awareness of affective states

13/09/2012 EEE Meeting 2012 5

Proposed solution

● Complete Learning Analytics cycle

DataCollection

Process /Analysis

Inter-ventions

Events fromVirtual

Machine

Detectingaffective

state

RecommendLearning

Resources

13/09/2012 EEE Meeting 2012 6

Classification of emotions

● Two prevalent ways:

– Two-dimensional features (valence, arousal)

– Discrete sets (e.g. Ekman's basic emotions: anger, disgust, fear, joy, sadness, surprise)

● Learning-domain set proposed by D'Mello et al. (2007): Boredom, confusion, frustration, eureka, motivation, neutral

13/09/2012 EEE Meeting 2012 7

Processing events

● Input: Events collected during a working session

● Methods: Hidden Markov Models, GSP

13/09/2012 EEE Meeting 2012 8

Intervention: Recommend resources

● Resource recommendation

● User-based (vs item) collaborative filtering

● Modified process:

– Define preference model

– Calculate user similarity including emotions

– Select neighbors of learner

– Obtain list of recommended items

– Filter recommendations according to current emotion

● Implementation based on Apache Mahout

13/09/2012 EEE Meeting 2012 9

Evaluation

● Affective state detection

– Learner feedback

– Instructor feedback

– External sensors:

● Galvanic skin response

● Facial gestures

● Resource recommendation

– Information system metrics:

● Precision and recall

13/09/2012 EEE Meeting 2012 10

First Prototype

● Widget: Learning Resource Affective Recommender

● Among the winners of the 3rd ROLE Widget Enchantment

● Paper accepted in workshop MATEL 2012

● Demo available!

13/09/2012 EEE Meeting 2012 11

Future work

● Implementing two sensors for emotion: Galvanic skin response (Arduino-based), facial gestures (OpenCV-based)

● Analyzing the inclusion of other variables:

– Grades, session duration, amount of events

● Experiments planned:

– Detection: Madrid's Science Week 2012

– Recommendation: C programming class

● Deployment on Amazon EC2

● Integration with LearnGLASS

13/09/2012 EEE Meeting 2012 12

Thank you!

Questions?