Architectures and experiences with design and monitorization

María Jesús Rodríguez-Triana( chus@gsic.uva.es )

GSIC/EMIC research groupUniversidad de Valladolid

May, 2013

Architectures and experiences withdesign and monitorization

What is this presentation about?

EEE Global Objective 3

Evaluation of learning situations in

ubiquitous learning and reflected spaces

(Iván)

Architectures for supporting learning

across web & physical spaces using AR(Luis P. & Juan)

2

EEE GO3:To define methods and

technologies to support the evaluation of ubiquitous learning,

taking into account the students’ actions in different spaces.

• Introduction• Proposal• Use in real settings• Adding new spaces• Work in Progress & Future work

3

Index

• Orchestration framework for TEL scenarios

4

5 + 3 aspects[Prieto et al. 2011]

Introduction (1/3)

• Orchestration framework for TEL scenarios– Awareness for regulation, management and assessment

5EEE Global Objective 3

UVA Specific Objective 3

5 + 3 aspects[Prieto et al. 2011]

EEE GO3

UVA SO3

Introduction (2/3)

6

• Synergies may appear when the analysis is aligned with the design [Carter, 2008] [Locyer & Dawson, 2011] [Looney & Siemens, 2011] [Martínez-Monés et al., 2011]

Scripting

Monitoring

… however, the integration is not straightforward:

design & technological problems may

appear[Martínez-Monés et al., 2011]

Introduction (3/3)

7

Monitoring-awaredesign processof CSCL scripts

Monitoring-awaredesign modelof CSCL scripts

Script-aware monitoring process

of CSCL scenarios

Architecturefor data gathering & integration in DLEs

Provide teachers with design and enactment support capable of linking pedagogical intentions with monitoring needs for orchestrating

blended CSCL scenarios in distributed learning environments

Provide teachers with coarse-grained information about the evolution of the CSCL scenario,

based on the constraints obtained from the learning

design.

Support teachers to identify and include the

monitoring issues of pedagogical significance throughout the design

process of CSCL scenarios.

Automatize the data gathering and integration

tasks in technologically distributed and

heterogeneous CSCL environments.

Proposal (1/7)

8[Rodríguez-Triana et al., 2012a] [Rodríguez-Triana et al., 2013a]

Pattern-based design process for CSCL scripts[Villasclaras-Fernández

et al., 2009]

Pattern-based design process for CSCL scripts[Villasclaras-Fernández

et al., 2009]

Proposal (2/7)Monitoring-aware design of CSCL scripts

[Soller et al., 2005] Main steps for the management of collaboration

(1) Collect Interaction Data

(2) Construct a Model of

Interaction

(3) Compare the Current State of Interaction to

the Desired State

(4) Advise/ Guide the Interaction

(5) Evaluate Interaction

Assessment and Diagnosis

Proposal (3/7)Script-aware monitoring of CSCL scenarios

9

ScriptingMonitoring

[Rodríguez-Triana et al., 2011a][Rodríguez-Triana et al., 2013b]

[Soller et al., 2005] Main steps for the management of collaboration

(1) Collect Interaction Data

(2) Construct a Model of

Interaction

(3) Compare the Current State of Interaction to

the Desired State

(4) Advise/ Guide the Interaction

(5) Evaluate Interaction

Assessment and Diagnosis

Guided by the script definition

(participants, resources,

deadlines, …)

Based on the script constraints

(obtained form the definition of the

activities)

i.e comparing the gathered evidence with

the script definition

Teacher are informed about the comparison

between the script and the gathered

evidenced

Proposal (4/7)Script-aware monitoring of CSCL scenarios

10

11

GLUE! – Architecture for implementing DLEs [Alario-Hoyos, et al. 2013]

Proposal (5/7)Architecture for data gathering and integration in DLEs

12[Rodríguez-Triana et al., 2011b] [Rodríguez-Triana et al., 2011c]

GLUE!-CAS (GLUE! - Collaboration Analysis Support)

GLUE! – Architecture for implementing DLEs [Alario-Hoyos, et al. 2013]

Proposal (6/7)Architecture for data gathering and integration in DLEs

13

Proposal (7/7)Architecture for data gathering and integration in DLEs

14

FPAD(170 students)

MASUP 2013(15 students)

OyT(60 students)

MASUP 23(14 students)

MASUP 32(14 students)

Blended CSCL scenarios supported by VLE’s & Web tools

Use in real settings (1/10)

15

EVALUATION

ENACTMENTDESIGN INSTANTIATION

Monitoringconfiguration

Use in real settings (2/10)

Monitoring-aware design process:Monitoring configuration of the activities

Use in real settings

16

Use in real settings

Monitoring-aware design process:Monitoring enhancement

Use in real settings

18

Use in real settings

20

EVALUATION

ENACTMENTDESIGN INSTANTIATION

Monitoringconfiguration

Deploy

Use in real settings (8/10)

21

EVALUATION

ENACTMENTDESIGN INSTANTIATION

Monitoringconfiguration

Deploy

Use in real settings (8/10)

22

EVALUATION

ENACTMENTDESIGN INSTANTIATION

Logs

Teacherobservations Students

feedback

Monitoringconfiguration

Deploy

Use in real settings (8/10)

Monitoring configuration + Deploy

Use in real settings (9/10)Script-aware monitoring process:Monitoring reports

Use in real settings (10/10)

24

“Identifying the dependences with other activities is not trivial “

“Focusing the analysis around the activity and the pattern constraints is very useful for

teachers, even in situations like these where there were not many eventualities, it allows you

to check whether the original idea is being verified.”

“… information from face-to-face activities was helpful for the evaluation purposes. An overview of the whole learning situation facilitates the review of the learning process.“

25

• The same problems appear again: [Martínez-Monés et al., 2011] – Design problems:

• Which tools provide awareness information? There are not many choices– Technological problems:

• Data gathering issues: – Some tools do not register any kind of data about the user activity – Does not exist a standard format to store information from user’s interaction– Tools do not provide documentation explaining how the information can be obtained

• Data interpretation issues:– Some applications do not provide ready to use data– Many applications do not store data for analysis purposes but for others such as debugging.

» Need of taking into account the monitoring requirements when designing and developing ICT tools [Carter, 2008].

• Data integration issues: – Data synchronization– User identities

• It’s necessary to identify teacher’s awareness needs in each space:– http://www.encuestafacil.com/RespWeb/Qn.aspx?EID=1520807

Adding new spaces

• [WIP] Adapt GLUE!, GLUE!-PS, GLUEPS-AR to better provide awareness information

• [WIP] Integration of AR data sources• [WIP] Analysis of teacher’s awareness needs-------------------------------------------------------------------------------• [FW] Integrate the monitoring-aware design process with

authoring tools• [FW] Provide an integrated view of the different data

sources• [FW] Student’s awareness needs

26

Work in progress & Future work

?Questions, doubts?

THANK YOU!For further information, please contact chus@gsic.uva.es

GSIC-EMIC research groupUniversidad de Valladolid, Spain