‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning
-
Upload
cies -
Category
Economy & Finance
-
view
1.224 -
download
4
description
Transcript of ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning
![Page 1: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/1.jpg)
‘Eyes Wide Shut’: The Challenge of Evaluating E-
Learning
Alan Masson Vilinda Ross
University of Ulster
SERA 2006
![Page 2: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/2.jpg)
Session Overview
CETL (NI) Overview E-Learning Issues Evaluation of E-Learning VLE Data Data Health Risks Data Usage – Realistic Scenarios Making Sense of the Data Wrap
![Page 3: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/3.jpg)
CETL Overview
Aim: “promote, facilitate and reward the adoption of a “learner centred” reflective practice approach to the development of teaching and learning, in particular wrt the use of e-learning technologies”
Posts: 1x Centre Director, 1x Academic Staff Developer, 1x Research Associate, 2 Tech Posts, 4 Content Developers, Learner Advocate
Cultural challenge: effecting changes in “teaching” practices - key to learning experience
CETL (NI): Utilising Institutional E-Learning Services to Enhance the Learning Experience
![Page 4: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/4.jpg)
Promoting reflective practice
• Full “cycle” support– Reflect– Inspire– Develop– Evaluate
• Learner focus emphasised
![Page 5: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/5.jpg)
![Page 6: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/6.jpg)
e-learning “evaluation” issuesExpectations
(still) emergent and high profile – demand for evaluation
Quantitative “data” readily available
Tensions between: reporting and evaluationevaluation and reflection
![Page 7: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/7.jpg)
Evaluation of e-learning
Scope & Activity? institutional structures, policies etc. e-teaching student learning
CETL perspectiveAssist evaluation of practiceExamine macro impact on teaching AND
learning
![Page 8: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/8.jpg)
VLE - a source of useful data? WebCT (CE, Vista)
Logging (user centric)
Tool Usage (Communication, Assessment) (resource centric)
Reports - individual, module, cohort, subject, demographics……
Data Extraction (Log files, Review periods, Extensive data, Filter out
variables, Statistical analysis)
User location (IP address), time… session profile
![Page 9: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/9.jpg)
Extensive VLE Reporting Options
![Page 10: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/10.jpg)
Evaluating VLE reports?
![Page 11: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/11.jpg)
Data Health Risks
Data requires context - Library book “impact” issue
Confidence in Data / Relevance Eyes Wide Shut vs. Open
‘Rule In’ (not trust anything except) - Eyes Open
‘Rule Out’ (trust everything except)
- Eyes Shut
![Page 12: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/12.jpg)
Data Usage – Realistic Scenarios
Targeting subjective work (thresholds and trends) Reinforcing studies (indicative)
CETL priority - cultural change (different agenda) Raise awareness Promote reflection Challenge practices Provide new perspectives Requires careful presentation
![Page 13: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/13.jpg)
Making Sense of the available data resourceData at Face Value vs. Probing DataDifficult to quantify (needs contexts)Making Inferences (meaningful trends,
identifying potential sample groups)Experience, Expectation (reinforcing
theory)
![Page 14: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/14.jpg)
Benefits of approach so far
• Can better focus on learner experience• Complementary approach - allows focus
on practices and interactions• Detailed review of “data” - identifying
number of added value uses
• Language / context important to manage expectations
![Page 15: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/15.jpg)
Complementary tools
• Survey methodologies– Teacher focused– Learner focused
• Learner reflective journals• Focus groups• Exploring conversational approaches to evaluation
– Teacher / learner models of practice
Resource intensive activities - getting best impact?
![Page 16: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/16.jpg)
Wrap
Evaluation of E-Learning
Eyes Wide Open
![Page 17: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/17.jpg)
Development of a transparent method of
recording practice
• Hybrid Learning Model drawing on Verpoorten’s 8LEM model and Bennett’s 30 learning verbs
![Page 18: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/18.jpg)
![Page 19: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/19.jpg)
Example of flash cards used to facilitate modeling of practice
![Page 20: ‘Eyes Wide Shut’: The Challenge of Evaluating E-Learning](https://reader036.fdocuments.in/reader036/viewer/2022062616/549a5d37b47959d31d8b45c3/html5/thumbnails/20.jpg)
Early example: hybrid learning model of T&L practice