Conole eden _mooc_evaln

22
Evaluating MOOCs – what is really happening? Gráinne Conole, University of Leicester 11 th June 2014 EDEN conference, Zagreb National Teaching Fellow 2012 Ascilite fellow 2012 EDEN fellow 2013

Transcript of Conole eden _mooc_evaln

Page 1: Conole eden _mooc_evaln

Evaluating MOOCs – what is really happening?

Gráinne Conole, University of Leicester

11th June 2014EDEN conference, Zagreb

National Teaching

Fellow 2012 Ascilite fellow 2012EDEN fellow 2013

Page 2: Conole eden _mooc_evaln

Outline

• Emergence of MOOCs• FutureLearn• UoL MOOC evaluation• Methodology• A new MOOC classification• Survey findings• Interview themes• Recommendations• Conclusion

Page 3: Conole eden _mooc_evaln

The emergence of MOOCs• CCK08

– Connectivist MOOC (cMOOC)– Siemens, Downes and Cormier– Evaluation (Fini, 2009)

– http://www.irrodl.org/index.php/irrodl/article/view/643/1402• What are MOOCs?

– http://www.youtube.com/watch?v=eW3gMGqcZQc• Emergence of large-scale xMOOCs• Recent developments

– UK-based FutureLearn– Launch of Massey on Open2Study

• List of MOOCs– http://www.mooc-list.com/

• EFQUEL series of blogs– http://mooc.efquel.org/

• ICDE list of MOOC reports– http://tinyurl.com/gconole-MOOC

Page 4: Conole eden _mooc_evaln

Massive Open Online Courses (MOOCs)

FreeDistributed global community

Social inclusion

High dropout ratesLearning income not learning outcome

Marketing exercisehttp://alternative-educate.blogspot.co.uk/2012/12/audio-ascilite-2012-great-debate-moocs.html

JOLT, Vol. 9, No. 2, http://jolt.merlot.org

Page 5: Conole eden _mooc_evaln

cMOOCs or xMOOCs?

• Weekly centred• Participant reflective

spaces• Social and networked

participation• Hashtag: #etmooc• Use of a range of social

media

• Linear learning pathway• Mainly text and video• Formative feedback

through MCQs• Individually focused

Page 6: Conole eden _mooc_evaln

Dimension Characteristics

ContextOpen Degree to which the MOOC is open

Massive How large the MOOC is

Diversity The diversity of the learners

Learning

Use of multimedia Extent of use of rich multimedia

Degree of communication Amount of communication incorporated

Degree of collaboration Amount of collaboration incorporated

Amount of reflection Ways in which reflection is encouraged

Learning pathway Degree to which the learning pathway is supported

Quality assurance Degree of quality assurance

Certification Mechanisms for accreditation

Formal learning Feed into formal learning offerings

Autonomy Degree of learner autonomy

A taxonomy of MOOCs

http://e4innovation.com/?p=727

Page 7: Conole eden _mooc_evaln

A new MOOC classificationDimension Connectivist Siemens MOOC

ContextOpen 3Massive 2Diversity 3LearningUse of multimedia 2Degree of communication 3Degree of collaboration 2Amount of reflection 3Learning pathway 1Quality assurance 1Certification 1Formal learning 1Autonomy 3

For each dimension, give the MOOC a score: Low=1, Medium=2High=3

Page 8: Conole eden _mooc_evaln

A new MOOC classificationDimension Connectivist Siemens MOOC

ContextOpen 3Massive 2Diversity 3LearningUse of multimedia 2Degree of communication 3Degree of collaboration 2Amount of reflection 3Learning pathway 1Quality assurance 1Certification 1Formal learning 1Autonomy 3

How to rate Open?

It’s free = 1

At least some CC materials = 2

All materials CC, and non-registered students can view materials=3

How to rate Massive?

Under 500=1500-10,000=2Over 10,000=3

http://tinyurl.com/OEWBirdConole

Page 9: Conole eden _mooc_evaln

FutureLearn

• Launched Dec 2012• 37 partners• Inspire learning for life• Diverse range of courses• Globally connecting learners

with experts• Bite-size steps of learning• Learning through

storytelling• Mix of text, video, audio and

activities

https://www.futurelearn.com/

Page 10: Conole eden _mooc_evaln

Principles

• Open• Listen to learners• Tell stories• Provoke conversation• Create connections• Keep it simple• Learn from others• Celebrate progress• Embrace FutureLearners

Page 11: Conole eden _mooc_evaln

UoL MOOCs

• Courses– England in the time of King

Richard III– Forensic Science and

Criminal Justice

• Key features– 6 weeks, 3 hours a week– Bite-size chunks of learning– Text, audio, videos, forums

and activities– Certificate of participation

Page 12: Conole eden _mooc_evaln

Methodology

• Focus: – Interviews

• developers, tutors, learners, UoL lead

– Survey (participants)– Analysis of courses– Learning analytics– No. registered– Drop out rates

Page 13: Conole eden _mooc_evaln

Aims

• Rationale for joining• Patterns of interactions• Perceptions• Development time• Time on different

components• Reasons for participating• Reasons for drop out• Recommendations

Page 14: Conole eden _mooc_evaln

Activities

Page 15: Conole eden _mooc_evaln

Survey findings

• Cost £28, 500 – 130 days of staff time

• 52% had prior MOOC experience

• Richard III (10, 066)• Most visited a few times a

week• Average No. posts 8• 87% no contact with tutors• 47% still active in final week

Page 16: Conole eden _mooc_evaln

Survey findings• 97% structure clear• 91.5% Engaging• About right

– 67% level– 85% time– 69% length

• Participating– Learn new things (85%)– Try a MOOC (53%)– Online learning (46%)

• 91% a positive experience • Interesting, enjoyable,

informative

Page 17: Conole eden _mooc_evaln

Interview themes• Marketing • Use of video and chunks• Learner analytics• Need for Learning Technologists• Unknown audience• Platform

– Good for learners– Not good for developers

• Expectation management• Not much discussion• Good profile• Learner discussions and content generation• Public good• Pedagogical innovation • Expert learners

Page 18: Conole eden _mooc_evaln

http://wp.europeanmoocs.eu/

Related projects

http://wikieducator.org/Emundus

http://vmpass.eu/

Page 19: Conole eden _mooc_evaln

Formal

Informal

Individual Social

Blended courses

DL+ social media

Trad. campus courses

DL courses

OER

xMOOCs

OER + Social media

cMOOCs

APELePortforliosOERuBadges

Formal/informal landscape

Page 20: Conole eden _mooc_evaln

Recommendations

• More strategic choice of courses

• Support from LTs• Effective use of learner

analytics• Explore new business

models• Feed into campus courses• More pedagogical

innovation

Page 21: Conole eden _mooc_evaln

MOOCs as a disruptive technology• Disruptive technologies are

challenging traditional institutions

• New business models emerging• New approaches needed for

designing and delivering MOOCs• Blurring of boundaries:

– formal/informal, real/virtual, teacher/learner, cross cultural

• Need for new pedagogies• Disaggregation of education

– High quality resources– Learning pathways– Support– Accreditation

Page 22: Conole eden _mooc_evaln

http://www.le.ac.uk/ilihttp://www.slideshare.net/GrainneConole

[email protected]://e4innovation.com

@gconole