UK OER 2:synthesis and evaluation

Allison Littlejohn, Helen Beetham, Lou McGill, Isobel Falconer, Claire Carroll

Caledonian AcademyGlasgow Caledonian University, UK

www.academy.gcal.ac.uk

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

Allison Littlejohn

Isobel Falconer

Helen Beetham

Lou McGill

Evaluation and Synthesis Team

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

Introduce the evaluation & synthesis framework developed during the pilot phase

Identify pertinent evaluation questions for this phase

Consider how you will gather evidence to answer your questions

How we will support you to do this...

Prioritise issues for future elluminate sessions

Aims of the session

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0 Focus areas

Approaches to OER releaseDeveloping, managing and sharing OERs Expertise Business cases and benefits realisation Cultural issues Roles, rewards and division of labour Legal issues Technical and hosting issues Quality issues Pedagogy and end-use issues

Which areas do you need more support/information about?Which areas do you expect to enhance with new knowledge and evidence?10 mins to think about this and comment in text chat box.

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

Structure to reflect key dimensions of the programme

provides a foundation and common language for collating data

offers a range of questions/issues to support evaluation and review

supports the collation of key messages, challenges, solutions and outputs

enables the identification of key areas of interest and highlights useful approaches

Rationale for a Framework

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

Issues for JISC/HEA programme managers: questions that have an answer for this programme; problems that have a solution for this programmee.g. 'we're having problems getting our consortium agreement signed off – do you have any suggested wording?'

Issues for programme support: problems that need expert advice from those working on OERe.g. 'we have a number of approaches to hosting that we're considering – can you advise us of the pros and cons?'

Issues for evaluation: questions that don't have a clear answer (yet) but that the programme is investigatinge.g....

What do we mean by 'issues'?

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputationEvaluation exampleglobal issues = benefits, uses and impacts

What are sustainable business/benefit models, building on Pilot phase and Good intentions report?

What are the benefits of OER in the current financial climate? Do they offer economies of scale?

How can we ensure learners make the best use of OER? What are the opportunities for learners to share content?

How are OERs used by academics in their own teaching? What new skills and expertise are needed?

Will OER contribute to a growth in online/distance learning and new pedagogies (will the HEFCE Online Learning Task Force work enable this)?

Who are OERs showcased to and with what impact? What kinds of communities benefit from OER sharing

and/or have existing ‘open’ practices

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0 What is the impact of cascade activities on capacity and organisational readiness

How are Web 2.0 tools best used to maximise discoverability and re-use?

How are repositories including JorumOpen, local and community repositories being used and how could they be used more effectively?

What is the impact of collection/collation on discovery and re-use?

What is the impact of OER release in specific topic areas to meet sector challenges?

What can case studies and tracking data teach us about uses, benefits and impacts?

Extending reputationEvaluation exampleLocal issues = strand specific and contextual issues

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

Which evaluation issues are you interested in?Try to re-frame questions in a way that is specific to

your project. Make the question tractable – ie. Something you can realistically find evidence to answer through the work of your project (NOT a separate research study!)

What evidence do you expect to be able to gather to help answer or enrich our understanding of this issue?

Make your proposals for evidence gathering realistic within the time and resources you have available

See our handout on evidence for the kind of things we mean. https://oersynth.pbworks.com/Which-Evidence

Activity

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

Projects will be paired-up to provide mutual support, sharing

experience, light touch Each pair will be offered support from one of us

Activity in task barHow could pairing up with a project help you? What activities would be useful?What support do you think you need?

lou.mcgill@gmail.com

Approaches – supporting evaluation activities

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputationProject pairing – possible activities

check evaluation plans are sensible, credible, valid, do-able, relevant to programme aims

provide peer review of content, interfaces, and other practical outputs from the projects

share ideas, experiences and interim outcomes of evaluation

swap time and expertise, e.g. analyse data for one another (extra objectivity)

share resources e.g. external consultants buddy up for e.g. workshops, critical moments (writing up etc)

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

Support projects in evaluation work Wiki, blog, evaluation resources, one-to-one Collate evidence across projectsEvaluate programme overall against original objectives: OERs released and collected Practices around OER reviewed/reformed/cascaded Lessons learned about OER release, management, discovery

and use Benefits in terms of challenges & stakeholdersReport on findings e.g. alternative approaches to OER,

approaches that are sustainable and usable, evidence of uptake and use, identified benefits to users

Link with other members of support team and across strands – including with 'users' study

Wiki - https://oersynth.pbworks.com/Blog - http://oersynthesis.jiscinvolve.org/wp/

Role of the evaluation team

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Re-visit the evaluation questions you listed earlier

Are these already answered on our wiki?

Go and find one Pilot Phase output that will help you?

Do you need to refine your questions?Do we need to refine the framework?

Have you already made connections with another project – would it be useful to pair up with them?

Vision Extending reputationActivity

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0 Vision Extending reputationInnovating e-Learning JISC online conference

23rd-26th November 2010Bringing innovation to life: From adversity comes opportunity

25th NovemberSustaining OER innovation through collaboration and partnership Simon Thomson (Leeds Metropolitan University) and Andy Beggan (University of Nottingham)

http://www.jisc.ac.uk/whatwedo/programmes/elearningpedagogy/elpconference10/programme.aspx

OE

R s

ynth

esis

and

eva

luat

ion

func

tion

2nd

Tues

day

Virt

ual s

essi

on –

Oct

ober

201

0

Vision Extending reputation

What should future Elluminate sessions focus on?

Priorities for Future Elluminate Session