Self-evaluative practices to

enhance e-pedagogy

Claire Raistrick

University of Warwick April 2014

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Raistrick, C (2014) Self-evaluative practices to enhance e-pedagogy. IN: Bayne, S, Jones, C, De Laat,

M, Ryberg, T & Sinclair, C. Proceedings of the Ninth International Conference on Networked Learning

2014. John McIntyre Conference Centre, University of Edinburgh. 245-252.

Link to paper: http://networkedlearningconference.org.uk/abstracts/raistrick.htm

“the capacity of technology to satisfy real

learning which bolsters the student experience” (Raistrick 2014:245)

E-pedagogy:

HOW

Research practice

Enhance learning

Transformational change

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

?

What do we know?

TEL innovations +++

Share practice +++

Evaluation +++

• Judgements

– Worth

– What works

What do self-evaluative

practices actually look

like when educators make a

TEL innovation?

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Self-evaluative practice

• Evaluations to inform the professional

practice of solitary practitioners (or groups)

• The development of new knowledge “as

people engage in a process of reflection

related to real problems and issues in their

own context” (Saunders, 2011:14)

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Research strategy

• Context:

– Educators (x7)

– Postgraduate award

– Make & evaluate

• e-learning innovation

• Co-construction:

– Dialogical conversation (Knight & Saunders, 1999)

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

SEPT4TEL Framework (Self-evaluative practices typologies for technology enhanced learning)

Reasons & purposes

Uses

Focus

Data & evidence

Audience

Timing

Agency

25

Gu

idin

g p

rin

cip

les

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

RUFDATA: An established evaluation tool (Saunders, 2000)

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

“try out a new e-learning

innovation … get more

active research wise”

prevent learners’ “struggling”, to show

them “gaining” from the innovation

“how [does the innovation contribute] to the

learning [is it] worth getting to work properly”

“the meaningfulness of the learning

… how useful is it … does it have an

impact … that’s the evaluation”

Reasons & purposes: To benefit student learning

"[a person] says

something [and that]

will make you think”

“trying to adapt … so that

[changes are] useful”

•Establish worth

•Improve pedagogy via

evaluative acts

•Sustain evaluative

creep

•Achieve CPD

SEPT4TEL: Guiding principles

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Reasons & purposes: To benefit student learning

“more confident” because

the innovation “ran

smoother, probably

[because I] took out parts

that definitely didn’t work.”

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

“use it as it is until … then

do an overall evaluation”

“to be improved next time”

“I’m going to ensure that I

communicate more effectively … my

excitement … might have put them off”

Uses: To inform change processes

“telling and showing

something that works”

•Reveal new knowledge: 3 stages

•Gain clarity re changes

•Enable responsiveness:

•Increasing confidence &

competence

SEPT4TEL: Guiding principles

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Uses: To inform change processes

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

“students are fully informed … when [X

occurs] that’s not a shock … real success

[will be] I don’t actually do [name of task]

… I would be overjoyed with that”

“that students have

done it [i.e. used the

TEL innovation]” “because we can’t get the [TEL

innovation work] done” ... want it

“to look reasonably professional

before [others see it]”

Focus: The effects of the TEL innovation

•Effect(s) at different stages

•Work out what interests you

•Identify convincing forms of data:

•Metadata

•Stakeholder feedback

•Changes/improvement/activity

•Indicators of success

•Problematic gaps

SEPT4TEL: Guiding principles

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Focus: The effects of the TEL innovation

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

“reflective chat

[to oneself]"

Data & evidence: Accessible, straightforward, manageable

“asking more

questions” “what [learners] liked”

“I made notes afterwards”

“the only way you can make the [TEL

innovation] a success is continuously

monitoring what’s going on, if

something’s not quite right changing it”

“wobble between …

hardly got any data

… other extreme …

got loads of data”

•Recognise, capture, analyse: Use

to inform decision-making: •Physical & non-physical forms

•Feedback: diverse sources, including

reflexively

•Recognise potency of multiple

knowledge resources

•Prioritise knowledge with practical

potential: •Influence change processes

•Attribute worth •Be tenacious, alert, dedicated

•Sufficient for its current purpose –

no more

SEPT4TEL: Guiding principles

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Data & evidence: Accessible, straightforward, manageable

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Audience: Learners, close colleagues & others in/beyond institution

“responses from

[colleagues] … to the

findings” ... ... ...

“obviously interested”

“one or two comments towards

the end [of the course]” “evaluation of the

students’ abilities”

the boss who wants

something “to happen”

Colleagues speak about the TEL

innovation “in glowing terms” and “learn”

from using the innovation “as well”

•Accept your central role •Engage users’ attention to influence development •Connect with what makes you curious •Use outputs to influence other educators’ practice

SEPT4TEL: Guiding principles

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Audience: Learners, close colleagues & others in/beyond institution

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Timing: Award submission date & other provisional endpoints

“on-the-hoof” evaluation

contrasts with reflective

elements which: “need

time to emerge”

“I’m planning, I’m doing … I’m

reflecting … saying yes this is

good keep going, or no …

back to the planning stage”

“read … responses …

make up a task list”

“very few people to work

with … very little time”

“... how I’m going to be doing my

evaluation … I’ve been working it

out for the past half an hour”

•Select an important (to you) project •Identify provisional endpoint 1:

•Continually ‘nudge’, achieving multiple provisional endpoints

•Keep tuned-in, maximising momentum •Build-in staging posts:

•Space •Time

SEPT4TEL: Guiding principles

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Timing: Award submission date & other provisional endpoints

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Agency: Myself, as educator & interested outsiders

“no-one else” will obtain the

feedback [I] accomplish

[I] do not “lock the door”

“I’ll be evaluating”

•Take responsibility:

•Progress

•Involving stakeholders

•Potential bias

•Recognise external entities

provide valuable input

SEPT4TEL: Guiding principles

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Agency: Myself, as educator & interested outsiders

Self-evaluative practices “transformational effect of 'evaluative creep' on change

processes ...

constellations of routine behaviours ... intrinsically worthwhile ...

probable that self-evaluators become ‘carriers’ of practices

[influencing the] effectiveness of technology for learning ...

[Use of the] SEPT4TEL framework [by] educators [may]

improve connections between technology and learning ...

a sufficient constellation of self-evaluative practices is

potentially powerful ... acting as a catalyst — with the power

to leverage more widespread change.”

(Raistrick, 2014:252)

c.g.raistrick@warwick.ac.uk @ClaireRaistrick

Thank you

Claire Raistrick

University of Warwick April 2014

c.g.raistrick@warwick.ac.uk @ClaireRaistrick