Evaluating Practice

Nigel Russell SFHEA

&Farzana Latif SFHEA

Tuesday 28 June 2016

TELFest2016.

The aim of this workshop is to encourage thinking around evaluation of:

• our practice as teachers• our students’ learning

and the role technology can play in supporting this.

Aim

Why evaluate?

Evaluation focuses on ‘level of quality’ at the moment of evaluation

Evaluation is not concerned with attainment level

Assessment focuses on quality of performance and how to improve the ‘level of quality’

Discussion

How do you evaluate your practice?

How you know your students are learning what you want them to learn?

When do you evaluate?

Discussion

What techniques or tools do you use to help evaluate:

• your practice, and/or

• your students’ learning?

CATsAngelo and Cross (1993): Classroom Assessment Techniques

Resource outlining fifty different techniques to help assess student learning

• at the point of learning (a ‘temperature check’)

• gives the teacher immediate, useful, feedback

• encourages active learning• results easily and quickly analysed

Muddiest PointSimplest CAT – assess prior knowledge or comprehension

Help explore concepts not understood

Turn around to the “lightbulb moment”: what was the key concept?

Discussion boardsSurveys or polls

Google Forms or Google DocsPolleverywhere

One sentence summaryStudents write single, long, summary sentence - assess higher cognitive skills

Stimulates creativity

Teacher can see originality of thought

Discussion boardsGoogle Forms or Google Docs

Blogs (audio or text-based)Padlet

HEA Toolkit

https://www.heacademy.ac.uk/resource/evaluating-teaching-development-higher-education

Points to rememberUse e-technology when it helps facilitate your students’ learning

Use techniques you have experienced or tried on colleagues – if new to you, use quick and easy techniques to start with

Report findings back to students

Adapt techniques to suit you/your students

It doesn’t help you achieve what you want, don’t use it!

Further Resourceshttp://cft.vanderbilt.edu/guides-sub-pages/cats/

http://pages.uoregon.edu/tep/resources/newteach/fifty_cats.pdf

http://www.crlt.umich.edu/sites/default/files/resource_files/ClassroomAssessmentTechniquesHopkins.pdf

http://flexiblelearning.net.au/plan-and-deliver/design-e-learning/gallery/concept-maps-andmind-maps/

http://www.pcrest2.com/institute_resources/PAI/4_1_2.pdf

http://files.eric.ed.gov/fulltext/ED317097.pdf

TEL Hub

http://tel.group.shef.ac.uk/

MOLE Turnitin | PebblePad | Lecture Recording | Google Apps | TurningPoint | LectureTools | Adobe Connect | WebPA | uPlayer Analytics | Mobile Apps

Technology Examples

Evaluation of TELFest and Changes to Practice

• 58 out of 398 Potential Participants (19%) responded to the questionnaire

• Focus groups with 21 members of staff• 81% (47) intend or plan to make changes

Evaluating Practice

“I made changes to my course, it made me feel better but I’m not sure if it offered value to my students” Lecturer 1

• How many plan to make change?• 34% (16 out of 47)

Evaluation Methods• Standardised Feedback e.g. module/course evaluation

(although there was an acknowledgment that this might not necessarily surface feedback related to specific changes)

• changes to grades• direct informal feedback• no longer receiving bad feedback• analysing analytics related to online access of

resources/completion of activities• develop specific questionnaires for students to complete• Peer review of use of TEL

General Trend

• Literature review of TEL practices, which found that a scholarly approach to enhancing teaching and learning is seldom by teachers and policy-makers

• 2014 UCISA TEL report: found that less than half of institutions evaluate impact of TEL tools, 45% (43 of 96) (Walker et al., 2014)

Case Study discussion

“I made changes to my course, it made me feel better but I’m not sure if it offered value to my students” Lecturer 1

– What would be the factors that we could use to evaluate this?

– (see next slide for responses)

Case Study discussion

Case Study discussion

“I used twitter in my teaching for 3 years, the first couple of times I used it, it just didn’t hit off/ It really worked this year, I’m not sure why.” Lecturer 2

– What evaluation techniques could have been used?

• How could evaluation help to sustain this engagement and impact?

§(see next slide for responses)

Case Study discussion

Importance of Evaluation

• Students change• Technology changes• Enhance your own practice• Reflect on how to improve

Use your evaluationtechniques as examples

in your application for professional recognition

AFHEA FHEA SFHEA

http://www.shef.ac.uk/lets/cpd/ltprs

Thank you!

Nigel Russell SFHEA

n.russell@sheffield.ac.uk

Farzana Latif SFHEA

farzana.latif@sheffield.ac.uk