7 Research-Based Principles

For Learning & TeachingAdapted from ‘How Learning Works’

Pedagogy with technology :

Getting the horse out in front of the cart

Simon Bates University of Birmingham, 10th annual T&L conf June 2013

Overview

I. The space of technology-enhanced

II. Case study: PHYS101 at UBC

Source: http://suifaijohnmak.wordpress.com/2013/05/04/finally-it-is-coming-and-coming-fast/

Learning content

Digital, open and at-scale

- 1 in 8 courses available as a MOOC, somewhere

- 370 courses, 70 institutions on Coursera

- 4 million students on MOOC platforms last year (Coursera, edX, Udacity)

Source: Coursera, edX, Udacity websites

Students

Source: ECAR Study of Undergraduate Students and Information Technology,2012.http://educause.edu/ecar-student-2012

Evidence

From FL site

Source: Educause meta analysis and review of online learning studieshttp://www.educause.edu/library/resources/evaluation-evidence-based-practices-online-learning-meta-analysis-and-review-online-learning-studies

EvidenceMeta-analysis of >1000 studies (1996-2008)

Included studies that - contrasted online to f-2-f- measured learning outcomes- rigorous research design- calculated effect size

On average, students performed better in online learning situations

Difference in outcomes largest in blended situations, but not the media per se.

Source: Educause meta analysis and review of online learning studieshttp://www.educause.edu/library/resources/evaluation-evidence-based-practices-online-learning-meta-analysis-and-review-online-learning-studies

Evidence

From FL site

Source: Changing Course, Sloan Consortium 2012http://www.pearsonlearningsolutions.com/assets/downloads/reports/changing-course-survey.pdf

Acceptance

Source: Changing Course, Sloan Consortium 2012http://www.pearsonlearningsolutions.com/assets/downloads/reports/changing-course-survey.pdf

Uptake

From FL site

Source: Changing Course, Sloan Consortium 2012http://www.pearsonlearningsolutions.com/assets/downloads/reports/changing-course-survey.pdfEducause meta analysis and review of online learning studieshttp://www.educause.edu/library/resources/evaluation-evidence-based-practices-online-learning-meta-analysis-and-review-online-learning-studies

But….

Source: ECAR Study of Undergraduate Students and Information Technology,2012.http://educause.edu/ecar-student-2012

A case study PeerWise in PHAS 101

• Web-based Multiple Choice Question repository built by students

• Students:– develop new questions with

associated explanations– answer existing questions and rate

them for quality and difficulty– take part in discussions– can follow other authors

peerwise.cs.auckland.ac.nz

>100,000student contributors

>500,000unique questions

>10,000,000answers

18

As a question author…..

19

20

23

As a question answerer …..

24

25

26

27

Timeline

2010-11: UoE pilot study

2011-12: Multi-institution, multi-course

2012-13: UBC PHYS 101

Coursera MOOC

28

Previous research

• Good engagement and participation beyond the minimum requirements

• Correlation between use and end-of-course outcome

• Replication study in 3 institutions, 5 courses,3 disciplines

1st year Physics N=172University of Edinburgh

1st year Physics N=172University of Edinburgh

2nd year Physics N=143

University of Glasgow 2nd year Biology N=279University of Edinburgh

1st year Chemistry N=172University of Edinburgh

1st year Chemistry N=163University of Nottingham

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

1 2 3 4 5 6

Taxonomic Category

Per

cen

tag

e o

f S

ub

mit

ted

Qu

esti

on

s

Previous research

• Question quality: mapped onto levels in cognitive domain of Bloom’s taxonomy

• Surprisingly high overall quality, evenfrom ‘novices’

First semester N = 350

Second semester N = 252

Implementation in PHAS 101

8 sections (4-3-1) N(total)~1800

2012 W2 3 sections N=791

All non majors.

2 PeerWise assignments

- scaffolded ahead of the first

- support woven into 4 subsequent tutorials33

Assessment requirements

As a minimum:

• Write one question• Answer 5• Comment on & rate 3

Contributed ~2.5% to course assessment

(mostly participation, small bonus for performance)

Student questions on midterm and final exams.

34

We were deliberately

hands off.

• No moderation• No corrections• No interventions at all

But we did observe…..

35

Scaffolding in tutorials

36

Scaffolding in tutorials

37

Engagement

First assessment exercise:

• 664 active students (out of 790!) • 1340 Q, 11000 A, 5000 C• x1.75, x17, x7 minimum requirements

Engagement

Score

Examples

42

43

but

WHYis this eff ective?

Take homes

• There exist a huge variety of ways you can harness technology to support student learning.

• Horse-before-cart

Acknowledgements

Physics 101 course team

Georg RiegerFiras MoosviEmily Altiere UBC CWSEI

simon.bates@ubc.ca

@simonpbates

These slides:

http://bit.ly/Bates_Bham2013

Ross GallowayJudy HardyKaron McBrideAlison KayKeith BruntonJonathan RiiseDanny Homer

Chemistry – Peter Kirsop

Biology – Heather McQueen

Physics – Morag Casey

Comp Sci – Paul Denny

Resources - PeerWiseSite: http://peerwise.cs.auckland.ac.nz

Community: http://www.PeerWise-Community.org

JISC-funded multi institution study:https://www.wiki.ed.ac.uk/display/SGC4L/Home

UoE Physics Pilot Study: AIP Conf. Proc. 1413, 359 http://dx.doi.org/10.1063/1.3680069

RSC overview articlehttp://www.rsc.org/Education/EiC/issues/2013January/student-generated-assessment.asp

UoE Physics scaffolding resources http://www2.ph.ed.ac.uk/elearning/projects/peerwise/

Question quality analysis (1st year Physics University of Edinburgh) Assessing the quality of a student-generated question repository, submitted to Phys Rev, ST Phys Educ Res.

Multi-institution, multi-course studyStudent-generated content: Enhancing learning through sharing multiple-choice questions, submitted to International Journal of Science Education

Scaffolding Student Learning via Online Peer Learning, submitted to International Journal of Science Education

Publications in preparation / review / press