Creating highly engaging online discussions to support sustainable formative learning

37
Creating highly engaging online discussions to support sustainable formative learning Brant Knutzen Learning Designer Faculty of Education University of Hong Kong

description

Creating highly engaging online discussions to support sustainable formative learning. Brant Knutzen Learning Designer Faculty of Education University of Hong Kong. Overview. Social Constructivism Transactivity Building engagement Assessment methods Participation rating - PowerPoint PPT Presentation

Transcript of Creating highly engaging online discussions to support sustainable formative learning

Page 1: Creating highly engaging online discussions to support sustainable formative learning

Creating highly engaging online discussions to support sustainable formative learning

Brant Knutzen

Learning DesignerFaculty of EducationUniversity of Hong Kong

Page 2: Creating highly engaging online discussions to support sustainable formative learning

Overview

Social ConstructivismTransactivityBuilding engagementAssessment methodsParticipation ratingMeasuring engagement

Page 3: Creating highly engaging online discussions to support sustainable formative learning

Computer-mediated ConferencingCMC : Computer-mediated

Conferencing Synchronous : “chat”, or instant

messaging Asynchronous : “discussion forum”

Asynchronous Participants take turns posting Minutes, hours, or days apart More time to digest, research, compose

response

Page 4: Creating highly engaging online discussions to support sustainable formative learning

Learning Interactions using CMC

Page 5: Creating highly engaging online discussions to support sustainable formative learning

Social ConstructivismThe effective construction of

knowledge is a product of a functional collaborative group producing artifacts for public display and use

Efficacy has been found to be linked to the process that learners utilize in working on the task together (Fischer et al 2002)

Process: social negotiation of arguments and argument sequences (Leitão 2000; Voss & Dyke 2001)

Page 6: Creating highly engaging online discussions to support sustainable formative learning

Social Constructivism

John Biggs captured the educational value of discussion when he stated:

"Good dialogue elicits those activities that shape, elaborate,

and deepen understanding“ (Biggs 1999 p. 5)

Page 7: Creating highly engaging online discussions to support sustainable formative learning

Transactivity

Transactivity: the method by which students build on the contributions of their fellow learners

(Berkowitz & Gibbs 1983)

Page 8: Creating highly engaging online discussions to support sustainable formative learning

Transactive online discussionsTransactive communication:

Participants respond to and build on each other’s contributions

Peer exchange of information and ideas Social negotiation of knowledge Each participant brings their own

experiences to apply to a common educational goal

Page 9: Creating highly engaging online discussions to support sustainable formative learning

Transactivity

A key theoretical construct for measuring collaboration

How can we formulate the instructional design conditions which consistently result in more productive and transactive learning activities?

How can we describe it in easily grasped ways? Quantitative Qualitative

Page 10: Creating highly engaging online discussions to support sustainable formative learning

Online discussion – the potentialAn online discussion is a great

formative learning activity for building depth of student understanding, or supporting project work

Highest potential for social construction of knowledge and transactive collaboration

Unfortunately, this activity is also the most likely to FAIL! Why?

Page 11: Creating highly engaging online discussions to support sustainable formative learning

Study #1 – 2007 – Building Engagement

Case study of one course (Knutzen, 2007)

▪ International school in Hong Kong – secondary level

▪ 1-to-1 laptop blended learning environment▪ Introduction to Psychology course▪ Sample size = 24

Investigation of instructional design conditions to achieve a highly productive online discussion

At start of study, average student production in optional online discussions = 1 post

Page 12: Creating highly engaging online discussions to support sustainable formative learning

Productive Communication

Four conditions to achieve productive online discussions:1. Teacher facilitated social formation of small

groups2. Class time to initiate oral and online discussion

interaction3. Setting open-ended, challenging topic

questions that encourage discussion and debate

4. Assessment system that reinforces production and peer interaction

At end of study, average studentproduction: over 10 posts per discussion!

Page 13: Creating highly engaging online discussions to support sustainable formative learning

Extensive use of discussion designOver the past six years:

Extensive use of the online discussion design

Full-time instruction of secondary students▪ 1-to-1 laptop environment▪ IT classes▪ Psychology

Implemented at University of Hong Kong: 2009▪ MSc IT in Education course on eLearning

Implemented in 12 courses at Lingnan Univ : 2010▪ Business, History, Social Studies, Philosophy,

EnglishDesign continues to result in good

production

Page 14: Creating highly engaging online discussions to support sustainable formative learning

Designing discussion topic questionsOne to three questions around one topic

or area of content / conceptsAdvantage of multiple questions:

Instructor can design a “gradient” of difficulty which can elicit a range of student answers▪ From basic knowledge -> higher-order thinking

skills (HOTS)▪ Use a taxonomy of active verbs to specify the

levels of understanding expected in answers (Blooms Revised, SOLO)

▪ Examples: Multi-structural (list, describe, classify)Relational (compare/contrast, explain, analyze, relate)Extended Abstract (hypothesize, generate, reflect)

Page 15: Creating highly engaging online discussions to support sustainable formative learning

Designing discussion topic questionsObjective of multiple questions:

Make discussion accessible to all students Challenge the advanced students

Other topic question gradients found to be useful: Concrete facts -> abstract concepts Textbook context -> personal context

(unique answers!)

Page 16: Creating highly engaging online discussions to support sustainable formative learning

Designing discussion topic questionsExample of a Topic Question gradient:

Can you demonstrate what you have learned in your study of the Porter management models?

1. Can you list and describe the Porter models? (Basic understanding ).

2. How can you compare Porter's models? (Relational understanding)Can you relate these models to each other in several ways, or on several dimensions?.

3. Based on these models, can you create your own model?

What factors do you theorize are important, and why? (Extended abstract understanding)

Page 17: Creating highly engaging online discussions to support sustainable formative learning

Discussion marking schemes

Traditional – teacher-assessed subjective marking Review contributions by each student Award mark based on:

▪ Participation – any contribution to discussion▪ Interaction - responding and seeking feedback▪ Transaction – sharing / exchanging useful information and

resources▪ Transformation – perspective change due to interaction

with others

Best method for summative assessment

Should a discussion be a summative activity?

Page 18: Creating highly engaging online discussions to support sustainable formative learning

Using a Rubric for assessmentCriteria examples:

▪ Locate, organize, analyze, evaluate, synthesize, and ethically use information from a variety of sources and media

▪ Interact, collaborate, and publish with peers, experts, or others employing a variety of digital environments and media

▪ Develop cultural understanding and global awareness by engaging with learners of other cultures

▪ Contribute to project teams to produce original works or solve problemsSource: ISTE NETS for Students

Page 19: Creating highly engaging online discussions to support sustainable formative learning

Problems with teacher assessmentA highly productive discussion can

easily produce over 200 posts a week!

A teacher can become a victim of their own success How much time can they devote to

quantitative marking? How much time remains for qualitative

feedback?

Page 20: Creating highly engaging online discussions to support sustainable formative learning

Peer-assessment

Desired graduate attributes: Critical thinking skills Excellent cooperative skills

▪ Integrity▪ Personal responsibility

Subjective peer-assessment can directly address the development of these attributes Requires student training Requires review and evaluation by

teacher

Page 21: Creating highly engaging online discussions to support sustainable formative learning

Peer-assessment - SubjectivePossible problems:

Revenge grading : 報復▪ “you gave me a low grade, I will give you a low

grade”

Back-scratching : 賄賂 ▪ “If you give me a high grade, I will give you a

high grade “

Page 22: Creating highly engaging online discussions to support sustainable formative learning

Sustainable forum assessment?

Teacher evaluation of forums: time intensive

Peer evaluation of forums: requires monitoring

Subjective evaluation of formative work tends to distort student learning

Optional forums do not get sufficient participation to create viable discussions

Page 23: Creating highly engaging online discussions to support sustainable formative learning

Leveraging the power of the LMSMy solution: automate the rating based

on participation No subjective judgment, just rating using a

systematic method:

Moodle can automatically average these grades!

Page 24: Creating highly engaging online discussions to support sustainable formative learning

Rating based on Participation

Moodle averages the ratings Grades produced by participation:

▪ One post = 6 -> D-▪ Two posts = 8 -> B-▪ Three posts = 8.6 -> B▪ Four posts = 9 -> A-▪ Five posts = 9.2 -> A-▪ Six posts = 9.33 -> A▪ Seven posts = 9.42 -> A▪ Eight posts = 9.5 etc

More Q&A participation = higher grade

Page 25: Creating highly engaging online discussions to support sustainable formative learning

More participation -> higher grade

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 210

1

2

3

4

5

6

7

8

9

10

Number of posts -> Forum Grade

Number of discussion posts

Forum Grade

Page 26: Creating highly engaging online discussions to support sustainable formative learning

A foundation of participationThis assessment method for

assessing a forum creates a foundation of participation: on average over 90% of students will engage

As always, the teacher’s role is to create the educational value of the activity: Set up challenging topic questions to

guide exploration Maintain a “visible presence” in the

forum activity Successfully moderate the discussion

Page 27: Creating highly engaging online discussions to support sustainable formative learning

Discussion moderation techniquesSalmon’s 5-stage

model ofeModeration

Teacher guidesstudents upthe levelstowardknowledgedevelopment Source: http://www.atimod.com/e-moderating/5stage.shtml

Page 28: Creating highly engaging online discussions to support sustainable formative learning

Participation-based forum gradeThe Moodle LMS handles the awarding

of quantitative marks for participation

Teacher can focus on quality: Monitor progress Guide the discussion Challenge students Time for thoughtful qualitative

feedback▪ Minimum: 30 mins to 1 hour each week per

discussion

Page 29: Creating highly engaging online discussions to support sustainable formative learning

Motivating students to engageMotivation on three levels:

1. Constructive alignment between formative discussions and summative final assignmentsExamples:

▪ Reflective posts build towards a Learning Portfolio blog

▪ Group collaboration on final project development

2. Social Constructivism: a sense of social obligation to support the group

3. Quantified Participation: each contribution is counted as participation towards a forum grade

Page 30: Creating highly engaging online discussions to support sustainable formative learning

The Participation ForumAutomates the awarding of

participation points Reduces technical requirements

(modifying roles) No need to train the students No need to monitor rating accuracy No need to motivate student ratings of

posts Finished development and testing in Dec

2011 Currently in trials at HKU

Planned release as open-source project soon

Page 31: Creating highly engaging online discussions to support sustainable formative learning

Measures of discussion activity

Quantitative:▪ Production = Total number of discussion posts / n▪ Interactivity = Total number of feedback posts / n▪ Group Activity = Total number of discussion posts

/ # topics▪ Transactivity = Production × Interactivity

Qualitative: a new type of graphical representation – the “Participation Map” Developed at Lingnan University in 2011 Continued development as open source

project

Page 32: Creating highly engaging online discussions to support sustainable formative learning

The Participation MapNew Moodle plug-in automates the

production of the map of discussion activity: Quantitative statistics Qualitative graphic display: a “data portrait”

Useful for displaying discussion activity Feedback to students in the class (who is

working?) Feedback to teacher (what approach is

working?) Anonymous mode for reporting research

results

Page 33: Creating highly engaging online discussions to support sustainable formative learning

Example Participation Map

Student profile picturesStudents can interact in multiple groups

Tim

e sc

ale

from

Day

1 to

Day

14

Red line shows rating cutoff

Group comparison stats

Overall discussion statistics

Page 34: Creating highly engaging online discussions to support sustainable formative learning

Quantitative Data

Quantitative ratings of overall discussion

Page 35: Creating highly engaging online discussions to support sustainable formative learning

Participation Map is a teacher toolAfter install on Moodle server, this

plugin is only available to those with Teacher roles

New option on CourseAdministration block

Page 36: Creating highly engaging online discussions to support sustainable formative learning

Participation Map operationSelect the

forum, andthen plot type: Normal plot

for feedbackto teacher andstudents

Anonymousfor reportingresults outside

Page 37: Creating highly engaging online discussions to support sustainable formative learning

Questions and DiscussionCurrently setting up a website to make

these open source projects available to the Moodle educational community: Brant.Knutzen.se

Goal is to release both as free open source plugins for Moodle 2 by Sep 2012

Q & A ?? My email is [email protected]