The 21st Century Educator
-
Upload
simon-bates -
Category
Education
-
view
75 -
download
1
Transcript of The 21st Century Educator
The 21st Century EducatorAnatomy & a case study from introductory science
Simon Bates [email protected] @simonpbates bit.ly/batestalks
Symposium for Effective Teaching & Learning in the Sciences — UOIT — Sept 1 2016
2
A talk in two parts
“Anatomy” of skills and values
Example driver of change: technology
Case study example
Deeper engagement with assessment,
learning
CC BY-NC-SA 2.0 https://flic.kr/p/2ZdABF
4
Technology - scale and pace
Slide credit: Eric Grimson (MIT)
5
Technology - scale and pace
Slide credit: Eric Grimson (MIT)
6
7
Technology - reach and unbundling
https://www.youtube.com/watch?v=QWEq3xifCDw
8
Technology - disruptions
Graph extracted from http://vikparuchuri.com/blog/on-the-automated-scoring-of-essays/
9
Technology - implications
Changing the
of many aspects of life, …and learning is included
what, where, when, how, from whom and with whom
10
So what are the
we need to embrace, develop and refine?
skills, values and habits
Your suggestions
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Case study - students as producers
“How can I get my students to engage more effectively with formative assessment opportunities in the course?”
a"web&based"MCQ"repository"created"by"students"
Ins$tu$ons((signing(up(per(year:(!
2009:! ! !22(2010:! ! !66(2011:! ! !204(2012:! ! !266(2013!(Jan,Jun):! !214(
Growing(content(repository:(!Courses:! !2,500(Logins/month:! !75,000(Ques<ons:! !600,000(Answers:! !12,000,000(
Answers (20,000,000)
Questions (1,000,000)
Student'ownership'over'learning'resource'
Student'familiarity'with'social'so7ware'
Leveraging'student'energy'and'crea9vity'
Badges' Points' Leaderboards'
Selected results & analysis
Engagement - how do students use the system?
Benefits - what is the impact on learning?
Question quality - how good is what students produce?
Relevant publications:
Scaffolding student engagement via online peer learning - European Journal of Physics 35 (4), 045002 (2014)
Student-Generated Content: Enhancing learning through sharing multiple-choice questions. International Journal of Science Education, 1-15 (2014).
Assessing the quality of a student-generated question repository - Phys Rev ST PER (2014) 10, 020105
Student-generated assessment - Education in Chemistry (2013) 13 1
Typical implementation
Minimum participation requirements for each of two assessment exercises (PW1, PW2)
Write 1 Answer 5 Rate / comment 3
5% course credit
Physics 101, Energy & Waves Winter Semester: 3 sections, ~800 students
Not so typical implementationTOCCLT'.' -
Qest\I\sct
How 1o...
q/t4osTer
Submit ond onswer questionson topics in lhe torget region,just obove lhe physicsyou hove olreody mostered.
Dislroclers
) l*"*"ò
rNrrltngIh s region corrk:ins llte
plrysics knowledgo ond colrco¡lsyou connol leorn yel
becouse the louncalionscrrc nol in plocc
PHYSICS TOPICS IN YOUR TARG T REGION
buo'5ot't1 'Den¡¡lu
B¿:o\v\vì 1 JF:rcq - YOUR CHOSEN TOPIC
à..\auC'q^ x@bôo'osro
pn ro, d logroo, \o 9tS
COMMON MISCONCEPÎIONS AND ERRORS(Sec hHÞ:/jÞhy¡.udofo¡.cdu/CJP/trêconcoÞllo¡t.pdt
tor o llil ol common mhconcepllons)
¿K N(,qht %cce Açxs no\ exi¡þ, c.rrrçr\J On oblec'-t \5 õÞYJrnr cìü?ÀI in rtuiò. ü
lrh'brot4ont eprç,e .I¡1enÀs crqden-s,\{^ c>Ç c.,þ¡ec\ , cìo+ .\*¿n:r
oÇ Çtu,8,fr- btrrqon! $crCe c¿c\s Àoc¡:'cr
fr¿lt{e- r *hcrn ú?
Trr¿e. $ììfifflsz hrrr: ¿lÕoolr-g lnnS
(**= tcpo þ/rnl {otr-* qrÇ bq.\\:. 0" Ooz
Physics knowledgeond conccpluol underslonding
you lrove olrcodyconslrucleC ¡n your heod
¡,<'t.lerJ rt)\^,¿z! Þ.rrp \r- \rr't\¡ ,l?
Chcck lhol yovr on3Íêr ¡t rêo!ànoblê ðnd potrlble
\'f<,,cc\
Photo by Seth Casteelhttp://www.littlefriendsphoto.comPermission to use agreed
Writing original questions is a demanding activity
Extensive scaffolding exercises
Revisited in subsequent tutorials
Engagement with PeerWise
Number Multiplier Number MultiplierQuestions 1105 [1.7] 998 [1.6]
Answers 11393 [17.2] 11807 [18.7]
Comments 4901 [7.4] 5509 [8.7]
PW 1 PW 2
Engagement with PeerWise
Engagement with PeerWise
Engagement with PeerWise
Engagement with PeerWiseGenerally, students did
• Participate beyond minimum requirements • Engage in community learning, correcting errors • Create problems, not exercises • Provide positive feedback
Generally, students did not
• Contribute trivial or irrelevant questions • Obviously plagiarize • Participate much beyond assessment periods • Leave it to the last minute (sort of….)
30
Correlation with learning
31
Correlation with learningQuartiles Q4 – top 25%
Q3 – upper middle
Q2 – lower middle
Q1 – bottom 25%
22 students did not take the FCI
32
Correlation with learning
33
1st year Chemistry N=172 University of Edinburgh
34
Question/Explanation QualityBloom’s Taxonomy of levels in the cognitive domain
Score Level Description
1 Remember Factual knowledge, trivial plugging in of numbers
2 Understand Basic understanding of content
3 Apply Implement, calculate / determine. Typically one-stage problem
4 Analyze Typical multi-step problem; requires identification of strategy
Evaluate Compare & assess various option possibilities; often conceptual
Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem.
35
Textp>0.05, NS
0%
5%
10%
15%
20%25%
30%
35%
40%
45%
50%
1 2 3 4 5 6
Taxonomic Category
Per
cent
age
of S
ubm
itte
d Q
uest
ions
First semester N = 350
Second semester N = 252
36
Question/Explanation QualityScore Level Description
0 Missing No explanation provided or explanation incoherent/irrelevant
1 Inadequate Wrong reasoning and/or answer; trivial or flippant
2 MinimalCorrect answer but with insufficient explanation/justification/ Some aspects may be unclear/incorrect/confused.
3 Good Clear and detailed exposition of correct method & answer.
4 ExcellentThorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution
370 1 2 3 4 0 1 2 3 40
20
40
60
Num
ber o
f que
stio
ns
Assessment 1 Assessment 2
Explanation Quality
38
Results (UoE 2010-11)2 successive years of the same course (N=150, 350)
‘High quality’ questions: 78%, 79%
Over 90% (most likely) correct, and 3/5 of those wrong were
identified by students.
69% (2010) and 55% (2011) rated 3 or 4 for explanations
Only 2% (2010) and 4% (2011) rated 1/ 6 for taxonomic level.
39
Bottomley & Denny Biochem and Mol Biol Educ. 39(5) 352-361 (2011)
107 Year 2 biochem students 56 / 35 / 9 % of questions in lowest 3 levels.
Momsen et al CBE-Life Sci Educ 9, 436-440 (2010)
“9,713 assessment items submitted by 50 instructors in the United States reported that 93% of the questions asked on examinations in introductory biology courses were at the lowest two levels of the revised Bloom’s taxonomy”
Comparison with literature
40
41
42
43
Resources
Student-facing system http://peerwise.cs.auckland.ac.nz/
All the research studies referenced and scaffolding materials referred to are accessible through the PeerWise community site http://www.peerwise-
community.org/
Beyond MCQs
Why not short answer Qs?
Why not …. anything?
Beyond MCQs
Why not short answer Qs?
Why not …. anything? LEARNING OBJECTS
Adaptive Comparative Judgement