Subverting MCQs for Deep LearningLee-Ann Sequeira

SHULT 2015

15/04/2023 LS-SHULT 2015 2

Learning outcomes

• Understand how formative MCQs can be used to promote deep learning in face-to-face and online learning

• Take a short formative MCQ quiz to see how it works from a participant’s perspective

• Begin to construct an online formative MCQ quiz

15/04/2023 LS-SHULT 2015 3

Deep approach v. surface approach

15/04/2023 LS-SHULT 2015 4

LearnersSurface approach Deep approach

Passive listeners Active participants

Understanding information only as it is presented

Trying to make linkages between information presented with other bits of (previously received) information

Processes and retains information as separate facts or bits of information

Looking for an underlying rationale/overarching structure to connect the (independent) facts

Focuses on breadth of content coverage Focuses on depth of content coverage

Assessment-oriented view of education Academically-inclined

Non-academic priorities or interests Intrinsic curiosity or a preference to engage with material at a conceptual level

Is expecting a predictable learning experience and not looking to be stretched

Is looking to be stimulated and challenged

15/04/2023 LS-SHULT 2015 5

TeachersSurface approach Deep approach

Presenting and expounding (explain) Eliciting (evoke) and probing

Learners as passive listeners Learners as active participants

Learner’s role: listen, take notes, ask the odd question

Learner’s role: active listening, discussion with peers/lecturer, query and counter-argue?

Breadth of coverage is a priority Depth of coverage is a priority

Views unwieldy questions as a distraction from lecture

Welcomes questions and views them as a teachable moment

Provides answers to students’ questions Facilitates students in seeking out the answers to their questions

15/04/2023 LS-SHULT 2015 6

The multiple choice question (MCQ) as we know it

• What is it?

• How is it currently used?• Why is it not a favoured form of assessment?

What is the most widely spoken language in the world?

i. Englishii. Chineseiii. Spanishiv. Mandarin

15/04/2023 LS-SHULT 2015 7

Can MCQs be used to promote deep learning?

15/04/2023 LS-SHULT 2015 8

Formative MCQs• Improve conceptual understanding• Target areas of potential confusion• Provide feedback• Prime learners• Activate prior knowledge• As a diagnostic test/JIT• In concert with other active learning methods –

peer instruction, discussion fora, etc.• Enable self-regulated and reflective learning

15/04/2023 LS-SHULT 2015 9

Anatomy of a formative MCQ

TopicQuestion set

DistractorsFeedback

Marking scheme

Integrated learning activity

15/04/2023 LS-SHULT 2015 10

Topic

• A key topic or fundamental concept• Lends itself to MCQ-style questions• Not likely to require frequent changes or updates

15/04/2023 LS-SHULT 2015 11

Stem and options

Which of the following statements are true of learners adopting a deep approach?i. They retain information and knowledge longerii. They are able to make more connections

between different pieces of informationiii. They experience confusion and uncertainty

when trying to understand new knowledgeiv. They focus on the details when mastering a

topic

stem

Correct options

distractor

15/04/2023 LS-SHULT 2015 12

Feedback

• Adaptive• Interactive• Deferred

15/04/2023 LS-SHULT 2015 13

Marking scheme

• Multiple correct answers• Negative marking• Certainty-based marking

15/04/2023 LS-SHULT 2015 14

Results of the self-assessment

15/04/2023 LS-SHULT 2015 15

Evaluating the effectiveness of MCQs

15/04/2023 LS-SHULT 2015 16

Trivia v. deep learning

15/04/2023 LS-SHULT 2015 17

Do MCQs promote deep learning?

• Biggs’ theory of learning• Deep/surface/strategic

• Bloom’s taxonomy• Knowledge, comprehension, application, analysis, synthesis,

evaluation

• Nicol’s principles of feedback• Clarifying goals, criteria and standards; self-assessment and

reflection, high-quality feedback; peer dialogue, feedback and motivation; closing the gap; just-in-time teaching

15/04/2023 LS-SHULT 2015 18

How can you increase the power of a formative MCQ?

15/04/2023 LS-SHULT 2015 19

Limitations of MCQs for deep learning

• The deep approach is not often used in everyday teaching• Difficulty finding an appropriate topic• Pedagogical design

• Learning aims• Integrating it with other learning activities

• Constructing the test• Item ambiguity• Technology

• Student expectations need to be managed

15/04/2023 LS-SHULT 2015 20

Can you use MCQs in your teaching?

• In which part of the curriculum could you use an MCQ?• What is the underlying rationale? • What would be the benefits for the students, for yourself?• How would it promote deep learning and better learning

outcomes?• Could you integrate it with other learning activities? • What potential challenges do you foresee with this

approach?

15/04/2023 LS-SHULT 2015 21

DIY

• Get into groups of three and design a formative MCQ quiz• Visit https://www.coursesites.com/s/_SHULT2015 and login

TopicQuestion set

DistractorsFeedback

Marking scheme

Integrated learning activity

15/04/2023 LS-SHULT 2015 22

Wrap up and review

15/04/2023 LS-SHULT 2015 23

References• Biggs, J. and Tang, C., 2007. Teaching for Quality Learning at University. 3rd ed. Berkshire: SRHE

and OUP• Bloom’s Taxonomy. Centre for Teaching. Vanderbilt University. [online] Available at:

http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ [Accessed 24 June 2014]• Lublin, J., 2003. Deep, surface and strategic approaches to learning. Good Practice in teaching

and Learning. Centre for Teaching and Learning, University College Dublin. [online] Available at: http://www2.warwick.ac.uk/services/ldc/development/pga/introtandl/resources/2a_deep_surfacestrategic_approaches_to_learning.pdf [Accessed 18 June 2014]

• Nicol, D., 2007. E assessment by design: using multiple choice tests to good effect. ‐ ‐ Journal of Further and Higher Education. 31:1, 53-64. Available at http://ewds.strath.ac.uk/REAP/public/Papers/MCQ_paperDN.pdf [Accessed on 30 May 2014