WRITING GREAT CLICKER QUESTIONS

Make Clickers Work for You

Dr. Stephanie V. Chasteen Physics Department

& Science Education InitiativeUniv. of Colorado at Boulder

http://colorado.edu/sei

Web and blog: http://sciencegeekgirl.com

Email: stephanie.chasteen@colorado.edu

What do you teach?

A. ScienceB. Engineering or MathC. Social sciencesD. HumanitiesE. Administration / faculty supportF. Other

Show of hands

Have you used response systems (clickers) in your teaching?

A. Not at all, and I haven’t seen them usedB. Not at all, but I’ve observed their use

somewhatC. I’ve used them a littleD. I’ve used them a lotE. I could be (should be?) giving this workshop

Take a clicker & turn it on If the green light flashes,

your vote has been counted

How familiar are you with Mazur’s “Peer Instruction”

A. Fairly familiar, and I like itB. Fairly familiar, but I’m not sure that I like itC. I’ve heard of it but only have a vague idea

what it isD. Not familiar at allE. Not sure

Colored cards

Introducing Me

Applying scientific principles to improve science education – What are students learning, and which instructional approaches improve learning?

Science Education Initiative

http://colorado.edu/SEI

Physics Education Research Group

One of largest PER groups in nation, studying technology, attitudes, classroom practice, & institutional change.

http://PER.colorado.edu

Blogger

http://blog.sciencegeekgirl.com

5

Learning goals

Clic

kers

& p

eer

inst

ruct

ion

Why question?

How many times have you given a lecture and found that students hadn’t followed you?

Can you rely on students to ask questions if they don’t understand something?

Can you rely on students to know if they don’t understand something?

What are the benefits of questioning?

whiteboardCredit: Rosie Piller

8

Agenda

1. When and how we can ask questions2. About clickers as a way to ask questions,

including some common challenges3. Writing good questions. Example

questions, writing our own.4. Action plan

10

Learning goals: Participants will be able to….A. Explain several benefits of questioning and of using clickers to

questionB. Defend the use of best practices in questioning to overcome

common challengesC. Formulate and revise clicker questions to target student

learning goals

Exercise #1: Question brainstorm

What questions could you ask to help students achieve your assigned learning goal -- to test mastery and stimulate learning?

Brainstorm as a group

whiteboard

11

5 minutes

When can we ask questions?

Credit: Rosie Piller and Ian Beatty.

12

BEFORESetting up instruction

MotivateDiscoverPredict outcomeProvoke thinkingAssess prior knowledge

DURINGDeveloping knowledge

Check knowledgeApplicationAnalysisEvaluationSynthesisExercise skillElicit misconception

AFTER Assessing learning

Relate to big pictureDemonstrate successReview or recapExit poll

Some methods of asking questions

Ask rhetoricallyTarget the class (how?)Target someone in particular (in what order?)Wait and then… (call on whom?)Answer your own questionLeave the question unanswered

Or ask out of classBlogsDiscussion boardsHomework…Credit: Rosie Piller

13

Why use clickers to target the class? An outline of Peer Instruction.

14

15

Ask Question

Peer Discussion

Vote

Class Discussion

…Lecture… (Maybe vote)

* See also: Peer Instruction, A User’s Manual. E. Mazur.

Anatomy of Peer Instruction

16

Note: Grading for Formative Assessment

Motivate students to participate, without stressing over the right answer

We recommend extra credit for:•Mostly participation (eg., 2 points)•Some for correctness (eg., 1 point)

A new research study (James & Willoughby, 2011) shows:Giving points for correctness creates less productive classroom conversations! See http://theactiveclass.com

Note: Timing / Groups

◦2-5 questions spaced through an hour

◦Discussion with peers (usually nearest neighbors)

17

18

Ask Question

Peer Discussion

Vote

Class Discussion

…Lecture… (Maybe vote)

* See also: Peer Instruction, A User’s Manual. E. Mazur.

Questions about this process?

19

But not a magic bullet!

Clickers are a tool for questioning

Peer instruction helps students learn

Research shows that:Students can better answer a similar

question after talking to their peersPeer discussion + instructor explanation

works better than either one aloneStudents like peer instruction, from intro to

the junior levelStudents in courses using peer instruction

outperform those in traditional lecture courses on a common test

21

See http://STEMclickers.colorado.edu for various references

Do you want to see the details of some of this

research?A. Yes, cut back on other

stuffB. No, let’s just get to the

question writingC. I’d like to see it in the

Q&A portion afterwards

How is a clicker question the same or different?*

Similar in terms of goalsMultiple choiceAnonymous (to peers)Every student has a voice –

the loud ones and the shy ones

Forced wait timeYou can withhold the answer

until everyone has had time to think (choose when to show the histogram)

26

* From other types of in-class questions

What does this tool help us to do?

U. Colorado clicker resources…27

Videos of effective use of clickers

http://STEMclickers.colorado.edu

Clicker resource page

http://STEMvideos.colorado.edu

2-5 mins long

• Instructor’s Guide• Question banks• Workshops• Literature / Articles

Which of these could be clicker questions?

Credit: Rosie Piller and Ian Beatty.

28

BEFORESetting up instruction

MotivateDiscoverPredict outcomeProvoke thinkingAssess prior knowledge

DURINGDeveloping knowledge

Check knowledgeApplicationAnalysisEvaluationSynthesisExercise skillElicit misconception

AFTER Assessing learning

Relate to big pictureDemonstrate successReview or recapExit poll

Let’s try it.

A. Writing good questionsB. Getting students to engage with the

questionsC. Getting students to share their answers with

the whole class / the same students always share

D. It takes too long for me to learn to do thisE. I have a lot of content to cover, it takes too

much class time

I think that the toughest thing about using clickers and peer instruction must be:

30

A science-related example…

Which superpower would you rather have? The ability to…

A. Change the mass of thingsB. Change the charge of thingsC. Change the magnetization of thingsD. Change the boiling point of things

Question: Ian Beatty, UNC Greensboro Image: Thibault fr on Wikimedia

30

31

Example question: Math

Your sister in law calls to say that she’s having twins. Which of the following is the most likely? (Assume she’s having fraternal, not identical, twins)

A.Twin boysB.Twin girlsC.One girl and one boyD.All are equally likely

Derek Bruff, Vanderbilt

Example Question: Survey

Which of the following are you least comfortable using to solve problems?

A. KinematicsB. Newton’s LawsC. Work-Energy TheoremD. Momentum-Impulse TheoremE. Angular Momentum-Angular Impulse Theorem

32

Ian Beatty, UMass Amherst

Two things to pay attention to in your questions

What is the goal of my question? What am I trying to accomplish?

Is my question at the right level / variety of depth?

Question goals

Credit: Rosie Piller and Ian Beatty.

34

BEFORESetting up instruction

MotivateDiscoverPredict outcomeProvoke thinkingAssess prior knowledge

DURINGDeveloping knowledge

Check knowledgeApplicationAnalysisEvaluationSynthesisExercise skillElicit misconception

AFTER Assessing learning

Relate to big pictureDemonstrate successReview or recapExit poll

COLTT CU 2009

Completely useless

Mostly useless

Somewhat useful

Useful

Very useful

Challenging conceptual

Recalling a previous fact

Recalling a recent fact

Plugging numbers into equation

Types of clicker questions:

0% 10% 20%

30% 40% 50% 60%% of students

91%

35%36%18%

N=4 courses, 66 students

Question Writing Depth

36

But how do we increase the depth of questioning?

Question Writing Depth: Bloom’s Taxonomy

Handout with handy verbs and question stems for different levels, e.g.: UNDERSTAND: match,

paraphrase, restate APPLY: choose, explain, show ANALYZE: compare, classify,

categorize EVALUATE: judge, criticize,

defend SYNTHESIS: combine,

develop, design

37

Handout: Bloom’s Taxonomy

Preparing to Write Questions

Read briefly over the “tips for writing clicker questions” handout.

Which is going to be most challenging for you?Which would you tell a colleague about?

3 minutes

Preparing to Write Questions

In groups of 2-3, choose one of the questions that you brainstormed at the beginning of the workshop. You will write a multiple choice version of this question.

3 minutes

Gallery Walk

As a table, look at the “example questions” trio that I have given you. What’s a common theme(s)?

Write the themes you find down on the sheet so that other groups will be able to read it.

After 5 minutes, circulate to see the themes of questions on other tables.Shop for ideas for your own questions!See handouts for a place to jot your notes.

10 minutes

Gallery Walk: Report Out

What was the theme of your question trio?

When would you use such a type of question?

Exercise #3: Writing Questions42

Using ideas you’ve learned, write a multiple choice version of your question in groups of 2-3.

Show your question to another group (and to me) for suggestions on revising it.

If you have time, write another question from another part of the questioning cycle.

10 minutes

Share-Out about Question Writing

What was challenging?

What worked well for you?

What questions or concerns do you have about writing questions?

How might you write questions that integrate with your lectures?

This workshop can’t do it all

There are great books to readPair up with other instructorsI give free webinars (see iclicker.com)Next workshop, 4:30-6:00, Weds Feb 1st. (4:00-4:30

refresher course for new folks)

44

Making Clickers Work for You: Facilitation

I.e., taking off the rose-colored glasses. What goes wrong? How can this technique work best? BRING YOUR HANDOUTS!

Action Plan

Take a few minutes to write down your action plan to implement ideas you heard about in the workshop

45

Many materials in this workshop (particularly the questioning cycle and the participant

exercises) were adapted from Rosie Piller, Making Students Think: The Art of Questioning. Short papers published in: Computer Training & Support Conference, 1995; ISPI International Conferences, 1991 and 1996; ASTD National Conference on Technical & Skills Training, 1990. Related workshop description at http://www.educationexperts.net/nstworkshop.html. Other materials (particularly

sample clicker questions and goals of clicker questions) adapted from Ian Beatty’s Technology Enhanced Formative Assessment (TEFA) program. http://ianbeatty.com/crs

Resource Page: http://STEMclickers.colorado.eduWeb and blog: http://sciencegeekgirl.comEmail: stephanie@sciencegeekgirl.com

Thanks!

NOW: Q&A, continued work on questions and revision, individual consultations.

NEXT WEEK, 4:30-6pm – Facilitation Tips & Techniques

Learning Goals

Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell

Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected).

Chemistry: Explain trends in boiling points in terms of intermolecular interactions

Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle.

Math: Solve a system of linear equations in two variables using algebra or graphing.

What Do I do if…?

What can you do if you ask questions and..There is no responseThe same people keep raising their handsThe answers are called out before everyone

has a chance to thinkThe answers take too longSomeone gives a wrong answerOnly some students are prepared?

48

We’ll discuss in Workshop #2.

For now: Many of these challenges are addressed by clickers