New Research in the Pedagogy of Learning with Clickers

Stefanie Mollborn

Angel Hoekstra

University of Colorado at Boulder

Acknowledgements

We thank the President’s Teaching and Learning Collaborative at the University of Colorado for their support of our research.

Learning goals

Goals-driven pedagogy What do you want your students to

have learned after taking one of your courses?

Brainstorm

Learning goals in sociology

Attending class and participating Understanding/applying concepts and

theories Critically evaluating concepts and

theories Using them to understand the life

experiences of self and others Other goals in other disciplines

How to make students’ learning “stick”?

Problems with retention of material in traditional lectures, listening fatigue

Active learning as a solution Practical issues with getting students

to attend, keep up with readings, and participate in large classes

“Clickers” (audience response systems)

Increasingly popular as a way to increase active learning, monitor attendance, motivate reading

Audio and visual stimulation Emerged in the natural sciences Flexible pedagogical tool, but they

are not themselves a pedagogical strategy—you have to do that

Sample question

Have you taught using clickers in the past?

A. No

B. Yes, and I’ve used i>Clickers

C. Yes, and I’ve only used other types of clickers (e.g. infrared)

D. Don’t know/other

Sample question

If you are considering trying clickers in your classroom, what is your most important concern about using them?

A. I am already using clickers or don’t want to use them

B. I’m worried about technical issuesC. I’m worried about how to use them to

achieve my goals for the classD. I’m worried that they will make my

preparations too time-consumingE. Other/don’t know

Prevalent clicker strategy

ConcepTests (Mazur, 1997) to encourage problem-based learning

Proven pedagogical strategy Usually includes points for correct

answers Points for trying can also function as

attendance tracking

What would happen to the seasons if the earth’s tilt was increased from 23 degrees to 40 degrees?

A. Seasons would be less extreme.

B. Seasons would be pretty much the same as now

C. Seasons would become much more extreme.

D. None of the above

What would happen to the seasons if the earth’s tilt was increased from 23 degrees to 40 degrees?

A. Seasons would be less extreme.

B. Seasons would be pretty much the same as now.

C. Seasons would become much more extreme.

D. None of the above

Amending clicker use to address other learning goals

Moving beyond ConcepTest paradigm of understanding and applying ideas

Encouraging critical thinking Students as data points: past

experience of self and others Encouraging reading comprehension

and participation in whole-class discussion

Reading quiz question

What gender combination leads to the lowest likelihood of negotiating, as well as a poor evaluation if the candidate does negotiate?

A. Female evaluator-female candidateB. Female evaluator-male candidateC. Male evaluator-female candidateD. Male evaluator-male candidateE. There were no differences

Concept test

Does the “sex labeling” of occupations affect supply-side gender segregation, demand-side gender segregation, or both?

A. Supply side

B. Demand side

C. Both

D. Neither

E. Don’t know/other

Critical thinking question

How much do you think cultural factors explain the differences in violent behaviors that we see between men and women?

A. Not much at all

B. A little

C. They are sometimes useful

D. They explain most of what we see

E. Don’t know/other

Critical thinking question

How well do you think England’s findings match your own and/or your friends’ sexual experiences in college?

A. I’m female: Pretty/very wellB. I’m female: Not very/not at all wellC. I’m male: Pretty/very wellD. I’m male: Not very/not at all wellE. Don’t know/don’t want to answer

Past experience question

When you were growing up, which of your parents earned the most money?

A. Don’t have 2 parents/one or both don’t work/varied from year to year

B. Dad usually earned a lot more

C. Dad usually earned a little more

D. Mom usually earned a little more

E. Mom usually earned a lot more

Demographic question

What is your race?

A. White

B. Black or African American

C. American Indian or Alaska Native

D. Asian or Pacific Islander

E. Other or multiple categories

2000 U.S. Census data

0

10

20

30

40

50

60

70

80

White Black AIAN API Other

% o

f p

op

ula

tio

n

Comprehension question

How well do you feel you understand the reading assignment due today?

A. Very well—don’t go over it more

B. Pretty well, but wouldn’t mind more

C. Just OK—I’d like more review

D. I’m pretty lost and need help

E. I didn’t read the assignment

Students’ own questions

Can be an assignment or an open invitation

Submitted “on the fly” or ahead of time Learning to design hypotheses, figure

out what empirical information would test them, design an instrument to collect that information

Putting it all together

10-minute chunks of lecture Questions with small-group discussions Show histogram, then large-group

discussions Other active learning activities:

chalkboard lists, small group work Questions form transitions between

mini-lectures

Our study

How do students perceive that this combination of clicker techniques affects their class experiences and learning?

6 sections of 3 sociology courses 385 surveys, 456 free writes, 10

interviews, 100 pages of field notes Angel has even more data from other

disciplines

General findings

Most students have past experience using clickers, many already own one

A majority approve of this pedagogical strategy of using clicker questions for diverse purposes

Strong positive effect on attendance, but some backlash

Complicated effect on motivating students to keep up with readings

General findings

All of the students interviewed said that if they were the instructor, they would keep using clickers this way

Findings on question types

Love/hate relationship with reading quiz questions

Most find other question types (concept tests, critical thinking, and past experience questions) to be useful

Many enjoy getting immediate feedback on their own comprehension of readings and others’ opinions

Student quotes

“I think the clickers are very helpful in helping me to understand the material. I like how they stop and force me to think about the material that has just been presented to me, instead of just writing aimlessly throughout class.”

Student quotes

“I enjoy the clickers. It saves time and helps [with] participation. For some students, speaking or being called on in class is uncomfortable … Clickers enable every student to participate equally without being singled out.”

Student quotes

“It is cool to see your own experience in relation to the rest of the class. It allows us to gauge the differences between each other and opens our eyes.” (e.g., gender questions)

Other question types to address other learning goals?

Brainstorm

Important issues to consider when using clickers, based on student assessments

Classroom environment

Importance of explaining what you’re doing and why

Effects of a right/wrong answer vs. no right answer focus

Importance of encouraging discussions

Confidentiality and ethics

Benefits of asking confidential questions

Effects in the classroom What you can and can’t promise Ethical boundaries on what to ask

Conclusions

Clickers are a pedagogical tool, and it’s up to you to find a good pedagogical strategy

A variety of question types can address different learning goals

Can make students feel like they’re engaged and learning in large lectures

Thank you! Questions?

Stefanie Mollborn

mollborn@colorado.edu

Angel Hoekstra

angel.hoekstra@colorado.edu

Slides and handout will be posted at:

http://www.colorado.edu/ibs/hb/mollborn