Using Online Quizzing to Improve Student Preparedness and

Learning Outcomes

John BroidaPsychology Department

University of Southern MaineBroida@usm.maine.edu

Topics I hope to cover• Why Quiz?• What is web-based (on-line) quizzing? Why quiz on-line?• When to quiz? • How many questions should be on a quiz?• How many points should each quiz be worth?• How many times should students be allowed to take a quiz?• How to prevent/reduce cheating on quizzes?• How much advance warning should students get before a

quiz?• How much time should students get to answer each

question? Should quizzes have time limits?

Why Quiz? Part 1• The simple answer: It helps students learn the material.– Quizzing has been used for many years, because it works– The average student does not do optional work, has other ways

to spend their time• Quizzes increase the time students spend studying– Students who expect a quiz may more than double the time

spent preparing for class– One result is that students come to class with more

understanding of the material• Quizzes spread study out through the entire semester

(reduces reliance on cramming)– Cramming improves performance on exams, but it does little to

foster long-term retention

Why Quiz, part 2

• Quizzes help those who are struggling, and reward those who understand the material– For students, they provide feedback about what they know

and do not understand– For instructors, who does not know (needs help), what

should be covered in class.• Quizzes have a down side.– They take time to create, to deliver, to grade.– Web-based quizzing deals with these issues very effectively

and efficiently.

What is web-based (on-line) quizzing? Why quiz on-line? Part 1

• In-class quizzes usually require students to answer the same questions at the same time.– In class quizzes are an evaluation, a “got ya”– In class quizzes are very time consuming, to administer and grade– This explains why many instructors do not use this tool

• Though it may seem obvious, web-based quizzing is quizzing done on-line, not in the classroom.– This means that class time can be devoted to something other

than the quiz.– Also, some, if not all, questions can be computer scored, reducing

faculty workload– And many publishers provide on-line quizzes with their texts,

further reducing workload

What is web-based (on-line) quizzing? Why quiz on-line? Part 2

• Web-based quizzing is a recent improvement over in-class quizzes.– Web-based quizzing enables quizzes to become “formative”• Multiple attempts facilitate learning/mastery

– As with in-classes, faculty can see who does not know, what should be covered in class

• Note, web-based quizzes can also be an evaluation, if students can take them only once.

• The drawback of web-based quizzing is you never know who is actually answering the questions– Are students relying on their text (control this by setting a time

limit)– Do they have help while they are taking the quiz?– As I use quizzing, the goal is to get students to read the text

When to quiz? Part 1• The most effective quizzing occurs before you talk about the material in class.

– Students get some understanding as the result of reading– Practiced teachers find that they eliminate between 30 and 50% of their

standard lecture,. Because the students already understand it.– We find that more students ask questions in class when they understand the

topic– They are more prone to ask about implication and analysis, not definition!

• I personally like to quiz students once a week, given I cover about 1 chapter each week. – The quiz is due an hour before the first class covering the chapter– If you are using a modular textbook, you may want to quiz students more

frequently – If you teach more than one chapter in a week, think about combining chapter

quizzes– That said, some colleagues like to offer online quizzes two or three times per

week.

When to quiz? Part 2• The best advice is, do what feels comfortable to you and

best matches your course organization.– I am not aware of research that suggests there is an optimal

quizzing frequency • The other best advice is to make quiz deadlines consistent—

– For example, all quizzes are due at 2 PM on Tuesday• Also, it is a good idea to have a consistent number of quizzes

each week.– Having 1 quiz this week, 2 quizzes the next and 3 two weeks later

is confusing• Communicate your expectations for quizzing clearly

How many questions should be on a quiz? Part 1

• There are a few general best practices but no absolute rules.– Consider my recommendations in the light of your own course

needs and preferences. • I typically use 20 questions

– I’ve tried 10, 25, and 30 but have found 20 to be most effective. – I have colleagues who use 10, 15, 20, 25 and 30. – Asking around, I have discovered that 20 is certainly the modal

value.• In part, the number of items on a quiz often is based on how

many topics are covered– Most chapters in the social sciences have fewer than 20 topics– The idea is to ask a question on each topic on every quiz, more or

less

How many questions should be on a quiz? Part 2

• Longer quizzes (with 30-50 questions) feel more like tests – They introduce compliance challenges.. – Data suggests that some students taking long quizzes respond

randomly towards the end.• Why not shorter quizzes?

– 5-10 questions is generally not representative enough for a full chapter quiz.

– Fewer items may work for a modular book.• I believe it important not to ask multiple questions on a

single topic while ignoring other topics. – Quizzes set up by Worth are designed such that there is about

1 question per topic

How many points should each quiz be worth? Part 1

• This really depends on you and your course structure – I can certainly talk what several early adopters have found

• Use a round number (10 or 20 rather than 17.436 points per quiz)– Fewer people use 13, or 7, because it makes it hard to

determine the percentage.• The number of points available on each relates directly to your

overall points in the course. – Some people use this as the determining factor in how many

points quizzes are worth.• Make sure that each quiz is worth the same number of points– The literature say students are confused when some quizzes

are worth more than others

How many points should each quiz be worth? Part 2

• The literature suggests that quizzes be “low stakes,” meaning than any 1 is not worth a lot– This reduces anxiety associated with the quiz without reducing

compliance – Missing a quiz (due to illness perhaps) is not disastrous to your

grade nor is failing a quiz.• Each quiz may not be worth much, – Multiplied by the number of quizzes, the total percentage of a

grade can be considerable. – An eighth to a third is typically used, but some use 100%.

How many points should each quiz be worth? Part 3

• In my intro class, I have 1000 points available, 300 from quizzes. – A colleague allots 50% of the course grade to quizzing – Another colleague has reported that students work hard for ¼

point. – His class had something like 10,000 possible points.– Students hear the word points

• To sum up:– make each quiz a low points activity, – quiz often and consistently throughout the term, – and determine what works best for quizzing vis a via

your overall course grade.

How many times should students be allowed to take a quiz? Part 1

• Exams grade what students know, quizzes assess what students need to know and can still learn. – Quizzes can be a learning tool and formative assessment tool, – They can also be a summative assessment, but this is not considered as

helpful• Quizzes are an ideal way to get students to read and study more

regularly.– As a result, I allow more attempts than people use quizzes as

summative assessments.• Other colleagues limit the number of possible attempts to 3-5

times. – Some faculty suggest that limiting the number of attempts focuses

student study – They know that they must answer all the items correctly soon

How many times should students be allowed to take a quiz? Part 2

• Offering multiple attempts is best executed with randomized quiz items. – Assume that you have a pool of 100 items. – The machine randomly picks, for example, 20 for the quiz.

• Thus, students do not see the same questions repeatedly

• They learn to focus on the topic, not the question.– In general, the more often unprepared students take quizzes, the

more concepts they master. – Better prepared students don’t need as many attempts to achieve

perfection– The better students often continue to take quizzes, because they

find them helpful

How many times should students be allowed to take a quiz? Part 3

• If students can take a quiz multiple times, think about how to credit their attempts. – There are several options – First quiz encourages student preparation, eliminates any

incentive to retake the quiz.– Average of all quizzes taken also encourages preparation,

but grade may go down! – Most recent quiz does not encourage preparation, and

again grade may go down– Most faculty use highest grade. No chance a student score

can go down.

How to prevent/reduce cheating on quizzes?

• Cheating is typically defined as having someone else do the work– Perhaps using the text while taking the quiz or having someone else do the work

• I do not see using the book as a problem while taking a quiz– The purpose of my quizzing is to get students to read the book.– If you want to limit reliance on the text, limit the amount of time to complete the

quiz• How to stop someone else from doing the work

– If you have many quizzes, few people outside of the class itself are willing to do the work

– And, if you make the questions hard, even advanced students will have a problem• Another tactic is to put quiz questions on the exam!

– Tell the students you are doing this – they will want to take the quizzes– At least 50% of the items on my exams come from the quizzes.

How much advance warning should students get before a quiz? Part 1

• Pop quizzes are helpful if you want students to study, but do not want to grade quizzes.– The threat of a quiz increases the likelihood that students will study, like quizzing

• There is, however, some data suggesting that they study less than when quizzed– If one uses quizzes to evaluate students, then giving little advance warning makes

sense– Those who do not prepare do not do well…– This makes no sense if quizzes are used as a study guide, to provide feedback

• My thinking is that there is no reason to use pop quizzes if you administer quizzes online– The machine does the work (creating the quizzes, scoring them, entering the

grades)– You need to set up the parameters and select the topics and/or questions– Thus, I schedule 1 quiz a week, all are available from the start of the semester– Many people make the quizzes due for a short time, a week or two before lecture.

How much advance warning should students get before a quiz? Part 2

• If you want to evaluate students, you should decide how much advance warning to give– If 24-48 hours makes sense to you, or a week… – Be consistent in your announcements, so that

students know what to expect.• My advice is that you decide what you want to

do and do it– Whatever makes sense given your purpose in

these quizzes.

How much time should students get to answer each question? Should quizzes have time limits?• There are two schools of thought on this one

– Limit time, to get at what students know.• I have a colleague who limits time to 1 minute per question. • Another colleague limits the time to 45 seconds per question,• and yet another 75 seconds.• The less time students have, the more anxious they are• Especially when they have a limited number of attempts.• The more time they have, the more likely they are to look up

answers• Limiting time creates the feeling of an assessment, an evaluation.

– Do not limit time, to get at what students can find out.• Makes sense if you see quizzing as a formative assessment• Encourages students to study more, • Enables students to identify their weak areas of knowledge

How much time should students get to answer each question? Should quizzes have time

limits? -2• I set very high time limits – 1 hour for 20 questions

– I want students to look up answers, to read and find the answers, with minimal stress.

– Quizzes are seen as a study guide, not an assessment.– My main goals are to get students to read the book and

come to class better prepared; – Extra time reduces the aversiveness of the task

• I do not recommend setting up quizzes with no time limits.– Students will start the quiz, and complete it after the

deadline!

Do not hesitate to contact me

• If you think that I can help you help your students.

• John Broida 207 780 4255• broida@usm.maine.edu