The Monte Carlo Quiz: Encouraging Punctual Completion and Deep Processing of AssignedReadingsAuthor(s): Peter S. FernaldSource: College Teaching, Vol. 52, No. 3 (Summer, 2004), pp. 95-99Published by: Taylor & Francis, Ltd.Stable URL: http://www.jstor.org/stable/27559190 .

Accessed: 10/10/2014 16:17

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.

.

Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to CollegeTeaching.

http://www.jstor.org

This content downloaded from 132.178.2.65 on Fri, 10 Oct 2014 16:17:39 PMAll use subject to JSTOR Terms and Conditions

THE MONTE CARLO QUIZ

ENCOURAGING PUNCTUAL COMPLETION AND DEEP PROCESSING OF

ASSIGNED READINGS

Peter S. Fernald

Abstract. The Monte Carlo Quiz (MCQ), a single item quiz, is so named because chance, with the roll of a die, determines (a) whether the quiz is administered;

(b) the specific article, chapter, or section of the

assigned reading that the quiz covers; and (c) the par ticular question that makes up the quiz. The MCQ

encourages both punctual completion and deep pro

cessing of assigned readings and is easy to implement. It is readily designed to address a wide variety of

learning objectives. Students' quiz scores and evalua

tions suggest that the MCQ is effective.

When

a student reads and studies

is important. Common practice

is to postpone assigned readings until the

day or night before an exam (Burchfield and Sappington 2000). This loaf-and

cram pattern, which runs counter to most

instructors' wishes and expectations, has

increased at an alarming rate. At one uni

versity, reading compliance, as measured

by passing a one-time surprise quiz,

decreased from more than 80 percent in

1981 to approximately 20 percent in 1997

(Burchfield and Sappington 2000). How

ever, the reading compliance rate is great

ly enhanced when quizzes are adminis

Peter S. Fernald is a professor of psychology at the

University of New Hampshire in Durham.

tered randomly and periodically through out the semester (Ruscio 2001).

How a student reads and studies also

is important. Ideally, the student studies

carefully and processes deeply the

assigned reading. Such studying and

processing enhances both understanding and retention. Superficial studying and

shallow or rote processing, on the other

hand, typically produce little, if any, sig nificant learning. Organizing informa

tion hierarchically (Bower 1970), ana

lyzing meanings (Craik and Tulving 1975), and applying concepts (Palmere

et al. 1983) are processes known to

enhance memory of subject matter.

These findings suggest that instructors

should encourage students to engage in

such processing.

Monte Carlo Quiz

With these thoughts in mind, I devel

oped the Monte Carlo Quiz (MCQ). The

city of Monte Carlo is noted for its gam

bling casinos, and the MCQ is so named because chance (that is, the rolling of a

die) is an important feature of the quiz. Before the semester begins, I prepare several quiz items such as the following:

1. Knowledge: Describe the major thesis, the central idea or set of ideas, in the

reading. Make certain that the thesis

you identify is primary. Also include one or two closely related ancillary or

secondary ideas or theses, clearly

identifying them as such.

2. Comparison: Identify two concepts or

principles presented in the chapter or

article and, when you first mention

each, underline and define it. Then,

show how the concepts or principles in

some way(s) are both similar to and

different from one another. If you

wish, one of the concepts or principles may be selected from another reading,

lecture, or discussion in this course.

3. Application: Select a concept or princi

ple in the chapter or article, clearly

define or describe it, and then indicate

how it applies to you or someone you

know. Provide sufficient details to justi

fy convincingly that the concept or

principle indeed applies as you suggest.

4. Critique: Write a critical perspective on some aspect of the chapter or arti

Vol. 52/No. 3 95

This content downloaded from 132.178.2.65 on Fri, 10 Oct 2014 16:17:39 PMAll use subject to JSTOR Terms and Conditions

ele, citing evidence that prompts you to

agree or disagree with the author's per

spective. Note that a critique may be

positive, negative, or some combina

tion of both. Your evidence may be

based on (1) personal experience, (2) observations of others, (3) reports of

others, (4) scientific findings, or (5)

logic. When citing evidence, identify the type(s) of evidence you are using.

5. Passion: Citing page number(s), quote verbatim a statement or brief passage

that elicits in you some type of emo

tional response: excitement, frustra

tion, pleasure, anger, sadness, surprise,

confusion, fear, delight, some combi

nation of the aforementioned, or what

ever. Then identify your emotional

response, describe the meaning(s) that

the statement or passage has for you,

and provide actual or possible reasons

for your response.

6. Student's Choice: Answer any of the

above five questions.

Items 1 to 4 are based on learning

objectives described by Bloom (1956). Item 5 is included to enhance students'

emotional engagement in the assigned

readings.

At each class meeting a student nomi

nated by the class rolls the die up to three

times. On the first roll, an odd number

indicates "no quiz" and an even number

dictates that a quiz will be administered.

The second roll of the die determines the

particular assigned reading (for example,

chapter or article). The third roll indicates

the particular question to be answered.

Instructions included in the syllabus are

as follows:

A quiz may or may not be administered at

each class meeting. Whether or not a quiz is

administered, the chapter or article on

which the quiz is based and the particular

question to be answered typically will be

determined by chance (that is, roll of the

die) and occasionally by decree (my deci

sion). The quiz is not open book. Therefore,

you may not refer directly to the assigned

reading. However, you may bring notes to

class and refer to them while taking the

quiz. Limit your answers to one paragraph or no more than eight sentences. For ques tion 5, be sure to include in your notes a

quotation with page numbers.

I spend one or two minutes reading,

commenting on, and grading each answer.

After experimenting with several grading

systems, I settled on the following four

category system:

E (excellent): Demonstrating sound under

standing of the selected concept(s), the

answer is clearly stated and accurate.

G (good): The answer is mostly or essen

tially correct but incomplete or

unclearly stated, as illustrated by the

following examples: a secondary thesis

is not identified (question 1); a concept is not clearly defined or a similarity is

either inaccurate or not identified

(question 2); an example does not pre

cisely fit the concept (question 3); type of evidence is not clearly identified

(question 4); an emotional response is

implicit but not clearly specified or the

quotation is not verbatim or has no

page reference (question 5).

S (satisfactory): The answer is incorrect,

yet it demonstrates some familiarity with and understanding of some por tion of the assigned reading.

U (unsatisfactory): The answer is incorrect

and demonstrates no familiarity with or

understanding of the assigned reading.

A (absent): Student was absent from class.

The grades of E, C, S, and U are equiv alent to respective percentile grades of

95, 85, 75, and 65. Intergrader reliability

(agreement between two graders), based on fifty quiz answers (ten answers for

each of the five questions), was 78 per cent. In only one instance did the graders

disagree by more than one grade level.

Method

During the fall 2001 and spring 2002

semesters, I evaluated the MCQ in two

courses, Counseling and Externship. The

two courses included upper-level psy

chology majors (70 percent women and

30 percent men) at the University of New

Hampshire and met weekly for three

hours. Performance on the quizzes count

ed for 25 percent of the final grade in

Counseling and 10 percent in Externship. At the end of the spring semester, I

administered a questionnaire in both courses (Counseling, N = \6; Externship,

N = 11). Punctual completion of reading and

deep processing were assessed using stu

dent quiz grades. The statistical index for

punctual completion was the percentage of quiz answers receiving grades of E, G, or S; the index for deep processing was

the percentage receiving the grade of E.

The questionnaire included eight rating items (table 1). For each item, students

circled one of six numbers (ratings) rang

ing from 1 (very strongly disagree) to 6

TABLE 1. Means and Standard Deviations of Students' Ratings of the MCQ

Externships Counseling

Item M SD M SD

1. The MCQ motivated me to complete the assigned

readings on time. 5.35 .85 4.76 .84

2. My ability to contribute to discussions of the assigned

readings was enhanced by the MCQ. 4.69 1.28 4.19 .86

3. If I had to choose between the MCQ and taking a quiz at every class meeting, I would choose the MCQ. 5.72 .76 4.82 1.47

4. As employed in this questionnaire, the acronym MCQ refers to multiple-choice questionnaire. 2.02 1.67 1.33 .77

5. I took notes on the assigned readings. 5.37 .93 4.87 .94

6. I recommend that the MCQ be used in future offerings of the course. 5.48 .84 4.26 1.20

7. When I come to class prepared to take a quiz, I some

times am disappointed when the roll of the die

indicates there will be no quiz. 2.11 1.43 2.82 1.77

8. The possibility of being asked five different questions

encouraged me to read carefully and take notes on the

assigned chapters/papers. 4.62 1.03 3.96 1.23

Note: Ratings were made on 6-point scales ( 1 = very strongly disagree, 6 = very strongly agree).

96 COLLEGE TEACHING

This content downloaded from 132.178.2.65 on Fri, 10 Oct 2014 16:17:39 PMAll use subject to JSTOR Terms and Conditions

(very strongly agree). Students indicated

their reasons for their ratings in a space

provided immediately below each item.

Item 4 was included to detect possible

positive response bias.

Results

Table 2 contains the frequencies for

quiz answers receiving each grade and

absence from class. The percentage of

answers meeting the criteria for punctual

ity and deep processing were 95.8 and

59.8, respectively.

Table 1 shows means and standard devi

ations for students' ratings of the MCQ.

Ratings higher than the midpoint (3.5)

suggest favorable attitudes. Except for

item 4, the control item, and item 7, which concerns disappointment when no quiz

was given, the average rating for each item

was above 3.5.1 examined ratings provid

ed by students who did not give a rating of 1 for item 4, and I did not observe positive response bias for any of these students.

Discussion

I consider here five topics relevant to

the MCQ: empirical findings, learning

objectives and memory, implementation,

adjustments in teaching and learning, and

design and flexibility.

Empirical Findings

As indicated by the high percentage

(95.8 percent) of quiz answers receiving a

grade of S or higher, as well as the ratings on item 1, students completed most of the

assigned readings for the prescribed dates. This outcome compares most

favorably with the 20 percent compliance rate for completing assigned readings

when no quizzes are administered

(Burchfield and Sappington 2000). It also

is consistent with the observation that stu

dents motivated by randomly adminis

tered quizzes complete assigned readings at impressive rates (Ruscio 2001).

I did not include a control group receiv

ing no MCQ, and there are no relevant

TABLE 2. Frequencies of Quiz Grades

Quiz grades

Course E G S U A Total

Counseling'01 33 11 17 4 5 70

Counseling'02 35 10 12 3 8 68

Externship'01 11 5 4 1 1 22

Externship'02 36 8 2 0 5 51 Total 115 34 35 8 19 211

Note: E = excellent, G = good, S =

satisfactory, U = unsatisfactory, A = absent from class.

TABLE 3. Selected Statements about the MCQ

The MCQ put excitement into the class.

I liked the MCQ. It insured that what I read stuck in my head.

Without the MCQ I would have fallen behind in the reading. The MCQ encouraged reading, note-taking, and critical thinking. The random nature of the MCQ provided motivation always to do the reading. There always was a chance for taking a quiz, which insured that I did the reading. Yet sometimes there was no quiz, which, although it was random, gave me a feeling of control.

Even though we had only one quiz, I completed all of the assigned readings, took

notes on them, and was always prepared for a quiz. The MCQ is a great way to make sure everyone does the assigned reading. I liked the MCQ. It forced me to do the reading, yet occasionally it provided a break

from taking a quiz. The process of having a quiz or not was unique and smart. It ensured that we did the

required reading with understanding.

baseline data for deep processing of

assigned reading in college courses. It is

impossible, therefore, to determine the

absolute level of the effects of the MCQ on deep processing. However, more than

half (59.8 percent) of students' quiz answers received a grade of E, a perfor

mance level suggestive of something more than rote or superficial processing.

The students indicated that they took

notes on (item 5) and carefully read (item

9) the assigned readings. By themselves, these activities do not constitute deep pro

cessing. However, one reasonably might

expect that the task of preparing notes that

adequately address the five quiz questions would foster some measure of deep pro

cessing. The data are suggestive, yet

inconclusive, that the MCQ encourages information processing.

Students preferred the MCQ to quizzes administered at every class meeting (item

3); they were not disappointed when they

prepared for a quiz and there was none

(item 7); and they strongly recommended

that I include the MCQ in future course

offerings (item 6). Nearly all of the stu

dents' written comments about the MCQ were positive. A selected sample of their

comments is shown in table 3.

Learning Objectives and Memory

Different types of processing known to

enhance information retention are rele

vant for each quiz question included in

the study. The knowledge question

encourages students to identify primary and secondary ideas. Such hierarchical

organization enhances memory (Bower

1970). The comparison question requires students to show how two concepts are

both similar to and different from one

another; it encourages students to grapple

with precise meanings. Information pro

cessing that involves the analysis of

meanings enhances memory (Craik and

Tulving 1975). The application question

promotes the use of examples that

enhance memory (Palmere et al. 1983).

Requiring students to elaborate condi

tions prompting them to agree or disagree with the author, the critique question involves some measure of each principle

already mentioned: organization, analysis

of meanings, self-referencing, and elabo

ration with examples. Elaboration

improves both comprehension and memo

Vol. 52/No. 3 97

This content downloaded from 132.178.2.65 on Fri, 10 Oct 2014 16:17:39 PMAll use subject to JSTOR Terms and Conditions

ry (Bradshaw and Anderson 1982). The

passion question requires self-referencing

and involves affect, both of which

enhance memory (Rogers, Kuiper, and

Kirker 1977). Preparing outlines of

assigned readings, a practice required by the MCQ, can enhance retention

(McDaniel, Waddill, and Shakesby 1996).

Finally, research on massed versus distrib

uted practice suggests that knowledge

acquired over many small regular study intervals (for example, weekly quizzes) is

retained more readily than knowledge

acquired in one study interval (such as for a final exam only) or a few very large intervals (Dempster 1988; Glenberg 1992;

Leeming 2002; Payne and Wenger 1996). The MCQ clearly incorporates proven

principles of cognitive processing.

Empirical support for MCQ-prompted

deep processing, however, has yet to be

demonstrated. Future studies should

directly assess the types and levels of

processing.

Implementation

Students initially are quite anxious

about the MCQ. To ease them into the

procedure, at the second class meeting I

announce that I will administer a quiz, that the quiz will consist of question 5

(passion), and that the die will be rolled

only to determine the reading on which

the quiz is based. Once the students have

completed their answers, I inform them

that the quiz was only for practice. I then

invite them to read their answers to the

class. Most students do so. Sharing answers seems to enhance their ease with

both the MCQ and one another. At the third class meeting, I repeat the

procedure, except that I have students

answer question 2 (comparison), which I

believe is the most challenging of the five

questions. With some assistance from me,

students are able to grade accurately their

quiz performance and both identify and understand any difficulties they experi enced in answering the question. Imple

menting the MCQ with the two practice quizzes, I discovered, gets most students

off to a favorable start with the MCQ. Even after two practice quizzes, how

ever, some students experience difficulty

with the MCQ. Below her quiz answer, one student wrote, "I spent a lot of time

reading and taking notes. I still do poorly.

I hate the MCQ." Covering ten entire

large-size lined pages, the student's notes

were comprehensive and detailed. How

ever, her note-taking was rote. When it

was time to write an answer to the

chance-determined question, she did not

know where to begin, and she complained about not having enough time. I encour

aged her to take fewer notes and to direct

her note-taking toward answering the five

questions. I pointed out that questions 2

through 5 permitted (and even encour

aged) her to direct her reading, studying, and note-taking efforts toward concepts

and principles of particular interest to her, and I suggested that she take full advan

tage of this opportunity. Lastly, I suggest ed that for some readings she might take

notes (for example, verbatim quotations) that would assist her in answering not

only question 5 (passion) but also one or

more of the other questions. Her reaction

to my suggestions seemed to be one of

surprise and relief. The surprise, I imag

ine, had to do with her belief that I was

telling her to be less thorough in her note

taking. I believe, of course, that my sug

gestions encouraged deeper information

processing. Whatever the case, she fol

lowed my suggestions. Thereafter, spend

ing less time and effort reading and taking

only two pages of notes, she performed

well on the quizzes. Coaching of this sort

is helpful for many students.

Adjustments in Teaching and Learning

With continued use of the MCQ, I expe rienced a gradual and profound evolution

in my teaching from transmitting informa

tion to encouraging information process

ing or, stated otherwise, from lecturing to

designing and implementing classroom

activities that promote reflective thinking and problem solving. Designing such

activities requires teaching ingenuity unlike that required to prepare and deliver a lecture (Giordano and Hammer 1999;

Meyers 1997; Zachry 1985). With problem-solving activities replac

ing lectures, students take fewer notes.

However, they engage in more discussions

and reflective thinking, ask more questions,

and demonstrate a greater willingness to

struggle with the complexity of the materi al. The classroom has a more interactive

and coll?gial atmosphere. Students are

more engaged with both the subject matter

and one another. Some students require

support, encouragement, and guidance in

adjusting to this new learning environment

(Hart?n et al. 2002). Informing them about

the different types and levels of informa

tion processing helps them understand the

rationale for the MCQ and take full advan

tage of the various classroom activities that

replace the lecture.

Design and Flexibility

The MCQ is not a singular technique. Rather, it is a set of principles taken from

psychology's two major learning para

digms, behavioral analysis and cognitive

psychology. From behavioral analysis,

the MCQ borrows the principle of rein

forcement (Skinner 1953). A high quiz

grade is contingent on students' punctual

reading, careful studying, question-driven

note-taking, and other learning activities.

As outlined in the section on memory and

learning objectives, principles from cog nitive psychology are useful for promot

ing particular types of information pro

cessing. The five questions presented are

but one set of questions. Other learning

and information processing objectives can

be addressed through other questions. An instructor willing to take the time

and effort can create a question designed

to elicit a particular form of reflective

thinking relevant to each assigned read

ing. One of my colleagues moves infor

mation processing beyond note-taking to

formal writing in her child development course. Students type answers for all five

questions. On the day of the quiz, the roll

of the die determines which answer stu

dents will submit for evaluation. Whatev

er form the MCQ takes, it provides stu

dents with an ongoing structure and

guidelines for thinking about and reflect

ing on the assigned readings. The advan

tages of such guidelines are well docu

mented (Ausubel 1960; Corkill 1992). The MCQ also can be used with

assignments unrelated to reading. Anoth

er of my colleagues, who requires his stu

dents to attend a public lecture every

week, includes an MCQ question that

asks students to report on the lecture.

Although I have not yet included such a

question, for my Externship course I have

considered including a question that addresses the externs' experiences at their

respective agencies. With the decree

98 COLLEGE TEACHING

This content downloaded from 132.178.2.65 on Fri, 10 Oct 2014 16:17:39 PMAll use subject to JSTOR Terms and Conditions

option, when other instructional activities

take priority, the instructor can elect not

to roll the die, that is, not to administer a

quiz. Or, if students' learning is better

served otherwise, the instructor can

decree a quiz without the roll of the die.

A flexible procedure, the MCQ is readily

designed to address a wide variety of

teaching and learning objectives.

Key words: quiz, evaluation, learning

objectives

NOTE I wish to thank Victor Benassi, Jennifer

Feenstra, Dodge Fernald, Russell Kosits, Donna Perkins, Tracey Martin, Edward

O'Brien, Michael Root, and William Stine for

their critical reading of the manuscript and

assistance with data analysis. Correspon dence concerning this article should be

addressed to Peter S. Fernald, Conant Hall,

University of New Hampshire, Durham, NH

03824, or e-mail psf8@cisunix.unh.edu.

REFERENCES

Ausubel, D. P. 1960. The use of advance orga nizers in the learning and retention of

meaningful verbal material. Journal of Edu

cational Psychology 51:267-72.

Bloom, B. S. 1956. Taxonomy of educational

objectives, handbook I: Cognitive domain.

New York: David McRoy.

Bower, G. H. 1970. Organizational factors in

memory. Cognitive Psychology 1:18^4-6.

Bradshaw, G. L., and J. R. Anderson. 1982.

Elaborative coding as an explanation of lev

els of processing. Journal of Verbal Learn

ing and Verbal Behavior 21:165-74.

Burchfield, C. M., and J. Sappington. 2000.

Compliance with required reading assign ment. Teaching of Psychology 27:58-60.

Corkill, A. J. 1992. Advance organizers: Facil

itators of recall. Educational Psychology Review 4:33-72.

Craik, F. I. M., and E. Tulving. 1975. Depth of

processing and the retention of words in

episodic memory. Journal of Experimental

Psychology: General 104:268-94.

Dempster, F N. 1988. The spacing effect: A

case study in the failure to apply the results

of psychological research. American Psy

chologist 43:627-34.

Giordano, P. J., and E. Y. Hammer. 1999. In

class collaborative learning: Practical sug

gestions from the teaching trenches. Teach

ing of Psychology 26:42-44.

Glenberg, A. M. 1992. Distributed practice effects. In Encyclopedia of learning and

memory, ed. L. R. Squire, 138-42. New

York: Macmillan.

Hart?n, H. C, D. S. Richardson, E. B. Ricar

do, M. J. Rockloff, and B. Latane. 2002.

Focused interactive learning: A tool for

active class discussion. Teaching of Psy

chology 29:10-13.

Leeming, F. C. 2002. The exam-a-day proce dure improves performance in psychology

classes. Teaching of Psychology 29:210-12.

McDaniel, M. A., P. J. Waddill, and P. S.

Shakesby. 1996. Study strategies, interest, and learning from text: The application of

material appropriate processing. In Basic

and applied memory research: Theory in

context, ed. D. J. Hermann, C. McEvoy, C.

Herzog, P. Hertel, and M. K. Johnson, vol.

1, 385-97. Mahwah, NJ: Erlbaum.

Meyers, S. A. 1997. Increasing student partic

ipation and productivity in small-group activities for psychology classes. Teaching

of Psychology 24:105-15.

Palmere, M., S. L. Benton, J. A. Glover, and

R. Ronning. 1983. Elaboration and recall of

main ideas in prose. Journal of Educational

Psychology 25:898-907.

Payne, D. G, and M. J. Wenger. 1996. Prac

tice effects in memory: Data, theory, and

unanswered questions. In Basic and applied

memory research: Practical applications, ed. D. J. Hermann, C. McEvoy, C. Hertzon, P. Hertel, and M. K. Johnson, vol. 2, 123-37. Mahwah, NJ: Erlbaum.

Rogers, T. B., N. A. Kuiper, and W. S. Kirker.

1977. Self referencing and the encoding of

personal information. Journal of Personal

ity and Social Psychology 35:677-88.

Ruscio, J. 2001. Administering quizzes at ran

dom to increase student's reading. Teaching

of Psychology 28:204-06.

Skinner, B. F 1953. Science and human

behavior. New York: Macmillan.

Zachry, W. H. 1985. How I kicked the lecture

habit: Inquiry teaching in psychology.

Teaching of Psychology 12:129-31.

Vol. 52/No. 3 99

This content downloaded from 132.178.2.65 on Fri, 10 Oct 2014 16:17:39 PMAll use subject to JSTOR Terms and Conditions