8/14/2019 Writing Effective Tests
1/28
Writing Effective Tests
The following slides summarize the workof Lucy Jacobs and Clinton Chase fromtheir book Developing and Using Tests
Effectively: A Guide for Faculty
The book is a compilation of years of
research on how to develop effectivetest instruments.
8/14/2019 Writing Effective Tests
2/28
Writing Tests
Most higher education faculty never took acourse in how to write tests. Their experiencecomes from the many tests they have taken overtheir many years in school. This personal
experience is not a reliable way to learn todevelop tests that will determine so manyimportant outcomes for our students.
Even with diligence, time and practice writingtests, most tests given in college haveweaknesses and are not highly reliable.
However, the following presentation can
significant help to improve the quality of the tests
8/14/2019 Writing Effective Tests
3/28
Writing Fair and EffectiveTests
Purpose ofTests
Grades Planning content
presentations
Facilitate learning
Motivate students
Benefits toStudents
Learn how theyare tested
Engage in deep orsurface learning
based on tests Feedback on
learning
8/14/2019 Writing Effective Tests
4/28
Test Benefits to Instructors
Focus on essentialcourse content
Message to
students on whatsimportant
Valuable feedback
on learning
Feedback if courseobjectives wereachieved
Feedback oneffectiveness ofinstruction
A motivationalstrategy
8/14/2019 Writing Effective Tests
5/28
Problems with College Tests
Dont focus onwhats important
Too little feedback
given to students Questions
ambiguous or
unclear
Too short tosample contentfairly
Purpose of test notmade clear-sostudents see it as
penalToo few tests-
dont have a broadview of students
knowledge
8/14/2019 Writing Effective Tests
6/28
How Long Should a Test Be?
True-False 30sec.
Multiple choice 1
min. Completion 1
min.
Short answers 2min.
Multiple choice90sec. higherlevel thinking
Matching 30sec.
Short Essay
10-15 minutes.
Extended essay
30 minutes
8/14/2019 Writing Effective Tests
7/28
8/14/2019 Writing Effective Tests
8/28
Test Item Difficulty
Tests shouldproduce a spreadof scores
Test items shouldbe of averagedifficulty
Items that 50-70%of the studentscan answercorrectly
If they didnt study
Difficulty can be:
A. content it asks
B. cognitive skilllevel beingmeasured
C. Construction ofthe
item
8/14/2019 Writing Effective Tests
9/28
Test Reliability
Long enough tosample contentwell
Most studentshould be able tofinish
Score rangeshould be wide
Items free ofambiguity and
tricks
Directions shouldbe clear.
Scoring procedure
needs to be free oferrors.
All students should
take the same test
8/14/2019 Writing Effective Tests
10/28
8/14/2019 Writing Effective Tests
11/28
Multiple Choice Tests
Write a stem in theform on anincomplete
statement orquestion thatcould be answeredwithout look at the
four choices Always put a verb
in the stem
Do not pad thestem-write themsimple and clear.
Distracters mustbe plausible
Avoid using humor
Make certain thereis only one bestanswer
8/14/2019 Writing Effective Tests
12/28
Multiple Choice Tests
Make distractersfairlyhomogeneous-this
will force studentsto bediscriminating
Dont giveirrelevant clues ortricks-thesereduce validity
Make answers
Avoid givinggrammar clues
careful with
specific modifiers--ie. Never, always
Limit use of all of
the above andnone of the above
8/14/2019 Writing Effective Tests
13/28
Multiple Choice Tests
Arrange options inlogical(alpha )order
Keep correctresponse letter( number)balanced
Dont lift stemsverbatim from text
Arrange answers
in vertical columns
Avoid asking trivia
Ask students tochoose the best
answer not thecorrect answer.
8/14/2019 Writing Effective Tests
14/28
True-False Tests
Advantages
A. Sample manymore bits of
information B. Ask more
question in 50
minute timeperiod helpsreliability
C. Research
affirms True-False
Limitations
A. Difficult to writeso there not
ambiguous orobvious
B. Difficult to make
clearly true orclearly falsestatements
C. 50-50 chance
D. Educated guess
8/14/2019 Writing Effective Tests
15/28
True-False Tests
Avoid use ofspecificdeterminers ( all,
never) Avoid use of
qualifiers(sometimes, usually)
Avoid use ofindefinite terms(long time ago)
Write questionsthat are clear andsimple to read.
Make it clearwhere the answersare to be placed--dont use + or -
8/14/2019 Writing Effective Tests
16/28
True-False Higher LevelThinking Use propositional
logic--if thenapproach.
Example-If thegross nationalproduct goes upand inflation is3.4%then thestock market willlikely go up.
Setting up aproblem thatrequires analysis.
Example --Johnbought a caritwont start. Whichare possiblereasons?
8/14/2019 Writing Effective Tests
17/28
Short Answer or CompletionItems Requires recall
rather thanrecognition of
information. Doesnt test
higher levelthinking
Best used to seewhat informationstudents have
collected.
Questions shouldhave a single wordanswer.
Make certain thereis only one rightanswer
Delete only keywords
Make blanks samelength
Dont lift
8/14/2019 Writing Effective Tests
18/28
Matching Tests
Stimulus columnon the leftnumbered--
response columnon right lettered.
Provide space toright answer to theleft of stimuli.
Use onlyhomogeneous
material
Put stimuli inalphabetical order
Use 10-15 items
Keep entire test onone page.
Have at least five
more stimuli thanresponses
8/14/2019 Writing Effective Tests
19/28
Essay Tests
Advantages
1. Can assesscomplex learning
2. Easy to write 3. Emphasize
communication
skills asfundamental
4. Cant simplyrecognize ananswer
5. Doesnt allowfor guessing--except bluffing
6. Can see howstudents select,organize andevaluate ideas and
apply them
8/14/2019 Writing Effective Tests
20/28
Essay Tests
Limitations
1. Difficult to score
2. Scores are less
reliable 3. Sample only a
limited amount of
content
4. Scoreinfluenced byreaders overall
impression of thestudent
5. Not favorable todeveloping goodwriting skills
8/14/2019 Writing Effective Tests
21/28
Essay Reliability Concerns
Less reliable thanobjective tests
Environmental
factors influencescoring ie. Time ofday, mood ofreader where
paper is in thestack Previous paper can
influence current
paper
A second readingoften results in adifferent grade.
Expectations ofstudentsperformance
Physical elements
of the paper--handwriting
Limited number ofquestions luck of
the draw
8/14/2019 Writing Effective Tests
22/28
Making Essay Tests Better
Restrict essays toassess onlyoutcomes that
require complexhigher levelcognitivefunctions.
Examples 1. Compare
-Contrast
2.Present
Keep questionfocused on asingle objective
All students shouldrespond to thesame questions-dont give choices
Grammar andspelling count onlyif taught as an
objective in the
8/14/2019 Writing Effective Tests
23/28
Making Essay Tests Better
Directions need tobe very clear andinclude type of
responseexpected--Exampleoutline, list,discussion
List the point valueof each question
8/14/2019 Writing Effective Tests
24/28
Scoring Essay Tests
Conceal studentsnames
Skim a few before
starting to gradeto get a sense ofresponses
Read only oneitem across allpapers
Reshuffle the stack
Use a prescribedreading procedure
A. Key procedure
lays out ideas thatshould have beenin the essay
B. RankingProcedure
Makes piles basedon quality
8/14/2019 Writing Effective Tests
25/28
Open-Book Tests
Advantages
Used in classeswhere formulas,
tables or graphsare needed to findsolutions
Studentdemonstrates theyknow how to usereferences
Reflect real life
Limitations
Students tend notto prepare well
No evidence tosuggest studentsdo any better
8/14/2019 Writing Effective Tests
26/28
Take Home Tests
Advantages
1. More thoroughexploration of
ideas can takeplace
2. No time
restrictions 3. Usually better
written
Limitations
1. Question rarelysample adequately
course material 2. Much more
grading time
3. Students oftenunsure of howthorough to be.
8/14/2019 Writing Effective Tests
27/28
Re-takes of Tests
Advantages
Retakes do resultin better learning
Lower test anxiety Students learn
from their
mistakes It eliminates the
need for make-uptests
Limitations
A large test bankis needed
Grading timeincreases
Students dont
often prepare wellfor the first test
8/14/2019 Writing Effective Tests
28/28
Helping Students Prepare forthe Tests
Review the scopeof the test
Use a practice test
Clarify time limits Clarify what
material will be
needed andallowed
Review gradingprocedure
Review make-uppolicy
Provide a review
session Make provisions
for last minute
questions Allow for breaks
during long exams
Top Related