Post on 12-Apr-2017
© 2013 Springer Publishing Company, LLC.
Chapter 10Assembling and Administering TestsOermann & GabersonEvaluation and Testing in Nursing Education4th edition
© 2013 Springer Publishing Company, LLC.
Effects on Reliability and Validity
♦ Test appearance and administration procedures can affect reliability and validity– Design flaws and clerical errors may ↑
measurement error and ↓ reliability of scores– Test administration problems may ↓ reliability
of scores and ↓ validity of inferences from those scores
2
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Allow enough time– Designing and assembling the test are not simply
clerical tasks– Teacher should make all decisions about the
appearance of the test– Allow enough time for this phase to avoid errors
that could affect scores
3
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Arrange test items logically– Group items of the same format together– Within each item format, arrange items according
to order in which content was taught– Within each content area, arrange items in order
of difficulty (easy to difficult)
4
© 2013 Springer Publishing Company, LLC.
Test Design Rules (cont’d)
♦ Write directions– General directions• Time allowed• How to record answers• Whether students may ask questions• Whether students may write on the test booklet
5
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Write directions (cont’d)– Item-specific directions• Multiple-choice: correct or best response• Matching: basis for the match • Completion: whether spelling counts• Computation: degree of precision• Essay: length of answer; whether organization, spelling,
grammar will count
6
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Use a cover page– For general directions– Can be numbered to maintain test security– Protects test contents from view during
distribution• Allows all students to have equal testing time
7
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Avoid crowding– Helps students to read efficiently and avoid errors
in recording answers– Ample space between and within items– Indent certain elements– Avoid printing multiple-choice options in tandem
8
© 2013 Springer Publishing Company, LLC.
Poor Spacing
Which method of anesthesia involves injection of an agent into a nerve bundle that supplies the operative site?
a. General; b. Local; c. Regional; d. Spinal; e. Topical
9
© 2013 Springer Publishing Company, LLC.
Better Spacing
Which method of anesthesia involves injection of an agent into a nerve bundle that supplies the operative site?a.Generalb.Localc.Regionald.Spinale.Topical
10
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Keep related material together– Multiple-choice stem and responses on same page– Both columns of matching exercise side by side on
one page, with the related directions– Stimulus and related items for context-dependent
and interpretive exercises on same page, if possible
11
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Facilitate scoring– Hand scoring• Separate answer sheet• Answers in test booklet—arrange items to
permit easy scoring– e.g., columns of Ts and Fs to left of true-false items for
students to circle
12
© 2013 Springer Publishing Company, LLC.
Poorly Arranged for Hand Scoring
1. A stethoscope is required to perform auscultation. T F
13
© 2013 Springer Publishing Company, LLC.
Arranged for Easy Scoring
T F 1. A stethoscope is required to perform auscultation.
14
© 2013 Springer Publishing Company, LLC.
Poorly Arranged for Hand Scoring
1. List 3 responsibilities of the circulating nurse during induction of general anesthesia. ________, ________, _________
15
© 2013 Springer Publishing Company, LLC.
Arranged for Easy Scoring
1-3. List 3 responsibilities of the circulating nurse during induction of general anesthesia.
1. _____________________________2. _____________________________3. _____________________________
16
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Random pattern of correct answers to objectively scored items – Teachers may inadvertently favor certain keyed response
positions.– Teachers may arrange test items so that correct answers
form a pattern for easy scoring.– Students who detect a pattern of correct answers may use
this information to obtain higher scores than their knowledge would merit.
– Use response positions with approximately equal frequency.
17
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Arrange response alternatives in logical order– Alphabetical or chronological order, or order
of size or degree– Reduces reading time – Tends to randomly distribute the correct
answer position
18
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Number items continuously throughout test– Helps students find items they may have skipped– Helps to prevent student errors when recording
answers on separate answer sheet
19
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Proofread– Assure that test is free of spelling, structural,
and typographical mistakes that contribute to measurement error
– Spell-checking programs not sufficient– May ask another teacher to proofread
20
© 2013 Springer Publishing Company, LLC.
Test Design Rules
♦ Prepare and verify an answer key– Facilitates efficient scoring – Verifies test item accuracy– Teacher should:• prepare ideal responses to essay items.• identify intended responses to completion items.• decide point values of required answer elements if
scoring essays analytically.
21
© 2013 Springer Publishing Company, LLC.
Reproducing the Test
♦ Assure legibility– Poor-quality copies may affect student
performance– Uppercase and lowercase lettering (not all capitals)– Letter-quality original– Copies with crisp, dark print; no artifacts
22
© 2013 Springer Publishing Company, LLC.
Reproducing the Test
♦ Print on one side of page– Easier to score if students record their answers
on the test itself– Printing on both sides of each page could cause
student-response errors
23
© 2013 Springer Publishing Company, LLC.
Reproducing the Test
♦ Reproduce enough test copies– More test copies than the number of students – Allows extra copies for proctors or to replace
defective copies distributed inadvertently to students
24
© 2013 Springer Publishing Company, LLC.
Reproducing the Test
♦ Maintain test security– Protect test materials from unauthorized access – Secure the test during preparation, duplication,
storage, administration, scoring– Preparing alternate forms of the test by
scrambling the order of items or options not recommended without evidence of true alternative-forms equivalence
25
© 2013 Springer Publishing Company, LLC.
Test Administration: Environment
♦ Face-to-face– Well-ventilated room– Limited distractions– Signage indicating testing in progress
♦ Online– Students have computer capabilities and Internet
access for allotted exam time
26
© 2013 Springer Publishing Company, LLC.
Test Administration: Distribution
♦ Distribute answer sheet before test booklet♦ Students keep booklet closed until told
to begin♦ Students check booklet for completeness♦ Proctor carefully for test security
27
© 2013 Springer Publishing Company, LLC.
Test Administration: Distribution
♦ Proctor reads general directions aloud while students read silently– Helps English language learners and students with
learning disabilities♦ Proctor answers any questions about the test
procedures♦ Students begin the test all at once
28
© 2013 Springer Publishing Company, LLC.
Test Administration: Questions During Testing♦ Student questions are distracting to others♦ Proctors may give inadvertent cues to correct answer♦ May decide not to allow questions– Students record questions on comment sheet to be
reviewed later
♦ If allowing questions:– Student raises hand, proctor goes to student’s seat and
answers quietly and briefly
29
© 2013 Springer Publishing Company, LLC.
Test Administration: Cheating
♦ Activity with intent to gain a higher test score than student is likely to earn by achievement – Acquiring test materials in advance– Sharing test materials with others– Using a substitute to take the test– Using unauthorized materials during the test – Exchanging information with others during the test,
including use of hand-held devices– Copying or retaining test materials to share later
30
© 2013 Springer Publishing Company, LLC.
Test Administration: Cheating
♦ Controlling cheating—face-to-face testing– Teacher’s responsibility– Establish standardized testing procedures, such as• Personal belongings left outside of testing room• No sunglasses, earbuds, caps with brims or bills
– Most effective through proctoring• Ideally minimum of two proctors• Devote full attention to proctoring
31
© 2013 Springer Publishing Company, LLC.
Test Administration: Cheating
♦ Responding to observed cheating– Be aware of and apply program policy on
academic integrity– If certain, collect test materials and remove
student from room quietly– If uncertain, observe student closely, make eye
contact, verify observation with other proctor
32
© 2013 Springer Publishing Company, LLC.
Test Administration: Cheating
♦ Establish an appropriate penalty for cheating on a test– Example: assign highest possible failing score for that test
♦ Discovery of test security breach in advance of scheduled test date – Attempt to obtain verifiable evidence that some students
have seen the test– If so, prepare another test or another way of assessing
student learning
33
© 2013 Springer Publishing Company, LLC.
Test Administration:Collecting Test Materials♦ Advise students of procedure during general
instructions♦ Keep distractions to minimum♦ Verify that students turn in all test materials♦ If large group, designate one proctor to collect
materials
34
© 2013 Springer Publishing Company, LLC.
Test Administration:Collecting Test Materials♦ Enforce time limit– Don’t give students extra time unless they have
been granted that specific accommodation for a disability
– Announce time left near the end of the test period and encourage students to finish quickly
– At end of time limit, collect materials from remaining students
35