Barry Williams 1
Developing Assessment Developing Assessment InstrumentsInstruments
Dick & Carey
Chap. 7
Barry Williams 2
Criterion-Referenced TestsCriterion-Referenced Tests
Designed to measure explicit behavioral objectives
Used to evaluate:– learner performance– effectiveness of the instruction
Barry Williams 3
Criterion-ReferencedCriterion-Referenced
Also called objective-referenced Refers directly to explicit “criterion” or
specified performance “Criterion-Referenced Test” must:
– match test item and performance objective– stipulate degree of mastery of the skill
Barry Williams 4
Types of Criterion TestsTypes of Criterion Tests
Pretest: 1. Consists of items that:
- measure entry behavior skills- test skills to be taught- draw from skills below the entry behavior line
2. Helps determine appropriateness of required entry skills.
3. Used during formative evaluation process. May be discarded in final version of instruction.
Barry Williams 5
Types of Criterion TestsTypes of Criterion Tests
Posttest: 1. Assesses all the objectives, focusing on
terminal objectives 2. Helps identify ineffective instructional
segments 3. Used during the design process and may
be eventually modified to measure only terminal objectives
Barry Williams 6
Designing Tests for Learning Designing Tests for Learning DomainsDomains
Intellectual & Verbal Information– paper & pencil
Attitudinal– state a preference or choose an option
Psychomotor– performance quantified on checklist– subordinate skills tested in paper-and-pencil
format
Barry Williams 7
Determining Mastery LevelsDetermining Mastery Levels
Approach # 1– mastery defined as level of performance
normally expected from the best learners– arbitrary (norm-referenced)
Approach # 2– defined in statistical terms, beyond mere chance– mastery varies with critical nature of task
example: nuclear work Vs. paint a house
Barry Williams 8
Writing Test ItemsWriting Test Items
What should test items do? Match the behavior of the objective
– Use the correct “verb” to specify the behavior Match the conditions of the objective
Barry Williams 9
Writing Test ItemsWriting Test Items
How many test items do you need?Determined by learning domains
Intellectual requires three or more– Wide range use random sample
Barry Williams 10
Writing Items Writing Items (continued)(continued)
What types (true / false, multiple choice, etc..) to use?clues provided by the behavior listed in the
objectivereview “Types of Test Items” this chap. p 148
Barry Williams 11
Writing Items Writing Items (continued)(continued)
Item types tempered by: amount of testing time ease of scoring amount of time to grade probability of guessing ease of cheating, etc. availability of simulations
Barry Williams 12
Writing Items Writing Items (continued)(continued)
What types are inappropriate?– true / false for definition
discrimination, not definition
Acceptable alternatives from “best possible”– for simulations
list steps
Barry Williams 13
Constructing Test ItemsConstructing Test Items
Consider: vocabulary setting of test item (familiar Vs. unfamiliar) clarity
– all necessary information trick questions
– double negatives, misleading information, etc.
Barry Williams 14
Other FactorsOther Factors
Sequencing Items– Consider clustering by objective
Test Directions– Clear and concise– General– Section specific
Evaluating Tests / Test Items
Barry Williams 15
Measuring Performance, Measuring Performance, Products, & AttitudesProducts, & Attitudes
Write directions to guide learner activities and
Barry Williams 16
Evaluating Performance, Evaluating Performance, Products, & AttitudesProducts, & Attitudes
Construct an instrument to evaluate these activities– a product, performance, or attitude
Sometimes includes both process and a product – For example -- TRDEV 518
Barry Williams 17
Test Directions for Test Directions for Performance, Products, & Performance, Products, &
AttitudesAttitudes Determine the
– Amount of guidance?– Special conditions
time limits, special steps, etc.– Nature of the task (i.e., complexity)– Sophistication level of the audience
Barry Williams 18
Assessment Instruments for Assessment Instruments for Performance, Products, & Performance, Products, &
AttitudesAttitudes
Identify what elements are to be evaluated– cleanliness, finish, tolerance of item, etc.
Paraphrase each element Sequence items on the instrument Select the type of judgment for rater Determine instrument scoring
Barry Williams 19
Formats for Assessments of Formats for Assessments of Performance, Products, & Performance, Products, &
AttitudesAttitudes
Checklist Rating Scale Frequency Counts Etc.
Barry Williams 20
Performance, Products, & Performance, Products, & Attitudes -- ScoringAttitudes -- Scoring
Guidelines?
Top Related