Construct Test Item

8/3/2019 Construct Test Item

1/39

Evaluation of Student Learning:

Test Construction &Other Practical Strategies


2/39

Guiding Principles for Evaluation

Evaluation should relate directly to instructional

objectives

Each evaluation activity should be designed to promote

student growth

The actual activity should be useful practice in itself

Feedback should be useable by the student

Multiple evaluation strategies should be provided to

master achievement of X objective/competency

Student should clearly understand the methods of

evaluation for X test or activity


3/39

Questions to Ask yourself in Designing

a Test

What objectives will (should) I be testing?

What types of items will be included in the

test?

How long will the test be in terms of time andnumber of items?

How much will each objective be worth in

terms of weighting and number of items?


4/39

Tests as Diagnostic Tools

Students demonstrate learning

Instructor effectiveness modify teaching

strategies or activities


5/39

Different Types of Tests & Learning

Paper & Pencil/Web Testing

Limited Choice Questions (MC, T/F, Matching)

Open-Ended Questions (Short Answer, Essay)

Performance Testing Acquisition of skills that can be demonstrated

through action (e.g., music, nursing, etc.)


6/39

Planning a Test

First step: Outline learning objectives or

major concepts to be covered by the test

Test should be representative of objectives

and material covered Major student complaint:Tests dont fairly

cover the material that was supposed to be

canvassed on the test.


7/39

Planning a Test

Second Step: Create a test blueprint

Third Step: Create questions based on blueprint Match the question type with the appropriate level

of learning Fourth Step: For each check on the blueprint,

3-4 alternative question on ideas and itemtypes which will get at the same objective

Fifth Step: Organize questions and/or ideas byitem types


8/39

Planning a Test

Sixth Step: Eliminate similar questions

Seventh Step: Reread all of the items try

doing this from the standpoint of a student


9/39

Planning a Test

Eight Step: Organize questions logically

Ninth Step: Time yourself actually taking the

test and then multiply that by about 4

depending on the level of students Tenth Step: Analyze the results (item analyses)


10/39

Translating Course

Objectives/Competencies into

Test Items Syllabus

Specification table- what was taught/weight areas to be tested

Creating a Test Blueprint (see handout)

Blueprint- this is the test plan, i.e., which questions test whatconcept

Plotting the objectives/competencies against some hierarchyrepresenting levels of cognitive difficulty or depth ofprocessing


11/39

Thinking Skills

What level of learning corresponds to the

course content

Blooms Taxonomy of Educational Objectives

Knowledge (see handout)

Comprehension

Application

Analysis

Synthesis

Evaluation


12/39

Practical Considerations

Representative sample of the course content

not random purposeful based on blueprint

Representative sample of skill or cognitive

levels across content Analyze results by level AND content area


13/39

Question Arrangement on a Test

Group by question type

Common instructions will save reading time

Limit the number of times students have to

change frame of reference Patterns on test must be logical

Arrange from a content standpoint

Keep similar concepts together

Group by difficulty (easy to hard)


14/39

Selecting the Right Type of evaluation

How do you know what type of question to useand when?

It depends on the skill you are testing.

Evaluation should always match as closely aspossible the actual activity youre teaching. Examples: Teaching Speech, should evaluate an oral

speech

If testing ability to write in Spanish, better give anessay.

Testing reading MC, TF

Wouldnt use MC to test creative writing


15/39

Question Types verses

Cognitive Levels of Learning

Knowledge

Comprehension

Application Analysis

Synthesis

EvaluationMultiple Choice (MC)

True/False (TF)

Matching

Completion

Short Answer

MC

Short Answer

Problems

Essay

Performance

MC

Short Answer

Essay


16/39

Constructing the Test

Types of Test Questions:

Multiple-Choice Items

True-False Items

Matching Items Fill-In, Completion or Short-Answer Items

Essay Questions


17/39

Multiple Choice Items

Advantages:

Extremely versatile-can measure the higher level

mental processes (application, analysis, synthesis

and evaluation)

A compromise between a short answer/essay and

T/F item

Can cover a wide range of content can be sampled

by one test Disadvantages

Difficult to construct plausible alternative responses


18/39

Types of Multiple Choice Items

Four Basic Types

Question Type

Incomplete Statement Type

Right Answer Type Best Answer Type

Which Type is Best?

Question Type vs. Incomplete Statement

Right Answer vs. Best Answer Type


19/39


1. Writing the stem first:

A. Be sure the stem asks a clear question

B. Stems phrased as questions are usually easier to

writeC. Stems should not contain a lot of irrelevant info.

D. Appropriate reading level/terms

E. Be sure the stem is grammatically correct

F. Avoid negatively stated stems


20/39


Writing the correct response

Use same terms/reading level

Avoid too many qualifiers

Assign a random position in the answer sequence Read the stem and correct response together

Generate the distractors/alternative responses


21/39


Other Tips for Constructing MC Items: Items should have 3-4 alternatives.

Stem should present a single, clearly formulatedproblem

Simple, understandable, exclude extraneous wordsfrom both stem and alternatives

Include in the stem any word that are repeated ineach response

Avoid all of the above (can answer based on partialinformation)

Avoid none of the above


22/39


Alternative responses/distractors should be plausible

and as homogeneous as possible

Response alternatives should not overlap

Two synonymous terms (arithmetic average/mean)

Avoid double negatives

None of the following are part of the brain except which one?

Emphasize negative wording

Each item should be independent of other items in thetest

Information in the stem of one item should NOT help answer

another item.


23/39

True-False Test Items

Best suited for testing 3 kinds of info.: Knowledge level learning

Understanding of misconceptions

When there are two logical responses

Advantages: Sample a large amount of learning per unit of

student testing time

Disadvantages:

Tends to be very easy 50-50 chance of guessing

Tends to be low in reliability


24/39

Tips for Constructing True/False Items

Tips for constructing True-False Items

Avoid double negatives

Avoid long or complex sentences

Include only one central idea in each statement

Exact quantitative (two, three, four) language is betterthan qualitative (some, few, many)

Avoid a pattern of answers


25/39

Objective Test Item Analyses

Evaluating the Effectiveness of Items..

Why?

Scientific way to improve the quality of tests and test items

Identify poorly written items which mislead students

Identify areas (competencies) of difficulty

Item analyses provided info. on:

Item difficulty

Item discrimination

Effectiveness of alternatives in MC Tests


26/39

Short-Answer Items

Two Types: (Question and Incomplete Statement)

Advantages:

Easy to construct

Excellent format for measuring who, what, when, and where

info.

Guessing in minimized

Student must know the material- rather than simply recognize

the answer

Disadvantages: Grading can be time consuming

More than one answer can be correct


27/39

Short Answer Items

Tips for Constructing Short Answer Items Better to supply the term and require a definition

For numerical answers, indicate the degree ofprecision expected and the units in which they are

to be expressed. Use direct questions rather than incomplete

statements

Try to phrase items so that there is only one possible

correct response When incomplete statements are used, do not use

more than one blank within an item.


28/39

Essay Questions

Types of Essay Questions

Extended Response Question

Great deal of latitude on how to respond to a question.

Example: Discuss essay and multiple-choice type tests.

Restricted Response Question

More specific, easier to score, improved reliability and validity

Example: Compare and contrast the relative advantages of

disadvantages of essay and multiple choice tests with respect

to: reliability, validity, objectivity, & usability.


29/39

Essay Items

Advantages: Measures higher learning levels (synthesis,

evaluation) and is easier to construct than anobjective test item

Students are less likely to answer an essay questionby guessing

Require superior study methods

Offer students an opportunity to demonstrate their

abilities to: Organize knowledge

Express opinions

Foster creativity


30/39

Essay Items

May limit the sampling of material covered Tends to reduce validity of the test

Disadvantages

Subjective unreliable nature of scoring halo effect good or bad students previous level of

performance

Written expression

Handwriting legibility

Grammatical and spelling errors

Time Consuming


31/39

Essay Questions

Give students a clear idea of the scope & directionintended for the answer Might help to start the question with the description of the

required behavior (e.g., compare, analyze)

Appropriate language level for students Construct questions that require students to

demonstrate a command of background info, but do notsimply repeat that info.

If question calls for an opinion, be sure that the

emphasis is not on the opinion but on the way itspresented or argued.

Use a larger number of shorter, more specific questionsrather than one or two longer questions so that more

information can be assessed.


32/39

Essay Questions

You might Give students a pair of sample answers to a question of the

type you will give on the test.

Sketch out a rubric (grading scheme) for each question beforereading the papers OR randomly select a few to read and make

up the grading scheme based on those answers Give students a writing rubric

Detach identifying information and use code numbers insteadto avoid letting personality factors influence you.

After grading all the papers on one item, reread the first few to

make sure you maintained consistent standards Be clear to student the extend to which factors other than

content (e.g., grammar, handwriting, etc.) will influence thegrade.


33/39

Essay Questions

Tips for constructing Essay Questions

Provide reasonable time limits for each question

thinking and writing time

Avoid permitting students a choice of questions Will not necessarily get a representative sample of student

achievement. Only be requiring all students to answer all

questions can their achievement be compared

A definite task should be put forth to the student

Critical words: compare, contrast, analyze, evaluate, etc.


34/39

Scoring Essay Items

Write an outline of the key points (use outline to design

a rubric)

Determine how many points are to be assigned to the

question as a whole and to the various parts within it.

If possible, score the test without knowledge of the

students name

Face Sheet

Score all of the answers to one question before

proceeding to the next question Consistent standard


35/39

Scoring Essay Exams

If possible, score each set of answers within the

same time frame

Handwriting, spelling & Neatness

Two separate grades? Mastery of material

Other


36/39

Alternative Methods of Assessment

Research/Term Papers

Research Reviews

Reports

Case Studies Portfolios

Projects

Performances

Peer evaluation Mastery

Simulations


37/39

Cheating

Preventing Cheating

Reduce the pressure (multiple evaluations)

Make reasonable demands (length/content of exam)

Use alternative seating Use alternative forms

Be cautious with extra copies


38/39

Using Assessment & Evaluation to

Improve Student Learning Outcomes

Providing feedback to student

Closing the assessment & evaluation loop

Maximizing student learning


39/39

Questions?

Construct Test Item

Documents

Transcript of Construct Test Item