September 10, 2011UIC

TEST ITEM WRITING and ITEM ANALYSIS

The Systematic Planning Process:

Identify Instructional Goals

Analyze Learners

Identify Objectives

Plan instructional activities

Choose Instructional Media

Design Assessment Tools

Implement Instruction

Revise Instruction

Steps in Test Construction

Identify Objectives Develop TOS Write Items

Illustrations on Sampling

Population(Universe of

Characteristics)

Sample( Best estimates of the

Population)

Test

Illustrations on Sampling

Universe of Characteristics

A Better Model

Best Model

A Poor Model

Illustrations on SamplingInvalid and Unreliable Test•Too many errors

Valid but Unreliable Test• Errors causes test scores to be unstable and inconsistent

Valid and Reliable Test• Observed errors are not significant (or negligible)

THREE-STAGE MODEL OF CLASSROOM MEASUREMENT

CONTENT VALIDITY

Stage 1 Stage 2 Stage 3

OBJECTIVES ACTIVITIES TEST

Test items must validly measure instructional

objectives

A. Research Designs 1. Differentiate Quantitative and Qualitative reseach.2. Classify the various research designs3. Identify common research designs4. Contruct research problems in each research designs.

total

Topics / Objectives K C A AN S E total

THINKING SKILLS

1,2

3,4,5,67,8,9,1011,12

Item Placement

2 2

2 2 41 2 1 4

2 2

60

Hint:# of items = # of days per topic / total # of days X total # of items

1 6

2 121 61 6

10

DAYS IDEAL

SAMPLE TABLE OF SPECIFICATION

WORKSHOP 1

You need to have:a. Sample syllabusb. Test Questionnairec. Paper and Pen

Instructions:Choose a syllabus and a

questionnaire from your compilation

Match the items in the questionnaire with the competencies measured in the syllabus

Make an evaluation about the content validity of the test – whether it measures what it is suppose to measure.

Volunteers will share their summary of evaluation

Matching Requirements:

Item versus objectiveThinking Skill versus

objectiveProportion of items per

objective versus the total number of items, total number of objectives, and the time element.

TemplateObjectiv

esList of Items

Skills Time Frame

Total

A. Solve the ….

5, 3, 5

ok 3

What do you see in the picture?

How many legs does this elephant have?

Multiple Choice Type of

Test

AdvantagesA large number of ideas can

be addressed in a short period of response time.

These questions are easily and quickly scored.

Questions can elicit responses from all cognitive levels, from knowledge to evaluation.

Questions can be improved over time by analyzing them in light of student performance.

DisadvantagesIt is time-consuming to write good items, especially those at higher cognitive levels.

Test-wise and English fluent students tend to be favored.

varieties of multiple-choice items

A. The correct answer variety

Who invented the sewing machine?

a. Fulton*b. Howec. Singerd. Whitee. Whitney

B. The best answer variety

What was the basic purpose of a test?

a.Measure teacher’s abilityb.determine students’ achievementc. Assess learning outcomesd. Identify students’ weaknesses

The multiple response variety

What factors are principally responsible for the clotting of blood?

a.contact of blood with a foreign substance*b. contact of blood with injured tissuec. oxidation of hemoglobin*d. presence of unchanged pro thrombin

Use of Analogies

Man is to woman as boy is to

a.Fatherb.Motherc.Girld.Lady

The incomplete statement variety

In 2008, millions of pesos worth of corn, rice, and wheat are destroyed in the Philippines by

a. ratsb. grasshoppersc. birds*d. pests

The negative variety

Which of these is NOT true of viruses?

a. Viruses live only in plants and animals.b. Viruses reproduce themselves.*c. Viruses are composed of very large living cells.d. Viruses can cause diseases.

The substitution varietyPassages to be readSurely the forces of education should be fully utilized to acquaint youth with the real nature of the dangers to democracy, (1) for no other place offers (2) as good or better opportunities than the school for; a (3) rational consideration of the problems involved.

Items to be answered1. *a. , for

b. . Forc. —ford. no punctuation needed

2. a. As good or better opportunities thanb. as good opportunities or better thanc. as good opportunities as or better than*d. better opportunities than

3. *a. rationalb. radicalc. reasonabled. realistic

The combined response variety

In what order should these sentences be written in order to make a coherent paragraph?A. A sharp distinction must be drawn between table manners and sporting manners.B. This kind of handling of a spoon at the table, however, is likely to produce nothing more than an angry protest against squirting grapefruit juice about.C. Thus, for example, a fly ball caught by an outfielder in baseball or a completed pass in football is a subject for applause.D. Similarly, the dexterous handling of a spoon in golf to release a ball from a sand trap may win a championship match.E. But a biscuit or a muffin tossed and caught at the table produces scorn and reproach.a. A. B, C. D. E*b. A, C. E, D. Bc. A, E. C. D, Bd. B, E, D. C. A

Basic Guidelines

Adolescents obtain information about marriage

A. in order to reduce the possibility of pregnancyB. from parentsC. from same-sex friends*D. in a haphazard fashion

PRESENT A CLEAR PROBLEM IN THE STEM

Most adolescents obtain information about marriage from

A. same sex friends*B. parentsC. health service personnelD. teachers

Before the Civil War, the South's _____was one of the major reasons manufacturing developed more slowly than it did in the North.

A. emphasis on staple-crop production*B. lack of a suitable supply of raw materialsC. short supply of personnel capable of operating the necessary machinery

PUT ALTERNATIVES AT THE END OF THE QUESTION

Before the Civil War, manufacturing developed more slowly in the South than in the North. One of the major causes of this was the South's___

A. emphasis on staple-crop production*B. lack of a suitable supply of raw materialsC. short supply of personnel capable of operating the necessary machinery

In objective testing, the term objective

A. refers to the method of identifying the learning outcomesB. refers to the method of selecting the test contentC. refers to the method of presenting the problemD. refers to the method of scoring the answers*

PUT MOST OF THE WORDING IN THE STEM

In objective testing, the term objective refers to the method of

A. identifying the learning outcomesB. selecting the test contentC. presenting the problemD. scoring the answers*

For almost a century, the Rhine river has been used by Europeans for a variety of purposes. However, in recent years, the increased river traffic has resulted in increased levels of diesel pollution in the waterway. Which of thefollowing would be the most dramatic result if, because of the pollution, the Council of Ministers of the European Community decided to close the Rhine to all shipping?

A. closure of the busy river Rhine ports of rotterdam, Marseilles and GenoaB. increased prices for Ruhr products*C. reduced competitiveness of the French Aerospace IndustryD. shortage of water for Italian industries

AVOID UNNECESSARY WORDINESS

Which of the following would be the most dramatic result if, because of diesel pollution from ships, the river Rhine were closed to all shipping?

A.closure of the busy river Rhine ports of Rotterdam, Marseilles and GenoaB. increased prices for Ruhr products*C. reduced competitiveness of the French Aerospace IndustryD. shortage of water for Italian industries

Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, which of the following should not happen?

A.Children should not believe all of their behavior is bad.B. Children should understand the reason(s) why they are being punished.C. Children should understand that the teacher, not the children, controls when the punishment will end.*

AVOID NEGATIVELY WORDED STEMS

Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, it is important that the children understand:

A.that it may be a long time before happy times return to the classroomB.the reason(s) why they are being punished*C. that the teacher, not the children, controls when the punishment will end

Good

Thurstone's 7 primary mental abilities include all of the following EXCEPT:

A. word fluencyB. reasoningC. social interaction*D. perceptual speed

Which of the following men contributed most towards he defeat of Hitler‘s Germany in World War II?

A. Winston ChurchillB. Josef StalinC. Franklin D. RooseveltD. George Patton

AVOID REQUIRING PERSONAL OPINION

What is the area of the rectangle described in Question 1?

A. 1050 sq. cm.B. 7396 sq. cm.C. 7654 sq. cm.*D. 8188 sq. cm.

AVOID LINKED OR CLUED ITEMS

What is the official state bird of Pennsylvania?

A.mountain laurelB.B. PhiladelphiaC. ruffed grouse*D. ibex

ALL OPTIONS SHOULD BE HOMOGENEOUS

What is the official state bird of Pennsylvania?

A. goldfinchB. robinC. ruffed grouse*D. wild turkey

Who succeeded Giscard d'Estaing as President of France in l981?

A. Georges PompidouB. Charles De GaulleC. Francois Mitterand*D. Mick Jagger

ALL OPTIONS SHOULD BE PLAUSIBLE

Which of the following is the best definition of a seismograph ?

A. An apparatus for measuring sound waves.B. An apparatus for measuring heat waves.C. An apparatus for measuring earthquake waves.*D. An apparatus for measuring ocean waves.

PUT REPEATED WORDS IN THE STEM, NOT THE

OPTIONSA seismograph is an apparatus for measuring:

A. earthquake waves*B. heat wavesC. ocean wavesD. sound waves

What type of waves does a seismograph measure?

A. earthquake*B. heatC. oceanD. sound

An angle of 90o is called a

A.acute angleB.obtuse anglesC.right angle*

MAKE ALL OPTIONS GRAMMATICALLY

CONSISTENT WITH STEM

Angles of 90o are called

A.acute anglesB.obtuse anglesC.right angles*

The average difference between males and females in the attainment of pubertyis:

A. 1 yearB. 2 years*C. 3 yearsD. no difference

ORDER OPTIONS LOGICALLY

RULES FOR WRITING MULTIPLE-CHOICE ITEMS

Design each item to measure an important learning outcome.

Present a single clearly formulated problem in the stem of the item.

Put the alternatives at the end of the question, not in the middle nor at the beginning.

Put as much of the wording as possible in the stem.

Eliminate unnecessary wordiness

Avoid negatively worded stems. "Which of the following is not..........“

Avoid requiring personal opinion. Other item types are more suitable for this.

Avoid textbook wording. Do not have linked or clued items. All options should be homogeneous.

All options should be plausible. Put repeated words in the stem, not in the options

Punctuation should be consistent. Make all options grammatically consistent with the stem of the item.

List options vertically.

Order options logically. Use the option "all of the above" sparingly.

Use the option "none of the above" sparingly.

Make All distracters the same length

Exercise:Joseph Estrada was an

ActorFormer PresidentDriverScientist

In what year did humans first set foot on the moon?

1975195719691963

Some test items Are too difficultAre objectiveAre poorly constructedHave multiple defensible answers

Ferdinand Magellan came to the Philippines in a

CarBoatAirplaneballoon

WORKSHOP 2

You need to have:a. Sample syllabusb. Test Questionnairec. Paper and Pen

Instructions: Evaluate your items using the guidelines in item writing as your criteria.

Revise poor items.Identify the objective/competence in every revised item.

Build an item bank