USING MULTIPLE-CHOICE QUESTIONS TO MEASURE PROBLEM SOLVING AND CRITICAL THINKING

USING MULTIPLE-CHOICE QUESTIONS TO MEASURE PROBLEM SOLVING AND CRITICAL THINKINGA step-by-step procedure for creating problem-solving measures:Should be known principles but the situation in which the principles are to be applied should be new.Should involve significantly important principlesShould be pertinent to a problem or situation common to all students.Should be within the range of comprehension of all students.Should use only valid and reliable sources from which to draw data.Should be interesting to students.

Step 1: Decide on the principle or principles to be tested. Criteria to be considered

Step 2: Determine the phrasing of the problem situation so as to require the student in drawing conclusions to be one of the following:Make a prediction.Choose a course of action.Offer an explanation for an observed phenomenon.Criticize a prediction of explanation made by othersStep 3: Set up the problem situation in which the principle or principles selected operate. Present the problem to a class with directions to draw a conclusion or conclusions and give several supporting reasons for their answerStep 4: Edit the students responses, selecting those that are most representative of their thinking. These will include conclusions and supporting reasons that are both acceptable and unacceptableStep 5: To the conclusions and reasons obtained from the students, the teacher now adds any others that he or she feels are necessary to cover the salient points. The total number of items should be at least 50 percent more than is desired in the final form to allow for elimination of poor items. The following list is a guide to the type of statements that can be used:True statements of principles and factsFalse statements of principles and factsAcceptable an unacceptable analogiesAppeal to acceptable or unacceptable authorityRidiculeAssumption of the conclusionTeleological explanationsStep 6: Submit test to other judges for criticisms. Revise test in view of criticismsStep 7: Administer test. Follow with thorough class discussionStep 8: Conduct an item analysisStep 9: In the light of steps 7 and 8, revise the test.Norris and Ennis (1989) write about four kinds of critical thinking questions:DeductionCredibilityInduction Assumption-identificationFollowing are examples of these types of items:DEDUCTIONIf these beings are from Earth, then another space ship must have landed on Nicoma. But no other space ship has landed on Nicoma.Suppose that what the speaker says is true. Then must this be true?These beings are not from Earth.

A.Yes, it must be true.B.No, it cannot be true.C.We cannot tell for sure from the information given

CREDIBILITYWhich underlined statement is more believable?

A.The health officer says, This water is safe to drink.B.Several others are soldiers. One of them says, This water supply is not safe.C.A and B are equally believable.INDUCTIONMr. Brown, who lives in the town of Salem, was brought before the Salem municipal court for the sixth time in the past month on a charge of keeping his pool hall open after 1A.M. He again admitted his guilt and was fined the maximum , $500, as in each earlier instance.3.On some nights it was to Mr. Browns advantage to keep his pool hall open after 1 A.M., even at the risk of paying a $500 fine. This statement isA.TrueB.Probably trueC.Probably falseD.FalseE.Insufficient dataASSUMPTION IDENTIFICATION4.Since about half the villagers have very short hair, I think that at least half are male.A.Half are femaleB.All males have short hairC.Only males have short hairFat MiceThin MiceBlack TailWhite Tail

GENERAL TYPES OF ESSAY QUESTIONS1. Extended Versus Restricted ResponseComplex learning outcomes can be measured effectively with questions.There are the abilities, for example, toexplain cause-effect relationshipsdescribe applications of principlespresent relevant argumentsFormulate tenable hypothesesFormulate valid conclusionsState necessary assumptionsDescribe the limitations of dataExplain methods and proceduresProduce, organize, and express ideasCreate original forms (e.g., design an experiment)Evaluate the worth of ideas

2. Content Versus Mechanics of ExpressionThe essay question or test allows the student to present his or her knowledge and understanding and to organize the material in a unique form and styleIf the instructor has attempted to develop students expressive skills, and if this learning outcome is included in his or her table of specifications, evaluating such skills is legitimateIf expressive skill are not part of the instructional program, it is unethical to evaluate them.If the score of each essay item includes an evaluation of the mechanics of English, this should obviously, be brought to the students attention.Separate scores should be computed for content and expressionThe decision to include either or both of these elements in a score, and the relative weighting of each, should be dictated by the table of specifications.3. Specific Types of Essay QuestionsThe following set of essay questions illustrates how the phrasing of an essay item could be framed to elicit particular behaviors and levels of response.1. RecallSimple recall1. What is the chemical formula for hydrogen peroxide?2. Who wrote the Emergence of Lincoln?

Selective recall in which a basis for evaluation or judgment is suggested1.Which three individuals in the 19th century had the most profound effect on contemporary life?

2.UNDERSTANDINGComparison of two phenomena on a single designated basis1.Compare the writers of the English Renaissance to those of the 19th century in their abilities to describe nature.

Comparison of two phenomena in general1.Compare the French and Russian Revolutions

Explanation of the use or exact meaning of a phrase or statement1.The Book of John begins In the beginning was the word From what philosophical system does this statement derive?

Summary of a text or some portion of it1.State the central thesis of the Communist Manifesto.

Statement of an artists purpose in the selection or organization of material1.why did Hemingway describe in detail the episode in which Gordon, lying wounded, engages the oncoming enemy?

2.What was Beethovens purpose in deviating from the orthodox symphony form in Symphony No. 6?2.UNDERSTANDING3. APPLICATIONIt should be clearly understood that whether or not a question elicits application depends on the preceding educational experience. If a particular analysis has been taught explicitly, a question involving that analysis is a matter of simple recall.Causes or effects

1.Why can too frequent reliance on penicillin for the treatment of minor ailments eventually result in its diminished effectiveness against major invasion of body tissues by infectious bacteria?

2.Why did fascism flourish in Italy and Germany but not in England and the United States?Analysis (it is advisable not to use the word analysis in the question itself)1.Why was Hamlet torn by conflicting desires?2.Why is the simple existence of slavery an insufficient explanation for the outbreak of the American Civil War?

Statement of relationship1.Intelligence is said to correlate with school achievement at about .65. Explain this relationship.

D.Illustrations or examples of principles1.Name three examples of uses of the lever in typical American homes

3. APPLICATIONE.Application of rules or principles in specified situations1.I would you weigh more or less on the moon? On the sun? Explain.

F.Reorganization of facts1.Some writers have concluded that the American Revolution was not merely a political revolution against England bur also a social revolution, within the colonies, of the poor against the wealthy. Using the available evidence to support this assertion, what other conclusion is possible3. APPLICATION4. JUDGMENTDecision for or against1.Should members of the Communist Party be allowed to teach in American colleges? Why or why not?2.Is nature or nurture more influential in determining human behavior? Why?

Discussion1.Discuss the likelihood that four-year private liberal arts colleges will gradually be replaced by junior colleges and state universitiesCriticism of the adequacy, correctness, or relevance of a statement1.The discovery of penicillin has often been called an accident. Comment on the adequacy of this explanation.

Formulation of new questions1.What should one find out to explain why some students of high intelligence fail in school?2.What questions should a scientist ask to determine why more smokers than non-smokers develop lung cancer?4. JUDGMENT