Making Competition 2009 - Educational InitiativesA. Their beaks are short and sharp like those of a...

National QuestionMaking Competition 2009

TIPS FOR MAKINGMULTIPLE CHOICE

QUESTIONS

www.ei-india.com/asset-qmc

National Question-Making Competition

Better learning through good questions

Published by Educational Initiatives Pvt. Ltd., Ahmedabad, 2009

This document is not copyrighted. You may make copies of it and distribute it, but must acknowledge the source. You are requested to inform Educational Initiatives in such cases.

This document has been developed and published with the objective of helping teachers improve classroom testing and evaluation.

Please send us comments and feedback at or call at 09990499504 (Atul Pandey) for more details

You can also contact us at:

[email protected]

CONTENTS:

Chapter 1 : Basics of question Making

Chapter 2 : Key principles of Question Making

chapter 3 : Different Formats of Questions

Chapter 4 : Making Multiple choice questions

chapter 5 : Using passage in Hindi

chapter 6 : Question Rating Exercise

The one real goal of education

is to leave a person asking

questions.

- Max Beerhohm

Published by Educational Initiatives Pvt. Ltd., Ahmedabad, 2009

This document is not copyrighted. You may make copies of it and distribute it, but must acknowledge the source. You are requested to inform Educational Initiatives in such cases.

This document has been developed and published with the objective of helping teachers improve classroom testing and evaluation.

Please send us comments and feedback at or call at 09990499504 (Atul Pandey) for more details

You can also contact us at:

[email protected]

CONTENTS:

Chapter 1 : Basics of question Making

Chapter 2 : Key principles of Question Making

chapter 3 : Different Formats of Questions

Chapter 4 : Making Multiple choice questions

chapter 5 : Using passage in Hindi

chapter 6 : Question Rating Exercise

The one real goal of education

is to leave a person asking

questions.

- Max Beerhohm

1 : Basics of question Making

Welcome to the 1st chapter of the Question Making Tips series. In this chapter, we look at the purpose of asking questions, and the different levels of cognitive skills that questions can test.

WHY LEARN ABOUT MAKING QUESTIONS?

A good question is one that challenges and stimulates a child to think deeply and to apply concepts learnt. The ability to ask questions that make students think both at the time of instruction and assessment, is the hallmark of a good teacher. A good question, correctly framed, can help a teacher understand the thought processes of students and how well a child has internalized a concept or mastered a skill.

Questions are very important because of the following reasons-• They influence what and how students study, and what they learn (thus they provide an opportunity to

focus on what's really worth learning). • They provide feedback on what students are learning. • They can stimulate thinking and develop higher level thinking skills, strategies and habits. • They can help to distinguish Mechanical Learning (apparent learning) from Learning for

Understanding (real learning) • They can help to identify common misconceptions, common errors.

To appreciate these points further, we need to first understand how questions can test different levels of thinking- and thereby promote or discourage certain levels of thinking. The Blooms taxonomy levels are used to explore this.

HIGHER AND LOWER ORDER SKILLS.

Bloom's Taxonomy (Benjamin Bloom (ed)., Taxonomy of Educational Objectives: Handbook I Cognitive Domain (New York: David McKay Co., 1956)) is a hierarchical system of ordering thinking skills from lower to higher, with the higher levels including all of the cognitive skills from the lower levels.Below are the levels of the taxonomy, a brief explanation of each one, and examples of questions, which require students to use thinking skills at each level.

>KNOWLEDGE: Remembering previously learned material

Skills Demonstrated • Observation and recall of information • Knowledge of dates, events, places • Knowledge of major ideas mastery of subject matter

Question Cues:List, define, tell, describe, identify, show, label, collect, examine, tabulate, quote, name, who, when, where, etc.

Question

Define a “Verb”

Example: 2

Question

State Newton's first law of motion

Example: 1

by explaining in one's own words or citing examples.

Skills Demonstrated• Understanding information • Grasp meaning • Translate knowledge into new context interpret

facts, compare, contrast • Order, group, infer causes predict consequences

Example: 1

Example: 2

>APPLICATION: Using information in a new context to solve a problem, to answer a question, or to perform another task.

Skills Demonstrated• Use information • Use methods, concepts, theories in new situations • Solve problems using required skills or knowledge

Example: 1

Example: 2

COMPREHENSION: Understanding the meaning of remembered material, usually demonstrated

Question Cues:Summarize, describe, interpret, contrast, predict, associate, distinguish, estimate, differentiate, discuss, extend

Question Cues:Apply, demonstrate, ca lcu late, complete, illustrate, show, solve, examine, modify

Question

Give two examples of adjectives.

Question

Explain the process of digestion.

Question

How does the law of supply and demand explain the current increase in vegetable prices?

Question

All living things can be grouped as PLANTS or ANIMALS. Which of these in the list below are ANIMALS?

A. All are animals.B. Lion and crocodile are animals.C. Lion, man and crocodile are animals.D. Lion, man, crocodile, fly and fish are animals

GRASSLIONFLY CROCODILEMANFISH

The unexamined life is not worth living.

– Socrates

1 : Basics of question Making

Welcome to the 1st chapter of the Question Making Tips series. In this chapter, we look at the purpose of asking questions, and the different levels of cognitive skills that questions can test.

WHY LEARN ABOUT MAKING QUESTIONS?

A good question is one that challenges and stimulates a child to think deeply and to apply concepts learnt. The ability to ask questions that make students think both at the time of instruction and assessment, is the hallmark of a good teacher. A good question, correctly framed, can help a teacher understand the thought processes of students and how well a child has internalized a concept or mastered a skill.

Questions are very important because of the following reasons-• They influence what and how students study, and what they learn (thus they provide an opportunity to

focus on what's really worth learning). • They provide feedback on what students are learning. • They can stimulate thinking and develop higher level thinking skills, strategies and habits. • They can help to distinguish Mechanical Learning (apparent learning) from Learning for

Understanding (real learning) • They can help to identify common misconceptions, common errors.

To appreciate these points further, we need to first understand how questions can test different levels of thinking- and thereby promote or discourage certain levels of thinking. The Blooms taxonomy levels are used to explore this.

HIGHER AND LOWER ORDER SKILLS.

Bloom's Taxonomy (Benjamin Bloom (ed)., Taxonomy of Educational Objectives: Handbook I Cognitive Domain (New York: David McKay Co., 1956)) is a hierarchical system of ordering thinking skills from lower to higher, with the higher levels including all of the cognitive skills from the lower levels.Below are the levels of the taxonomy, a brief explanation of each one, and examples of questions, which require students to use thinking skills at each level.

>KNOWLEDGE: Remembering previously learned material

Skills Demonstrated • Observation and recall of information • Knowledge of dates, events, places • Knowledge of major ideas mastery of subject matter

Question Cues:List, define, tell, describe, identify, show, label, collect, examine, tabulate, quote, name, who, when, where, etc.

Question

Define a “Verb”

Example: 2

Question

State Newton's first law of motion

Example: 1

by explaining in one's own words or citing examples.

Skills Demonstrated• Understanding information • Grasp meaning • Translate knowledge into new context interpret

facts, compare, contrast • Order, group, infer causes predict consequences

Example: 1

Example: 2

>APPLICATION: Using information in a new context to solve a problem, to answer a question, or to perform another task.

Skills Demonstrated• Use information • Use methods, concepts, theories in new situations • Solve problems using required skills or knowledge

Example: 1

Example: 2

COMPREHENSION: Understanding the meaning of remembered material, usually demonstrated

Question Cues:Summarize, describe, interpret, contrast, predict, associate, distinguish, estimate, differentiate, discuss, extend

Question Cues:Apply, demonstrate, ca lcu late, complete, illustrate, show, solve, examine, modify

Question

Give two examples of adjectives.

Question

Explain the process of digestion.

Question

How does the law of supply and demand explain the current increase in vegetable prices?

Question

All living things can be grouped as PLANTS or ANIMALS. Which of these in the list below are ANIMALS?

A. All are animals.B. Lion and crocodile are animals.C. Lion, man and crocodile are animals.D. Lion, man, crocodile, fly and fish are animals

GRASSLIONFLY CROCODILEMANFISH

The unexamined life is not worth living.

– Socrates

Skills Demonstrated • Seeing patterns • Organization of parts • Recognition of hidden meanings identification

of components

Example: 1

Example: 2

Example: 3

>SYNTHESIS: Putting parts together to form a new whole, pattern or structure.

Skills Demonstrated • Use old ideas to create new ones • Generalize from given facts • Relate knowledge from several areas • Predict, draw conclusions

ANALYSIS: Breaking a piece of material into its parts and explaining the relationship between the parts.

Question

What factors in the Indian economy are affecting the current price of petrol?

Question

What parts of the Goldilocks story could not have actually happened?

Question Cues:Analyze, separate, order, explain, connect, classify, arrange, divide, compare, select, explain, infer

Question

A trained monkey sits quietly or rubs its head when shown coloured cards in 3 positions as shown below. Here are some results.

One of these rules describes when the monkey rubs its head. Which one could it be?A. when there is a red card in the left-most position B. when there is a red card in the central position C. when there is a blue card in the right-most position D. when there is a blue card to the right of a red one

R R B R B R B R R

R Red card B Blue card

Left Middle Right Left Middle Right Left Middle Right

Question Cues:Combine, integrate, modify, rearrange, substitute, plan, create, design, invent, what if? Compose, formulate, prepare, generalize

Example: 1

Example: 2

Example: 3

>EVALUATION: arrive at a reasoned judgment.

Skills Demonstrated • Compare and discriminate between ideas assess

value of theories, presentations• Make choices based on reasoned argument • Verify value of evidence recognize subjectivity

Example: 1

Example: 2

Using a set of criteria, established by the student or specified by the instructor, to

Question

How is a family like a government?

How successful would the law to wear seat belts in cars and helments be in controlling accidents?

Question

Emperor Penguins in Antarctica are sea birds which dive under water to feed on fish, squids and krill. Which of the following MOST LIKELY helps it to catch its prey in the water without being caught by other predators ?

A. Their beaks are short and sharp like those of a sparrow.B. They have thick webbed feet which helps them to move fast over land.C. Their black and white colours makes it difficult for them to be seen in the

sea.D. They have plump bodies which do not allow them to dive deep into the

waters.

Question

Design an experiment to check whether the flowering of a plant is affected by the duration of sunlight exposure.

Given a list of three potential investments, including their interest rates, lengths of investment, and compounding schedule, select the best option, and defend you decision.

Question Cues:Assess, decide, rank, grade, test, measure, recommend, convince, select, judge, explain, discriminate, support, conclude, compare, summarize

The art and science of asking questions is the source of all knowledge.

– Thomas Berger

Skills Demonstrated • Seeing patterns • Organization of parts • Recognition of hidden meanings identification

of components

Example: 1

Example: 2

Example: 3

>SYNTHESIS: Putting parts together to form a new whole, pattern or structure.

Skills Demonstrated • Use old ideas to create new ones • Generalize from given facts • Relate knowledge from several areas • Predict, draw conclusions

ANALYSIS: Breaking a piece of material into its parts and explaining the relationship between the parts.

Question

What factors in the Indian economy are affecting the current price of petrol?

Question

What parts of the Goldilocks story could not have actually happened?

Question Cues:Analyze, separate, order, explain, connect, classify, arrange, divide, compare, select, explain, infer

Question

A trained monkey sits quietly or rubs its head when shown coloured cards in 3 positions as shown below. Here are some results.

One of these rules describes when the monkey rubs its head. Which one could it be?A. when there is a red card in the left-most position B. when there is a red card in the central position C. when there is a blue card in the right-most position D. when there is a blue card to the right of a red one

R R B R B R B R R

R Red card B Blue card

Left Middle Right Left Middle Right Left Middle Right

Question Cues:Combine, integrate, modify, rearrange, substitute, plan, create, design, invent, what if? Compose, formulate, prepare, generalize

Example: 1

Example: 2

Example: 3

>EVALUATION: arrive at a reasoned judgment.

Skills Demonstrated • Compare and discriminate between ideas assess

value of theories, presentations• Make choices based on reasoned argument • Verify value of evidence recognize subjectivity

Example: 1

Example: 2

Using a set of criteria, established by the student or specified by the instructor, to

Question

How is a family like a government?

How successful would the law to wear seat belts in cars and helments be in controlling accidents?

Question

Emperor Penguins in Antarctica are sea birds which dive under water to feed on fish, squids and krill. Which of the following MOST LIKELY helps it to catch its prey in the water without being caught by other predators ?

A. Their beaks are short and sharp like those of a sparrow.B. They have thick webbed feet which helps them to move fast over land.C. Their black and white colours makes it difficult for them to be seen in the

sea.D. They have plump bodies which do not allow them to dive deep into the

waters.

Question

Design an experiment to check whether the flowering of a plant is affected by the duration of sunlight exposure.

Given a list of three potential investments, including their interest rates, lengths of investment, and compounding schedule, select the best option, and defend you decision.

Question Cues:Assess, decide, rank, grade, test, measure, recommend, convince, select, judge, explain, discriminate, support, conclude, compare, summarize

The art and science of asking questions is the source of all knowledge.

– Thomas Berger

WHAT ARE LOWER/HIGHER ORDER QUESTIONS?

Lower order questions are those at the knowledge, comprehension, and simple application levels of the taxonomy. Higher-level questions are those requiring complex application (e.g., analysis, synthesis, and evaluation skills).

Usually questions at the lower orders are appropriate for: • Evaluating students' preparation and comprehension. • Diagnosing students' strengths and weaknesses. • Reviewing and/or summarizing content.

Questions at higher orders of the taxonomy are usually most appropriate for: • Encouraging students to think more deeply and critically. • Problem solving. • Encouraging discussions.

Key Takeways:

• Questions influence what and how students study, and what they learn (thus they provide an opportunity to focus on whats really worth learning).

• Use a judicious mix of lower order and higher order questions in assessments. Higher order questions can make learning enjoyable and help children's minds to reach their full potential. Higher order skills are also critical in today's life and work environment.

2 : Key principles of Question Making

Welcome to the 2nd chapter of the Question Making Tips series. In this chapter we present some of the key principles of question making. We will make use of examples to illustrate our points, and clarify how to identify good questions from average or poor ones.

FOCUS ON IMPORTANT CONCEPTS/ SKILLS:

a.) The question should test a skill or concept that is worth understanding:It is important to frame questions that help students to focus on the heart of a discipline or a concept rather than on trivial aspects, like terminologies or definitions. A good question is based on ‘Enduring Understandings’ or something that is important for the student to know or do. It should test a skill or concept that is worth understanding. It should not test mere recall. (‘Enduring Understandings’ are understandings that will remain with a student throughout his life.)

Language - Name the parts of speech in Grammar. This question tests only memory. Just knowing the names of the parts of speech without understanding their usage would make this exercise futile.

On the contrary, if a passage with gaps was given and students were asked to fill in the blanks with the most appropriate words to give the passage complete meaning, we would be testing whether students have learnt, not only the usage of the Parts of Speech, but have also comprehended an unfamiliar passage. See the example below: Complete the passage with suitable words to make the passage more meaningful.

Then he ____ on his knees and kissed the King’s feet _____. “Your _____, please forgive me,” ____ beseeched. “I made a _____ mistake.” The King smiled gently. “ ____! Why do you ask ____ forgiveness? You gave me an opportunity to see the problems ____ besiege my subjects. I am glad I was able to help you."

Science - ‘What is the temperature of the sun?’ This question tests a point which is of little importance to know and is just a factual detail, which can be obtained from different sources whenever it is required. In contrast ‘What would happen if the sun were cold and not hot?’ tests whether students understand the central role the sun occupies in their lives as well as stimulates their thinking.

It is not the answer that enlightens, but the question.

– Decouvertes

WHAT ARE LOWER/HIGHER ORDER QUESTIONS?

Lower order questions are those at the knowledge, comprehension, and simple application levels of the taxonomy. Higher-level questions are those requiring complex application (e.g., analysis, synthesis, and evaluation skills).

Usually questions at the lower orders are appropriate for: • Evaluating students' preparation and comprehension. • Diagnosing students' strengths and weaknesses. • Reviewing and/or summarizing content.

Questions at higher orders of the taxonomy are usually most appropriate for: • Encouraging students to think more deeply and critically. • Problem solving. • Encouraging discussions.

Key Takeways:

• Questions influence what and how students study, and what they learn (thus they provide an opportunity to focus on whats really worth learning).

• Use a judicious mix of lower order and higher order questions in assessments. Higher order questions can make learning enjoyable and help children's minds to reach their full potential. Higher order skills are also critical in today's life and work environment.

2 : Key principles of Question Making

Welcome to the 2nd chapter of the Question Making Tips series. In this chapter we present some of the key principles of question making. We will make use of examples to illustrate our points, and clarify how to identify good questions from average or poor ones.

FOCUS ON IMPORTANT CONCEPTS/ SKILLS:

a.) The question should test a skill or concept that is worth understanding:It is important to frame questions that help students to focus on the heart of a discipline or a concept rather than on trivial aspects, like terminologies or definitions. A good question is based on ‘Enduring Understandings’ or something that is important for the student to know or do. It should test a skill or concept that is worth understanding. It should not test mere recall. (‘Enduring Understandings’ are understandings that will remain with a student throughout his life.)

Language - Name the parts of speech in Grammar. This question tests only memory. Just knowing the names of the parts of speech without understanding their usage would make this exercise futile.

On the contrary, if a passage with gaps was given and students were asked to fill in the blanks with the most appropriate words to give the passage complete meaning, we would be testing whether students have learnt, not only the usage of the Parts of Speech, but have also comprehended an unfamiliar passage. See the example below: Complete the passage with suitable words to make the passage more meaningful.

Then he ____ on his knees and kissed the King’s feet _____. “Your _____, please forgive me,” ____ beseeched. “I made a _____ mistake.” The King smiled gently. “ ____! Why do you ask ____ forgiveness? You gave me an opportunity to see the problems ____ besiege my subjects. I am glad I was able to help you."

Science - ‘What is the temperature of the sun?’ This question tests a point which is of little importance to know and is just a factual detail, which can be obtained from different sources whenever it is required. In contrast ‘What would happen if the sun were cold and not hot?’ tests whether students understand the central role the sun occupies in their lives as well as stimulates their thinking.

It is not the answer that enlightens, but the question.

– Decouvertes

b.) The question should really test what it intends to test.

This is an important criterion to keep in mind when framing a question. Let us look at the examples given below:

CLARITY OF A QUESTION: The question should be clear in terms of being specific, and in simple and clear language (unless the intention is to test comprehension). The problem with the ambiguous question is that the teacher/ tester may have a certain expectation in terms of the length or the level of detail of the answer, which does not come across clearly. This may lead to some students not doing well on this question as the expectations were not stated clearly. State what is expected in clear and unambiguous terms.

Example: Science

Balance the see-saw.

The teacher in tended to tes t an understanding of levers, and her expected correct answer was that one of the weights be removed from the right hand side. However, there are several issues with this question-

a) The question does not specify whether weights can be removed or added to the lever, and some students might add a weight on the left hand side (or remove 2 weights from the right hand side and 1 weight from the left hand side)

b) Even if this is not an issue, and students do answer as per the expectation of the teacher, the question would ‘at most’ test the notion of equality, and not an understanding of levers, as the lever can be balanced by moving the weights on the right hand side ‘in’ towards the fulcrum.

This question could possibly be modified like this:

Here is a see-saw. A fat man is seated at one end and a thin man at the other end. How best should they seat themselves in order to balance the see saw?

CORRECTNESS OF QUESTIONS The question should be completely free of language and conceptual error as they can affect the test results negatively and confuse students as well.

Maths In the question on the right, the question maker expects students to choose the answer ‘8’, but that is incorrect, as more squares can be formed by counting the main squares inside which the small squares exist. A student who thinks deeply about this is not likely to select any answer as the correct answer is not in the options. To correct this, the question and options could be worded as:(See the question on the right in the table below.)

AGE APPROPRIATENESS OF LANGUAGE IN QUESTIONS Use language that is appropriate to the age level of the class in question. Let’s look at the Maths example given on the next page: Many primary school children would find it hard to understand the question because of the language and terminologies in the question.

An ambiguous question - Write a diary entry in your notebook.

Here, the question does not mention which day / time / period / situation, within which the student is expected to, write a diary entry. In other words the question lacks direction and will consume quite a bit of the students’ time in trying to decide what to write about.

A more specific question:

Your school took you to see the Science Exhibition. Write a diary entry in your notebook, describing the visit and your feelings therein, in not

more than 200 words.

A question with a conceptual error:

The number of squares formed in the given figure is____.A. 4B. 63. 34. 8

A question with no conceptual error:

The maximum number of squares formed in the given figure is____.A. 8B. 6C. 7D. 10

Question

A prudent question is one-half of wisdom.

– Francis Bacon

b.) The question should really test what it intends to test.

This is an important criterion to keep in mind when framing a question. Let us look at the examples given below:

CLARITY OF A QUESTION: The question should be clear in terms of being specific, and in simple and clear language (unless the intention is to test comprehension). The problem with the ambiguous question is that the teacher/ tester may have a certain expectation in terms of the length or the level of detail of the answer, which does not come across clearly. This may lead to some students not doing well on this question as the expectations were not stated clearly. State what is expected in clear and unambiguous terms.

Example: Science

Balance the see-saw.

The teacher in tended to tes t an understanding of levers, and her expected correct answer was that one of the weights be removed from the right hand side. However, there are several issues with this question-

a) The question does not specify whether weights can be removed or added to the lever, and some students might add a weight on the left hand side (or remove 2 weights from the right hand side and 1 weight from the left hand side)

b) Even if this is not an issue, and students do answer as per the expectation of the teacher, the question would ‘at most’ test the notion of equality, and not an understanding of levers, as the lever can be balanced by moving the weights on the right hand side ‘in’ towards the fulcrum.

This question could possibly be modified like this:

Here is a see-saw. A fat man is seated at one end and a thin man at the other end. How best should they seat themselves in order to balance the see saw?

CORRECTNESS OF QUESTIONS The question should be completely free of language and conceptual error as they can affect the test results negatively and confuse students as well.

Maths In the question on the right, the question maker expects students to choose the answer ‘8’, but that is incorrect, as more squares can be formed by counting the main squares inside which the small squares exist. A student who thinks deeply about this is not likely to select any answer as the correct answer is not in the options. To correct this, the question and options could be worded as:(See the question on the right in the table below.)

AGE APPROPRIATENESS OF LANGUAGE IN QUESTIONS Use language that is appropriate to the age level of the class in question. Let’s look at the Maths example given on the next page: Many primary school children would find it hard to understand the question because of the language and terminologies in the question.

An ambiguous question - Write a diary entry in your notebook.

Here, the question does not mention which day / time / period / situation, within which the student is expected to, write a diary entry. In other words the question lacks direction and will consume quite a bit of the students’ time in trying to decide what to write about.

A more specific question:

Your school took you to see the Science Exhibition. Write a diary entry in your notebook, describing the visit and your feelings therein, in not

more than 200 words.

A question with a conceptual error:

The number of squares formed in the given figure is____.A. 4B. 63. 34. 8

A question with no conceptual error:

The maximum number of squares formed in the given figure is____.A. 8B. 6C. 7D. 10

Question

A prudent question is one-half of wisdom.

– Francis Bacon

INTEREST AND STIMULATION The question should be interesting and thought provoking, so that it stimulates a student to think further about the question before arriving at a logical conclusion. Science - A group of students observed the following tree while hiking through a hilly area. What is the most likely cause for the angle of growth of this tree? A. The tree is reaching toward the afternoon sunlight. B. Water rolling down the hill uprooted the tree. C. Large rocks hit the tree when they rolled down the hill. D. Strong uphill winds have pushed against the tree. A question such as this forces students to think about what kind of reasons could be there for the tree to grow at such an angle, and make a judgment on why A, B and C are not plausible options. By attempting (and possibly discussing) such a question, the students’ power of thinking is stretched.

Key Takeaways:

In summary, these are the main points to keep in mind when framing a good question. A good question should: • test a skill or concept that is worth understanding. • test a skill or concept that is worth understanding. • really test what it intends to test. • be framed in such a way that it states what is expected in clear and unambiguous terms. • contain language that is appropriate to the age level of the class in question. • be completely free of language and conceptual error as they can affect the test results

negatively confuse students as well. • create interest so that it stimulates a student to think further about the question before

arriving at a logical conclusion.

A question with inappropriate language for the age level:

Manish writes a 3-digit number. The number has a 3 with a place value of 3, a 5 with a place value of 500 and the digit 7. Which of the following is Manish’s number?

A. 573 B. 357 C. 530 D. 573

A question with appropriate language for the age level:

Manish writes a 3-digit number. The number has 3 in the ones place, 5 in the hundreds place and 7 in the tens place. Which of the following is Manish’s number?

A. 573 B. 357 C. 530 D. 573

3 : Different Formats of Questions

Welcome to the 3rd chapter of the Question Making Tips series. In this chapter we present some of the different formats of questions which are commonly used. Different formats are useful for different purposes, and in particular contexts. These points will be explained along with examples.

The true/false question is a simple statement that is either true or false.

>Constructing True/False Questions- Key Points

• Make questions require application of knowledge as much as possible. • Always make certain the statement is unequivocally true or false. • Develop approximately the same number of true and false statements. • Do not consistently make true statements longer than false statements, or vice versa • Avoid negative and complicated statements. • Words such as "all," "only," "never," and "always" give hints about the answer and are usually false. Use

them only when the question is to measure understanding. • Statements with the words "generally" and "usually" tend to be true.

Ask students to justify their answers for statements that are false or provide counter examples to decrease the possibility of guessing.

Example:

True/False Questions:

Advantages • Quite easy to frame.• Scoring is easy and objective.

Disadvantages • Students have a 50 percent chance of

being correct or wrong.• May be perceived as an unfair judgment of

learning.• Need a larger number of items to

dist inguish stronger and weaker knowledge levels. -

Question framing without asking for justification

State whether the statements below are true or false.• Every number ending with 0 is divisible by

2. - True• Every multiple of 5 is a also a multiple of

10. - False

Question framing asking for justification

State whether the statements below are true or false, and if false, give an example to show it is so.• Every multiple of 5 is a also a multiple of

10. - False-Example: 15 is divisible by 5, but not by 10

To raise new questions,new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advance in science

– Albert Einstein

INTEREST AND STIMULATION The question should be interesting and thought provoking, so that it stimulates a student to think further about the question before arriving at a logical conclusion. Science - A group of students observed the following tree while hiking through a hilly area. What is the most likely cause for the angle of growth of this tree? A. The tree is reaching toward the afternoon sunlight. B. Water rolling down the hill uprooted the tree. C. Large rocks hit the tree when they rolled down the hill. D. Strong uphill winds have pushed against the tree. A question such as this forces students to think about what kind of reasons could be there for the tree to grow at such an angle, and make a judgment on why A, B and C are not plausible options. By attempting (and possibly discussing) such a question, the students’ power of thinking is stretched.

Key Takeaways:

In summary, these are the main points to keep in mind when framing a good question. A good question should: • test a skill or concept that is worth understanding. • test a skill or concept that is worth understanding. • really test what it intends to test. • be framed in such a way that it states what is expected in clear and unambiguous terms. • contain language that is appropriate to the age level of the class in question. • be completely free of language and conceptual error as they can affect the test results

negatively confuse students as well. • create interest so that it stimulates a student to think further about the question before

arriving at a logical conclusion.

A question with inappropriate language for the age level:

Manish writes a 3-digit number. The number has a 3 with a place value of 3, a 5 with a place value of 500 and the digit 7. Which of the following is Manish’s number?

A. 573 B. 357 C. 530 D. 573

A question with appropriate language for the age level:

Manish writes a 3-digit number. The number has 3 in the ones place, 5 in the hundreds place and 7 in the tens place. Which of the following is Manish’s number?

A. 573 B. 357 C. 530 D. 573

3 : Different Formats of Questions

Welcome to the 3rd chapter of the Question Making Tips series. In this chapter we present some of the different formats of questions which are commonly used. Different formats are useful for different purposes, and in particular contexts. These points will be explained along with examples.

The true/false question is a simple statement that is either true or false.

>Constructing True/False Questions- Key Points

• Make questions require application of knowledge as much as possible. • Always make certain the statement is unequivocally true or false. • Develop approximately the same number of true and false statements. • Do not consistently make true statements longer than false statements, or vice versa • Avoid negative and complicated statements. • Words such as "all," "only," "never," and "always" give hints about the answer and are usually false. Use

them only when the question is to measure understanding. • Statements with the words "generally" and "usually" tend to be true.

Ask students to justify their answers for statements that are false or provide counter examples to decrease the possibility of guessing.

Example:

True/False Questions:

Advantages • Quite easy to frame.• Scoring is easy and objective.

Disadvantages • Students have a 50 percent chance of

being correct or wrong.• May be perceived as an unfair judgment of

learning.• Need a larger number of items to

dist inguish stronger and weaker knowledge levels. -

Question framing without asking for justification

State whether the statements below are true or false.• Every number ending with 0 is divisible by

2. - True• Every multiple of 5 is a also a multiple of

10. - False

Question framing asking for justification

State whether the statements below are true or false, and if false, give an example to show it is so.• Every multiple of 5 is a also a multiple of

10. - False-Example: 15 is divisible by 5, but not by 10

To raise new questions,new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advance in science

– Albert Einstein

Matching questions involve paired lists that require students to correctly identify, or "match," the relationship between the items.

>Constructing Matching Questions- Key Points

• Avoid equal-sized lists by including a few "distracter" items in the second (answer) column, to decrease the probability of guessing.

• Use it to assess related items like: states and capitals, terms and definitions, tools and uses, and events and dates.

>Short-Answer Questions:

Short-answer questions are open-ended questions that require students to construct a response. Short-answer items typically require responses of one word to a few sentences.

"Fill in the blank" and "Completion" questions are examples of short-answer question types.

Matching Questions

Advantages • Assesses a large amount of information in a

confined space.• If developed carefully, the probability of

guessing is low.

Advantages • Short-answer questions reduce the

possibility of guessing.• They are relatively easy to write.

Disadvantages • Matching assesses recognition rather than

recall of information.

Disadvantages • May end up testing memorization ability• Only suitable for questions that can be

answered with short responses.

Fill in the blank.

A mouse started from its hole and went along the path shown to the cheese.

On the way, it made ________ RIGHT turns?

Complete

Put the correct number in the empty box.

2 × = 18

CHEESE

• Frame short-answer questions, in such a way that the desired student response is clear. Aim at eliciting a limited number of possible correct responses (or only one) to be able to pinpoint which responses deserve higher scores/ indicate a higher ability.

• Design for recall of specific names, facts, and basic knowledge. • Ensure that the question provides enough information so that students know what constitutes a

correct response. For example, if you discussed four reasons for World War II and you want students to name them, be specific on how many reasons constitute a complete answer (i.e., Name two of four/Name any three, etc.).

• Ensure that the answer will require only a few words. • Write the stem in your own words; that is, don’t repeat the words used in the textbook or your class notes. • When the stem is an incomplete statement, place the "blank" for the correct choice at the end of the statement. • When a negative item is used, emphasize the negative word or phrase, that is, underline, capitalize, or

italicize, for example, "DOES NOT."

>Essay type Questions

An essay question indicates a specific goal. It is a short, clear question that can be answered quickly with students’ knowledge or opinions. A one minute essay question can be used to check understanding, provide feedback, and promote reflection.

>Constructing Essay-type questions- Key Points

• Use them to provide reasons for true/false or short answer questions. • Use this to get students to state their opinions based on logical reason. • Provide guidelines to enable students to compare and contrast clearly.

Illustration of how to increase focus while using essay type questions:Which of the following questions is the most effective in focusing students’ thought?

A asks for an opinion. It basically asks what the student thinks should be done. Any answer can be a correct one. B asks for an opinion but also forces the student to recognize some of the implications of his/her decision.

Constructing Short Answer Questions- Key Points

Advantages of Essay-type questions• Promotes reflection and shows what

students THINK. • Checks on background knowledge. • Reveals misconceptions. • Can be composed quickly.

Example A:What is your opinion of mercy killing (euthanasia)?

Disadvantages of Essay-type questions• Does not check in-depth understanding • Takes time to correct and poses

subjectivity in evaluation.

Example B:Assisted suicide is legal in some countries. Are you in favour of, or opposed to, making it legal in our country? In your answer include the impact on various age groups, the medical profession, and the economic situation of the country.

It is error only ,and not truth , that shrinks from inquiry.

– Thomas Paine

Matching questions involve paired lists that require students to correctly identify, or "match," the relationship between the items.

>Constructing Matching Questions- Key Points

• Avoid equal-sized lists by including a few "distracter" items in the second (answer) column, to decrease the probability of guessing.

• Use it to assess related items like: states and capitals, terms and definitions, tools and uses, and events and dates.

>Short-Answer Questions:

Short-answer questions are open-ended questions that require students to construct a response. Short-answer items typically require responses of one word to a few sentences.

"Fill in the blank" and "Completion" questions are examples of short-answer question types.

Matching Questions

Advantages • Assesses a large amount of information in a

confined space.• If developed carefully, the probability of

guessing is low.

Advantages • Short-answer questions reduce the

possibility of guessing.• They are relatively easy to write.

Disadvantages • Matching assesses recognition rather than

recall of information.

Disadvantages • May end up testing memorization ability• Only suitable for questions that can be

answered with short responses.

Fill in the blank.

A mouse started from its hole and went along the path shown to the cheese.

On the way, it made ________ RIGHT turns?

Complete

Put the correct number in the empty box.

2 × = 18

CHEESE

• Frame short-answer questions, in such a way that the desired student response is clear. Aim at eliciting a limited number of possible correct responses (or only one) to be able to pinpoint which responses deserve higher scores/ indicate a higher ability.

• Design for recall of specific names, facts, and basic knowledge. • Ensure that the question provides enough information so that students know what constitutes a

correct response. For example, if you discussed four reasons for World War II and you want students to name them, be specific on how many reasons constitute a complete answer (i.e., Name two of four/Name any three, etc.).

• Ensure that the answer will require only a few words. • Write the stem in your own words; that is, don’t repeat the words used in the textbook or your class notes. • When the stem is an incomplete statement, place the "blank" for the correct choice at the end of the statement. • When a negative item is used, emphasize the negative word or phrase, that is, underline, capitalize, or

italicize, for example, "DOES NOT."

>Essay type Questions

An essay question indicates a specific goal. It is a short, clear question that can be answered quickly with students’ knowledge or opinions. A one minute essay question can be used to check understanding, provide feedback, and promote reflection.

>Constructing Essay-type questions- Key Points

• Use them to provide reasons for true/false or short answer questions. • Use this to get students to state their opinions based on logical reason. • Provide guidelines to enable students to compare and contrast clearly.

Illustration of how to increase focus while using essay type questions:Which of the following questions is the most effective in focusing students’ thought?

A asks for an opinion. It basically asks what the student thinks should be done. Any answer can be a correct one. B asks for an opinion but also forces the student to recognize some of the implications of his/her decision.

Constructing Short Answer Questions- Key Points

Advantages of Essay-type questions• Promotes reflection and shows what

students THINK. • Checks on background knowledge. • Reveals misconceptions. • Can be composed quickly.

Example A:What is your opinion of mercy killing (euthanasia)?

Disadvantages of Essay-type questions• Does not check in-depth understanding • Takes time to correct and poses

subjectivity in evaluation.

Example B:Assisted suicide is legal in some countries. Are you in favour of, or opposed to, making it legal in our country? In your answer include the impact on various age groups, the medical profession, and the economic situation of the country.

It is error only ,and not truth , that shrinks from inquiry.

– Thomas Paine

The 'Multiple Choice' format will be presented in the next chapter.

Key Takeaways:

• Different question types are suited for different purposes and contexts. Understanding this helps us to choose the appropriate question type in a given situation.

• It may be necessary to have a mix of different question types in a test paper to achieve the desired assessment objective.

• We should make sure that the necessary information is provided in the question/ stem, and be clear about the desired student response.

• Finer points should be kept in mind while constructing questions, so that clues are not given away inadvertently.

Welcome to the 4th chapter of the Question Making Tips series. In this chapter, we present an extension of the different formats of questions presented in the previous chapter. Multiple-Choice is a form of assessment in which respondents are asked to select one or more choices from a list. Writing good Multiple Choice Questions (MCQs) requires specialised expertise.

Multiple choice items consist of two parts: (1.) a stem and (2.) a set of options.

The stem is the beginning part of the item that presents the item as a problem to be solved, a question asked of the respondent, or an incomplete statement to be completed, and contains any other relevant information. The options are the possible answers that the respondent can choose from, with the correct answer called the key and the incorrect answers called distracters.

For advanced items, such as an applied knowledge item, the stem can consist of multiple parts. The stem can include authentic material such as a graph, a table, or a detailed description which has multiple elements to it. In such cases, the stem ends with a lead-in question explaining how the respondent must answer.

>Advantages

• MCQs can be designed with a diagnostic end in mind, in order to find out whether specific areas of a given subject are adequately known, or in order to detect misconceptions. This can provide feedback on the effectiveness of instruction.

• MCQs can be scored quickly and reliably using computer systems (or even manually for small numbers) as compared to other question types. Also, the wrong answer choices can provide valuable information about how students are thinking. For example, a teacher can give a 5-10 question MCQ test on a topic to a class of 40 students, and use the data powerfully to gain insights about the understanding levels in her class.

• MCQs can be set at different cognitive levels. For example, a question may simply challenge a student's ability to recall facts, while another may test a student's ability to apply factual knowledge to given situations; or, at a higher level, a question may test a student's ability to evaluate given information.

• Tests made up of MCQs can provide a better coverage of content than essay-type questions, thus broadening the scope of the test.

• MCQs provide an opportunity to assess the abilities of students with writing difficulties in a fair manner.

Disadvantages

• Challenging to write higher-level questions • Cannot test ability to organize and present knowledge • Time consuming to construct a good question with good distracters (wrong answer choices)

Multiple-Choice Questions

4 : Making Multiple choice questions

You can tell whether a man is clever by his answers. You can tell whether a man is wise by his questions.

– Naguib Mahfouz

To find the exact answer one must find the exact question.

– S. Tobin Webster