Types of Test QuestionsClayton R. Wright, crwr77[at]gmail.com

You can use several types of test or assessment questions to ascertain whether or not your students can achieve specified objectives. Select the question type that best meets your needs. Perhaps, I am old-fashion, but I believe that every student should write at least one essay question during a course as this type of question can require students to display their ability to sift through information, organize it, and display their critical thinking skills – I want to know if they can apply what they have learned, can they use their newly-acquired information and skills in new situations, can they discuss the ethical ramifications of what they have learned, and can they manage change effectively. However, in the digital world, instead of asking learners to write an essay, you might ask students to produce a multimedia presentation or to publish a blog or wiki.

Essay Questions

Essay questions are used to test the achievement of complex skills such as application, synthesis, and evaluation. They are used to assess thinking processes and creativity. You can use them to ask “What if…” questions to which there are no definitive answers. However, you evaluate how they construct their answer, how it is organized, and the information they use to support their point of view.

Although these test questions can be constructed quickly, essays can take a long time to mark and to mark in a reliable manner. Because marking can be very subjective, you should construct an answer key that contains a number of acceptable responses. Some instructors modify this answer key after they have sampled a number of the students’ papers. If this procedure is followed, one must question whether an adequate acceptable standard was established for the completion of the course objectives. However, this latter procedure may be acceptable if all students faced an interruption to the course schedule so that adjustments had to be made to the course or if the instructor was aware that he or she was not able to complete the course objectives as originally planned.

When constructing essay questions, consider the items below. Since marking essay questions can be subjective and thus the results can be

unreliable, use essay questions when other forms of question types cannot be used. Use essays for assessing organization skills, thought processes, and the ability to think “outside of the box”. Naturally, they are also used to test writing and communication skills. Do not use them to assess knowledge and comprehension skills. Use then to address “why” questions, to assess a process, and to evaluate specific circumstances.

Carefully, identify the area that you want to cover. Ensure that it can be addressed in the time you will provide to answer the question.

Consider the language capability of the students and write the question at a level that is appropriate to them. Expect them to respond at a similar level.

Write clear directions. Give them some idea as to what is expected. For example, in the directions tell them that they should identify at least five key factors that influence global warming instead of saying “Identify factors that influence global warming and explain how these affect the local climate.”

Indicate what will be taken into account as the question is graded. For example, if you will take note of their spelling or punctuation mistakes, tell them.

Draft an answer key before issuing the question. Ensure that your expectations are reasonable. Assign a weight to each of the items that should be included in the response.

When marking essay questions, score the same question submitted by all students at the same time. Then, score the second question on all papers.

If you are unable to evaluate a question the first time you read it, set it aside until you have evaluated all the remaining essay responses to the same question.

If possible, score the responses anonymously so that you focus on the content rather than the person who wrote it. Have the students place their name on a cover sheet rather than next to the essay response. However, it is acknowledged that after a time, you will begin to recognize the patterns of words some students may use. This is likely to happen if you ask them to submit a number of writing assignments.

If an essay examination is administered that will be a significant part of the students’ formal assessment, consider having several instructors review each essay paper in order to improve the reliability of the final outcome.

If you decide to ask students to complete an “open-book” essay question, set a short specific time limit and state that plagiarism will not be tolerated. All content taken from another source must be referenced appropriately. Provide them with sample references or direct them to use reference standards provided by organizations such as APA. Indicate that when they attach their signature to the essay, they are stating that the work they are submitting is their own and no one assisted them while writing the essay.

Unless written work is essential to the application of the skills taught, you might consider

conducting an oral essay test. However, oral tests take time and must be administered to each student individually. If you need to measure communication skills and interactions with others, then use oral tests and videotape or digitally record them.

Matching Questions

Matching questions can be used to test associations or relationships and, if well constructed, they can be used to verify that students have knowledge of a specific sequence of events. Usually matching questions are used to check the attainment of lower levels of learning. They can address questions that begin with who, when, where, and what. Matching questions are best for knowledge, comprehension and some application-type questions, while multiple-choice questions are better for application, analysis, synthesis, and evaluation.

When constructing matching questions, consider the items below.

Clearly define the basis on which the matching should occur. Students should not have to figure out how items are related. If they make the wrong assumption or determine a relationship different than the one you intended, they will waste their time. In addition, they will loose marks, not because of their ability, but because you have stated the question poorly.

Use at least six pairings and no more than twelve. If there are fewer than six, a multiple-choice question might be preferable. If more than twelve pairings are required, it may be difficult to keep track of questions that deal with the sequencing of events.

Provide additional pairings or distracters in order to reduce the element of chance. _____________________________________________________________________________________Types of Questions by Clayton R. Wright, crwr77[at]gmail.com 2

Unless instructors allow students to select a response more than once, or unless one list of items is longer than the other, students may achieve success simply by eliminating items rather than by knowing the correct answer. Normally, the longest list appears on the left-hand side.

All the items in a list should be similar in form. For example, if you are matching a country to its currency, then the list of currencies should all be in the same form – symbols or words.

Arrange the lists in some logical order. The list could contain items arranged according to size, date, or the alphabet.

Place all lists to a single question on the same page. It can be confusing and distracting if students must flip the page back and forth to find possible matches.

Frequently, matching questions are constructed in a written form. However, depending on the course content, it may be better to use different three-dimensional objects and ask students to demonstrate how objects are related.

Multiple-Choice Questions

Multiple-choice questions can be used in a variety of situations and can test all domains of learning in a relatively short period of time. If well constructed, they can be highly reliable and objective. These questions can be scored quickly and allow users to apply statistical analysis that can determine what questions are easy, what questions are difficult, what questions are best predictors of a student’s final score, which distracters work effectively, and so forth.

Poorly constructed multiple choice questions have often been called "multiple-guess"

questions. Quality questions, with plausible alternative choices, take time to develop. One of the best ways to construct good multiple-choice questions is to write a short scenario or case study. Then, ask a series of questions based on this scenario. Consider all the errors students are likely to make and include these errors in the possible choices.

It can take time to construct an effective multiple-choice question as it can be difficult to construct a number of suitable distracters. It can also be difficult to construct a number of questions that do not contain clues which students can use to answer other questions on the test.

When constructing multiple-choice questions, consider the items below. The question should focus on one a problem or one aspect of a problem if the

problem is complex. How will you know what portion of the problem a student understands if you include a variety of aspects in your problem statement or try to address a variety of concepts at the same time? Focus on one issue at a time.

Place most of the information in the first part or introductory statement which is referred to as the “stem”. Only when you are seeking the definition of a term should the stem be one or two words. Thus, the stem should be long and the alternatives brief.

If you include a controversial opinion in your stem, ensure that you provide a reference for it. You do not want your students to think that it is your opinion. If they do, it might colour their opinion of you and how they answer the other questions on the test.

Write concisely. Remove any superfluous words from your question. _____________________________________________________________________________________Types of Questions by Clayton R. Wright, crwr77[at]gmail.com 3

Do not repeat information in the distracters or the alternative statements. This will reduce the amount of time it takes students to complete a question.

Do not place information in the distracters that may alert readers to the correct response. For example, if the stem ends with “an”, then only those distracters that begin with a word that needs an “an” before it will be correct.

Ensure that all distracters are almost the same length. Usually, the longer stems are correct as they include qualifiers to describe the item and to minimize the incorrect interpretation of the meaning.

Do not include clues in one distracter that could help with eliminating other possibilities.

Make the distracters or alternatives plausible. Do not include items that are obviously wrong to your average student.

Write questions that contain reasonable or plausible scenarios. Even if you are asking a “what if” question, the scenario should have some basis of reality.

If there is a negative statement in the question, highlight the negative in some way. Bold or italicize it.

Construct four of five distracters for each question. Three distracters may encourage guessing and six or more distracters means that a student spends a significant amount of time responding to one question that only covers a small portion of a course.

Ensure that only one distracter is correct unless your answering scheme allows for more than one answer. For example, you could assign 5 marks for the best response; 3 marks for a correct response, but not the best response considering the conditions; 0 marks for a response that is neutral, it does not cause any harm; -3 for a response that is incorrect, but would cause harm or damage; -5 for a response that is gravely wrong as it would cause serious injury or yield significant negative ramifications

There are many sources for multiple-choice questions. Textbook publishers often provide test questions on paper or on computer discs. However, these questions must be reviewed carefully because they frequently test knowledge skills only.

Several professional and vocational organizations have developed multiple-choice tests that they use to determine if a student should receive professional accreditation. These organizations may not be willing to release their exams unless the trainers can guarantee that the questions will be kept secure.

Consider using computer software that can generate questions from a testbank, score student responses, provide feedback, and analyze the quality of test items. There are a number of online test creation tools. Some do not charge for their services, while others offer a premium service for a price. Before using these services, note that those that offer free services usually limit the number of questions you can ask and the manner in which you can ask them. Further, you might not be able to export any statistics generated by the software to an Excel spreadsheet or an electronic grade book. Thus, you might have all this information available to you, but you must transfer each statistic or grade “by hand” – a time-consuming process.

Short-Answer Questions

Short-answer or completion questions are used to test low-level skills and to minimize guessing. Students must complete a small task or insert the words or numbers that

_____________________________________________________________________________________Types of Questions by Clayton R. Wright, crwr77[at]gmail.com 4

correctly complete a statement. Consequently, a number of skills can be tested in a short period of time. Short answer questions can also be used to determine if a student can follow a sequence of events.

Short-answer type questions are usually associated with sentence completion exercises, but they can also involve labelling a diagram or completing a puzzle.

Although short answer questions can be objective and can be scored quickly, they are prone to ambiguity. Unless these test questions are well-constructed, students spend more time trying to determine what is being asked rather than formulating the correct answer. If you make the questions clear so that the answers will be brief, you will likely only stress the recall of factual information. If you provide a number of blank spaces in a statement to be filled in, you might really be testing the ingenuity of the students to provide responses that can be interconnected in some way. You might not obtain the responses you are seeking.

When constructing short answer or completion questions, consider the items below. Provide clear direction to focus or limit the number of possible responses. Omit only key words from the statements.

Avoid using articles such as “an” before a blank as that will restrict the possible responses that may be given.

Place blanks at the end of the statement so that students get some idea of what you are saying before they are required to provide a response.

Blanks should replace words rather than complete phrases or sentences. Thus, it will be easier to score as the responses will be limited.

Provide a reasonable length for each blank. The length of the blanks should be consistent so as not to give away the possible

answer as would occur if one blank is short, indicating you are looking for a one word response.

Draft an answer sheet and note alternative responses. Are there any responses to which you would assign partial marks?

True-False Questions

True-false questions can be used to cover a large number of items in a short period of time. These questions are excellent for assessing previous learning or summarizing new material. True-false questions test recognition and are suitable for assessing low-level skill attainment. Since they also force students to make choices, they can be used to evaluate the ability of students to apply principles to new situations.

Good true-false questions are not easy to construct. It can be extremely difficult to construct questions that are definitely or unequivocally true or false. Students are prone to guess at the answer to true-false questions. When true-false questions are employed, some instructors devise complicated marking schemes, such as the subtraction of wrong responses from correct responses. It would seem more appropriate to spend the time making good test items than to spend the time to devise various schemes to re-adjust the scores for the possibility of guessing.

When constructing true-false questions, consider the items below. Make each statement absolutely true or false. Include all the qualifiers that would

make the statement absolutely true or false. _____________________________________________________________________________________Types of Questions by Clayton R. Wright, crwr77[at]gmail.com 5

Be precise. Avoid using terms such as smaller or larger. Instead use specific measurements.

Note that some words such as “generally”, “may”, “probably”, and “sometimes” are usually found in true statements. Word such as “all”, “always”, “never”, “none”, and “no” are usually found in false statements.

Avoid double negatives as the meaning of the statement can be unclear. Do not make trivial changes to a statement in order to make it wrong. For example,

you could change an article from “a” to “the” in order to make a statement wrong. But then you would be testing how well someone can read and not whether they could understand the learning objective.

Do not lift sentences from the text or PowerPoint slide and change one word to make it false. Usually, these statements will be taken out of context. Devise statements specifically for true and false questions.

If you use controversial or opinionated statements, indicate the source so that the student can place the statement in context with the author’s work.

Ensure that some questions are true and some are false, but do not have an equal number of true and false questions.

_____________________________________________________________________________________Types of Questions by Clayton R. Wright, crwr77[at]gmail.com 6