Developing Effective Probes - McGraw Hill Education Effective...Developing Effective Probes ... KEEP...

THEMCGRAW‐HILLCOMPANIES

DevelopingEffectiveProbes

AuthoringGuide

Last updated

3/7/2012

This document is the property of McGraw‐Hill companies, Inc. and its subsidiaries. The information contained herein is trade‐secret and cannot be used for any purpose other than conducting McGraw‐Hill business. When disposing of this document, please do so by either deleting, destroying, shredding or placing the document in a secure paper recycling receptacle.

D E V E L O P I N G E F F E C T I V E P R O B E S

M A R C H 7 , 2 0 1 2

2

TableofContents

DEVELOPING EFFECTIVE PROBES .................................................................................................................... 3

RULE # 1: START WITH THE LEARNING OBJECTIVE .................................................................................. 3

RULE #2 AVOID AMBIGUITY ................................................................................................................................ 5

RULE #3: AVOID NEGATIVE PROBES ................................................................................................................ 7

RULE #4: THE CORRECT RESPONSE SHOULD COMPLETE THE QUESTION/STATEMENT (AS OPPOSED TO BEGINNING IT) ............................................................................................................................... 8

RULE # 5: AVOID REDUNDANT INFORMATION .............................................................................................. 9

RULE #6: CREATE PLAUSIBLE DISTRACTORS ............................................................................................. 10

RULE #7: KEEP GRAMMAR CONSISTENT ...................................................................................................... 11

RULE # 8: AVOID UNNECESSARY OR IRRELEVANT INFORMATION ..................................................... 12

RULE #9: MAKE ANSWER CHOICES HOMOGENOUS .................................................................................. 13

RULE #10: MINIMIZE USE OF “WHICH OF THE FOLLOWING…” PHRASES ........................................ 15


M A R C H 7 , 2 0 1 2

3

Developing Effective Probes Most probes can be effectively written by keeping in mind a few general principles.

Rule # 1: Start with the Learning Objective The learning objective is your roadmap to developing probes. LearnSmart probes are designed to assess whether students have been able to achieve specific learning objectives. So, when designing a probe it’s important to start with the learning objective.

Poor Probe Design: This probe’s options identify several characteristics of carbohydrates, but proteins aren’t even mentioned.

This probe would be better suited for the following learning objective:


M A R C H 7 , 2 0 1 2

4

Improved Probe Here’s an example of an improved probe. Notice that the new probe is very specific in differentiating between carbohydrates and proteins. This probe is better designed to assess the learning objective.


M A R C H 7 , 2 0 1 2

5

Rule #2 Avoid Ambiguity This rule is especially important when designing the question stem. When students finish reading the question stem, they should know exactly what they are expected to answer.

Poor Probe Design: In this example, the question stem is ambiguous. Students may not understand the type of information being assessed.

Improved Probe Design: This question stem is clear, direct, and avoids ambiguity.


M A R C H 7 , 2 0 1 2

6

You’ll want to be especially clear when creating fill-in-the-blank probes.

Poor Probe Design: In this example, the phrase “biggest state” is ambiguous. Is the question asking for the biggest state in the United States? Is the probe asking for the state with the greatest area? Or is it asking for the state with the greatest population?

Improved Probe Design:

In the improved probe, the statement is much more specific.


M A R C H 7 , 2 0 1 2

7

Rule #3: Avoid Negative Probes A negative probe is a probe that asks the student to identify the item that is incorrect. Most researchers suggest that stems containing negatives are not effective tools for measuring achievement. Thus you should avoid probes that contain negative words such as NOT or EXCEPT.

Poor Probe Design:


Here’s a better example of the probe, where the question is asked in the affirmative.


M A R C H 7 , 2 0 1 2

8

Rule #4: The Correct Response Should Complete the Question/Statement (as Opposed to Beginning It) Poor Probe Design:

In this example the blank is positioned at the beginning of the sentence. This should be avoided as much as possible. Asking for a response before the student understands the question can cause confusion. These types of questions might also require more reading, and thus take longer than necessary to complete.


Here’s a better example of the probe, where the blank is placed at the end of the probe.


M A R C H 7 , 2 0 1 2

9

Rule # 5: Avoid Redundant Information Poor Probe Design:

Notice the redundancy in the example below. In this example, the student has to read “The cell will typically” three separate times.


Notice that we’ve placed as much information in the stem as possible to avoid redundant information in the alternative answers. This decreases the burden of reading, and more clearly defines the question or problem.


M A R C H 7 , 2 0 1 2

10

Rule #6: Create Plausible Distractors Poor Probe Design:

The probe below poses the question “Who was the 2nd president of the United States?” The distractors “George Bush” and “Bill Clinton” are not plausible distractors for most students- so students can quickly eliminate these options as possible answers.


The improved version of the probe makes all the alternatives are plausible, thus more effectively testing the student’s knowledge.


M A R C H 7 , 2 0 1 2

11

Rule #7: Keep Grammar Consistent Poor Probe Design:

Notice the grammatical clue that is given in this example. Because the article “a” was used in the probe, the only alternative that makes this statement grammatically correct is the first choice, “cell”.


Notice that in the question stem “a(n)” is used before the blank. This removes the grammatical clue from the probe.


M A R C H 7 , 2 0 1 2

12

Rule # 8: Avoid Unnecessary or Irrelevant Information Poor Probe Design:

The stem in this probe contains a lot of irrelevant information. The information regarding the year of birth, and place of birth is unnecessary, and only seeks to distract the student.


Here’s a better alternative. In this example, the irrelevant information has been removed.


M A R C H 7 , 2 0 1 2

13

Rule #9: Make Answer Choices Homogenous This rule is especially relevant when creating matching probes.

Poor Probe Design:

Look at the options in this matching probe. Note that students are expected to differentiate between dates, and individuals. This can cause confusion.


A better option would be to create a matching probe with just dates…


M A R C H 7 , 2 0 1 2

14

…and a separate matching probe with just individuals.


M A R C H 7 , 2 0 1 2

15

Rule #10: Minimize Use of “Which of the following…” Phrases Poor Probe Design:

Take a look at this example. Notice that the stem begins with the “Which of the following…” phrase. Try to avoid probes that begin with this phrase, as it forces students to look at the options presented first, before considering answers they might think up on their own.

Improved Probe:

A better option might be to recreate the question as a fill-in-the-blank probe, such as in the example here. The fill-in-the-blank structure encourages students to think up answers on their own, instead of simply choosing from a list of options.


M A R C H 7 , 2 0 1 2

16

Below is a recap of the Top Ten Rules for Developing Effective Probes:

Developing Effective Probes - McGraw Hill Education Effective...Developing Effective Probes ... KEEP...

Documents

Transcript of Developing Effective Probes - McGraw Hill Education Effective...Developing Effective Probes ... KEEP...