1 Testing and Grading Ensuring that we meet the objectives.
-
Upload
charla-lloyd -
Category
Documents
-
view
215 -
download
0
Transcript of 1 Testing and Grading Ensuring that we meet the objectives.
2
Desired Learning Levels
TeachingPhilosophy
Students Abilities Course Level and Scope
Write Course and Topic Objectives
Select Effective Teaching Methods
How Students Learn
BA 9200 Course Overview
The Lecture
Active LearningTesting and Grading
3
Testing Overview
A. Preparing Exams 1. Type of Tests2. Number of Tests3. Questions per Test, and4. Table of Specification
B. Writing Questions 1. Effective Stems and Distracters
2. Structured Essay Questions and Answers
C. Grading 1. Scoring Key and PartialCredit for Multiple Choice
D. Improving Exams 1. Survey and Difficulty and Discrimination Indices
4
A. Preparing Exams
1. Type of Tests
Norm Based Criterion Based
Goal Sort and Classify Students Assess Student Competency
Scores Must Have Large Spread in Scores
Spread in Scores Not Relevant
Difficulty Questions Must Range from Easy to Difficult
Questions Must Reflect Demands of Profession
Timing All Students May Not Finish Exam
All (or Most) Should Finish Exam
5
A. Preparing Exams
2. Number of Tests
Not Effective: One Exam at the end Effective: Unit Tests/Quizzes followed by a final exam.
6
A. Preparing Exams
3. Criterion-Based Tests: Number of Questions per Test
RoteMultiple choiceShort Answer
2 minutes4 minutes
MI LevelMultiple ChoiceShort Answer
4 minutes8-10 minutes
Critical ThinkingMultiple ChoiceShort Answer
10-15 minutes15-30 minutes
Rough guideline for time per question at each level.
7
A. Preparing Exams 4. Table of Specification
Rote Exercise MI
Measures of Center
15% 25%
Measures of Spread
10% 15% 10%
Shape of Distribution
10% 15%
8
Testing Overview
A. Preparing Exams 1. Type of Tests2. Number of Tests3. Questions per Test, and4. Table of Specification
B. Writing Questions 1. Effective Stems and Distracters
2. Structured Essay Questions and Answers
C. Grading 1. Scoring Key and PartialCredit for Multiple Choice
D. Improving Exams 1. Survey and Difficulty and Discrimination Indices
9
B. Writing Questions
1. Multiple Choice Questions
Essay Questionsa. Require the Student to Remember Information.b. Are Scored More Reliably.c. Are More Difficult to Construct.d. Are Easier to Construct.
The Median isa. the Average Value.b. the 25% Percentile Value.c. the 75% Percentile Value.d. that Number such that Approximately 50% of the Numbers Are Higher and 50% are Lower than the Median Value.
10
B. Writing Questions Multiple Choice Questions
The Term Used in Connection with Sound that Corresponds to Color with Reference to Light is a. Pitch.b. Hue.
c. Timbre. d. Intensity.
One Basic Advantage of Using Short Answer Essay Questions as Opposed to Long Answer Ones is that Short Answer Essay Itemsa. Generating Questions is Easier.b. Allow for a Broader Sample of Questions.c. Students Prefer Them.d. They Relate More Closely to the Course Objectives.
11
B. Writing Questions Writing Effective Stems Ideal Stem Answerable Without Seeing Choices. Questions Address Important Concepts. Relate Each Question to One or More Topic Objectives.
Question Stem Followed by Cognitive Objectives that Question Addresses{O12, O15}
Cite Authority in Stem where applicaple
12
B. Writing Questions Writing Effective Stems
Solve Math Problems to Ensure All Data in Stem. State Stem in Positive Language. Same Phrase Appears in Each Answer, Shift It to Stem. Use Complete Sentence Format.
13
B. Writing Questions
Writing Effective Distractors Use Best-Answer Format. Write Best or Correct Answer First. For Concept Question
Correct Answer to Similar Concept. Have Correct Elements, But Also Incorrect Elements From Students' Answers to Essay Version of Question.
14
B. Writing Questions
Writing Effective Distractors
For Math Question Reflect Mistakes That Students Make. Use "Not Sufficient Information to Solve Problem"
Option. Avoid “None of Above” for Math Problems
15
B. Writing Questions
Writing Effective Distracters
Correct Answer Should Not Be Longest Answer. All Distractors Should Form Complete and Grammatically-Correct
Sentences With Stem. Avoid Using Never or Always. Avoid an “All of the Above” Option. Avoid Repeating Words From Stem. Vary Correct Answer Among Four Possible Answer Positions.
16
B. Writing Questions
2. Writing Short-Answer Questions
Base on Objectives. Use “Structured Essay” Format.
Uses Precise Verbs. Helps Students Frame Answer.
Use Iterative Procedure. Write Ideal Answer. Could Students Achieve Ideal Given
Question Wording? Questions Address Important Concepts. Solve Math Problems.
17
Compare and Contrast “Amadeus” and “The Shawshank Redemption”.
John Locke: The Key to David Hume? Discuss.
Is the Central Limit Theorem Critical to Constructing a Confidence
Interval on a Population Mean? Discuss and Defend.
B. Writing Questions
Unstructured Short-Answer Questions
18
B. Writing Questions
The All-Bluff Answer and the Need for the Structured
FormatIs Central Limit Theorem Critical to Constructing a Confidence Interval on a Population Mean? Dr. Deming Noted that the Central Limit Theorem is Essential.
As You Said in Class, “The Central Limit Theorem is a Critical Topic.”
The Central Limit Theorem Says that ……..
Finally, Hypothesis Testing is Similar to Confidence Intervals in that ...
19
B. Writing Questions Structured Answer Format
Compare and Contrast “Amadeus” and “The Shawshank Redemption”.
Basis 1 is _________ Basis 1 is Important Because _________
Basis 2 is __________ Basis 2 is Important Because __________
20
B. Writing Questions
Structured Answer Format Students then Complete the Table.
Bases Amadeus Shawshank 1 2 3 4
21
Can the Bias in a forecast ever be greater than the MAD value? [O15] 10 Points
Select One Response Below and Complete.
Yes Because
No Because
B. Writing Questions Structured Answer Format
22
Suppose the Central Limit Theorem was not true.Would It be Possible to Construct a Confidence Intervalon a Population Mean? [O23] [15 Points]
Select One of the Responses Below and Complete.Would Never be Possible Because
Would Always be Possible Because
Would Only be Possible When
B. Writing Questions Structured Answer Format
23
Suggest a modification to the analogy method that would increase the likelihood that a Sensing-dominant manager would use the Method. [O45, O56] [25 Points]
Describe the Modification.
This Modification Increases the Likelihood that aSensing-Dominant Manager Would Use theAnalogy Method Because …
B. Writing Questions Structured Answer Format
24
Testing Overview
A. Preparing Exams 1. Type of Tests2. Number of Tests3. Questions per Test, and4. Table of Specification
B. Writing Questions 1. Effective Stems and Distracters
2. Structured Essay Questions and Answers
C. Grading 1. Scoring Key and PartialCredit for Multiple Choice
D. Improving Exams 1. Survey and Difficulty and Discrimination Indices
25
C. Grading
Grading Issues for All Types of Exams
Written Comments Are Highly Motivating. “Excellent Improvement” “Sorry Did So Poorly. Please See Me.”
Return Within One Class (MC) or One Week (Short Answer).
Provide Key After Quiz, if Possible.
26
C. Grading
Grading Issues on Multiple Choice Exams
Partial Credit for Multiple Choice?
On Numerical Problems: Provide Optional White Space and Only Check if Wrong Option Selected.
Defend Concept Answer to Complex Question: Provide Optional White Space and Only Check if Wrong Option Selected.
27
C. Grading
Grading Issues on Short Answer Exams
For Small Classes, Provide Audio Feedback. Students Hand In Audio Cassette to Instructor. Write Annotated Comments on Exam.
Use a Detailed Scoring Key. Grade One Question at a Time. Shuffle Papers After Each Question.
28
Testing Overview
A. Preparing Exams 1. Type of Tests2. Number of Tests3. Questions per Test, and4. Table of Specification
B. Writing Questions 1. Effective Stems and Distracters
2. Structured Essay Questions and Answers
C. Grading 1. Scoring Key and PartialCredit for Multiple Choice
D. Improving Exams 1. Survey and Difficulty and Discrimination Indices
29
D. Improving Exams Use Test Assessment Instrument During Term.1. Have exams reflected topic objectives?2. Have exams reflected important subject content?3. Has there been sufficient time to complete exams?4. Has instructor reacted positively to student concerns on questions,
grading, etc.? __________________________________________1 2 3 4 5
Not at All Moderately Very Much
5. How could the exams be improved?
30
Additional Readings
Bloom, B., Hastings, T., and Madaus, G. Handbook of Formative and Summative Evaluation for Student Learning. New York: McGraw Hill, 1971.
Jacobs, L and Chase, C. Developing and Using Tests Effectively: A Guide for Faculty. San Francisco, CA.: Jossey-Bass, 1992.