Post on 25-Dec-2015
HISD EXPECTATIONS FOR 2012-13
Continue work on realigning curriculum and updating curriculum documents
Continue and refine use of assessment for learning strategies with emphasis on student self-assessment and ownership of learning
Create quality assessments using research to inform your design
Use CIITS to create and store assessments and to track and analyze assessment data
Design ConcernsDesign ConcernsClassroom AssessmentClassroom Assessment
Purpose Audience Content Evidence of learning Process Meaning
Key No. 1 STANDARDS-BASEDWhat do we want students to know and be able to do at the end?
Key No. 3 VALIDITYDoes the assessment measure what it’s supposed to measure?
Key No. 2 FAIRDoes the assessment allow students to show what they know?
Key No. 4 MULTIPLE SOURCESIs there sufficient evidence to judge whether a student has mastered the standard?
KEYS
TO
QU
ALIT
Y
AS
SES
SM
EN
T
Objectivity (no one gets unfair advantage) Absence of bias (cultural, gender, socio-
economic) Opportunity for each student to demonstrate
what he/she knows
FAIRNESS MEANS…
ISSUES RELATED TO VALIDITY
Do the items form a representative sample?
Do the items represent a fresh experience or are they recycled?
Do the items rely on skills that students may lack which are not part of the assessment?
DO ITEMS FORM A REPRESENTATIVE SAMPLE?
Items are not aligned with the SFLTs Items represent teacher convenience
rather than systematic design Items focus on exceptions rather than
key topics Items are not appropriately distributed
among topics Items do not reflect the amount of time
devoted to them during instruction
“The essential characteristic of well-designed assessments is that the processes used to collect and interpret data are consistent with the purpose of the assessment. That match of purpose and process is achieved through thoughtful planning…”
—National Science Education Standards
TEST BLUEPRINTS
•Select targets to be assessed•Select methods to be used•Create items•Reasonable number of items per target (based on type of item)•Group items by targets•Assure appropriate balance among items and among targets •Consider if redo/retake will be permitted and what form it will take
“Any item may function at the rote learning level if the concept involved has been taught in the same way that it is used in the testing situation. In view of this, we must do our utmost to increase the likelihood that an item requires reasoning and understanding.”
—Harry D. Berg
Do the items represent a fresh experience?
SKILLS ISSUESHeavy reliance on a skill in which student is
not proficient (e.g., reading level, drawing)Sentence structure and vocabularyExtraneous knowledge Format (complexity; unfamiliar to student)Student’s ‘testwiseness’Student test anxiety
PRECISE UNAMBIGUOUS LANGUAGE
“It is not sufficient to write a question that can be understood. To paraphrase the advice of Gen. Douglas MacArthur to the Corps of Cadets: Write questions that cannot be misunderstood, not merely questions that can be understood.”
—John P. Sevenair
" The content of assessments should match challenging subject matter standards and be connected to contexts of application. This means assessing learning based on observations, oral questioning, significant tasks, projects, demonstrations, collections of student work, and students' self-evaluations, and it means that teachers must engage in systematic analysis of the available evidence."
—Lorrie A. Shepard
Knowledge
Reasoning
Product
Skills
Constructed response
Performance tasks
Observation
Selectedresponse
Personal Communication
ASSESSMENT METHODSASSESSMENT METHODS
True FalseMultiple True False (MTF)Multiple Choice & Alternate ChoiceMatchingKeyed Lists
SELECTED SELECTED RESPONSE RESPONSE
1.1. Grass is green.Grass is green.
2.2. California is the largest state.California is the largest state.
3.3. The U.S. national debt is too high. The U.S. national debt is too high.
TRUE/FALSE
Benefits• Ease of use and
coverage of material
• Assessing student misconceptions
• Useful in formative assessment
Pitfalls• Trivia, trickery, and
truisms• Overemphasis on
recall • 50% chance of
guessing correctly
TRUE/FALSE
WRITING GOOD T/F ITEMS
Focus statements on ONE fact or concept. Create statements of relatively equal length. Word statements positively using precise,
unambiguous language. Use NEW language (don’t lift statements from
textbook or from class). Don’t use absolutes (e.g., always, never, all) or
conditional verbs (“can, may”). Avoid vague qualifiers like “some,” “usually,”
“typically,” “several,” “many,” “large,” “small”). If it is necessary to test an opinion, attribute it to
a source.
WRITING GOOD T/F ITEMS
Lessen the power of guessing by: having slightly more “false” items than “true”
(when students don’t know, they tend to opt for “true”)
mixing up the sequence of responses so that they do not form patterns
“A major distinction between the [true-false] item and items in multiple-choice or two-choice formats, is that the [true-false] item contains no criterion for answering the item. The criterion is outside of the item, rooted in the characteristics and experiences of each individual examinee. Each examinee must ask the question, true or false with respect to what? It follows that each … true-false item must be unequivocally true or unequivocally false.”
https://www.msu.edu/dept/soweb/writitem.html#m
citem
Use T/F to test CONCEPTUAL UNDERSTANDING
1. Inflation leads to a lower cost of living.
2. Physical change results in formation of a new substance.
3. Profit is the money a store takes in when goods are sold.
LESSEN POWER OF GUESSING
1. Higher prices are one of the results of deflation.
2. The rhythmical succession of single tones producing a distinct musical phrase or idea is referred to as harmony.
•Students can correct entire statement (focuses on the complete proposition) OR
•They can correct an underlined word or phrase (allows teacher to focus questions on particular concepts or facts)
•Correction diminishes the power of guessing
JUSTIFIED TRUE FALSE
Consider the equation a=12+b (when b≠0)
T or F
Why I (we) think so
a is always greater than b
a and b could be equal
etc.
from Mathematics Formative Assessment by Page Keeley and Cheryl RoseTobey, ©2011 Corwin Press
MULTIPLE TRUE FALSE (MTF)MULTIPLE TRUE FALSE (MTF) Consists of a context or situation and several
statements which the student is to evaluate in terms of that information.
Is an alternative way to deal with “All of the above” multiple choice items
EXAMPLE OF MTF
Which of these are characteristics of a square?
Check (√) all that are:Sides are equal in lengthSides join to form acute anglesHas four sidesIs a type of polygonIs a three-dimensional figure
ANOTHER MTF EXAMPLE
Which of these are effects of inflation? Check (√) all that are: Shortage of capital to invest in new businesses Drop in the purchasing power of money Decline in cost of goods and services People on fixed incomes can buy less than before Interest rates favor borrowers rather than lenders
A THIRD MTF EXAMPLE
Place a checkmark (√) by each example of chemical change:
1. Butter melting on warm bread2. Tarnish on a silver charm3. Black edges on overdone toast4. Creases in pants from being ironed5. Rust on a saw blade
Identify each item on the left by matching it with an item from the right.
1. Ship that brought Pilgrims to America
2. Indian who helped the Pilgrims
3. Name of the Pilgrim settlement
4. Important food crop of the Pilgrims
5. Key Pilgrim leader6. Date Pilgrims arrived in
America
A. William BradfordB. SquantoC. Henry HudsonD. MayflowerE. JamestownF. 1620G. PlymouthH. RiceI. Corn
CREATING QUALITY MATCHING ITEMS
Target items should be of the same class Directions indicate the relationship
between target items and choices Have at least one extra choice or allow choices to be used more than
once Sequence choices to make matching
easier (e.g., ABC order) Reasonable number of items
1. “It’s better to have it and not need it, than to need it and not have it.”
2. “He is smart as an owl.”3. “George has been put out to
pasture.”4. “Henry was a lion in battle.”5. “This vase will make a good
centralpiece.”6. “If I don’t make an A, I will just die.”
What type of figure speech is represented by each quote? Match the correct term from the right to the quote on the left. There is one extra term.
A. chiasmusB. euphemismC. hyperboleD. malapropismE. metaphorF. simileG. synedoche
BEYOND RECOGNITION/RECALL
KEYED LIST
Sentence rolesA. SubjectB. Direct objectC. Indirect objectD. Predicate
nominative
1. Is that your car?2. She told him to come in.3. Joe is a great student.4. Which book do you need?5. To whom did you give the
package?6. Why didn’t Mr. Howe
return my call?
Use the key to identify the role the underlined word plays in the sentence.
Key
ANOTHER KEYED LIST
Functions of moneyA. Unit of accountB. Store of valueC. Medium of
exchange
Isabella deposits $25.00 in her savings account at the bank
The supermarket sign that says “Strawberries $2.99 per pint”
Mrs. Wallace writes a check to her beautician for her perm.
The school board receives a bid from Acme Supply to provide ten cases of copy paper for $375.
Dr. Greene finds a hundred dollar bill on the street.
Classify each example using the key:Key
ALTERNATE CHOICE ALTERNATE CHOICE Two choices Useful in formative assessment Can highlight particular misconceptions or
errors students are likely to make Can be expanded to conventional MC by using
“both of these” and “neither of these” as options
1. If you were making shirts, which of these would be a capital resource?
A. the person who designed the shirtsB. the sewing machine used in sewing
them
2. People who make things for others to use are called:A. consumersB. producers
1. The______ lasted from 1861 to 1865.
A. Revolution
B. Civil War
C. Spanish-American War
2. It marked the turning point of the war.
A. Ft. Sumter B. Antietam C. Gettysburg
3. The Civil War
A. Was the bloodiest war in American history
B. Was fought mainly in the South
C. Could have been avoided
EFFECTIVE STEMS
Focus the question through the stem, not the choices. (Writing the stem as a question helps do this.)
Avoid non-functional language. Use simple, direct language as much as
possible. If stem contains a negative, highlight it in
some manner: e.g., Which of these is NOT true of paintings by Monet?
Focus the question through the stem:
What does a seismologist study?
A) Earthquakes and earth’s internal structure
B) Human social organization and institutions
C) Physical features of earth and human response to them
(preferred form)
Seismology
A) Study of human social organization and institutions
B) Study of earthquakes and internal structure of the earth
C) Study of physical features of earth and human response to them
Make all choices plausible & attractiveUse common misconceptions and errors as
basis for distractorsKeep distractors consistent in content, form,
and lengthUse similar degree of specificity Use “new” language rather than phrases
from the text or class3-4 good choices are enough
EFFECTIVE CHOICES
AVOID COMPLEX MULTIPLE CHOICE ITEMS:
Composers of the Baroque period include:1. J.S. Bach2. Frederic Chopin3. Georg Philipp Telemanna) 1 & 2 b) 1 & 3 c) 2 & 3 d) 1, 2, 3
•Harder for students to process
•Don’t discriminate better than simple choice questions
AVOID “ALL OF THE ABOVE” AS OPTION
• If student finds one exception, then he knows it is NOT “all of the above”
• If he finds at least two choices that are correct, then he knows it IS “all of the above”
ALTERNATIVES1.Stem…… A) ------- B) ------ C) both of these D) neither of these2. Use MTF
“NONE OF THE ABOVE” AS CHOICE
• Useful if the correct response is an absolute• If it is the answer, you can’t be certain that the
student actually knows the information• Better than a weak distractor• Should be the correct answer about half the time
Which of these adds value to Choco-Bars as a product?A) the elegant blue and gold wrappers in which they are soldB) the short travel time they have from factory to storeC) the free gift that comes with a purchase of a five pound boxD) none of the above
SUGGESTIONS FOR MANAGING OPTIONS
• Logical sequencing (e.g., ABC, magnitude)• Random patterns in arrangement of options• Relatively equal use of each option on every
test• Use vertical formatting for longer options• Have students use capital letters rather than
small case if they have to write the letters
1. Which of these artists is known primarily for his work as a sculptor?A) Leonardo da VinciB) Auguste RodinC) George BellowsD) Rembrandt van Rijn
2. Which of these is an example of
Baroque sculpture?A) Donatello’s DavidB) Rodin’s The Age of BronzeC) Bernini’s Apollo and DaphneD) Miró’s Woman and Bird
1. “Pharaohs were to Egypt as Emperors were to Rome” is an example of an - - -:
A) simile B) chiasmus C) analogy D) metaphor
2. The poetic meter with a foot consisting of an unaccented and an accented syllable is known as:
A) lyricB) epicC) iambicD) trochaic
3. Who composed Rhapsody in Blue?A) Judy GarlandB) Frank SinatraC) Cole PorterD) George Gershwin
GIVEAWAYS Grammatical clues (e.g., a, an) Words from the stem repeated in the
correct choice Correct choice is much longer or
more complicated in wording Absolutes: always, never Choices form pairs Interlocking items (first item answers
second)
• Options not selected by anyone should be replaced as they are not working
• An option chosen by almost everyone over the other options including the right answer may signal an instructional problem
• Items that everyone gets right are not functioning as discriminators
• Items missed by virtually everyone may signal an assessment problem or an instructional problem
Item Analysis
MULTIPLE CHOICE AND HIGHER LEVEL THINKING
Identify the verb you would like to use
Change it to a noun and pair it with a convergent verb
You can also use “which” with the noun
Explain: explanation Select the best
explanation for …. Describe: description
Identify the most accurate description of…
Solve: solution Choose the most
viable solution for….
“Bury the Verb” Mike Dickinson
• Common factor• Example or
illustration• Problem/solution• Conclusion from
data (e.g. trends)• Explanation of a
sequence• Image-based
items
FRAMING QUESTIONS
• Identification/recognition
• Application• Comparison/
contrast• Premise/
consequence; cause/effect
• Case or scenario• Interpretation
CLOSE DECISION MULTIPLE CHOICE (MYRON DUECK)
Student may choose a second answer and provide an explanation “Lifeline”
Reveals student’s thinkingReveals flaws in your questions
CONSTRUCTED RESPONSE TYPES
Short response• Fill in the blank• Graphic organizers and visuals• Relationships (e.g., form pairs, which
doesn’t belong, word splashes)• Exit Cards• Prompts
• Extended response (formerly called “open response”)
• Essay
GRAPHIC ORGANIZERS AND VISUALS
Simple webs Concept maps Venn diagrams Graphs and data
displays Labeling features
in a picture or diagram
Promote “chunking”
Show relationships
between ideas
Promote categorization
USING GRAPHS
Number apples eaten
1 2 3 4 5 6
Kate
Ashley
Ethan
Steve
Ethan ate apples. Show this number on the graph.
1.Who ate more apples: the girls or the boys?2.How many more apples did Ashley eat than Steve?3.Write a number sentence than shows how many apples Kate and Ashley ate.
FILL IN THE BLANK
a. ________________ is the current President of the United States.
b. A good source of protein is __________________.c. The steps in solving a quadratic equation are
1) ________________ 2) __________________, 3) _________________, 4) __________________, and 5) _________________.
Item should have only one right answer Blank should be placed at or near the end Multiple blanks should be avoided
Writing Quality Fill in the Blank Items
1. The name of the current Governor of Kentucky is ________________________.
2. Based on the data in the graph, the month with the highest number of absences was __________________.
SHORT RESPONSE PROMPTS
Focus question on one point. Use clear, unambiguous language. Assess knowledge and reasoning
Compare the role of producers and consumers in the economy.
Sam shaded part of this circle. (Diagram shown)What fraction of the circle is shaded? Explain your answer.
K-PREP sample
1. (Based on reading passage)Tommy learned that watching birds build their nests is a good way to understand them. How does watching the birds build their nests help Tommy understand birds better?
Sample K-PREP short response promptsSample K-PREP short response prompts
2. A bird called the arctic tern flies about 18,600 miles each year. What is this distance rounded to the nearest ten thousand? Show your work or explain how you found your answer.
3. Draw a closed figure with 5 line segments. Your figure must contain 2 right angles. Label the right angles with the correct angle marks.
DIAGNOSTIC QUESTIONS
“The crucial feature of…diagnostic questions is based on a fundamental asymmetry of teaching; in general, is better to assume that students do not know something when they do than it is to assume they do know something when they don’t. What makes a question useful as a diagnostic question, therefore, is that it must be very unlikely that the student gets the correct answer for the wrong reason.”
—Dylan Wiliam
DIAGNOSTIC QUESTIONS (DYLAN WILIAM, PAGE KEELEY)
Ask “Why is…?” rather than “What is…?”“Why is 17 prime and 15 not?” rather than “What
is a prime number?” or “Is 15 a prime number?”
“Why is a square a rectangle?” rather than “Is a square a rectangle?”
“What makes the veto an example of checks and balances?” rather than “What are “checks and balances?”
Take a straight recall question and transform it by phrasing it to require explanation or justification.
DIAGNOSTIC QUESTIONS (DYLAN WILIAM)
Ask for reactions to statements:“All squares are rectangles.”“Light travels from the eye to the object.”Use MTF/MC with multiple correct responses:“In which of these diagrams is one-fourth of the
area shaded?”
“Just as there can be no set formulas for producing a good story or a good painting, so there can be no set of rules that will guarantee production of good test items Principles can be established and suggestions offered, but it is the item writer’s judgment in the application…that determines whether good items or mediocre ones will be produced.”
—R.L.Ebel
CAVEAT