Historical Meap

106
Item Jurying Training Assessing Michigan’s Content Expectations

Transcript of Historical Meap

Page 1: Historical Meap

Item Jurying TrainingItem Jurying TrainingAssessing Michigan’s Content

ExpectationsAssessing Michigan’s Content

Expectations

Page 2: Historical Meap

“Today’s large-scale science tests simply aren’t that good. Too often, the questions are confusing or not well aligned to the key science ideas and skills that students are expected to learn.”

Jo Ellen Roseman, Director

Project 2061

American Association for the Advancement of Science (AAAS)

“Today’s large-scale science tests simply aren’t that good. Too often, the questions are confusing or not well aligned to the key science ideas and skills that students are expected to learn.”

Jo Ellen Roseman, Director

Project 2061

American Association for the Advancement of Science (AAAS)

The Need:The Need:

Page 3: Historical Meap

Characteristics of Assessments:

Validity

Characteristics of Assessments:

Validity…“When an item is well designed, students

should choose the correct answer only when they know the targeted idea and they should choose an incorrect answer only when they do not know the idea.

“Students should be able to demonstrate their science knowledge without being tripped up by confusing language, inaccurate information, unclear diagrams, or contexts that are unfamiliar or unnecessarily complex.”

Project 2061 Today, Winter 2007

…“When an item is well designed, students should choose the correct answer only when they know the targeted idea and they should choose an incorrect answer only when they do not know the idea.

“Students should be able to demonstrate their science knowledge without being tripped up by confusing language, inaccurate information, unclear diagrams, or contexts that are unfamiliar or unnecessarily complex.”

Project 2061 Today, Winter 2007

Page 4: Historical Meap

Basic Item Writing Principles

Basic Item Writing Principles

The following slides give some

Guidelines that should be considered when writing items; and

Criteria for writing valid items;

A description of an item’s Depth of Knowledge

The following slides give some

Guidelines that should be considered when writing items; and

Criteria for writing valid items;

A description of an item’s Depth of Knowledge

Page 5: Historical Meap

The Big Bang Theory states that Earth is

A. moving away from the center of the universe.*B. spiraling in towards the center of the universe.C. drifting in a random path throughout the universe.D. traveling in an orbit around the center of the

universe.

(MEAP Released Item 2004)

The Big Bang Theory states that Earth is

A. moving away from the center of the universe.*B. spiraling in towards the center of the universe.C. drifting in a random path throughout the universe.D. traveling in an orbit around the center of the

universe.

(MEAP Released Item 2004)

“STEM”

Four “OPTIONS” – one correct, three equally plausible, but

incorrect

Item Format – Multiple Choice

Item Format – Multiple Choice

Page 6: Historical Meap

1. Item “stem” should be longer (a complete sentence – i.e. a “closed stem,” if possible), the “options” should be shorter.

Open-ended stems are sometimes acceptable if it is very clear from the stem what the item is asking.

1. Item “stem” should be longer (a complete sentence – i.e. a “closed stem,” if possible), the “options” should be shorter.

Open-ended stems are sometimes acceptable if it is very clear from the stem what the item is asking.

GuidelinesGuidelines

Page 7: Historical Meap

Change to

GuidelinesGuidelines

When considering an iron as an example of a system, When considering an iron as an example of a system, A. electricity is the input.A. electricity is the input.B. temperature is the input.B. temperature is the input.C. motion on clothes is the input.C. motion on clothes is the input.D. steam from the plate is the input.D. steam from the plate is the input.

Page 8: Historical Meap

GuidelinesGuidelines

When considering an iron as an example of a system, what is the input?When considering an iron as an example of a system, what is the input?A. electricityA. electricityB. temperatureB. temperatureC. motion on clothesC. motion on clothesD. steam from the plateD. steam from the plate (Massachusetts released item, 2006)(Massachusetts released item, 2006)

Page 9: Historical Meap

2. Grammar & Vocabulary: Keep the grammar (e.g. verb tense) consistent.

2. Grammar & Vocabulary: Keep the grammar (e.g. verb tense) consistent.

GuidelinesGuidelines

Page 10: Historical Meap

Example: A certain species of bird can be brown or white. White color is a recessive trait, while brown color is a dominant trait. When two brown birds mate, is it possible for them to have white offspring?

A. No, because both parents will have only genes for being brown.

B. Yes, because offspring color does not depend on the genes of the parents.

C. Yes, because both parents may have and pass on the gene for being white.*

D. No, because the parents passed on only the dominant trait to their offspring.

Example: A certain species of bird can be brown or white. White color is a recessive trait, while brown color is a dominant trait. When two brown birds mate, is it possible for them to have white offspring?

A. No, because both parents will have only genes for being brown.

B. Yes, because offspring color does not depend on the genes of the parents.

C. Yes, because both parents may have and pass on the gene for being white.*

D. No, because the parents passed on only the dominant trait to their offspring.

GuidelinesGuidelines

Page 11: Historical Meap

Example: A certain species of bird can be brown or white. White color is a recessive trait, while brown color is a dominant trait. When two brown birds mate, is it possible for them to have white offspring?

A. No, because both parents will have only genes for being brown.

B. Yes, because offspring color does not depend on the genes of the parents.

C. Yes, because both parents may have and pass on the gene for being white.*

D. No, because the parents passed on only the dominant trait to their offspring.

Example: A certain species of bird can be brown or white. White color is a recessive trait, while brown color is a dominant trait. When two brown birds mate, is it possible for them to have white offspring?

A. No, because both parents will have only genes for being brown.

B. Yes, because offspring color does not depend on the genes of the parents.

C. Yes, because both parents may have and pass on the gene for being white.*

D. No, because the parents passed on only the dominant trait to their offspring.

GuidelinesGuidelines

Page 12: Historical Meap

2. Grammar & Vocabulary: Don’t repeat a word in an option that was used in the stem.

2. Grammar & Vocabulary: Don’t repeat a word in an option that was used in the stem.

GuidelinesGuidelines

Page 13: Historical Meap

Example: Which of the following is a consideration when planning a nuclear power facility?a. Emission of chemicals that cause acid rainb. Disposal of nuclear wastes*c. Production of greenhouse gasesd. Removal of ash and soot from smokestacks

(“Test-wiseness”)

Example: Which of the following is a consideration when planning a nuclear power facility?a. Emission of chemicals that cause acid rainb. Disposal of nuclear wastes*c. Production of greenhouse gasesd. Removal of ash and soot from smokestacks

(“Test-wiseness”)

GuidelinesGuidelines

Page 14: Historical Meap

2. Grammar & Vocabulary: Keep the readability appropriate.

2. Grammar & Vocabulary: Keep the readability appropriate.

GuidelinesGuidelines

Page 15: Historical Meap

Example: Which device can be utilized to generate electromotive force thereby increasing the voltage from a dc source? A. Generator B. Electroscope C. Induction coil D. Mass spectrometer

(Any troublesome vocabulary here?)

Example: Which device can be utilized to generate electromotive force thereby increasing the voltage from a dc source? A. Generator B. Electroscope C. Induction coil D. Mass spectrometer

(Any troublesome vocabulary here?)

GuidelinesGuidelines

Page 16: Historical Meap

Example: Which device can be utilized to generate electromotive force thereby increasing the voltage from a dc source? A. Generator

B. Electroscope (used instructionally, but not an assessable vocabulary term)

C. Induction coil (“induction” used in electrostatics only)

D. Mass spectrometer

(Any troublesome vocabulary here?)

Example: Which device can be utilized to generate electromotive force thereby increasing the voltage from a dc source? A. Generator

B. Electroscope (used instructionally, but not an assessable vocabulary term)

C. Induction coil (“induction” used in electrostatics only)

D. Mass spectrometer

(Any troublesome vocabulary here?)

GuidelinesGuidelines

Page 17: Historical Meap

3. (a) Avoid the use of “all of the above,” or “none of the above” in the options.

3. (a) Avoid the use of “all of the above,” or “none of the above” in the options.

GuidelinesGuidelines

Page 18: Historical Meap

Example: What is the percent composition of carbon in carbon dioxide, CO2?

A. 12%B. 32%C. 44%D. none of the above*

Example: What is the percent composition of carbon in carbon dioxide, CO2?

A. 12%B. 32%C. 44%D. none of the above*

GuidelinesGuidelines

Page 19: Historical Meap

Example: What is the percent composition of carbon in carbon dioxide, CO2?

A. 12% A. 12%B. 32% B. 27%C. 44% C. 32%D. none of the above* D. 44%

Example: What is the percent composition of carbon in carbon dioxide, CO2?

A. 12% A. 12%B. 32% B. 27%C. 44% C. 32%D. none of the above* D. 44%

Change to

GuidelinesGuidelines

Page 20: Historical Meap

3. (b) Usually avoid the use of absolute terms, e.g. “always,” “only” or “never.”

Advice taken from a “Test Taking Strategies” website:

Be aware of words like "always," "never," "only," "must," and "completely."  These are usually the wrong answers…

http://www.shsu.edu/~counsel/hs/teststrats.html

3. (b) Usually avoid the use of absolute terms, e.g. “always,” “only” or “never.”

Advice taken from a “Test Taking Strategies” website:

Be aware of words like "always," "never," "only," "must," and "completely."  These are usually the wrong answers…

http://www.shsu.edu/~counsel/hs/teststrats.html

GuidelinesGuidelines

Page 21: Historical Meap

3. (b) Usually avoid the use of absolute terms, e.g. “always,” “only” or “never.”

Example:

Earth’s atmosphere can containA. gases onlyB. gases and solidsC. gases and liquidsD. solids, liquids and gases

3. (b) Usually avoid the use of absolute terms, e.g. “always,” “only” or “never.”

Example:

Earth’s atmosphere can containA. gases onlyB. gases and solidsC. gases and liquidsD. solids, liquids and gases

GuidelinesGuidelines

Page 22: Historical Meap

Example: The earth’s moon is

A. always much closer to the sun than it is to the earth

B. always much closer to the earth than it is to the sun*

C. about the same distance from the sun as it is from the earth

D. sometimes closer to the sun than it is to the earth and sometimes closer to the earth than it is to the sun

Science Framework for the 2009 NAEP

Example: The earth’s moon is

A. always much closer to the sun than it is to the earth

B. always much closer to the earth than it is to the sun*

C. about the same distance from the sun as it is from the earth

D. sometimes closer to the sun than it is to the earth and sometimes closer to the earth than it is to the sun

Science Framework for the 2009 NAEP

“Always” & “Never” – Exception

“Always” & “Never” – Exception

Page 23: Historical Meap

4. Make all the options “parallel,” i.e. the same length, with similar detail. Avoid making the correct option unique or conspicuous in any way.

More advice from a test-taking website:

“Usually the correct answer is the choice with the most information.”

http://www.testtakingtips.com/test/multiple.htm

4. Make all the options “parallel,” i.e. the same length, with similar detail. Avoid making the correct option unique or conspicuous in any way.

More advice from a test-taking website:

“Usually the correct answer is the choice with the most information.”

http://www.testtakingtips.com/test/multiple.htm

GuidelinesGuidelines

Page 24: Historical Meap

Example: A local car dealership wants to know how many people hear their advertisements on radio. Which method provides the most valid results?

A. Survey the next 20 customersB. Survey all the people living within ½ mileC. Survey a large random sample of people

living within the listening range of the radio station*

D. Survey customers at a nearby auto repair shop

Example: A local car dealership wants to know how many people hear their advertisements on radio. Which method provides the most valid results?

A. Survey the next 20 customersB. Survey all the people living within ½ mileC. Survey a large random sample of people

living within the listening range of the radio station*

D. Survey customers at a nearby auto repair shop

Guidelines – Parallel Options

Guidelines – Parallel Options

Page 25: Historical Meap

Improved: A local car dealership wants to know how many people hear their advertisements on radio. Which method provides the most valid results?

A. Survey the next 20 customers who come into the dealership.

B. Survey all the people living within ½ mile of the dealership.

C. Survey a random sample of people in the range of the station.*

D. Survey customers at a nearby competitor’s repair shop.

Improved: A local car dealership wants to know how many people hear their advertisements on radio. Which method provides the most valid results?

A. Survey the next 20 customers who come into the dealership.

B. Survey all the people living within ½ mile of the dealership.

C. Survey a random sample of people in the range of the station.*

D. Survey customers at a nearby competitor’s repair shop.

Guidelines – Parallel Options

Guidelines – Parallel Options

Page 26: Historical Meap

Which of these is a mineral most commonly found in igneous rocks on Earth?A. GneissB. Quartz*C. SandstoneD. Clay

(Quartz is the only mineral listed, The others are rocks or types of soil particles.)

Which of these is a mineral most commonly found in igneous rocks on Earth?A. GneissB. Quartz*C. SandstoneD. Clay

(Quartz is the only mineral listed, The others are rocks or types of soil particles.)

Guidelines – Parallel Options

Guidelines – Parallel Options

Page 27: Historical Meap

5. Present numerical options consistently.

Usually in ascending, or descending order. Don’t mix specific values with ranges, etc.

5. Present numerical options consistently.

Usually in ascending, or descending order. Don’t mix specific values with ranges, etc.

GuidelinesGuidelines

Page 28: Historical Meap

Example: In certain breeds of dogs, deafness is due to a recessive allele (d), and normal hearing is due to the dominant allele (D). What is the probability that the offspring of a normal heterozygous (Dd) dog and a deaf dog (dd) will have normal hearing?          A. 100%B. 0.25C. less than 1/3D. One half*

Example: In certain breeds of dogs, deafness is due to a recessive allele (d), and normal hearing is due to the dominant allele (D). What is the probability that the offspring of a normal heterozygous (Dd) dog and a deaf dog (dd) will have normal hearing?          A. 100%B. 0.25C. less than 1/3D. One half*

GuidelinesGuidelines

Page 29: Historical Meap

Example: In certain breeds of dogs, deafness is due to a recessive allele (d), and normal hearing is due to the dominant allele (D). What is the probability that the offspring of a normal heterozygous (Dd) dog and a deaf dog (dd) will have normal hearing?          A. 100% A. 0%B. 0.25 B. 25%C. less than 1/3 C. 50%*D. One half* D. 100%

Example: In certain breeds of dogs, deafness is due to a recessive allele (d), and normal hearing is due to the dominant allele (D). What is the probability that the offspring of a normal heterozygous (Dd) dog and a deaf dog (dd) will have normal hearing?          A. 100% A. 0%B. 0.25 B. 25%C. less than 1/3 C. 50%*D. One half* D. 100%

Change to

GuidelinesGuidelines

Page 30: Historical Meap

6. Make sure there is only ONE correct response.

6. Make sure there is only ONE correct response.

GuidelinesGuidelines

Page 31: Historical Meap

Guidelines – only one correct response

Guidelines – only one correct response

Example: According to the graph, what percent of the city’s residents own pets?

A. Less than 20% B. Less than 30%C. More than 50% D. More than 75%

(‘B’ overlaps ‘A’ and ‘D’ overlaps ‘C’)

Example: According to the graph, what percent of the city’s residents own pets?

A. Less than 20% B. Less than 30%C. More than 50% D. More than 75%

(‘B’ overlaps ‘A’ and ‘D’ overlaps ‘C’)

0

10

20

30

40

50

60

0 1 2 3

Number of Pets

Per

cen

t o

f C

ity'

s R

esid

ents

Page 32: Historical Meap

Example: According to the graph, what percent of the city’s residents own pets?

A. Less than 20% A. Less than 20%B. Less than 30% B. Between 20% and 40%C. More than 50% C. Between 40% and 60%D. More than 75% D. More than 60%

Make the options mutually exclusive, not overlapping

Example: According to the graph, what percent of the city’s residents own pets?

A. Less than 20% A. Less than 20%B. Less than 30% B. Between 20% and 40%C. More than 50% C. Between 40% and 60%D. More than 75% D. More than 60%

Make the options mutually exclusive, not overlapping

Guidelines – only one correct response

Guidelines – only one correct response

Page 33: Historical Meap

Example: If Karen measures the length of each of the butterflies in her collection, which would be the best measurement unit to use?

A. meterB. centimeter*C. millimeter*D. kilometer

Example: If Karen measures the length of each of the butterflies in her collection, which would be the best measurement unit to use?

A. meterB. centimeter*C. millimeter*D. kilometer

Guidelines – only one correct response

Guidelines – only one correct response

Page 34: Historical Meap

Carol asked her 4 best friends, Alice, Clark, Ruth, and Matt, if she could measure their heights in inches. She found that Alice was 58 inches tall; Clark was 62 inches tall; Ruth was 60 inches tall; and Matt was 61 inches tall. To the nearest inch, what was the mean height of Carol’s 4 friends?

Consider this alternative:

Carol measured the heights of each of 4 friends. Her results are shown below.

Alice: 58 inches Ruth: 60 inches

Clark: 62 inches Matt: 61 inches

To the nearest inch, what is the mean height of Carol’s 4 friends?

Use a chart, a bulleted list, a graphic, etc. to break up reading.

7. Word Economy & Formatting

7. Word Economy & Formatting

Page 35: Historical Meap

Additional Guidelines :Additional Guidelines :

8. Write questions that measure understanding, insight, and higher level skills. Avoid continually asking trivial questions. (DOK)

9. Use concise, straightforward language.

10. Present the problem or question as clearly as possible in the stem. Don’t include unnecessary or redundant information. Don’t use the item stem to instruct!

11. Don’t try to be tricky or cute.

12. Avoid bias toward or against any group of individuals. Include multicultural contexts and names.

8. Write questions that measure understanding, insight, and higher level skills. Avoid continually asking trivial questions. (DOK)

9. Use concise, straightforward language.

10. Present the problem or question as clearly as possible in the stem. Don’t include unnecessary or redundant information. Don’t use the item stem to instruct!

11. Don’t try to be tricky or cute.

12. Avoid bias toward or against any group of individuals. Include multicultural contexts and names.

Page 36: Historical Meap

13. When calling for a judgment or conclusion, the student must be able to infer the correct answer from the text. Don’t pose questions such as “What do you think? / What would you do?” because any option could be defensible. The stems should be worded “According to the author,” or “According to the selection,…”

14. If possible, avoid using negative stems that ask the students to choose the one wrong answer, such as those that use the terms “except” and “not”.

15. “Which of the following…?” is preferred over “What is…?”

13. When calling for a judgment or conclusion, the student must be able to infer the correct answer from the text. Don’t pose questions such as “What do you think? / What would you do?” because any option could be defensible. The stems should be worded “According to the author,” or “According to the selection,…”

14. If possible, avoid using negative stems that ask the students to choose the one wrong answer, such as those that use the terms “except” and “not”.

15. “Which of the following…?” is preferred over “What is…?”

Additional Guidelines :Additional Guidelines :

Page 37: Historical Meap

End of “Guidelines”End of “Guidelines”

Page 38: Historical Meap

Criteria for Valid ItemsCriteria for Valid Items

Always BEGIN with the Content Expectation!

Study the Content Expectation and be sure you know which knowledge and skills are expected before examining the actual assessment item.

Always BEGIN with the Content Expectation!

Study the Content Expectation and be sure you know which knowledge and skills are expected before examining the actual assessment item.

Page 39: Historical Meap

Six Criteria for Valid Items

Six Criteria for Valid Items

Adapted from Project 2061 Criteria used by Michigan Mathematics Leadership Academy (MMLA)

1. The CONTENT of the item matches the CONTENT of the expectation.

2. The PERFORMANCE required in the item matches the VERB of the expectation.“Underline the key concepts (important nouns and noun phrases) and circle the skills (the verbs).” Larry Ainsworth, Unwrapping the Standards, 2003

3. The item cannot be solved merely by TEST-WISENESS.

Adapted from Project 2061 Criteria used by Michigan Mathematics Leadership Academy (MMLA)

1. The CONTENT of the item matches the CONTENT of the expectation.

2. The PERFORMANCE required in the item matches the VERB of the expectation.“Underline the key concepts (important nouns and noun phrases) and circle the skills (the verbs).” Larry Ainsworth, Unwrapping the Standards, 2003

3. The item cannot be solved merely by TEST-WISENESS.

Page 40: Historical Meap

4. The item addresses ONLY ONE content expectation (may not be required in some instances).

5. ALL the content in the expectation is addressed in the item bank.

6. The CONTEXT of the item is APPROPRIATE and ENGAGING.

4. The item addresses ONLY ONE content expectation (may not be required in some instances).

5. ALL the content in the expectation is addressed in the item bank.

6. The CONTEXT of the item is APPROPRIATE and ENGAGING.

Six Criteria for Valid ItemsSix Criteria for Valid Items

Page 41: Historical Meap

1. The CONTENT of the item matches the CONTENT of the expectation

(i.e., “Alignment”)

1. The CONTENT of the item matches the CONTENT of the expectation

(i.e., “Alignment”)

Chemistry prerequisite HSCE:

P4.p2D Recognize that the properties of a compound differ from those of its individual elements.

Chemistry prerequisite HSCE:

P4.p2D Recognize that the properties of a compound differ from those of its individual elements.

Page 42: Historical Meap

P4.p2D Recognize that the properties of a compound differ from those of its individual elements.

Example: Which of the following is true for chemical compounds that have been detected elsewhere in the universe? 

A. They have a greater average density than the same

compounds found on Earth. B. They are composed of the same elements that are found on Earth. * C. They are less reactive chemically than the same compounds found on Earth. D. Those with the greatest molar masses are found furthest away from our solar system.

Question: How well-aligned is this item to the Content Expectation?

P4.p2D Recognize that the properties of a compound differ from those of its individual elements.

Example: Which of the following is true for chemical compounds that have been detected elsewhere in the universe? 

A. They have a greater average density than the same

compounds found on Earth. B. They are composed of the same elements that are found on Earth. * C. They are less reactive chemically than the same compounds found on Earth. D. Those with the greatest molar masses are found furthest away from our solar system.

Question: How well-aligned is this item to the Content Expectation?

1. The CONTENT of the item matches the CONTENT of the

expectation

1. The CONTENT of the item matches the CONTENT of the

expectation

Page 43: Historical Meap

P4.p2D Recognize that the properties of a compound differ from those of its individual elements.

Example: Which of the following is a property of water that differs from its individual elements?

A. Water is combustible in airB. Water is metallicC. Water is less denseD. Water is a liquid at room temperature*

(Getting closer…)

P4.p2D Recognize that the properties of a compound differ from those of its individual elements.

Example: Which of the following is a property of water that differs from its individual elements?

A. Water is combustible in airB. Water is metallicC. Water is less denseD. Water is a liquid at room temperature*

(Getting closer…)

1. The CONTENT of the item matches the CONTENT of the

expectation

1. The CONTENT of the item matches the CONTENT of the

expectation

Page 44: Historical Meap

Based on the information in the table above, which is a reasonable hypothesis regarding elements and their compounds?  (NAEP item)

A. An element retains its physical and chemical properties when it is combined into a compound.

B. When an element reacts to form a compound, its chemical properties are changed but its physical properties are not.

C. When an element reacts to form a compound, its physical properties are changed but its chemical properties are not.

D. Both the chemical and physical properties of a compound are different from the properties of the elements of which it is composed.*

(Well-aligned)

Based on the information in the table above, which is a reasonable hypothesis regarding elements and their compounds?  (NAEP item)

A. An element retains its physical and chemical properties when it is combined into a compound.

B. When an element reacts to form a compound, its chemical properties are changed but its physical properties are not.

C. When an element reacts to form a compound, its physical properties are changed but its chemical properties are not.

D. Both the chemical and physical properties of a compound are different from the properties of the elements of which it is composed.*

(Well-aligned)

Form of Carbon Charcoal (C) Carbon Dioxide (CO2 )

State at Room Temperature

Solid Gas

Soluble in Water No Yes

Combustible in Air Yes No

Page 45: Historical Meap

P4.4A Describe specific mechanical waves (e.g. on a demonstration spring, on the ocean) in terms of wavelength, amplitude, frequency, and speed.

Example: Calculate the speed of a wave with a wavelength of 3.0 m and a frequency of 15 Hz.

A. 5.0 m/sec B. 12 m/sec C. 18 m/sec D. 45 m/sec*

(Poor Alignment)

P4.4A Describe specific mechanical waves (e.g. on a demonstration spring, on the ocean) in terms of wavelength, amplitude, frequency, and speed.

Example: Calculate the speed of a wave with a wavelength of 3.0 m and a frequency of 15 Hz.

A. 5.0 m/sec B. 12 m/sec C. 18 m/sec D. 45 m/sec*

(Poor Alignment)

2. The PERFORMANCE required in the item matches the VERB of the

expectation

2. The PERFORMANCE required in the item matches the VERB of the

expectation

Page 46: Historical Meap

Example: The figure above shows some ocean waves. Which of the labeled distances represents the wavelength?  (NAEP item)

a. A* b. B c. C d. D

(Better Alignment)

Example: The figure above shows some ocean waves. Which of the labeled distances represents the wavelength?  (NAEP item)

a. A* b. B c. C d. D

(Better Alignment)

2. The PERFORMANCE required in the item matches the VERB of the

expectation

2. The PERFORMANCE required in the item matches the VERB of the

expectation

Page 47: Historical Meap

Example: Nitrogen-fixing bacteria help cycle nitrogen through ecosystems. How do they do this?

A. They change nitrogen into forms usable by plants.*B. They make inorganic compounds from organic

compounds during decomposition.C. They release chemical energy during respiration.D. They convert sunlight into chemical energy during

photosynthesis.

Example: Nitrogen-fixing bacteria help cycle nitrogen through ecosystems. How do they do this?

A. They change nitrogen into forms usable by plants.*B. They make inorganic compounds from organic

compounds during decomposition.C. They release chemical energy during respiration.D. They convert sunlight into chemical energy during

photosynthesis.

3. The item cannot be solved merely by TEST-WISENESS

3. The item cannot be solved merely by TEST-WISENESS

Page 48: Historical Meap

4. The item addresses ONLY ONE content expectation

4. The item addresses ONLY ONE content expectation

Example: How is our solar system similar to an atom?

A. The force of gravity holds planets and electrons in their orbits.

B. Nuclear fusion constantly occurs in the sun and in the atom’s nucleus.

C. Electrons and planets both travel in well-defined paths.

D. The solar system and the atom are mostly empty space.*

What have we learned if a student misses this item?

Example: How is our solar system similar to an atom?

A. The force of gravity holds planets and electrons in their orbits.

B. Nuclear fusion constantly occurs in the sun and in the atom’s nucleus.

C. Electrons and planets both travel in well-defined paths.

D. The solar system and the atom are mostly empty space.*

What have we learned if a student misses this item?

Page 49: Historical Meap

Is it …

or

Is it …

or

4. The item addresses ONLY ONE content expectation

4. The item addresses ONLY ONE content expectation

Page 50: Historical Meap

5. ALL the content in the expectation is addressed in the

item bank

5. ALL the content in the expectation is addressed in the

item bank

B3.4d Describe the greenhouse effect and list possible causes.

Example: Which of the following is the main contributor to Earth’s greenhouse effect?

A. Earth’s distance from the sun B. the gases in Earth’s atmosphere* C. the tilt of Earth’s axis D. the number of active volcanoes on Earth

Will this item adequately assess the entire CE?

B3.4d Describe the greenhouse effect and list possible causes.

Example: Which of the following is the main contributor to Earth’s greenhouse effect?

A. Earth’s distance from the sun B. the gases in Earth’s atmosphere* C. the tilt of Earth’s axis D. the number of active volcanoes on Earth

Will this item adequately assess the entire CE?

Page 51: Historical Meap

B3.4d Describe the greenhouse effect and list possible causes.

Example: Which of the following facts about Earth’s temperature is due mainly to the greenhouse effect?

A. Earth’s average surface temperature is 30 Co

warmer than would be predicted based on its distance from the sun.*

B. Earth’s oceans warm and cool much more slowly than its land masses.

C. Earth’s average summer temperature is about 15 Co warmer than its winter temperature.

D. Earth’s surface temperature at the poles is much colder than at the equator.

B3.4d Describe the greenhouse effect and list possible causes.

Example: Which of the following facts about Earth’s temperature is due mainly to the greenhouse effect?

A. Earth’s average surface temperature is 30 Co

warmer than would be predicted based on its distance from the sun.*

B. Earth’s oceans warm and cool much more slowly than its land masses.

C. Earth’s average summer temperature is about 15 Co warmer than its winter temperature.

D. Earth’s surface temperature at the poles is much colder than at the equator.

5. ALL the content in the expectation is addressed in the

item bank

5. ALL the content in the expectation is addressed in the

item bank

Page 52: Historical Meap

B3.4d Describe the greenhouse effect and list possible causes.

Example: Which of the following atmospheric gases is NOT considered a “greenhouse” gas?

A. oxygen*B. carbon dioxideC. water vaporD. ozone

B3.4d Describe the greenhouse effect and list possible causes.

Example: Which of the following atmospheric gases is NOT considered a “greenhouse” gas?

A. oxygen*B. carbon dioxideC. water vaporD. ozone

5. ALL the content in the expectation is addressed in the

item bank

5. ALL the content in the expectation is addressed in the

item bank

Page 53: Historical Meap

The Danish astronomer Ole Roemer (1644-1710) was the first person to determine that light traveled with a measurable speed. As Earth revolved around the sun, he noticed a variation in the time Io emerged from behind Jupiter, and calculated that light took 22 minutes to cross a diameter of Earth’s orbit. If the diameter of Earth’s orbit is 3.0 x 1011 m, what value would Roemer have calculated for the speed of light?

A. 4.4 x 10-9 m/secB. 2.2 x 108 m/sec*C. 3.0 x 108 m/secD. 1.3 x 1010 m/sec

(Inappropriate and confusing context)

The Danish astronomer Ole Roemer (1644-1710) was the first person to determine that light traveled with a measurable speed. As Earth revolved around the sun, he noticed a variation in the time Io emerged from behind Jupiter, and calculated that light took 22 minutes to cross a diameter of Earth’s orbit. If the diameter of Earth’s orbit is 3.0 x 1011 m, what value would Roemer have calculated for the speed of light?

A. 4.4 x 10-9 m/secB. 2.2 x 108 m/sec*C. 3.0 x 108 m/secD. 1.3 x 1010 m/sec

(Inappropriate and confusing context)

6. The CONTEXT of the item is APPROPRIATE and ENGAGING

6. The CONTEXT of the item is APPROPRIATE and ENGAGING

Page 54: Historical Meap

Some items require a context,…

“When items are written to particular content statements, such as the history and nature of science, they may be framed in these contextual components of science content.”

Science Framework for the 2009 NAEP

Some items require a context,…

“When items are written to particular content statements, such as the history and nature of science, they may be framed in these contextual components of science content.”

Science Framework for the 2009 NAEP

6. The CONTEXT of the item is APPROPRIATE and

ENGAGING

6. The CONTEXT of the item is APPROPRIATE and

ENGAGING

Page 55: Historical Meap

Example: Ernest Rutherford found that when he fired alpha particles at a thin gold foil, some were scattered at large angles. What caused this scattering?A. The gold’s positive atomic nuclei attracted the negatively

charged alpha particles.B. The gold’s negative atomic nuclei repelled the negatively

charged alpha particles.C. The gold’s negative atomic nuclei attracted the positively

charged alpha particles.D. The gold’s positive atomic nuclei repelled the positively

charged alpha particles.*

Science Framework for the 2009 NAEP

Example: Ernest Rutherford found that when he fired alpha particles at a thin gold foil, some were scattered at large angles. What caused this scattering?A. The gold’s positive atomic nuclei attracted the negatively

charged alpha particles.B. The gold’s negative atomic nuclei repelled the negatively

charged alpha particles.C. The gold’s negative atomic nuclei attracted the positively

charged alpha particles.D. The gold’s positive atomic nuclei repelled the positively

charged alpha particles.*

Science Framework for the 2009 NAEP

6. The CONTEXT of the item is APPROPRIATE and

ENGAGING

6. The CONTEXT of the item is APPROPRIATE and

ENGAGING

Page 56: Historical Meap

Some items require no context,…

MEAP HST Science Released Items 2004

Some items require no context,…

MEAP HST Science Released Items 2004

6. The CONTEXT of the item is APPROPRIATE and

ENGAGING

6. The CONTEXT of the item is APPROPRIATE and

ENGAGING

Page 57: Historical Meap

End of Criteria for Valid Items

End of Criteria for Valid Items

Page 58: Historical Meap

Can be applied to Standards (Content Expectations), AND to Assessment Items

(Dr. Norman Webb, University of Wisconsin, 1997, 2002)

1. Recall2. Skills and Concepts3. Strategic Thinking4. Extended Thinking

Can be applied to Standards (Content Expectations), AND to Assessment Items

(Dr. Norman Webb, University of Wisconsin, 1997, 2002)

1. Recall2. Skills and Concepts3. Strategic Thinking4. Extended Thinking

Depth of KnowledgeDepth of Knowledge

Page 59: Historical Meap

Level 1 – RecallThe recall of information (fact, definition, or term), or

performing a simple procedure (a “recipe”), or applying a simple algorithm or formula. Requires only a rote response, a well-known formula, or following a well-defined procedure that typically involves only one step. Key words include “identify,” “recognize,” “use,” “calculate” and “measure.”

A student answering a Level 1 item either knows the answer or does not. The answer does not need to be “solved” or “figured out.”

(Verbs like “describe” and “explain” can be used at different levels depending on the complexity of

what’s being “described” or “explained.”)

Level 1 – RecallThe recall of information (fact, definition, or term), or

performing a simple procedure (a “recipe”), or applying a simple algorithm or formula. Requires only a rote response, a well-known formula, or following a well-defined procedure that typically involves only one step. Key words include “identify,” “recognize,” “use,” “calculate” and “measure.”

A student answering a Level 1 item either knows the answer or does not. The answer does not need to be “solved” or “figured out.”

(Verbs like “describe” and “explain” can be used at different levels depending on the complexity of

what’s being “described” or “explained.”)

Depth of KnowledgeDepth of Knowledge

Page 60: Historical Meap

B2.1A Explain how cells transform energy (ultimately obtained from the sun) from one form to another through the processes of photosynthesis and respiration. Identify the reactants and products in the general reaction of photosynthesis.

Level 1 Example: Which of the following is a product of the overall process of photosynthesis?A. Carbon dioxideB. ProteinC. Glucose*D. Water

B2.1A Explain how cells transform energy (ultimately obtained from the sun) from one form to another through the processes of photosynthesis and respiration. Identify the reactants and products in the general reaction of photosynthesis.

Level 1 Example: Which of the following is a product of the overall process of photosynthesis?A. Carbon dioxideB. ProteinC. Glucose*D. Water

DOK Level 1DOK Level 1

Page 61: Historical Meap

E4.p2A Describe the composition and layers of the atmosphere. (prerequisite)

Level 1 Example: Air is made up of many gases. Which gas is found in the greatest amount?A. Nitrogen*B. OxygenC. Carbon DioxideD. Hydrogen

Science Framework for the 2009 NAEP

E4.p2A Describe the composition and layers of the atmosphere. (prerequisite)

Level 1 Example: Air is made up of many gases. Which gas is found in the greatest amount?A. Nitrogen*B. OxygenC. Carbon DioxideD. Hydrogen

Science Framework for the 2009 NAEP

DOK Level 1DOK Level 1

Page 62: Historical Meap

Level 2 – Skills & Concepts

This level is more complex and involves mental processing beyond simply recalling or reproducing a response. Items require students to make some decisions and typically involve more than one step. Key words and activities include “classifying,” “organizing,” “estimating,” or “interpreting or comparing data” in tables graphs or charts.

Level 2 – Skills & Concepts

This level is more complex and involves mental processing beyond simply recalling or reproducing a response. Items require students to make some decisions and typically involve more than one step. Key words and activities include “classifying,” “organizing,” “estimating,” or “interpreting or comparing data” in tables graphs or charts.

Depth of KnowledgeDepth of Knowledge

Page 63: Historical Meap

From Science Framework for the 2009 NAEP

DOK Level 2DOK Level 2

Page 64: Historical Meap

P2.1D Describe and analyze the motion that a position-time graph represents, given the graph.

Example: The graph below shows the distance traveled over time by a student walking down a hall. During which time interval was the student moving the fastest?

a. Ab. Bc. Cd. D*

P2.1D Describe and analyze the motion that a position-time graph represents, given the graph.

Example: The graph below shows the distance traveled over time by a student walking down a hall. During which time interval was the student moving the fastest?

a. Ab. Bc. Cd. D*

DOK Level 2DOK Level 2

From Science Framework for the 2009 NAEP

Page 65: Historical Meap

Which of the following graphs best illustrates the relationship between the heat released by the foods listed in Table 1 and the change in water temperature?

a. b. c. d.

Which of the following graphs best illustrates the relationship between the heat released by the foods listed in Table 1 and the change in water temperature?

a. b. c. d.

DOK Level 2DOK Level 2

Preparing for the ACT 2006/2007

Page 66: Historical Meap

Level 3 – Strategic Thinking

This level is more demanding and requires planning, using evidence, and complex and abstract reasoning. In most instances, requiring students to explain their thinking is Level 3. Students are asked to draw conclusions, cite evidence, develop logical arguments, solve complex problems, explain concepts and justify their response.

Level 3 – Strategic Thinking

This level is more demanding and requires planning, using evidence, and complex and abstract reasoning. In most instances, requiring students to explain their thinking is Level 3. Students are asked to draw conclusions, cite evidence, develop logical arguments, solve complex problems, explain concepts and justify their response.

Depth of KnowledgeDepth of Knowledge

Page 67: Historical Meap

Level 3 Example: The main reason for Earth’s temperature being hotter in summer than in winter is:

A. The earth’s distance from the sun changes.

B. The sun is higher in the sky.*C. The distance between the northern

hemisphere and the sun changes.D. Ocean currents carry warm water north.

Level 3 Example: The main reason for Earth’s temperature being hotter in summer than in winter is:

A. The earth’s distance from the sun changes.

B. The sun is higher in the sky.*C. The distance between the northern

hemisphere and the sun changes.D. Ocean currents carry warm water north.

Depth of KnowledgeDepth of Knowledge

From Science Framework for the 2009 NAEP

Page 68: Historical Meap

DOK 3(From ACT Practice Test www.actstudent.org)

DOK 3(From ACT Practice Test www.actstudent.org)

26. A student claimed that “If the masses of 1 cm3 of any solid and 1 cm3 of any liquid are compared, the mass of the solid will be greater.” Do the data in Table 1 support his claim?

F. No; lead has a higher density than any of the liquids listed.

G. No; mercury has a higher density than any of the solids listed.

H. Yes; lead has a higher density than any of the liquids listed.

J. Yes; mercury has a higher density than any of the solids listed.

26. A student claimed that “If the masses of 1 cm3 of any solid and 1 cm3 of any liquid are compared, the mass of the solid will be greater.” Do the data in Table 1 support his claim?

F. No; lead has a higher density than any of the liquids listed.

G. No; mercury has a higher density than any of the solids listed.

H. Yes; lead has a higher density than any of the liquids listed.

J. Yes; mercury has a higher density than any of the solids listed.

Table 1

DensitySubstance Phase (g/cm3)Arsenic solid 5.73Glucose solid 1.56Iron solid 7.86Lead solid 11.34Zinc solid 7.14

Ethanol liquid 0.79Ethyl ether liquid 0.71Glycerol liquid 1.26Mercury liquid 13.59

Freon-12 gas 0.00495

Krypton gas 0.00343

Methane gas 0.00065

Table 1

DensitySubstance Phase (g/cm3)Arsenic solid 5.73Glucose solid 1.56Iron solid 7.86Lead solid 11.34Zinc solid 7.14

Ethanol liquid 0.79Ethyl ether liquid 0.71Glycerol liquid 1.26Mercury liquid 13.59

Freon-12 gas 0.00495

Krypton gas 0.00343

Methane gas 0.00065

Page 69: Historical Meap

Level 4 – Extended Thinking

This level requires complex reasoning, experimental design, and planning usually over extended periods of time. Students are asked to make connections within or among content areas. Level 4 tasks are typically assessed locally and often involve performance or open-ended assessments. Many on-demand instruments will not include any items at Level 4.

Level 4 – Extended Thinking

This level requires complex reasoning, experimental design, and planning usually over extended periods of time. Students are asked to make connections within or among content areas. Level 4 tasks are typically assessed locally and often involve performance or open-ended assessments. Many on-demand instruments will not include any items at Level 4.

Depth of KnowledgeDepth of Knowledge

Page 70: Historical Meap

End of Depth of Knowledge (DOK)

End of Depth of Knowledge (DOK)

Page 71: Historical Meap

Poorly written Well/poorly-written Well-written

Poorly aligned Well/Poorly aligned Well-aligned

Inappropriate DOK Appropriate/Inappropriate DOK Appropriate DOK

(Beyond hope) (Salvageable) (Acceptable)

Poorly written Well/poorly-written Well-written

Poorly aligned Well/Poorly aligned Well-aligned

Inappropriate DOK Appropriate/Inappropriate DOK Appropriate DOK

(Beyond hope) (Salvageable) (Acceptable)

“Jurying” and Editing Items:

“Jurying” and Editing Items:

Page 72: Historical Meap

Unnecessary information in the stemMutually exclusive optionsAlignment Issues (HSCEs)Vocabulary Issues (Companion

Documents)

Unnecessary information in the stemMutually exclusive optionsAlignment Issues (HSCEs)Vocabulary Issues (Companion

Documents)

“Jurying” and Editing Items:

“Jurying” and Editing Items:

Page 73: Historical Meap

Body cells of fruit flies contain only 8 chromosomes, compared to human cells that contain 46. Scientists used studies of fruit flies to discover how egg and sperm cells (gametes) are formed. What did they observe?A. Body cells of the offspring flies had 16

chromosomes.B. Sperm cells from the male had 8 chromosomes.C. Egg cells from the female had 4 chromosomes.*D. Body cells of the offspring flies had 4

chromosomes. Connecticut Released Item 2006

Body cells of fruit flies contain only 8 chromosomes, compared to human cells that contain 46. Scientists used studies of fruit flies to discover how egg and sperm cells (gametes) are formed. What did they observe?A. Body cells of the offspring flies had 16

chromosomes.B. Sperm cells from the male had 8 chromosomes.C. Egg cells from the female had 4 chromosomes.*D. Body cells of the offspring flies had 4

chromosomes. Connecticut Released Item 2006

“Jurying” and Editing Items:Examples (unnecessary information)“Jurying” and Editing Items:

Examples (unnecessary information)

Page 74: Historical Meap

The body cells of fruit flies each contain 8 chromosomes. Which of the following statements correctly describes how fruit flies reproduce sexually?

A. Body cells of the offspring flies have 16 chromosomes.

B. Sperm cells from the male flies have 8 chromosomes.C. Egg cells from the female flies have 4 chromosomes.*D. Body cells of the offspring flies have 4

chromosomes.

The body cells of fruit flies each contain 8 chromosomes. Which of the following statements correctly describes how fruit flies reproduce sexually?

A. Body cells of the offspring flies have 16 chromosomes.

B. Sperm cells from the male flies have 8 chromosomes.C. Egg cells from the female flies have 4 chromosomes.*D. Body cells of the offspring flies have 4

chromosomes.

“Jurying” and Editing Items:Examples (unnecessary information)“Jurying” and Editing Items:

Examples (unnecessary information)

Page 75: Historical Meap

“Jurying” and Editing Items:Examples (unnecessary information)“Jurying” and Editing Items:

Examples (unnecessary information)

California Released Item 2007

Page 76: Historical Meap

Under what conditions will a substance be likely to enter a cell through diffusion?

A. when the substance is a particle of foodB. when a molecule of the substance is very largeC. when the concentration of the substance is

greater outside the cell than inside*D. when the concentration of the substance is

greater inside the cell than outside

Connecticut Released Item 2006

Under what conditions will a substance be likely to enter a cell through diffusion?

A. when the substance is a particle of foodB. when a molecule of the substance is very largeC. when the concentration of the substance is

greater outside the cell than inside*D. when the concentration of the substance is

greater inside the cell than outside

Connecticut Released Item 2006

“Jurying” and Editing Items:Examples (mutually exclusive

options)

“Jurying” and Editing Items:Examples (mutually exclusive

options)

Page 77: Historical Meap

Under what conditions will a substance be likely to enter a cell through diffusion?

A. when the substance is a particle of food B. when a molecule of the substance is very

large C. when the concentration of the substance is greater outside the cell than inside D. when the substance is needed to store genetic information

Under what conditions will a substance be likely to enter a cell through diffusion?

A. when the substance is a particle of food B. when a molecule of the substance is very

large C. when the concentration of the substance is greater outside the cell than inside D. when the substance is needed to store genetic information

“Jurying” and Editing Items:Examples (mutually exclusive

options)

“Jurying” and Editing Items:Examples (mutually exclusive

options)

Page 78: Historical Meap

“Jurying” and Editing Items:Alignment?

“Jurying” and Editing Items:Alignment?

California Released Item 2007

“The Coriolis effect is the apparent deflection of moving objects from a straight path when they are viewed from a rotating frame of reference.”

http://en.wikipedia.org

Page 79: Historical Meap

E4.p2I Identify major global wind belts (trade winds, prevailing westerlies, and polar easterlies) and that their vertical components control the global distribution of rainforests and deserts. (prerequisite)

E4.2A Describe the major causes for the ocean’s surface and deep water currents, including the prevailing winds, the Coriolis effect, unequal heating of the earth, changes in water temperature and salinity in high latitudes, and basin shape.

E4.2f Explain how the Coriolis effect controls oceanic circulation.

E4.p2I Identify major global wind belts (trade winds, prevailing westerlies, and polar easterlies) and that their vertical components control the global distribution of rainforests and deserts. (prerequisite)

E4.2A Describe the major causes for the ocean’s surface and deep water currents, including the prevailing winds, the Coriolis effect, unequal heating of the earth, changes in water temperature and salinity in high latitudes, and basin shape.

E4.2f Explain how the Coriolis effect controls oceanic circulation.

“Jurying” and Editing Items:No Alignment to MI HSCEs

“Jurying” and Editing Items:No Alignment to MI HSCEs

California Released Item 2007

Page 80: Historical Meap

B2.4e Explain how cellular respiration is important for the production of ATP (build

on aerobic vs. anaerobic).

“cytokinesis?”

Massachusetts Released Item 2006

B2.4e Explain how cellular respiration is important for the production of ATP (build

on aerobic vs. anaerobic).

“cytokinesis?”

Massachusetts Released Item 2006

“Jurying” and Editing Items:Vocabulary Issues

“Jurying” and Editing Items:Vocabulary Issues

Page 81: Historical Meap

HSSCE Companion Document: “The companion document is intended to provide boundaries to the content expectations.”

Unit 9 Cell Division Vocabulary:

cancercarcinogenic jumping geneschromosome karyotypechromosome pair meiosiscrossing over mitosisdeletion mutationDNA replication new gene

combinationsdiploid progenyduplication of genes recombination

of haploid genetic

materialgametes sex cellgenetic variation sex

HSSCE Companion Document: “The companion document is intended to provide boundaries to the content expectations.”

Unit 9 Cell Division Vocabulary:

cancercarcinogenic jumping geneschromosome karyotypechromosome pair meiosiscrossing over mitosisdeletion mutationDNA replication new gene

combinationsdiploid progenyduplication of genes recombination

of haploid genetic

materialgametes sex cellgenetic variation sex

“Jurying” and Editing Items:Vocabulary Issues

“Jurying” and Editing Items:Vocabulary Issues

Page 82: Historical Meap

California Released Item 2007

E3.1A Discriminate between igneous, metamorphic, and sedimentary rocks and describe the processes that change one kind of rock into another.

California Released Item 2007

E3.1A Discriminate between igneous, metamorphic, and sedimentary rocks and describe the processes that change one kind of rock into another.

“Jurying” and Editing Items:Vocabulary Issues

“Jurying” and Editing Items:Vocabulary Issues

HSSCE Companion Document Unit 4 Rock Forming Processes

Vocabulary:

contact metamorphism magmacooling metamorphic rocks crystallization metamorphismdeposition molten rockerosion non-foliated

textureextrusive plate tectonic contextFoliation regional

metamorphismgrain shape rock cyclegrain size rock sequenceigneous rocks sedimentary rocksIntrusive sedimentationlithification weathering

HSSCE Companion Document Unit 4 Rock Forming Processes

Vocabulary:

contact metamorphism magmacooling metamorphic rocks crystallization metamorphismdeposition molten rockerosion non-foliated

textureextrusive plate tectonic contextFoliation regional

metamorphismgrain shape rock cyclegrain size rock sequenceigneous rocks sedimentary rocksIntrusive sedimentationlithification weathering

Page 83: Historical Meap

Fluorine is a gas at room temperature while iodine is a solid. This is because:

A. fluorine atoms have less attractive forces than iodine atoms*

B. iodine atoms repel each other more than fluorine atoms

C. fluorine can lose an electron to form -1 ionsD. iodine atoms are spread out more than fluorine atoms

C4.3A Recognize that substances that are solid at room temperature have stronger attractive forces than liquids at room temperature, which have stronger attractive forces than gases at room temperature.

Fluorine is a gas at room temperature while iodine is a solid. This is because:

A. fluorine atoms have less attractive forces than iodine atoms*

B. iodine atoms repel each other more than fluorine atoms

C. fluorine can lose an electron to form -1 ionsD. iodine atoms are spread out more than fluorine atoms

C4.3A Recognize that substances that are solid at room temperature have stronger attractive forces than liquids at room temperature, which have stronger attractive forces than gases at room temperature.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 84: Historical Meap

Guideline #1: Incomplete Stem Complete Stem:

Which of the following statements best describes why fluorine is a gas at room temperature while iodine is a solid?

A. Fluorine atoms have less attractive forces than iodine atoms.*

B. Iodine atoms repel each other more than fluorine atoms.

C. Fluorine atoms can lose an electron to form -1 ions.D. Iodine atoms are spread out more than fluorine atoms.

C4.3A Recognize that substances that are solid at room temperature have stronger attractive forces than liquids at room temperature, which have stronger attractive forces than gases at room temperature.

Guideline #1: Incomplete Stem Complete Stem:

Which of the following statements best describes why fluorine is a gas at room temperature while iodine is a solid?

A. Fluorine atoms have less attractive forces than iodine atoms.*

B. Iodine atoms repel each other more than fluorine atoms.

C. Fluorine atoms can lose an electron to form -1 ions.D. Iodine atoms are spread out more than fluorine atoms.

C4.3A Recognize that substances that are solid at room temperature have stronger attractive forces than liquids at room temperature, which have stronger attractive forces than gases at room temperature.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 85: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

In which state of matter would molecules move the fastest?

A. Gas*B. SolidC. LiquidD. None of the above

C2.2B Describe the various states of matter in terms of the motion and arrangement of the molecules (atoms) making up the substance.

In which state of matter would molecules move the fastest?

A. Gas*B. SolidC. LiquidD. None of the above

C2.2B Describe the various states of matter in terms of the motion and arrangement of the molecules (atoms) making up the substance.

Page 86: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

3 “plausible, incorrect” optionsIn which of the following substances do

molecules move the fastest? A. Gas*B. Crystalline SolidC. LiquidD. Aqueous solution

C2.2B Describe the various states of matter in terms of the motion and arrangement of the molecules (atoms) making up the substance.

(Other examples: “increase, decrease, remain the same…”)

3 “plausible, incorrect” optionsIn which of the following substances do

molecules move the fastest? A. Gas*B. Crystalline SolidC. LiquidD. Aqueous solution

C2.2B Describe the various states of matter in terms of the motion and arrangement of the molecules (atoms) making up the substance.

(Other examples: “increase, decrease, remain the same…”)

Page 87: Historical Meap

A rock is dropped from the top of a cliff. How far did the rock travel in 3.0 s? (Use the 10.0 m/s/s as the acceleration due to gravity.)

A. 30. mB. 45. m*C. 90. mD. 120 m

P2.1g Solve problems involving average speed and constant acceleration in one dimension.

A rock is dropped from the top of a cliff. How far did the rock travel in 3.0 s? (Use the 10.0 m/s/s as the acceleration due to gravity.)

A. 30. mB. 45. m*C. 90. mD. 120 m

P2.1g Solve problems involving average speed and constant acceleration in one dimension.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 88: Historical Meap

Guideline #10: Present problem clearly (Which 3 seconds??)

Guideline #2: Verb tense

A rock was dropped from the top of a cliff. How far did the rock travel during the first 3.0 s? (Use the 10.0 m/s/s as the acceleration due to gravity.)

A. 30. mB. 45. m*C. 90. mD. 120 m

P2.1g Solve problems involving average speed and constant acceleration in one dimension.

Guideline #10: Present problem clearly (Which 3 seconds??)

Guideline #2: Verb tense

A rock was dropped from the top of a cliff. How far did the rock travel during the first 3.0 s? (Use the 10.0 m/s/s as the acceleration due to gravity.)

A. 30. mB. 45. m*C. 90. mD. 120 m

P2.1g Solve problems involving average speed and constant acceleration in one dimension.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 89: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

E2.2C Describe natural processes in which heat transfer in the Earth occurs by conduction, convection, and radiation.

The Earth’s mantle is made up of very hot material that rises to the top of the mantle, cools, then sinks, reheats, and rises again, constantly repeating the cycle. This action, which causes the Earth’s crust to move, is known as —A convection currents B magnetic fieldsC hot spotsD advection forces Virginia Released Item 2006

E2.2C Describe natural processes in which heat transfer in the Earth occurs by conduction, convection, and radiation.

The Earth’s mantle is made up of very hot material that rises to the top of the mantle, cools, then sinks, reheats, and rises again, constantly repeating the cycle. This action, which causes the Earth’s crust to move, is known as —A convection currents B magnetic fieldsC hot spotsD advection forces Virginia Released Item 2006

Page 90: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Guideline #2: Grammar & VocabularyThe Earth’s mantle is made up of very hot

material that rises to the top of the mantle, cools, then sinks, reheats, and rises again, constantly repeating the cycle. This action, which causes the Earth’s crust to move, is known as —A convection currents B magnetic fieldsC hot spotsD advection forces Virginia Released Item 2006

Guideline #2: Grammar & VocabularyThe Earth’s mantle is made up of very hot

material that rises to the top of the mantle, cools, then sinks, reheats, and rises again, constantly repeating the cycle. This action, which causes the Earth’s crust to move, is known as —A convection currents B magnetic fieldsC hot spotsD advection forces Virginia Released Item 2006

Page 91: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Guideline #2: Grammar & VocabularyThe Earth’s mantle is made up of very hot

material that rises, cools, then sinks, reheats, and rises again, constantly repeating the cycle. Which of the following terms best describes this process?A a convection currentB a magnetic fieldC a seismic waveD elastic rebound

Guideline #2: Grammar & VocabularyThe Earth’s mantle is made up of very hot

material that rises, cools, then sinks, reheats, and rises again, constantly repeating the cycle. Which of the following terms best describes this process?A a convection currentB a magnetic fieldC a seismic waveD elastic rebound

Page 92: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

P.EN.04.41 Demonstrate how temperature can be increased in a substance by adding energy.

A substance is heated. Which of the following would NOT be correct?

A. The measured temperature of the substance would go up

B. The molecules in the substance would move faster

C. The average kinetic energy of the molecules would increase

D. The substance must undergo a phase change*

P.EN.04.41 Demonstrate how temperature can be increased in a substance by adding energy.

A substance is heated. Which of the following would NOT be correct?

A. The measured temperature of the substance would go up

B. The molecules in the substance would move faster

C. The average kinetic energy of the molecules would increase

D. The substance must undergo a phase change*

Page 93: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

“Test-Wiseness” Issues:A substance is heated. Which of the

following would NOT be correct?A. The measured temperature of the

substance would go upB. The molecules in the substance would

move fasterC. The average kinetic energy of the

molecules would increaseD. The substance must undergo a phase

change*

“Test-Wiseness” Issues:A substance is heated. Which of the

following would NOT be correct?A. The measured temperature of the

substance would go upB. The molecules in the substance would

move fasterC. The average kinetic energy of the

molecules would increaseD. The substance must undergo a phase

change*

Page 94: Historical Meap

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

“Test-Wiseness” Issues:Which of the following is necessary for

the temperature of a substance to increase?

A. The substance must undergo a change in state.

B. The particles in the substance must decrease speed.

C. The substance must be mixed with another substance.

D. The substance must gain heat energy.*

“Test-Wiseness” Issues:Which of the following is necessary for

the temperature of a substance to increase?

A. The substance must undergo a change in state.

B. The particles in the substance must decrease speed.

C. The substance must be mixed with another substance.

D. The substance must gain heat energy.*

Page 95: Historical Meap

A physics book is sitting on a table. Identify the action and reaction forces acting on the book.

A. The book pushes down on the table. The table pushes up on the book.*B. The book pulls up on Earth. Earth pulls down on the book. C. The table pulls the book down. The book pushes the table down.D. There are no action and reaction forces acting because the book is at rest.

P3.3A Identify the action and reaction force from examples of forces in everyday situations (e.g., book on a table, walking across the floor, pushing open a door).

A physics book is sitting on a table. Identify the action and reaction forces acting on the book.

A. The book pushes down on the table. The table pushes up on the book.*B. The book pulls up on Earth. Earth pulls down on the book. C. The table pulls the book down. The book pushes the table down.D. There are no action and reaction forces acting because the book is at rest.

P3.3A Identify the action and reaction force from examples of forces in everyday situations (e.g., book on a table, walking across the floor, pushing open a door).

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 96: Historical Meap

Guideline #6: Make sure there is only one correct answer.(A and B)

Letter ‘D’ is essentially “none of these.”

A physics book is sitting on a table. Identify the action and reaction forces acting on the book.A. The book pushes down on the table. The table pushes up

on the book.*B. The book pulls up on Earth. Earth pulls down on the book. C. The table pulls the book down. The book pushes the table

down.D. There are no action and reaction forces acting because

the book is at rest.

Guideline #6: Make sure there is only one correct answer.(A and B)

Letter ‘D’ is essentially “none of these.”

A physics book is sitting on a table. Identify the action and reaction forces acting on the book.A. The book pushes down on the table. The table pushes up

on the book.*B. The book pulls up on Earth. Earth pulls down on the book. C. The table pulls the book down. The book pushes the table

down.D. There are no action and reaction forces acting because

the book is at rest.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 97: Historical Meap

Which of the following statements best describes the action and reaction forces acting on a book which is sitting on a table.

A. The book pushes down on the table. The table pushes up on the book.*

B. The book pushes down on the table. The table pushes down on the floor.

C. The floor pushes up on the table. The table pushes up on the book.

D. The table pushes down on the floor. The floor pushes up on the table.

Which of the following statements best describes the action and reaction forces acting on a book which is sitting on a table.

A. The book pushes down on the table. The table pushes up on the book.*

B. The book pushes down on the table. The table pushes down on the floor.

C. The floor pushes up on the table. The table pushes up on the book.

D. The table pushes down on the floor. The floor pushes up on the table.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 98: Historical Meap

On a pool table the first shot is called the break. Motion of the balls in the break would be considered which type of motion?A. linearB. two-dimensional*C. periodicD. circular

P2.1E Describe and classify various motions in a plane as one dimensional, two dimensional, circular, or periodic.

On a pool table the first shot is called the break. Motion of the balls in the break would be considered which type of motion?A. linearB. two-dimensional*C. periodicD. circular

P2.1E Describe and classify various motions in a plane as one dimensional, two dimensional, circular, or periodic.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 99: Historical Meap

Guidelines #10 Don’t use the stem to instruct, and #12: Avoid bias

On a pool table the first shot is called the break. Motion of the balls in the break would be considered which type of motion?A. linearB. two-dimensional*C. periodicD. circular

P2.1E Describe and classify various motions in a plane as one dimensional, two dimensional, circular, or periodic.

Guidelines #10 Don’t use the stem to instruct, and #12: Avoid bias

On a pool table the first shot is called the break. Motion of the balls in the break would be considered which type of motion?A. linearB. two-dimensional*C. periodicD. circular

P2.1E Describe and classify various motions in a plane as one dimensional, two dimensional, circular, or periodic.

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 100: Historical Meap

Two rubber balls roll toward each other on a flat surface and collide, as shown in the figure. After the collision, each ball continues moving, but with a different speed and direction. Which term best describes this type of motion?A. linearB. two-dimensional*C. periodicD. circular

Two rubber balls roll toward each other on a flat surface and collide, as shown in the figure. After the collision, each ball continues moving, but with a different speed and direction. Which term best describes this type of motion?A. linearB. two-dimensional*C. periodicD. circular

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 101: Historical Meap

1. During the combustion of methane gas, methane combines with oxygen gas to form water and carbon dioxide. The heats of formations of methane, carbon dioxide and water in kJ/mol are -1200, -400 and -300 respectively. Use Hess’ Law to determine the amount of heat given off when 8 grams of methane are burned.

A. 200 kJB. 400 kJC. 100 kJD. 250 kJ

1. During the combustion of methane gas, methane combines with oxygen gas to form water and carbon dioxide. The heats of formations of methane, carbon dioxide and water in kJ/mol are -1200, -400 and -300 respectively. Use Hess’ Law to determine the amount of heat given off when 8 grams of methane are burned.

A. 200 kJB. 400 kJC. 100 kJD. 250 kJ

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 102: Historical Meap

1. During the combustion of methane gas, methane combines with oxygen gas to form water and carbon dioxide. The heats of formations are given below:

Compound Heat of Formation (kJ/mol) methane -1200 carbon dioxide -400 water -300

Use Hess’ Law to determine the amount of heat given off when 8 grams of methane are burned.

A. 200 kJB. 400 kJC. 100 kJD. 250 kJ

1. During the combustion of methane gas, methane combines with oxygen gas to form water and carbon dioxide. The heats of formations are given below:

Compound Heat of Formation (kJ/mol) methane -1200 carbon dioxide -400 water -300

Use Hess’ Law to determine the amount of heat given off when 8 grams of methane are burned.

A. 200 kJB. 400 kJC. 100 kJD. 250 kJ

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 103: Historical Meap

16. Look at the two pictures below. They show what happened when two solid blocks were each put in a jar containing a liquid. Based just on what you can see in the pictures, what can you say about the blocks and the jars?

       

A)  The liquid in the jars must be water. B)  The block in jar 1 weighs more than the block in jar 2. C)  The block in jar 1 is floating lower in its liquid than is the block in jar

2. D)  The block in jar 1 must be made of metal and the block in jar 2

must be made of wood. NAEP Released Item http://nces.ed.gov

16. Look at the two pictures below. They show what happened when two solid blocks were each put in a jar containing a liquid. Based just on what you can see in the pictures, what can you say about the blocks and the jars?

       

A)  The liquid in the jars must be water. B)  The block in jar 1 weighs more than the block in jar 2. C)  The block in jar 1 is floating lower in its liquid than is the block in jar

2. D)  The block in jar 1 must be made of metal and the block in jar 2

must be made of wood. NAEP Released Item http://nces.ed.gov

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 104: Historical Meap

16. Look at the two pictures below. They show what happened when two solid blocks were each put in a jar containing a liquid. Based just on what you can see in the pictures, what can you say about the blocks and the jars?

       

A)  The liquid in the jars must be water. B)  The block in jar 1 weighs more than the block in jar 2. C)  The block in jar 1 is floating lower in its liquid than is the block in jar

2. D)  The block in jar 1 must be made of metal and the block in jar 2

must be made of wood. NAEP Released Item http://nces.ed.gov

16. Look at the two pictures below. They show what happened when two solid blocks were each put in a jar containing a liquid. Based just on what you can see in the pictures, what can you say about the blocks and the jars?

       

A)  The liquid in the jars must be water. B)  The block in jar 1 weighs more than the block in jar 2. C)  The block in jar 1 is floating lower in its liquid than is the block in jar

2. D)  The block in jar 1 must be made of metal and the block in jar 2

must be made of wood. NAEP Released Item http://nces.ed.gov

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 105: Historical Meap

Guideline #13: When calling for a judgment or conclusion, the student must be able to infer the correct answer from the text. Don’t pose questions such as “What do you think? / What would you do?” because any option could be defensible.

16. Look at the two pictures below. They show what happened when two solid blocks were each put in a jar containing a liquid. Based just on what you can see in the pictures, what can you say about the blocks and the jars?

       

A)  The liquid in the jars must be water. B)  The block in jar 1 weighs more than the block in jar 2. C)  The block in jar 1 is floating lower in its liquid than is the block in jar 2. D)  The block in jar 1 must be made of metal and the block in jar 2 must be made of

wood. NAEP Released Item http://nces.ed.gov

Guideline #13: When calling for a judgment or conclusion, the student must be able to infer the correct answer from the text. Don’t pose questions such as “What do you think? / What would you do?” because any option could be defensible.

16. Look at the two pictures below. They show what happened when two solid blocks were each put in a jar containing a liquid. Based just on what you can see in the pictures, what can you say about the blocks and the jars?

       

A)  The liquid in the jars must be water. B)  The block in jar 1 weighs more than the block in jar 2. C)  The block in jar 1 is floating lower in its liquid than is the block in jar 2. D)  The block in jar 1 must be made of metal and the block in jar 2 must be made of

wood. NAEP Released Item http://nces.ed.gov

“Jurying” and Editing ItemsExamples

“Jurying” and Editing ItemsExamples

Page 106: Historical Meap

Contact InformationContact Information

Tom Wessels, DirectorGrand Traverse Regional Math and Science

CenterTraverse Bay Area Intermediate School

District1101 Red Drive PO Box 6020Traverse City, MI 49682231.922.7875

[email protected]

Tom Wessels, DirectorGrand Traverse Regional Math and Science

CenterTraverse Bay Area Intermediate School

District1101 Red Drive PO Box 6020Traverse City, MI 49682231.922.7875

[email protected]