Understanding Test Blueprints and Their Impact on Student Growth
Transcript of Understanding Test Blueprints and Their Impact on Student Growth
![Page 1: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/1.jpg)
Understanding Test Blueprints and Their Impact on Student Growth Holly Lavender and Cathryn Shaw
![Page 2: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/2.jpg)
Learning Targets
1. Understand the definition and purpose of test blueprints
2. Know how to build a test blueprint
3. Understand how assessment blueprints connect to other initiatives
![Page 3: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/3.jpg)
The Purpose of Test Blueprints
1
![Page 4: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/4.jpg)
The Purpose of Test Blueprints • Define the parameters of an assessment before you
begin the creation process. • Review a locally- or vendor-developed assessment for
alignment to learning expectations, instruction, and cognitive complexity.
• Confirm in advance the information and knowledge you are planning to assess, which can guide instructional activities.
• Share blueprints with students to help make learning targets clear.
![Page 5: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/5.jpg)
A Blueprint is a TOOL to ensure: alignment of assessment to standards
- content - depth of knowledge
alignment of assessment to instruction adequate “stretch” in assessment
![Page 6: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/6.jpg)
Building a Test Blueprint
2
![Page 7: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/7.jpg)
Where to Begin?
The End
Design the blueprint Create the assessment Plan for Instruc7on
![Page 8: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/8.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1
2
3
4
5
Total Points
Weight
IDENTIFIED STANDARDS
![Page 9: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/9.jpg)
Learning Priorities Selection
Unit Test
Quarterly Assessment
SLO Pre/Post-Assessment
![Page 10: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/10.jpg)
Identify Learning Priorities
Longevity
Leverage
Levels
Learning Priorities
Ainsworth, L. (2003). Power standards: Iden7fying the standards that maHer the most. Englewood, CO: Lead and Learn Press.
![Page 11: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/11.jpg)
Learning Priorities: Longevity
• The knowledge and skills that are important today, tomorrow, and in the future.
• Example: Reading Comprehension Longevity
![Page 12: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/12.jpg)
Learning Priorities: Leverage
• The knowledge and skills that are used in more than one content area at a specific grade level (horizontal).
• Example: Creating and analyzing charts and graphs
Leverage
![Page 13: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/13.jpg)
Learning Priorities: Levels • The knowledge and
skills that are key to success in subsequent grade levels or courses.
• Example: Strong understanding of fractions to prepare for work with rational numbers
Levels
![Page 14: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/14.jpg)
Learning Priorities
• Introduced early • Reinforced often • Multiple opportunities
for learning Learning Priorities
![Page 15: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/15.jpg)
Identify Learning Priorities
Clear Focus
Improved Validity
SLO Checklist
![Page 16: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/16.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2. Determine the lesson or morale of a story
3. Demonstrate understanding of key details in a text
4. Iden5fy differences in point of view
5. Compare and contrast two or more versions of a story
Total Points
Weight
IDENTIFIED STANDARDS
![Page 17: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/17.jpg)
Alignment Content: Are the expectations consistent for both curriculum and assessments? Depth of Knowledge (Rigor): Are the cognitive demands of curricular aims and assessments the same?
– For individual items? – For the assessment as a whole?
![Page 18: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/18.jpg)
Cognitive Complexity
• Common Core State Standards and Ohio’s New Learning Standards call for a higher level of cognitive complexity than previous assessments could measure
![Page 19: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/19.jpg)
©2013. Battelle for Kids. All Rights Reserved.
Hess’ Cognitive Rigor Matrix
Bloom’s Taxonomy Webb’s Depth of Knowledge
![Page 20: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/20.jpg)
DoK Level 1 Routine Thinking
Recall and Reproduction
Describe Measure Define Calculate Locate Memorize
Jen Jones (2013); helloliteracy.blogspot.com
![Page 21: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/21.jpg)
DoK Level 2 Conceptual Thinking
Basic Application and Reasoning
Classify Categorize Summarize Graph Infer Interpret
Jen Jones (2013); helloliteracy.blogspot.com
![Page 22: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/22.jpg)
DoK Level 3 Strategic Thinking
Higher Level Application
Discuss Revise
Investigate Assess Critique Argue/Justify
* Usually more than one right answer Jen Jones (2013); helloliteracy.blogspot.com
![Page 23: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/23.jpg)
DoK Level 4 Extended Thinking
Real World Application
Synthesize Create Design Plan Develop Propose
* Usually over an extended period of time Jen Jones (2013); helloliteracy.blogspot.com
![Page 24: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/24.jpg)
Is This Item Aligned? K.G2: Correctly name shapes regardless of their orientations or overall size. (CCSS)
• Show me the triangles. • Show me the pentagons. • Show me the cylinders.
![Page 25: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/25.jpg)
Is This Item Aligned?
LS 5.2: Food webs can be used to identify the relationships among producers, consumers and decomposers in an ecosystem. (ONLS)
![Page 26: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/26.jpg)
Is This Item Aligned?
Write the definitions for the following words:
1. Figure ______________________
2. Determine ______________________
3. Abode ______________________
3.L.4a: Use sentence-level context as a clue to the meaning of a word or phrase. (CCSS)
![Page 27: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/27.jpg)
Is This Item Aligned?
Analyze the causes and consequences of major political, economic and social developments of the 1930’s with emphasis on the Dust Bowl. (OACS)
![Page 28: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/28.jpg)
Is This Item Aligned?
Create a mixed media fantasy landscape that conveys a story, incorporating a well-known character from children’s fiction.
VA 6.4: Connect selected ideas, concepts and processes used in visual art with those used in other academic disciplines. VA 6.5: Use observations, life experiences and imagination as sources for visual symbols, images and creative expression.
![Page 29: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/29.jpg)
![Page 30: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/30.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
2. Determine the lesson or morale of a story
2
3. Demonstrate understanding of key details in a text
1
4. Iden5fy differences in point of view
3
5. Compare and contrast two or more versions of a story
3
Total Points
Weight
IDENTIFIED STANDARDS
![Page 31: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/31.jpg)
Determine Weight
What portion of the total test will be
assigned to each of the identified standards?
![Page 32: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/32.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
2. Determine the lesson or morale of a story
2
3. Demonstrate understanding of key details in a text
1
4. Iden5fy differences in point of view
3
5. Compare and contrast two or more versions of a story
3
Total Points
Weight
Weight
![Page 33: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/33.jpg)
How is Weight Determined?
Learning targets that are broad in scope
The importance of the standard
Standards that are emphasized in classroom instruction
![Page 34: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/34.jpg)
Why should I determine weight?
Ensures adequate coverage of content
Ensures emphasis is placed where intended
![Page 35: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/35.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2 5
points 20%
2. Determine the lesson or morale of a story
2 3
points 12%
3. Demonstrate understanding of key details in a text
1 3
points 12%
4. Iden5fy differences in point of view
3 6
points 24%
5. Compare and contrast two or more versions of a story
3 8
points 32%
Total Points
Weight
Weight
![Page 36: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/36.jpg)
Assessment Method
Many factors can affect the test format
![Page 37: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/37.jpg)
How Much Time is Available?
Time is a factor when it comes to preparing and scoring tests
Consider collaborating with others to write and/or evaluate test items
![Page 38: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/38.jpg)
What is to be Measured?
Choose the most appropriate format for
measuring the cognitive objectives of the test
FIP refers to this as the “target-method match”
![Page 39: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/39.jpg)
Which Assessment Method? Identify the eight major planets in the solar system.
Selected Response
Constructed Response
![Page 40: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/40.jpg)
Which Assessment Method? Make a line plot to display a data set of measurements in fractions of a unit (1/2, 1/4, 1/8).
Performance
![Page 41: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/41.jpg)
Which Assessment Method? Write arguments to support claims with clear reasons and relevant evidence.
Constructed Response
![Page 42: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/42.jpg)
Which Assessment Method? Work effectively in a group to complete a task or solve a problem for which the group is held accountable.
Performance
![Page 43: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/43.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
SR & CR 5
points 20%
2. Determine the lesson or morale of a story
2
CR 3
points 12%
3. Demonstrate understanding of key details in a text
1
SR 3
points 12%
4. Iden5fy differences in point of view
3
P 6
points 24%
5. Compare and contrast two or more versions of a story
3
P 8
points 32%
Total Points
Weight
Method
![Page 44: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/44.jpg)
Stretch
Assessments should demonstrate stretch Include both high and low-achieving students
![Page 45: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/45.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
SR & CR 5
points 20%
2. Determine the lesson or morale of a story
2
CR 3
points 12%
3. Demonstrate understanding of key details in a text
1
SR 3
points 12%
4. Iden5fy differences in point of view
3
P 6
points 24%
5. Compare and contrast two or more versions of a story
3
P 8
points 32%
Total Points
Weight
Cogni5ve Complexity
![Page 46: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/46.jpg)
Options to Demonstrate Stretch
Include items that assess content from adjacent grade levels
Include items that assess on-level standards at a greater or lesser depth
![Page 47: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/47.jpg)
How Much Stretch?
At least 50% of
assessment items should be at or
above the target DOK level
![Page 48: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/48.jpg)
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
SR & CR
2 points
3 points 5
points 20%
2. Determine the lesson or morale of a story
2
CR
1 point
2 points 3
points 12%
3. Demonstrate understanding of key details in a text
1
SR
3 points 3
points 12%
4. Iden5fy differences in point of view
3
P 2 points 4 points 6
points 24%
5. Compare and contrast two or more versions of a story
3
P
8 points 8
points 32%
Total Points 6 points 7 points 12 points 0 points
Weight 24% 28% 48% 0%
Cogni5ve Complexity
Rigor
![Page 49: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/49.jpg)
Putting It All Together
3
![Page 50: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/50.jpg)
Remember…. We are measuring the effects of teaching
Assessment must reflect what is taught
Teaching must reflect what is to be assessed
![Page 51: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/51.jpg)
Remember… Look for and evaluate items from banks and collaborate with others to write and evaluate high-quality items Use the data to improve instruction and Increase student learning. The quality of the data is only as good as the quality of the assessment!
![Page 53: Understanding Test Blueprints and Their Impact on Student Growth](https://reader033.fdocuments.in/reader033/viewer/2022051522/588c51251a28ab26218b47a3/html5/thumbnails/53.jpg)
Social Media
@OHEducation
ohio-department-of-education
Ohio Families and Education Ohio Teachers’ Homeroom
OhioEdDept
storify.com/ohioEdDept