Emily Jolley Let’s Measure Together Project – Constructed Response
description
Transcript of Emily Jolley Let’s Measure Together Project – Constructed Response
Emily JolleyLet’s Measure Together Project –
Constructed Response
Mentor: Dr. Tammy Howard, DPI August 6, 2013
kenanfellows.org dpi.state.nc.us/accountability
Kenan Fellows Externship
• 5 week externship this summer at the Testing and Accountability office of DPI
• Mentor is Dr. Tammy Howard – director of the testing and accountability division for DPI
• Attended department meetings, met with test development staff, item writers, content specialists, psychometricians, and just about everyone else involved in testing and accountability.
NC DPI Accountability Services Division
• The mission of the Accountability Services Division is to promote the academic achievement of all North Carolina public school students and to assist stakeholders in understanding and gauging this achievement against state and national standards.
NC DPI Accountability Services Division
The major thrust of this mission is three-fold: 1. the design and development of reliable
and valid assessment instruments2. the uniform implementation of and access
to suitable assessment instruments for all students
3. the provision of accurate and statistically appropriate reports.
EOCs and EOGs
• Fulfill requirements of Federal NCLB Act of 2001.• Developed by NCSU Technical Outreach for Public
Schools (TOPS) Program as directed by DPI• Questions are written by North Carolina teachers
and then edited by TOPS and DPI.• Standardized administration and grading statewide.• Rigorous development process that takes years to
complete.• Used to assess student learning and growth.
Measures of Student Learning/Common Exams
• Required by Race to the Top Grant.• Developed by NCSU Technical Outreach for Public
Schools (TOPS) Program as directed by NC D.P.I.• Questions are written by North Carolina teachers.• Not standardized, considered a local assessment
administered and graded by individual Local Education Authorities (LEAs)• Used to assess teachers for Standard 6
Item Writing Trainings
• Attended testing item writing training for teachers by TOPS.• Focused on breaking down the standards into
Revised Bloom’s Taxonomy verbs, standard objects, and supporting items.• After learning how to break down the
standards, basic instructions were given about writing items.
Item Writing Trainings
Revised Bloom’s Knowledge Dimensions:•Factual – basic elements•Conceptual – interrelationships•Procedural – how to do something•Metacognition – awareness of what you know
Revised Bloom’s Cognitive Processes:•Remember, Understand, Apply•Analyze, Evaluate, Create
Item Writing TrainingsFor example – breaking down a Civics & Economics standard: CE.C&G.2.7: Analyze contemporary issues and governmental responses at the local, state, and national levels in terms of how they promote the public interest and/or general welfare (eg: taxes, immigration, naturalization, civil rights, economic development, annexation, redistricting, zoning, national security, health care, etc)
Unifying Concept (found in unpacking doc)
ES/CO RBT Verb Object In Terms Of: Examples:
Government Systems, Structure, Function
CE.C&G.2.7
Analyze contemporary issues and governmental responses at the local, state, and national levels
in terms of how they promote the public interest and/or general welfare
taxes, immigration, naturalization, civil rights, economic development, annexation, redistricting, zoning, national security, health care, etc
Most essential standards for high school social studies fall under the understand or analyze range of conceptual knowledge. A few fall under the understand or evaluate range, and a couple standards are procedural knowledge (especially in World History & American History I & II).
Why Constructed Response?
• Constructed Response questions were included in MSLs for a variety of reasons:•Almost none of the new standards are based on
factual knowledge only. Because the new essential standards are more conceptual, having only multiple choice questions is not the best way to assess these standards•Many teachers have been asking for assessments
that more closely reflect students’ learning, and constructed response questions help facilitate that request
Why Constructed Response?
• As teachers, we need to ensure that the written, taught, and tested curriculum are aligned. •We can of course go beyond what the
standards require, but we need to make sure we’re teaching and assessing the students at least at the level the standards are written.
What about the Common Core State Standards?
• We also have literacy standards for history/social studies. These are not technically tested by MSLs, but we should still be teaching them with fidelity to help our students become better readers and writers. A sample reading standard:• CCSS.ELA-Literacy.RH.9-10.8: Assess the extent to which the
reasoning and evidence in a text support the author’s claims.
Unifying ConceptES/CO RBT Verb Object In Terms Of:
Reading in History/Social Studies
RH.9-10.8 Assess the extent to which reasoning and evidence in a text supports the author’s claims
n/a
Almost all of the Common Core State Standards for literacy in high school social studies fall under the apply, analyze, or evaluate ranges of procedural knowledge, with a few falling under the create range of procedural knowledge (especially the writing strand).
Essential Standards Breakdown
• Now, let’s practice breaking down our own standards into this basic format. Use the template from my site to start breaking down your standards:
• The unpacking documents can also be found on my site. It may be more useful (along with the examples in the standards) when figuring out what content to teach.
Unifying Concept (found in unpacking doc)
ES/CO RBT Verb Object In Terms Of: Examples:
Quick Assessment
•When you’ve broken down some of your standards, take the quick survey on my site to assess your comfort with this process.• The survey has some standards from each course. It
also has a few of the released constructed response questions from last fall. You should categorize each by RBT cognitive process and knowledge dimension.• I highly recommend breaking down all the standards
from your course this way (and looking through the unpacking document!)
Constructed Response Criteria
• Now we’re ready to write!• You need a few things to begin writing a
constructed response question. Basic format:• Stimulus or Context Statement (optional): chart,
map, picture, graph, text, quote, intro, or scenario•Prompt or Task Statement: directive to students•Requirements Statement: response specifics• Scoring Criteria: rubric, range 0-4 points
C&E Fall 2012 CR Released Example:
• Stimulus/Context Statement: Throughout its history, American democracy has upheld and strengthened citizens’ individual rights. • Prompt or Task Statement: Take a position that argues for or
against the above statement. • Requirements Statement: Provide at least two specific historic
examples (e.g., legislation, Supreme Court cases, executive orders, or other actions by the federal government) that support your position.
• Scoring Criteria: • Score 0 No response or the response does not address the prompt. • Score 1 Fulfills only 1 of 3 requirements of a level 3 performance • Score 2 Fulfills 2 of 3 requirements of a level 3 performance • Score 3 Takes a position that argues for or against the statement; provides one
historic example that supports the position; provides a second historic example that supports the position
C&E Fall 2012 CR Released Example:
• Throughout its history, American democracy has upheld and strengthened citizens’ individual rights. • Take a position that argues for or against the above
statement. Provide at least two specific historic examples (e.g., legislation, Supreme Court cases, executive orders, or other actions by the federal government) that support your position. • Standard Alignment: • CE.C&G.2.6 Evaluate the authority federal, state, and local
governments have over individuals’ rights and privileges (e.g., Bill of Rights, Delegated Powers, Reserved Powers, Concurrent Powers, Pardons, Writ of habeas corpus, Judicial Process, states’ rights, Patriot Act, etc.).• CE.C&G.3.8 Evaluate the rights of individuals in terms of how well
those rights have been upheld by democratic government in the United States.
Constructed Response Tips
• Start with the objective(s) you want to measure. Most constructed response questions will satisfy more than one objective. • You must use the same RBT level that the standard requires.
(You also must use the same content… obviously!)• You should adhere to the same basic scoring criteria
whether you are creating a 2, 3, or 4 point question. Remember, students don’t see this criteria on the MSL, so you should teach them how to pick out points (there’s a PowerPoint about this on my site). • Review the released questions and use the RBT question
stems if you need help getting started!
Constructed Response
• Choose at least 2 objectives from your course and write your own constructed response using the template provided.
• After writing your constructed response, please trade it with a partner. You should break down each other’s CR and determine if it fits the criteria and matches the content and RBT levels of the standard.
Public Google Spreadsheet
• Please load the constructed response question you created today into the Google spreadsheet found on my site: https://docs.google.com/spreadsheet/ccc?key=0AtoXZ1jIf_QadDJ5N3RDTC0zZlhsSnVpOHpJSEE4Wmc&usp=sharing • This is a public spreadsheet that anyone can
access. I will update it throughout the year with constructed response questions I write (mostly for C&E), and I hope you will do the same.• Please share with everyone!
Other Resources
• TOPS suggested these resources for writing constructed response questions and understanding concept-based instruction:• Lynn Erickson - Concept-Based Curriculum and
Instruction • Lorin Anderson & David Krathwohl - A Taxonomy
of Learning, Teaching, and Assessing•Robert Mager - Measuring Instructional Results •RBT Resources (Section 5):
http://tpri.wikispaces.com/space/content
Contact Info
• Please contact me if you have any questions:
Emily JolleyMillbrook High School
Preparing Students for the Real World
Takes Teachers Who Learn in the Real World
BECOME A KENAN FELLOW
•See first-hand how researchers and industry experts use STEM skills in real-world settings
•Apply that knowledge to better prepare students for the jobs of tomorrow
•Enrich your career through professional development opportunities
•Serve as an ambassador for educational excellence in your school, community and across the state
Find out more: kenanfellows.org
Kenan Fellows Program Timeline
Kenan Fellows Program Stay Connected
facebook.com/KenanFellows
@KenanFellows
Kenan Fellows Program Group
youtube.com/user/KenanFellows
pinterest.com/kenanfellows