University of Colorado at Boulder Lorrie A. Shepard School of Education University of Colorado at...
-
Upload
gertrude-hancock -
Category
Documents
-
view
218 -
download
0
Transcript of University of Colorado at Boulder Lorrie A. Shepard School of Education University of Colorado at...
University of Colorado at Boulder
Lorrie A. ShepardSchool of Education
University of Colorado at Boulder
Conceptual Coherence between Large-Scale and Classroom Assessments
CRESST ConferenceUCLA
September 9, 2004
University of Colorado at Boulder
Contrasts between large-scale and classroom level assessments that imply that different assessments are needed
• standardized vs. dynamic
• uniform date vs. variable dates
• independent performance vs. assisted performance
• delayed vs. immediate feedback
• stringent requirements for technical accuracy vs. less stringent requirements
The most important shared characteristics of the two levels of assessment should be alignment with the same curriculum standards and a shared model of learning.
University of Colorado at Boulder
For an assessment system to support learning, it must have coherence. The conceptual base or models of student learning underlying both external and classroom assessments must be compatible.
Knowing What Students Know
Pellegrino, Chudowsky, & Glaser NRC, 2001
University of Colorado at Boulder
Committee on Assessment in Support of Instruction and Learning A joint committee of Board on Testing and Assessment Committee on Science Education K-12 Mathematical Sciences Education Board
Bridging the Gap criteria that characterize an ideal system, distilled from:
NRC Science Education Standards NCTM Assessment Standards Commission on Instructionally Supportive Assessment
University of Colorado at Boulder
Shared Characteristics of Large-Scale and Classroom Assessments
A shared model of student learning including developmental progressions over time
A shared conception of disciplinary knowledge and competence Focus on evaluating understanding and reasoning Offer a clear vision of what constitutes mastery Target both general and domain-specific forms of cognition Select complex, context-based tasks that are open to multiple approaches
University of Colorado at Boulder
This idea of alignment between large-scale and classroom assessments refers
Not to the limited alignment where test publishers show fit within test blueprints,
But to the more complete and substantive alignment that occurs when the tasks, problems, and projects in which students are engaged represent the range and depth of what we say we want students to understand and be able to do.
Perhaps a better word would be embodiment.
Shepard, 2003, NSTA
University of Colorado at Boulder
A Sampler of Science Assessment© California Department of Education
Part B
University of Colorado at Boulder
A Sampler of Science Assessment© California Department of Education
University of Colorado at Boulder
© Wyoming Body Evidence Activities Consortium
University of Colorado at Boulder
© Wyoming Body Evidence Activities Consortium
University of Colorado at Boulder
Note that good assessment tasksare interchangeable with good instructional tasks.
(The exact same task should not be used for both purposes, however.)
We also have evidence that “teaching to”
problem types like these improves learning.
University of Colorado at Boulder
Shepard, Flexer, Hiebert, Marion, Mayfield, & Weston (1996).
University of Colorado at Boulder
Effect Size: AP – PaceSetter Students = .64 on SAT I, 1.58 on AP Calculus, and .21 on Open-Ended Tasks
University of Colorado at Boulder
Purposes Served by Large-Scale Assessment • Data for monitoring and accountability
• Exemplification of learning goals
• Program “Diagnosis”
• Professional Development
University of Colorado at Boulder
Program Diagnosisnotes from Delaware’s Rachel Wood and Julie Schmidt
…an 8th-grade weather assessment revealed that students across the state had over-generalized their knowledge of the movement of all air masses as having to go from west to east.
…lead teachers were stunned that their own students could not answer a question that involved drawing conclusions from a simple graph.
A commentary around a released item on Punnett squares attempts to turn teachers’ attention to thinking about why students have acquired only a mechanical sense.
University of Colorado at Boulder
Professional Development
Assessment-focused professional development deepens understanding of standards and directs attention to what students are getting from new activities.
Given time burdens, teachers can learn formative assessment strategies while working to raise achievement on accountability tests.
Assessment efforts only make sense if they are intimately tied to content learning.
University of Colorado at Boulder
A developmental framework and progress variables provide a substantive basis for monitoring student progress over time.
“Large-scale assessment tools and supporting instructional materials should be developed so that clear learning goals and landmark performances along the way to competence are shared…”
University of Colorado at Boulder
Towards Coherence Between Classroom Assessment and AccountabilityNSSE Yearbook 2004M. Wilson, Editor
Forster & Masters (Australia)
• Closely articulated, classroom-level developmental assessments
• National survey assessments
• Both tied to progress maps and accompanying curriculum materials
• Progress maps are broad, general descriptions of how proficiency typically develops in a curricular area.
• Accountability data collection embedded in classrooms with teachers as judges
University of Colorado at Boulder
NSSE Yearbook 2004
Wilson & Draney (BEAR Assessment System)
BEAR follows a developmental perspective, is built around a set of progress variables that reflect
the way that increasing proficiency develops.
(In contrast to cross-sectional and smorgasbord type assessments typical in the U.S.)
Coherence between classroom and external tests requires a shared understanding of the construct at the level of the progress dimension and the level of specific assessment tasks and scoring guides.
University of Colorado at Boulder
Note that KWSK said that progress variables provide a substantive basis for monitoring student progress.
Developmental continua should not rely on either statistically derived trajectories or aspirational benchmarks. Instead progress maps should represent the pathways of real students’ learning gains under conditions of reasonably adequate instruction.
Caution in Developing Growth Trajectories
University of Colorado at Boulder
University of Colorado at Boulder
Queensland
Shavelson
Queensland replaced A levels with formative assessments to meet students’ needs and to serve
as a basis for summative information beyond the classroom.
Queensland’s system is intended to audit local implementation of curriculum, assessment, and accountability.
A process for moderation of scores ensures consistency across classes and schools.
Australian examples are quite different from U.S. context because of the feasibility of a shared curriculum and trust in teachers’ judgments.
University of Colorado at Boulder
Delaware: Comprehensive Science Assessment
Rachel Wood
Curriculum materials developed to support state standards.
Prerequisites for meeting curricular objectives were identified grade-by-grade.
Then, end-of-unit assessments were developed.
Generic rubrics were not diagnostic, replaced with specific rubrics to identify misconceptions.
State, with some outside resources, provided intensive professional development for a cadre of teachers.
University of Colorado at Boulder
Déjà Vu All Over Again Assessment reformers used the terms
“Performance,” (Resnick & Resnick, 1992), “Direct,” (Frederiksen & Collins, 1989), and “Authentic” Assessment (Wiggins, 1989)
to convey the idea that assessments must capture real learning goals. Embodying worthy goals is still a core issue. KWSK’s ideal system will require much more than token
constructed-response items on accountability tests.