Post on 18-Aug-2018
Special Populations, Comprehensive Next Generation
Assessment Systems, and Learning Models: Transition to Assessment
for Learning
Friday, June 21, 2013: 8:00 AM-9:30 AM
Chesapeake G-H (Gaylord National Resort and Convention Center)
90 Min Session Agenda Timeframe Topic Presenters
15 minutes Introduction GSEGs and AA MAS Context WestEd GSEG Project
Dissemination Learning Models Colloquium
Susan Weigert Edynn Sato Pat Almond
15 minutes Low Performing SWD—Lessons Learned
Sue Bechard
15 minutes Learning Models/Learning Progressions
Margaret Heritage
15 minutes Smarter Balanced & Low Performing SWD
Vince Dean
30 minutes Discussion Questions Facilitation
Sandra Warren Edynn Sato
6/21/2013 NCSA 2013 Session 3447
Learning Models & Assessment of Special Populations 2
SUSAN WEIGERT, PH.D. OSEP US-DOE PROJECT OFFICER
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 3
OSEP US-DOE Projects
• General Supervision Enhancement Grants (GSEG) FY2007-2010
• Administration Change in 2008
• Advocacy of Several Disability Advocates
• Message of Inclusion
• Promise of Greater Test Access and Sensitivity
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 4
• Problem:
Promote better access to grade level instruction
for students with disabilities who are behind grade level.
How?
• One Solution:
– Formative assessments supporting Learning Analytics/Learning Progressions from the student’s access point toward the grade level standard.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 5
Tools
• CBM– fundamental skills
Off grade level
• Traditional Formative assessment
– Normative models of difficulty
• Can we Improve formative assessment?
– Reached out to Consortia
– Sponsored a series of Colloquia
– White Papers
6/21/2013
NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
6
LEAD PRESENTER: EDYNN SATO, PH.D.
.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 7
Colloquium on Learning Models, Instruction, and Next Generation Assessments that Include Special
Populations
October 26, 2012
Washington Marriott at Metro Center Washington, DC
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 8
Background:
General Supervision Enhancement Grant
• WestEd, the Kansas State Department of Education, and the Louisiana Department of Education (H373X070002)
• Project Officer: Susan Weigert
• Focus of grant (general):
– Technical Assistance on Data Collection
– Priority A––Modified Academic Achievement Standards
• Dissemination
1 Learning Models Colloquium 6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 9
Colloquium Purpose
Learning Models: • in development in the U.S.
• proposed as the foundation for designing comprehensive next generation assessment systems—both formative and summative
Colloquium will involve discussion of: • what is known,
• what is in the works, and
• what needs to be known or investigated
2 Learning Models Colloquium 6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 10
Learning progressions have been proposed for use in both large-scale and classroom assessments. In both cases, they may provide more detailed information about student thinking than more traditional models of assessment. This detailed information is particularly important in the classroom, where it can be used as the first step in a formative assessment process, to impact instructional decisions and provide feedback to students, ultimately improving student learning (Alonzo & Steedle, 2008, p. 419).
3 Learning Models Colloquium 6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 11
PATRICIA ALMOND, PHD CATE—UNIVERSITY OF OREGON
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 12
Agenda Time Session Topic
Lear
nin
g M
od
els 8:00 a.m. 0
Form Discussion Groups
Welcome & Introductions
9:00 a.m. 1 Learning Models, Instruction and Assessment
10:00 a.m. 2 Diversity of Special Populations
10:55 a.m. 3 Technical Considerations
Learning Analytics & Data Mining
12:00 p.m. Working
Lunch
Implications for the Future
Report out
Nex
t G
ener
atio
n 1:00 p.m. 4 Learning Progression Frameworks for Use with CCSS
1:20 p.m. 5 RTTT—Special Populations in Next Generation Assessments
2:25 p.m. 6 GSEG—Special Populations in Next Generation Assessment
3:25 p.m. 7 Implications for Continued Research and Development
4:25 p.m. 8 Wrap-Up and Thank You
4:30 p.m. Adjourn
5 Learning Models Colloquium
LMC Light
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 13
Basic Agenda Format
Focused Table Discussions 15 minutes
Presentation 40 minutes
7 Learning Models Colloquium
Think Speed Dating.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 14
Tom’s Proposition If you want one page . . .
. . . It’ll take 20 days.
If you want it today . . .
. . . It’ll take 20 pages.
6 Learning Models Colloquium 6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 15
Comprehensive Assessment One aspect of “comprehensive” systems was the idea that assessment would be closely linked to instruction through formative and interim assessment and would not be limited to summative, accountability assessment purposes.
Instructionally Embedded (Formative)
Interim
Summative/ Accountability
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 16
Of Particular Interest
• The learning characteristics of students eligible for Alternate Assessments based on Modified Achievement Standards
• Students whose actual achievement levels were thought to be misrepresented by ‘regular’ summative, accountability measures
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 17
Formative versus Summative?
“In CBAL they are integrated and based on a common set of models. There are strong reasons to use a common foundation for formative and summative assessment. How do you take the evidence from the formative assessment and use it in the classroom?”
“But, the formative and summative could be different. The standards are common, but the learning progression may not be the same for both assessments. I see it halfway between both of you. A summative [assessment] requires a definition of the scope and sequence.”
A common learning progression for both
formative & summative assessment.
The foundation is the CCSS—which assumed
progressions, an embedded scope & sequence.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 18
Selected Discussant Comments
• Next generation assessments need high correspondence between interpretation of students’ progress given the model and the outcomes measurement model.
• If there are variations in student pathways, there should be correspondence in the measurement model.
• Few learning models have specifically addressed learning and progress for students with disabilities.
• Learning models need to support both instruction and assessment to inform instruction.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 19
SUE BECHARD, PH.D. INCLUSIVE EDUCATIONAL ASSESSMENT
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 20
.
• 14 projects involving 26 states
• Lessons learned from their studies on alternate
assessments based on modified achievement
standards (AA-MAS).
• Funded by the U.S. Department of Education
in 2006-2007 under three funding sources:
• General Supervision Enhancement Grants
• Enhanced Assessment Grants
• Supplemental Funding
Lessons Learned in Federally Funded Projects That Can Improve
the Instruction and Assessment of Low Performing Students with
Disabilities (Thurlow, M., Lazarus, S., & Bechard, S. (Eds.))
Available at: www.nceo.info
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 21
Demographic Characteristics
• 4 projects looked at demographic characteristics of low
performing students. More likely to be:
– Male
– From a racial/ethnic minority
– From a low socio-economic background
– Have ELL status
• Both students with and without disabilities were low
performing.
6/21/2013 NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
22
Disability Categories
• 4 projects looked at the disabilities categories of students who might be candidates for an AA-MAS.
• All found that a majority of the students had specific learning disabilities.
• Other categories:
o Intellectual disabilities
o Speech and language impairments
o Other health impairments
o Emotional/behavioral disabilities
o Autism
o Other disability categories
6/21/2013 NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
23
Opportunity to Learn
4 projects looked at whether students who were candidates for
the AA-MAS had the opportunity to learn the content. All found that some students may not have had adequate opportunity to learn.
6/21/2013 NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
24
Learning Characteristics/Barriers
10 projects found one or more of these learning characteristics/barriers: • Difficulty interacting with print - poor reading skills • Difficulty solving problems that require multi-step solutions • Easily distracted • High vocabulary load • Lack of availability of needed accommodations • Limited meta cognition • Need clarification of instructions • Poor organizational skills • Self-monitoring skills • Slower work pace • Text structure (passage length and formatting) • Working memory capacity
6/21/2013 NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
25
Implications for an Inclusive
Comprehensive Assessment System
• Develop systems that support student achievement.
• Mine the data to learn more about what is happening
with low performing students.
• Develop clear participation guidelines that seamlessly
include all students in the assessment system.
• Develop assessments that incorporate the principles of
universal design in an online environment.
• Provide high-quality professional development.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 26
Suggestions for Further Research
• Research on growth and growth expectations for this population.
• Investigations into the effects of specific accommodations, supports, and scaffolding in an online environment
• Research on professional development needed for higher education training programs.
• Investigations into the experiences of this population during testing.
6/21/2013 NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
27
MARGARET HERITAGE, PH.D. ASST DIR PROFESSIONAL DEVELOPMENT (CRESST)
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 28
Special Populations, Comprehensive Next Generation Assessment Systems, and Learning Models: Transition to Assessment for Learning NCSA | National Harbor, MD June 19-22, 2013
Margaret Heritage
CRESST/UCLA
Learning Models to Support Instruction and Assessment
Internal:
What is going
on inside
students’ heads
What Is A Learning Model?
External:
Goals for
student
learning
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 30
Learning Models?
• Validated learning
progressions/trajectories
• Standards progressions
• Organized curricular
goals based on internal
logic/knowledge of
discipline
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 31
Learning Progressions
Learning progressions should be based on what we know from research about student learning and need to be empirically tested
(Mosher, 2012; Pellegrino, 2011)
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 32
Equipartitioning Trajectory
• Description of mathematics and cognition
• Description of achievement level
• Description of what to expect
when children are working
toward the level
• Examples of working toward
the level
• Assessment tasks
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 33
• Can equipartition a single whole. The difference
between Level 1 and Level 2 is that here the child
is partitioning a particular shape into smaller
shares with equal areas (instead of partitioning
collection of objects). In most case for young
children, being able to create equal-sized areas
is a more complex action than dealing
collections.
• Create multiple unequal-sized parts by
“chopping” the whole into many pieces and then
allocating them
• Create the correct number of parts, but the parts
are of unequal size
• Create equal-sized parts but create too many of
them and want to “throw away” the extra parts
Children who are
working towards this
level:
Children who have
achieved this level:
(Confrey, 2010)
Equipartition Level 2: Sharing a Whole
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 34
Standards Progressions
Students develop through a series of levels
of geometric and spatial thinking. Initially,
students cannot reliably distinguish between
examples and non examples of categories of
shapes, such as triangles, rectangles, and
squares. With experience, they progress to
the next level of thinking…
(CCSS Math Writing Group, 2012)
6/21/2013
NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
35
(Di Ranna et al., 2008)
Conceptual Flow
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 38
Put image of table from p. 69 of reflective teaching on your desk
(Di Ranna et al., 2008)
Assessment Plan
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 39
• Clear descriptions of learning pathways
• Assist understanding of how student thinking and
skills develop within domains
• Descriptions of performance: on the way to,
achieved a stage in the progression
• Clarify difficulties, common misconceptions and
challenges to learning in a domain
• Guide assessment tasks
• Provide a framework for interpretation
Utility for Instruction and Assessment
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 40
Moving Forward
• We need more research on learners’
thinking related to core ideas in
domains to develop curricula,
instruction and assessment
• We need to pay attention (more
research) on models for learners with
disabilities
• We have a lot to do!
6/21/2013
NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
41
VINCENT J. DEAN, PH.D., MI DIRECTOR, OFFICE OF STANDARDS & QUALITY ASSESSMENTS
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 43
Implications for Special
Populations in Next Generation
Assessment
State Perspective on Smarter Balanced Assessment Consortium
Vince Dean, Ph.D.
Director, Office of Standards & Assessment Michigan Department of Education
A Balanced Assessment System
Common
Core State
Standards
specify
K-12
expectations
for college
and career
readiness
All students
leave
high school
college
and career
ready
Teachers and
schools have
information and
tools they need
to improve
teaching and
learning
Interim assessments Flexible, open, used
for actionable feedback
Summative assessments
Benchmarked to college and career
readiness
Teacher resources for formative
assessment practices
to improve instruction
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 45
Building a Logical Argument
Student Response
Evidence
Assessment Target
Claim
Career & College Readiness/ Common Core State Standards
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 46
6 Key Components of Evidence-Centered
Design
6. Develop Items or Performance Tasks
1. Define the domain Common Core Standards Math/ELA
2. Define claims to be made ELA & Math Claims
Content Specifications
3. Define assessment targets Knowledge, Skills, & Abilities
4. Define evidence required Evidence to be Elicited from Student
5. Develop Task Models Methods for Eliciting Evidence
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 47
Formative Assessment Definition for Smarter Balanced Assessment Consortium
Formative Assessment is a deliberate process used by teachers and students during instruction that provides actionable feedback that is used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 48
Formative Assessment Process
Heritage, 2009 6/21/2013
NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
49
The Formative Assessment Process Is At Every Level of the RtI Process
Intensive Evidence-based Interventions
Targeted Evidence-based Interventions
Universal Instruction
1 to 5%
5 to 10%
80 to 90%
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 50
A System of Support
*Students &
Models
Formative Practices
Learning Goals Success Criteria Learning Activity
Interim Assessments
Claims Standards Learning activities
Summative Assessments
Claims Standards
Learning Models
Learning Maps Learning Progressions
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 51
CAT-Specific Opportunities
• Individualizing the student test event
• Increased measurement precision
• More appropriate indication of individual student growth
• Item difficulty, complexity, linguistic characteristics
• Item pool depth
• Cross-grade administration
• Administration time
53 6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations
CAT-Specific Challenges
• Content constraints
• Item pool size – Substantially more items are needed
– Complicated by the number of item types
• Item pool distribution shape – More rectangular than bell shaped
• Algorithm specifications – How many adaptive facets?
• Item type, difficulty, DOK, LDL, etc.
55 6/21/2013
NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations
Lessons Learned - Designing CAT for Special Populations
• Clarity of purpose is even more important when building a CAT assessment
• More comprehensive professional development is needed (e.g., item writers, test administration)
• Increase adaptive characteristics only when it adds substantial, reliable value
• Must have an item inventorying feature able to quickly report by multiple variables
• Cognitive Labs are essential
56 6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations
SANDRA WARREN, PH.D. ASES SCASS FACILITATOR
QUESTIONS FOR
DISCUSSION
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 57
ASES SCASS Perspective
“What is it we don’t know that is critical to the success of students with disabilities?” • Sometimes we don’t know what we don’t
know until we’ve started the journey. • Now that we are midway in this journey,
what are we learning we don’t know….and how might we move forward to support SEAs , LEAs, and schools?
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 58
ASES/NCEO NCSA Pre-Session Discussion – Action Steps/Areas
• Ensure accountability for student outcomes • Invest in strong LEA & school leadership • Use data to understand student trajectory • Ensure all students have access to AT and
professionals know how to support use • Ensure access to best practices regardless of
geographical location • Preserve integrity of graduation diploma with
multiple pathways • Clarify college vs. career readiness
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 59
Panel Discussion
To increase our understanding of: Research …. Policy …. and Practice
for special populations, comprehensive next generation assessment systems, and
learning models
“What is it we don’t know that is critical to the success of low performing students
with disabilities?”
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 60
DISCUSSION FACILITATORS: SANDRA WARREN, PH.D. & EDYNN SATO, PH.D.
6/21/2013 NCSA 2013 Session 3447 Learning Models
& Assessment of Special Populations 61