Landslide risk assessment along a major road corridor based on ...
Assessment: Not Just a Stop Along the Way · Assessment: Not Just a Stop Along the Way Barbara D....
Transcript of Assessment: Not Just a Stop Along the Way · Assessment: Not Just a Stop Along the Way Barbara D....
Our Roadmap
• The “disrupted” context for assessment of student learning
• Unbundling, rebundling, and assessment
• Closing the loop – and why it matters
• How assessment pulls it all together
• Assessment, equity, and GEMS
3/20/2015 CSU Fullerton 2
“Disruptions”
• Open courseware (e.g., Carnegie Mellon, MIT, Kahn Academy)
• Massive, open, online courses (MOOCs)
• Course Redesign (Carol Twigg)
• Competency-based education (CBE; USDOE)
– Southern New Hampshire University
– Western Governors University
– Capella, Brandman, others
June 2014 IGEA, Burlington, VT 3
“Disruptions,” cont.
• Challenges, “testing out” (CLEP, CSU system, University of Akron)
• DIY trends (A. Kamenetz)
• Alternatives to traditional degrees, e.g.,
– Certificates
– Badges
– Peer reviews
– Social networks
June 2014 IGEA, Burlington, VT 4
The Mother of All Disruptions:
• The shift from teaching to learning – From inputs to outcomes
– From teacher- to student-centered
– From covering the material to deep learning
– From time held constant and varied learning (bell curve) to time varied, learning constant
– From grading to assessing, credits to competency
– Competency or mastery learning as goal
– From filter to pump, i.e., sorting to supporting
June 2014 IGEA, Burlington, VT 5
3/20/2015 CSU Fullerton 6
“Colleges maintain fixed and fossilized structures and tell their students to be flexible and open to new things.” (John Tagg, The Learning Paradigm College, p. 352)
A fundamental challenge:
What’s happening to HE?
• Unbundling
• Deinstitutionalization
• Disintegration
• Incoherence
• Fragmentation
• “A tsunami of chaos”
June 2014 IGEA, Burlington, VT 7
Maybe that’s why we need maps and markers.
Unbundling is promoted by . . .
• A changing student population
• Students’ need for flexibility, alternatives
• Student mobility
• Cost, debt
• For-profit institutions
• Technology
• Trend toward externalization of services
• Society-wide: increasing personalization; growing skepticism toward traditional institutions
June 2014 IGEA, Burlington, VT 8
Unbundling Traditional Institutional Roles:
• Knowledge
• Instruction
• Validation of learning
• Degrees
• Social networking
• Cultural opportunities
• Personal development
June 2014 IGEA, Burlington, VT 9
Unbundling Traditional Faculty Roles:
• Faculty have traditionally
– Owned the knowledge, expertise
– Designed the curriculum
– Designed the course
– Delivered the course
– Conducted the testing/assessment of what’s learned
June 2014 IGEA, Burlington, VT 10
Unbundling the Student Experience:
• Commuting
• Part-time attendance
• Attendance at multiple institutions
• Swirling
• Transfer
• Face to face, hybrid, and online coursework
• Varying levels of preparation, strengths
June 2014 IGEA, Burlington, VT 11
There’s a countervailing trend: rebundling
• “Intentionality”
• Integrative learning
• High-impact practices
• From factoids to deep, inquiry-based learning
• LEAP
• The Degree Qualifications Profile (DQP)
• General Education Maps and Markers (GEMS)
June 2014 IGEA, Burlington, VT 12
The Mother of All Rebundling Tools: Assessment
3/20/2015 CSU Fullerton 13
Assessment is the key – done right
Assessment and GEMS Principles
• Proficiency
• Agency and Self-Direction
• Integrative Learning and Problem-Based Inquiry
• Equity
• Transparency and Assessment
3/20/2015 CSU Fullerton 14
June 2014 15
Assessment is the key to . . .
• Improving and validating students’ knowledge, skills, proficiencies
• Engaging all students in self-assessment, agency, self-direction
• Choosing, using the right HIPS
• Achieving equity
• Communicating with multiple audiences (transparency)
IGEA, Burlington, VT
Assessment is the key if . . .
• The approaches are direct, embedded
• They encourage deep, integrative learning
• They elicit, assess higher-order intellectual skills
• Demonstrate levels of achievement
• Are educative for students and faculty
• Promote self-assessment, life-long learning
• (Not checklists or standardized testing)
June 2014 IGEA, Burlington, VT 16
Assessment is the key if
• We actually close the loop at all levels
– Student
– Course
– Program
– Institution
– Institutional structures and practices (e.g., planning, budgeting, oversight)
3/20/2015 CSU Fullerton 17
Assessment vs. Evaluation:
• Inquiry
• Improvement
• Curiosity
• Candor, honesty
• Transparency
• Collegiality
• Rewards for quality, equity
• Judgment
• Consequences
• Anxiety
• Circling the wagons
• Concealment
• Avoiding blame
• Rewards for looking good, “excellence”
3/20/2015 CSU Fullerton 18
Key definitions . . .
• Criterion-referenced: testing or assessment in which student performance is judged in relation to pre-established standards and not in relation to the performance of other students.
• Norm-referenced: testing or assessment in which student performance is judged in relation to the performance of a larger group of students, not measured against a pre-established standard.
3/20/2015 CSU Fullerton 19
3/20/2015 CSU Fullerton 20
Thinking about standards of student performance . . .
• Absolute standards: knowledge/skill level of champions, award winners, top experts
• Contextual standards: appropriate expectations for, e.g., a 10-year old, a college student, an experienced professional
• Developmental standards: amount of growth, progress over time, e.g., 2 years of college, 5 years
• Benchmarked standards: performance goals set by a group or consortium
• Institutional, regional, national standards – emerging?
1. Outcomes, questions
2. Gathering evidence 3. Interpretation
4. Use
The Assessment Loop
3/20/2015 CSU Fullerton 21
Why is it so
hard to
close the
loop? Black
Black Hole
of
Assessment
3/20/2015 CSU Fullerton 22
THE
#3 Interpretation, or making meaning out of evidence. . .
• Is it included as an explicit step?
• Is the “community of interpretation” inclusive?
• Is it characterized by courage & candor?
• Is this a non-judgmental process of inquiry and understanding, not blaming?
3/20/2015 CSU Fullerton 23
Questions for Interpretation. . . . • What did we find?
• Did we disaggregate for subpopulations?
• What do the findings mean?
• Who can shed light on them?
• What do we want to do about them?
• Is the plan feasible (e.g., sustainable, aligned with campus culture)?
• Are we communicating with the campus?
• Where is institutional support – or not?
• What will it cost?
3/20/2015 CSU Fullerton 24
4. Using findings . . .
3/20/2015 CSU Fullerton 25
• Does the notion of “use” go beyond – Collecting data? – Writing reports? – Making recommendations? – Planning for changes? – Refining the assessment process?
• And “close the loop,” i.e.,
– Implement recommended changes? – Retrace the loop to determine the effect of
changes? On all student populations?
Using findings, cont.: Do we have
• Credible recommendations? • Ongoing communication, tailored to different
audiences? • Broad buy-in (e.g., faculty, administrators, other
educators, students, employers)? • Budget? • Structures for systematic reporting and feedback? • Steady, knowledgeable leadership? • A schedule for revisiting, adapting, moving on? • Data management and analysis capability? • Public praise and rewards for problems solved?
3/20/2015 CSU Fullerton 26
27
Making it happen . . .
• Students as partners in closing the loop – and a vast, untapped resource?
3/20/2015 CSU Fullerton 28
If it’s really all about learning for all students, then . . .
• Assessment is the key to dealing with disruptions
• Closing the assessment loop is the key to equity
3/20/2015 CSU Fullerton 29