Susan Agre-Kippenhan, Portland State University Professor, Art Department
27
Susan Agre-Kippenhan, Portland State University Professor, Art Department Evaluating the Effectiveness of Service Learning
description
Susan Agre-Kippenhan, Portland State University Professor, Art Department. Evaluating the Effectiveness of Service Learning. Why evaluate?. Assessment as a continual Service Learning improvement tool - PowerPoint PPT Presentation
Transcript of Susan Agre-Kippenhan, Portland State University Professor, Art Department
Susan Agre-Kippenhan,Portland State University Professor, Art Department
Evaluating the Effectiveness of
Service Learning
Why evaluate?Assessment as a continual Service
Learning improvement toolClarification of critical learning
objectives, ability to focus on Service Learning specifics
Yields useful information for programs, faculty, students
Evidence and support claims
SL Outcomes: Service Learning Claims:
Understanding of course contentTheory to practice Understanding of Community Community related skills Civic Engagement Diversity MORE?
Student Learning FocusValue of authentic work samples
Moving from self-report Moving from less integrated forms of assessment (i.e. observation, focus group)
Continual Improvement Feedback to actual courses, programs Applicable/actionable assessment Can be course/assignment based Can serve other programmatic goals
How do we know what works? A working model for assessment based on
the premise that there are:
Key questions to ask Logical steps to follow Usable data Transforms student learning and community experiences
1Assessment Cycle Model
4Develop Learning Objectives
Identify Work Samples
Locate Critical Junctures
DetermineAssessment Criteria/Quality
TestValidity /Reliability
DisseminateRevise/Revisit
56
23
Assessment Cycle Model
4Develop Learning Objectives
Identify Work Samples
Locate Critical Junctures Determine
Assessment Criteria/Quality
TestValidity /Reliability
DisseminateRevise/Revisit 561
2 3
Assessment Cycle Model What am I trying to assess?
Action: Define Student Learning Outcome
Student Learning Outcomes in the context of:
Mission Priorities Program Goals
1
What are appropriate work samples?
Action: Identify Student Work Samples
Qualitative vs. Quantitative Examples: Portfolios, Papers (writing samples, essays, reflective papers, editorials, journals…) Performances, Artwork, Lab reports
Assessment Cycle Model
2
When should I assess?
Action: Identify critical junctures
Critical Junctures Entering students Passages Culminating
Assessment Cycle Model
3
What am I looking for in student work?
Action: Define scoring criteria
Rubric Development: Scoring Criteria What indicators will I use to demonstrate desired student learning?
Examples: Bloom's taxonomy
4Assessment Cycle Model
How do I evaluate the range of student work?
Action: Define performance levels
Rubric Development: Performance standards How many levels will my rubric have and how will I define performance at each level?
Specific clear descriptors at each levelQualitative differencesCommon errors
4Assessment Cycle Model
Sample RubricWork Sample
Scoring Criteria 1
Scoring Criteria 2
Scoring Criteria 3
Performance Level 1- Low
Performance Level 2- Medium
Performance Level 3- High
Once I have a rubric how do I know it's valid?
How do I know if the rubric is reliable?
Evaluate the rubric against the learning objectives, learning opportunities etc… Is there clear alignment?
Pilot test with multiple evaluatorsDiscuss how well the rubric worksCompare scores for consistencyDetermine procedures for inconsistent scores
5Assessment Cycle Model
What's next?
Action: Back to the cycle
Revise and revisit - learning outcomes, learning opportunities, assignments, critical junctures, rubrics, program, course offerings/order, curriculum, pedagogy…
6Assessment Cycle Model
SL Case Study University Studies - General Education
4 Year integrated program Interdisciplinary program
Critical Features Clearly defined goals
Communication, Critical thinking, Diversity, Civic Responsibility
Common Pedagogy Required Courses
SL Case Study Student Learning Outcomes
Action: Define Student Learning OutcomeAppreciation and Understanding of Civic Responsibility
PSU Mission - Let Knowledge Serve the City University Studies Goal - Civic Responsibility
1
SL Case StudyWork samples
Action: Identify Student Work SamplesCommon assignment given in each class
Paper2
Critical Junctures
Action: Identify critical junctures Pilot in Senior Capstone Classes
Senior Capstone required course Culminating group experience for major and general education Application of expertise to a real project in the community
SL Case Study
3
Common Assignment “Please write a 3-5-page reflective analysis
summarizing the ways you have gained experience with the University Studies goal of Civic Responsibility. Use specific examples from your discussions and course readings, or in completing your project.”
SL Case Study
Scoring Criteria and Performance Levels
Action: Define scoring criteria/levelsCivic Responsibility Stages
Based on Bloom's taxonomy
SL Case Study
4
Valid Rubric
How do I know if the rubric is reliable?
PSU Mission - Let Knowledge Serve the City University Studies Goal - Civic Responsibility
Pilot test with multiple evaluators Compare scores for consistency. Discuss how well the rubric works.
SL Case Study
5
Revisiting ProcessSuggestions for improvements Revise assignment for clarity Revise rubrics for specificity Clarify pedagogy
Confirmed Program goals addressed in class Basis of assignment is valid Keep assessment in Capstone
SL Case Study
6
Focus on student learning objectives What are appropriate work samples? Are there existing critical junctures? How do we evaluate quality? How can this improve our practice?
SL Assessment in practice
Connect to goals and mission Knowledge of the process that is
transferable Assessment as a tool for partnerships Applicable/actionable assessment
Lessons Learned
For More InformationSusan [email protected]
