Portfolio Assessment in Clerkship
description
Transcript of Portfolio Assessment in Clerkship
Portfolio Assessment in Clerkship
Michelle Gibson - Geriatrics(thanks to Chris Frank and Melissa Andrew too)
Objective Participants will discuss the use of portfolios in assessing medical trainees (after a brief review of some basic principles…)
Background Change to the Internal Medicine clerkship in January 2007 4 weeks to 6 weeks
Opportune time to review the assessment process in geriatrics Previously, 8 assessment cards filled out by MD supervisors
Challenges with old system Cards were often filled out the last week of the rotation
Depended solely on supervisor impression
No direct observation was incorporated
Students were assessed using only one modality
Reasons for change Desire for more formative assessment opportunities
Desire for more robust assessment
(Need to meet course requirements!)
Formative assessment Assessment that occurs during instruction to provide feedback to teachers and students
Describes needs for future learning
Assessment for learning
Summative Assessment Assessment that occurs at the end of an instructional unit to document student learning
Certifies learning Assessment of learning
Is this portfolio formative or summative?
High Quality Assessment Clear & appropriate learning targets Appropriateness of assessment methods*
Validity Reliability Fairness Positive consequences* Alignment* Practicality & efficiency*
Reliability & Validity Reliability: concerned with the consistency, stability, and dependability of the scores
Validity: the appropriateness of the inferences, uses, and consequences of the assessment Is it assessing what you want it to assess?
Portfolio A systematic collection of student products to assess progress
Structured portfolio The assessor determines the mandatory content of the portfolio
Literature review Assessment of housestaff in clinical rotations Portfolio literature reviewed Specific instruments examined
Portfolio components were selected to provide multiple methods of assessment for each clerkship objective
Components by MethodologyMethodology Portfolio components
Foundational evaluations
Assessment cards - 6
Direct observations Mini-CEX - 2
Practice- & Data-based learning
Medication ReviewSelf-assessment of case write-up
Multi-source evaluations
TAB - 2
Miscellaneous (optional)
Student’s choice
Mini-CEX
TAB
Card
Medication Review - written Based on an actual patient, preferably one for whom the student has responsibility brief summary of patient, meds he/she is taking including whether or not they are appropriate
suggestion of other medications he/she should be taking
Self-assessment of write-up H & P from early in rotation Students asked to read and identify 2 things done well 2 things that could be improved
Reviewed with undergrad coordinator on the last day
Components & CPE CategoriesCPE Category EC MC TAB MR Case
Knowledge Base X X X X
Motivation to Learn X X X X X
Team Functioning X X
Communication X X X X X
Relationship with patients/families
X X X
Resource Management X X X X
Process Orientation with students on Day 1 Team, faculty, & residents are oriented to the portfolio
Students meet with undergrad coordinator at mid-term, review progress, identify any gaps, etc.
Last day - review portfolio with students, completion of CPE
Evaluation of Pilot Focus on acceptability
Students and assessors Student evaluations ongoing - not seen by the undergrad coordinator until collated in April 2008
Will redo faculty portion in December 2007
Challenges noted - 1st time First rotation ever for clerks
Time spent learning how to be a clerk Couldn’t assemble half the portfolio in 3 weeks
Exam stress “detracted from our learning”
Supervisors were absent more than usual
*all except exam stress have resolved
Strengths noted Multidisciplinary team members provided high-quality feedback
Direct observation occurred Forms were felt to be user-friendly, acceptable
Students appreciated discussing their medication review and being observed
Assessor ratings of forms TAB
5/5 for ease of use; 5/5 for usefulness Mini-CEX
5/5 for ease of use; 4/5 for usefulness Time spent by assessor: 13.5 minutes
Encounter card 4.5/5 for ease of use; 4.5/5 for usefulness
Time spent by assessor: 15 minutes
My time? 5-10 minutes on day 1 for orientation Plan to develop “talking power point”
15 min per student at the end of week 3 Mid-term review Could be any faculty
15 min per student on the last day Final evaluation Could be any faculty
My *impressions* Students receive more feedback in a timely fashion
Direct observation doesn’t take long, and is very valuable
TAB form is very useful, and team members are pleased to be filling them out
Mid-term meetings are very useful
Future directions Continue with student and assessor feedback
Incorporate other tools? Peer assessment? Patient assessment?
Study it more Implement it for residents
Factors to note Small unit, discrete group of nursing and other staff
Small division Very workable for us … but for others?
Questions to consider Is this high quality assessment? How could it be improved?
Can it be extended into other areas?
Is there a role for a “whole clerkship” portfolio? “Pre-clerkship” portfolio?
Residents & non-MDs as assessors? Electronic?
Comments about our students “ .. Is very attentive… he listens well to pt. & staff, & then goes off & evaluates what he’s been told”
“excellent verbal communication observed with all team members”
“better H & P than most residents”