8/2/2019 Reflective Feedback in GME
1/38
Reflective Feedback in GraduateMedical Education
Adah Chung, M.Ed, University of Colorado School ofMedicine Center for Advancing Professional Excellence, USA
5th Annual ASPE Conference
Tucson, AZAugust, 2006
8/2/2019 Reflective Feedback in GME
2/38
Agenda
Welcome and Introductions
Objectives
Background and challenges
Brief literature review
What we did Reflective feedback
Discussion
Concluding remarks
8/2/2019 Reflective Feedback in GME
3/38
Lets talk
What are your program experiences withworking with residencies?
None an area were interested in Objective Structured Clinical Examinations
Communications training
Small group teaching
Formative assessment with feedback
8/2/2019 Reflective Feedback in GME
4/38
Objectives
Participants will understand how using reflectivefeedback with standardized patients in graduatemedical education can help to meet the ACGMErequirements.
Participants will learn about the benefits of usingverbal feedback for residents.
Participants will learn about a subjective writtenpatient perception tool to guide discussion withresidents
Participants will learn how a formativeassessment using immediate reflective feedbackis implemented.
8/2/2019 Reflective Feedback in GME
5/38
A little history
ACGME Outcomes Project emphasis on educationaloutcomes assessment based on (6) competencies: Patient care Medical knowledge Practice based learning Communication Professionalism System Based practice
ACGME Toolbox: Checklist evaluation Global rating of live or recorded performance SP exam
OSCEs Patient Surveys
8/2/2019 Reflective Feedback in GME
6/38
The challenges faced
Resident competencies (skills,knowledge and attitudes) must betranslated into a set of complexbehaviors that result in the deliveryof high-quality medical care(Whitcomb, 2002)
how to measure this??
8/2/2019 Reflective Feedback in GME
7/38
The challenges faced
Checklist evaluation: Difficulty with consensus on essential
behaviors/action and sequence Scoring behavior using checklists is more
difficult when checklists assume a fixedsequence of actions because experiencedphysicians use various valid sequences(ACGME Toolbox, 2000)
Checklists unable to capture complex and
hierarchical problem-solving skills ofexperienced clinicians (Hodges et al, 2002)
8/2/2019 Reflective Feedback in GME
8/38
The challenges faced
SP Exams, OSCEs:
Perception of SP exams by resident
directors and residents Pervasive idea that residents should
have skills and behaviors at a morecomplex level than medical students
8/2/2019 Reflective Feedback in GME
9/38
The challenges faced
Patient Surveys:
Not enough specific behavior
information for formative evaluationand performance improvement(ACGME Toolbox, 2000)
8/2/2019 Reflective Feedback in GME
10/38
A possible solution a formativeassessment (educational experience)
Allows reflection from residents (meetscomplexity/professional competenceconcerns)
Non-threatening environment for learning(meets attitudes towards SPs)
Trained SPs can provide more specificbehavioral feedback than patients (meets
patient survey concerns)
8/2/2019 Reflective Feedback in GME
11/38
The purpose of assessment
The outcomes of assessmentshould foster learning, inspireconfidence in the learner, enhancethe learners ability to self-monitorand drive institutional self-assessment and curricular change.
Epstein, R.M. & Hundert, E.M. (2002) Defining andassessing professional competence, JAMA, 287(2):226-235.
8/2/2019 Reflective Feedback in GME
12/38
Professional competency
Epstein & Hundert (2002):
Professional competence developmental, impermanent andcontext dependent
More than demonstration of isolatedcompetencies
A competent physician possesses theintegrative ability to think, feel and actlike a physician
8/2/2019 Reflective Feedback in GME
13/38
Dimensions of competence
Barnett (1994), Talbot (2004):
Knows that knows how reflective
knowing
Reflective knowing: focus on dialogand argument for better practicalunderstanding
8/2/2019 Reflective Feedback in GME
14/38
Definition of reflection
Reflection:
6: A thought, idea or opinion
formed or a remark made as aresult of meditation, 7:consideration of some subjectmatter, idea or purpose
Merriam-Webster Dictionary
8/2/2019 Reflective Feedback in GME
15/38
Reflection
Donald Schon (1983):
Reflection-on-action reflecting onexperiences
Adds to professional repertoire
Difference between effective and non-effective practitioners
8/2/2019 Reflective Feedback in GME
16/38
Reflection contd
Reflection as a way to help
professionals integrate skills,knowledge, behavior and attitudes
Reflection as a higher levelcognitive and personal process as
one develops professionally
8/2/2019 Reflective Feedback in GME
17/38
My question:
If reflection is the key to the difference inlearning & it promotes integration of skillsand behaviors that residents shouldpossess, then can it be facilitated?
8/2/2019 Reflective Feedback in GME
18/38
The literature on SPs giving feedback
Howley & Martindale (2004):
Students who responded favorably toSP experience more likely to have
received feedback from SPs
Students agreed strongly that feedbackfrom SPs are clear, non judgmental,specific and provided them with
patients perspective of the encounter
8/2/2019 Reflective Feedback in GME
19/38
The literature on SPs giving feedback
Hodder, Rivington, Calcutt & Hood(1989):
Even brief feedback encounters (2minutes) can have significant influenceon clinical competency
8/2/2019 Reflective Feedback in GME
20/38
Feedback & Attitudes
Efficacy of feedback:students who receive feedback
regarding clinical performance report
more positive attitudes towards theclinical experience Howley, LD & Martindale, J. (2004). The efficacy of standardized
patient feedback in clinical teaching. A mixed methods analysis.Med Educ Online, 9(18): 1-10.
Attitudes are an important factor inadult learning!
8/2/2019 Reflective Feedback in GME
21/38
Questions?
Agree or disagree reflectionimportant for graduate level medicaleducation/professionals?
8/2/2019 Reflective Feedback in GME
22/38
What we did
A formative assessment based on 3aspects of the ACGMEcompetencies:
Patient Care
Communication and Interpersonal Skills
Professionalism
8/2/2019 Reflective Feedback in GME
23/38
Logistics
4 stations: 20 min. patient encounters
10 min. verbal feedback from SP/TA
Checklists were in the form of patientperception questionnaires helped toguide discussion with resident
Cases: Bad news, Smoking cessation,
Diabetes management, Urological exam(TA)
8/2/2019 Reflective Feedback in GME
24/38
Logistics contd
Feedback session: Invite resident to self assess
Provide a positive statement
Provide an area to improve
End with an overall positive statement
Use the PPQ as a tool to guidediscussion
Describing feelings from patientsperspective
8/2/2019 Reflective Feedback in GME
25/38
Reflective Feedback
Promotes reflection as part of adialog (both parties involved)
Focuses on observed behavior Descriptive vs. judgemental
Specific vs. general
Sharing information vs. advicegiving
8/2/2019 Reflective Feedback in GME
26/38
Training of SPs
Focused on feedback using afeedback manual: Purpose of feedback
Characteristics of feedback Tips for success
Methods for giving feedback (thesandwich, I statements)
Examples and practice scenarios Appendices
8/2/2019 Reflective Feedback in GME
27/38
Patient Perception QuestionnaireChecklist items Yes No
1.Would you do what this doctor asks you to do?Patient Care
2. Did this doctor demonstrate compassion?3.Was this doctor respectful?
Communication and Interpersonal Skills
4. Did this doctor communicate effectively?How?
______________________________________________________________________
______________________________________________________________________
_______________________________
5. Did this doctor demonstrate effective listening skills?How?
______________________________________________________________________
______________________________________________________________________
_______________________________
6. Did this doctor demonstrate nonverbal communication skills?How?
______________________________________________________________________
______________________________________________________________________
_______________________________
7. Did this doctor demonstrate the ability to establish a working relationship with
you?
8. Did this doctor elicit and answer your questions?Professionalism
9. Did this doctor demonstrate sensitivity and responsiveness to your needs?10. Did you feel you could trust this doctor to be your advocate?
8/2/2019 Reflective Feedback in GME
28/38
PPQ
Terminology and categoriesfollowing (3) ACGME competencies
Yes/No provide more useful,reproducible results (ACGMEToolbox, 2000)
Room for comments of specific
behaviors for performanceimprovement
8/2/2019 Reflective Feedback in GME
29/38
PPQ guide
Guide for training focused on:
Understanding items
Examples for things that could be written
under HOW? Ie. Effective communication
Invited me to tell my story
Summarized what I said
Allowed me to ask questions
Ie. Non-effective communication Interrupted me
Used closed ended questions
8/2/2019 Reflective Feedback in GME
30/38
An example
Hi thanks for coming in. How did you think things went? It really is uncomfortable to talk to someone when shes
crying I didnt know what to do. When you put your hand on my shoulder, I felt reassured. I
also liked that you gave me space to cry rather than trying totalk through my tears.
Oh Im glad I actually just didnt know what to say Saying nothing at all was very comforting to me What else could I have done? Several times you tapped your foot on the floor and looked
at your notes rather than me that made me feel like I wastaking up too much of your time.
I didnt even notice that I was doing that! Overall, your gentle demeanor and touch helped me to
understand the news you were giving me. Thank you.
8/2/2019 Reflective Feedback in GME
31/38
Resident response
Receiving feedback from standardized patients was very helpful Direct feedback Great food Good learning tool. I felt most of the feedback was useful, but there was no criticism The topics are relevant to practice Ability to ask afterward how they thought I did Immediate feedback Good feedback/Good cases. Immediate feedback was very helpful Excellent specific feedback Helpful feedback. It helps me to see my weak points. I think focusing on communication is best Very low pressure atmosphere. Real life 20 min appts
8/2/2019 Reflective Feedback in GME
32/38
Benefits of this approach
Uses ACGME recommended toolbox ideas:combining SP encounters with patientsurveys
Provides structured observation Allows for professional self reflection
(reflection-on-action)
Checklist data for residency directors
Direct feedback for residents from trainedstandardized patients (changing attitudes)
8/2/2019 Reflective Feedback in GME
33/38
The purpose of assessment
The outcomes of assessmentshould foster learning, inspireconfidence in the learner, enhancethe learners ability to self-monitorand drive institutional self-assessment and curricular change.
Epstein, R.M. & Hundert, E.M. (2002) Defining and
assessing professional competence, JAMA, 287(2):226-235.
8/2/2019 Reflective Feedback in GME
34/38
Discussion
What are the advantages anddisadvantages of using a reflective,
narrative, verbal discussion withstandardized patients?
What strengths and/or weaknesses do
standardized patients bring to this kind offormative assessment?
8/2/2019 Reflective Feedback in GME
35/38
Conclusion
A direct feedback method with a
subjective written tool can help todemonstrate competencies whileappreciating the complex learninginteraction that happens as one
becomes a professional
8/2/2019 Reflective Feedback in GME
36/38
Future considerations
An added written self reflection
exercise? Research on the efficacy of SP
feedback to promote reflection
8/2/2019 Reflective Feedback in GME
37/38
Final thoughts
Recognition of SP strengths andweaknesses and how to meet the needs ofthe residents and residency programs
Challenge to think about ways we canassist residency programs in creative ways,other than our usual OSCE and SP exams
Be intentional rather than accidental!
8/2/2019 Reflective Feedback in GME
38/38
Thank you!
Please contact Adah Chung, MEd. [email protected] or (303) 724-1209for more information
mailto:[email protected]:[email protected]Top Related