Post on 28-Dec-2015
The student’s world:The student’s world:
Learning and Evaluation Learning and Evaluation from the Learner’sfrom the Learner’s
PerspectivePerspective
Kevin W. Eva
Centre for Health Education Scholarship
University of British Columbia
Presenter DisclosurePresenter Disclosure
I have been unable to achieve I have been unable to achieve any conflicts of interest for the any conflicts of interest for the
material included in this material included in this presentationpresentation
\ˈfēd-ˌbak\\ˈfēd-ˌbak\
Providing constructive feedback Providing constructive feedback (1991)(1991)
1.1.Discuss the basis for evaluationDiscuss the basis for evaluation
2.2.Base feedback on specific Base feedback on specific observationsobservations
3.3.Use a balance of positive and negativeUse a balance of positive and negative
4.4.Et cetera …Et cetera …
Feedback as ImprovFeedback as Improv
““The dialogue, action, story, The dialogue, action, story, and characters are created and characters are created collaboratively as the story collaboratively as the story
unfolds.”unfolds.”
•Evaluating performanceEvaluating performance
•Incorporating feedbackIncorporating feedback
•Building professional identityBuilding professional identity
The student’s worldThe student’s world
What do these things What do these things have in common?have in common?
Running a triathlonRunning a triathlon
Writing a prescriptionWriting a prescription
Driving a carDriving a car
Offering a consultationOffering a consultation
Judging one’s own abilityJudging one’s own ability
““It’s a comfortability … when I It’s a comfortability … when I don’t have to think about it don’t have to think about it
that’s when I realize, more or that’s when I realize, more or less, [that I] have achieved my less, [that I] have achieved my
goal”goal”(Kindler, et al., 2013)(Kindler, et al., 2013)
TheThe self-assessment self-assessment literatureliterature
•Hundreds of articlesHundreds of articles
•Many literature reviewsMany literature reviews
•One conclusionOne conclusion
Self-assessment abilitySelf-assessment ability
is generally pooris generally poor
•Imagine yourself drivingImagine yourself driving
•It’s easy to do …It’s easy to do …• … … but details are inevitably left outbut details are inevitably left out• … … and those details are crucial to and those details are crucial to
determining the outcome of the actual determining the outcome of the actual experienceexperience
(Vallone, et al., 1990)(Vallone, et al., 1990)
WhyWhy can’t we self-can’t we self-assess?assess?
•Furthermore …Furthermore …
•Did anyone imagine themselves Did anyone imagine themselves having difficulty driving?having difficulty driving?
•When it’s easy to imagine an event When it’s easy to imagine an event we tend to overestimate its we tend to overestimate its likelihoodlikelihood
(The availability heuristic: Tversky and Kahneman, 1973)(The availability heuristic: Tversky and Kahneman, 1973)
WhyWhy can’t we self-can’t we self-assess?assess?
TheThe generalgeneral pointpoint
•When students self-assess they do so When students self-assess they do so by drawing inferences….by drawing inferences….
……just as we do when we make just as we do when we make judgments about their performance…judgments about their performance…
……and, as a result, they’re just as and, as a result, they’re just as falliblefallible
•Evaluating performanceEvaluating performance
•Incorporating feedbackIncorporating feedback
•Building professional identityBuilding professional identity
The student’s worldThe student’s world
Shute (2008)Shute (2008)
“formative feedback has been shown … to improve students’ learning and enhance teachers’ teaching
to the extent that the learners are receptive and the feedback is valid, objective, focused, and clear.
[BUT] there is no “best” type of formative feedback for all learners and learning outcomes
““Physician agreement Physician agreement with colleagues’ scores with colleagues’ scores
correlated positively correlated positively with overall mean scores with overall mean scores
from colleagues from colleagues (R=.48)”(R=.48)”
Sargeant et al. (2003)Sargeant et al. (2003)
““Physician agreement Physician agreement with colleagues’ scores with colleagues’ scores
correlated positively correlated positively with overall mean scores with overall mean scores
from colleagues from colleagues (R=.48)”(R=.48)”
Sargeant et al. (2003)Sargeant et al. (2003)
Which factors, personal or external, Which factors, personal or external, most influence students’ generation most influence students’ generation
of learning goals?of learning goals?
(Eva, et al., 2010)
What predicted generation of What predicted generation of learning goals?learning goals?
Overall R2=0.45
NOTE: Absolute values illustrated for ease of comparison (Eva, et al., 2010)
““It is impossible for It is impossible for a man to learn a man to learn
what he thinks he what he thinks he already knows”already knows”
Epictetus Epictetus
(circa 100AD)(circa 100AD)
““It is impossible for It is impossible for a man to learn a man to learn
what he thinks he what he thinks he already knows”already knows”
Epictetus Epictetus
(circa 100AD)(circa 100AD)
““I mean sometimes you’ll do it and I mean sometimes you’ll do it and somebody will watch you and it didn’t somebody will watch you and it didn’t work and they’re like, ‘You did everything work and they’re like, ‘You did everything right, right, it’s just one of those things.it’s just one of those things.’ But if ’ But if it’s successful then I think, yeah, I can do it’s successful then I think, yeah, I can do that.”that.”
““Some days I’ll go home and say I was a Some days I’ll go home and say I was a really good doc today and feel good about really good doc today and feel good about what happened in the day. And then other what happened in the day. And then other days you know, days you know, if I have a headacheif I have a headache, , maybe I wasn’t so good today.”maybe I wasn’t so good today.”
(Eva, et al. 2011; AHSE) (Eva, et al. 2011; AHSE)
Discounting the Discounting the negativenegative
Self EfficacySelf Efficacy
• InfluencedInfluenced by feedback, but also by feedback, but also influencesinfluences future performance future performance
• Not merely a passive reflection of Not merely a passive reflection of performance, but part of a self-performance, but part of a self-fulfilling prophecy that affects fulfilling prophecy that affects performanceperformance
(see Bandura, 1977)(see Bandura, 1977)
ImplicationsImplications
•External guidance is required …External guidance is required …
… … that is sensitive to the that is sensitive to the perceptions and perspectives of perceptions and perspectives of the recipient …the recipient …
•Evaluating performanceEvaluating performance
•Incorporating feedbackIncorporating feedback
•Building professional identityBuilding professional identity
The student’s worldThe student’s world
The need for The need for confidence/safetyconfidence/safety
““If you feel bad and you feel If you feel bad and you feel uncomfortable, you’re gonna not uncomfortable, you’re gonna not have the confidence to talk about have the confidence to talk about mistakes …you don’t get any kind mistakes …you don’t get any kind of constructive feedback because of constructive feedback because …you’re not comfortable saying …you’re not comfortable saying
it”it”(Eva, et al., 2012)(Eva, et al., 2012)
Self-perception
Performance
Learning goals
Motivation
AwarenessBenchmarkin
g
Feedback
A common perspectiveA common perspective
““Desirable difficulties”Desirable difficulties”
Creating situations that will Creating situations that will enable learners to discover enable learners to discover the limits of their the limits of their knowledge/abilityknowledge/ability
(see Eva, 2007 (BMJ); 2009 (AHSE))(see Eva, 2007 (BMJ); 2009 (AHSE))
““Education is Education is learning what you learning what you didn’t even know didn’t even know you didn’t know”you didn’t know”
(Boorstin, 1914)(Boorstin, 1914)
““Education is Education is learning what you learning what you didn’t even know didn’t even know you didn’t know”you didn’t know”
(Boorstin, 1914)(Boorstin, 1914)
““Studies that augmented Studies that augmented feedback with consultation feedback with consultation
produced substantially produced substantially larger differences, but larger differences, but other methodological other methodological
variations had little effect”variations had little effect”
(Marsh and Roche, 1997)(Marsh and Roche, 1997)
““Studies that augmented Studies that augmented feedback with consultation feedback with consultation
produced substantially produced substantially larger differences, but larger differences, but other methodological other methodological
variations had little effect”variations had little effect”
(Marsh and Roche, 1997)(Marsh and Roche, 1997)
CredibilityCredibility
Of relationships: Of relationships:
Engagement and awarenessEngagement and awareness
Of the process: Of the process:
Validity and authenticity Validity and authenticity
Of intent: Of intent:
Beneficence and non-maleficenceBeneficence and non-maleficence
(Sargeant, et al., 2010)(Sargeant, et al., 2010)
CredibilityCredibility
““It doesn’t mean it’s nice to hear It doesn’t mean it’s nice to hear it. It’s not so nice; you don’t feel it. It’s not so nice; you don’t feel
so good after it. But it’s a so good after it. But it’s a difference when it’s real, if what difference when it’s real, if what they’re saying is just to help you, they’re saying is just to help you,
or if it’s just to put you down”or if it’s just to put you down”
(Eva, et al., 2012)(Eva, et al., 2012)
ImplicationsImplications
•External guidance is required …External guidance is required …
… … that is sensitive to the that is sensitive to the perceptions and perspectives of perceptions and perspectives of the recipient …the recipient …
… … while taking into account the while taking into account the inherent risk/strain learners feelinherent risk/strain learners feel
Promoting receptivityPromoting receptivity
1.1. Self-assessment is vitalSelf-assessment is vital
2.2. Self-confidence is importantSelf-confidence is important
3.3. Self-assurance is powerfulSelf-assurance is powerful
4.4. Self-discovery is crucialSelf-discovery is crucial
5.5. Self-reliance can be Self-reliance can be detrimentaldetrimental
Moving beyond the scriptMoving beyond the script
Things to do:
•Avoid normative comparisons
•Use praise wisely
•Different styles for different tasks
(see Shute, 2008)
•Keep the relationship/dialogue in mind