Development of viva voce exam to assess clinical...
Transcript of Development of viva voce exam to assess clinical...
![Page 1: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/1.jpg)
Development of viva voce exam to assess clinical reasoning in the paraclinical phase of a BVSc program. HelenOwen,SeniorLecturerinVeterinaryPathologyandJus:neGibson,Senior
LecturerinVeterinaryBacteriologyandMycology
DrTamsinBarnesAssociateProfessorRowlandCobboldDrKatrinaGarreGDrStevenKopp
AssociateProfessorJoanneMeersDrDanSchullAssociateProfessorJennySeddonDrFrancesShapterAssociateProfessorRebeccaTraub
![Page 2: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/2.jpg)
Background on Program • TheBVScatUQisanundergraduate5yearprogram• First3yearsconsistpredominantlyofpreclinicalscienceswiththefourthandfi?hyearshavingmoreofaclinicalfocus.• Majorcoursesinthirdyear-ayear-longcourseonpathophysiology–stereotypicalwayswhichFssuesrespondtoinjury,prototypediseases-clinicalsigns,clinicalexamanddiagnosFcapproachtobodysystem-courseoninfecFousdisease(parasitology,virology,bacteriologyandmycology).
![Page 3: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/3.jpg)
Development of viva voce exam • In2012,weweretaskedwithdevelopingavivavoceoralexamasaprogramlevel“hurdle”assessmentattheendofthirdyear.
ObjecFves,toassess:• applicaFonoftheorytocasescenarios• diagnosFcclinicalreasoning• integraFonofmaterialfromboththepathophysiologyandinfecFousdiseasescourses• professionalismandoralcommunicaFonskills
• Exposuretooralexamformatbefore5thyearvivaexams
![Page 4: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/4.jpg)
Advantages of oral format
• Goodforassessingproblem-solvingabiliFes.• Canprovideinsightintostudents’cogniFveprocesses.• Itallowsprobingofthedepthandextentofstudents’knowledgewithfollow-upquesFons• UnclearorambiguousquesFonscanbere-expressedorimmediatelyclarified• Studentscanbeguidedbackontotherighttrackiftheystray
![Page 5: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/5.jpg)
Advantages of oral format • Allowsjudgementsaboutstudents’interpersonalcompetence-confidence,self-awarenessandaspectsofprofessionalism.• SimulaFonofclinicalpracFce–studentsneedto“thinkontheirfeet”• Itimprovesthequalityofstudentlearning–studentshavebeenreportedtostriveforthoroughnessinunderstandinginresponsetotheunpredictabilityandtoavoidembarrassingthemselvesinfrontoftheexaminer
• Itsuitssomestudents
![Page 6: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/6.jpg)
Disadvantages of oral format • Generatesalotofanxietyinstudentsandexaminers• Timeconsumingandrequiresalotofenergy• LackofanonymityandpotenFalforbias–unconsciousbiasbasedoncharismaFcpersonality,welldressed,beingarFculate• Reliability-Inter-case-differentstudentsgetdifferentcasestopreventsharingofinformaFon.
-Inter-rater-Intra-raterreliability
![Page 7: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/7.jpg)
Features of the initial exam • FormulaicseriesofquesFonsbasedonthediagnosFcprocess-ProvidedwithimageandsomeinformaFon–historyandclinicalexamfindings- bodysystem(s)- possiblepathologicalprocesses(highweighFng)–usingpathophysiologyforclinicalreasoning
- differenFaldiagnoses(lowweighFng!)- diagnosFcplan(highweighFng)- Interpretresults,treatment,control,prognosis
• professionalism
![Page 8: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/8.jpg)
Initial exam • Allstudentsreceivedonecase• StudentswhofailediniFalexamweregivenaresit• 10minutesperusal,20minutesoral• 2examinersperstudent-increasereliability–consensusmarksused
• Rubricandmodelanswers-increasereliability
![Page 9: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/9.jpg)
Changes over the years based on examiner and student feedback
• 2cases(usuallyonelargeanimal,onesmall)–increasesreliability• Moreexaminertraining• RemovedhistoryandclinicalexamsecFons• 2-stagepathophysiology• Adjustedrubric-moreemphasisonsystemaFcapproachforpathogenesisanddiagnosFctesFng,forcedtoprioriFseDDxs• Year-longpreparaFonandprovisionofformaFvefeedback.
![Page 10: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/10.jpg)
Youarepresentedwitha1yearolden3remaleDomes3cShortHaircat.Theownerno3cedthatthecatisoffitsfoodand‘quieter’thanusual,thishasbeenthecaseforaboutaweek.
Exam format – written section (30 minutes)
PartA.Descrip:onofcase,presen:ngsigns(images/videos)
![Page 11: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/11.jpg)
Exam format – written section • PartB.OutlinethepathologicalprocesseswhichmaybecausingthepresentaFon• StudentssFllrotelearningpathogenesisforparFcularclinicalpresentaFonwithoutemphasising/prioriFsingbasedoncaseparFculars• OrstudentsusingpacernrecogniFonandbeingabletodescribeonespecificpathogenesisbutnotbeingabletosuggestalternaFves• Breakdownintotwosteps,inthefirststep,theyneedtodemonstrateasystemaFcknowledgestructurethatincludesallpossiblepathogeneses• Secondstep,providedwithaddiFonalinformaFonaboutthecaseandneedtousethistoprioriFseDDxs
![Page 12: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/12.jpg)
• Encouragestudentstoconsiderallpossiblepathogenesesforabroadclinicalsignatthispoint• Mustdemonstratestructure,systemaFcapproach,encouragedtouseschemes
![Page 13: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/13.jpg)
Scheme-inductive knowledge structures
Harasymetal.2008KaohsiungJMedSci,24:7;341-355.
ResearchsuggeststhatexpertproblemsolversaredisFnguishedbythewayintheyorganiseandunderstandtheirknowledge
![Page 14: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/14.jpg)
Scheme-inductive knowledge structures • Mandinetal,UniversityofCalgaryFacultyofMedicine–“ClinicalPresentaFon”curriculum
• “The“hypotheFco-deducFve”strategytradiFonallyusedforPBLshouldbereplacedbyscheme-drivensearchstrategiessothatstudentsdevelopamoreorganisedandlogicalapproachtoproblem-solving”
SchemeshavemulFplepurposes:• ProvidesystemaFcapproachtodecreasetheriskofomidngDDxs
![Page 15: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/15.jpg)
• wayoforganisingknowledgeformoreefficientunderstanding,serveasascaffoldfornewinformaFon
![Page 16: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/16.jpg)
Thehistory,physicalexaminaFonandlaboratoryinvesFgaFonarealldrivenbythederivedscheme
![Page 17: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/17.jpg)
Paraclinical schemes • Probablydifferentcontenttoclinicalschemes,morerelianceonprinciplesofpathophysiology,lesssignalmentetc?
Nodescanconsistof:• locaFone.g.bodysystems-cardiovascularversusrespiratorytract;withinbodysystems-upperversuslowerrespiratorytract;• aformofgeneralpathologicalprocessintheFssuee.g.myocardiFs,hepaFcnecrosis,pulmonaryoedema• SpecificcausaFveagents(prototypic)• SomepresentaFonsabitdifferente.g.anaemia,jaundice• Lotsofschemesprovidedtothestudentsandtheyareencouragedtoimproveonthem
![Page 18: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/18.jpg)
Mnemonics as a system? • e.g.DAMNITV-DegeneraFve,Anomalous,Metabolicetc• Studentso?enpreferthisapproach• tryingtonotbetooprescripFve,flexibilityallowingfordifferentthoughtprocesses,clinicalpresentaFon
Don’tworktoowellinourhands….• Doesn’trequireexplicitidenFficaFonofbodysystems• Studentso?endon’tseemtounderstandpathogenesis• Doesn’tprovidestructureforsystemaFcdiagnosFcapproach• Moreappropriateasamemoryjoggerinclinicalyearsa?erstudentshavedemonstratedpathophysiologyknowledge?• UsefulaspartofamulFplestrategyapproach?
![Page 19: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/19.jpg)
YouwillbeprovidedwithaddiFonalhistoryorclinicalfeatures• YouperformaPCVandthereismarkedanaemia.
• IniFallyweassumethatgoodstudentswouldprioriFseDDxsbasedonallthefeaturesofthecasebutthisrarelyhappened
PartC.Giveaproblemdefini:on• Subacuteinappetence,depression,jaundiceandanaemia.
Exam format – written section
![Page 20: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/20.jpg)
Exam format – oral section PartD.Bodysystem(s)?• haematopoieFc
• PartE.UsetheaddiFonalinformaFontoidenFfymostlikelypathologicalprocess(es),describepathogenesisandprovideaeFologicalexamplesifappropriate
![Page 21: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/21.jpg)
![Page 22: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/22.jpg)
PartF.List5differenFaldiagnoses
PartG.DiagnosFcplan• IniFallyfoundthatstudentswouldjusttestandFckoffeachoftheirdifferenFals• EncouragesystemaFcapproach,useofschemetoguide,mostinformaFonfromfewesttests• EncourageuseofdiagnosFcstoguidetreatment• StudentsdorelaFvelypoorlyinthissecFon
Exam format – oral section
![Page 23: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/23.jpg)
Exam format – oral section
Providediagnosis,prognosis,treatmentplan,controlThissecFonalsopoorlydoneingeneral
![Page 24: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/24.jpg)
Exam format – oral section
• PartI–professionalism–useofprofessionallanguage,professionalbehaviouranddress.
Outcome of viva exams • Thisisamust-passhurdle,alargernumberofstudentsfailthiscomparedtofailingonbasisoftheoryexams(~5-9/120versus~2-3)• O?enabout10studentsonpass/failline,reluctanttoletthemgointo4thyear–whatshouldourexpectaFonsbeforthesenovicediagnosFcians?• Moststudentswhofailvivaareweakintheoryalsobutnotalways
![Page 25: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/25.jpg)
Preparation for viva exams • 6face-to-facetutorialsdispersedthroughtheyear• 5thyearstudentsactastutorsforsmallgroups• 3rdyearsexaminingeachotherinpairs• Includeallstepsinprocess,evenatthebeginning• 3on-linecase-basedtutorialswheregenericfeedbackgivenbutstudentscanelecttohavepersonalfeedback• “diagnosFcwork-shop”• Viva“hint-sheet”
![Page 26: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/26.jpg)
Lessons we have learnt • WetookalotforegrantediniFally:“goodstudentsshouldautomaFcallydemonstrateasystemaFc/structuredapproach,prioriFseDDxsbasedonparFcularsofacase…”• NeedtobecompletelyexplicitinexpectaFonsandspecificforeachstepinprocesswhichmeanshavingagoodunderstandingoftheminyourownmind• NeedtostructurerubricsothatitisnotpossibleforastudenttogetgoodmarkswithoutasystemaFcapproachandbeingabletoprioriFse
• NeedtostructurepreparaFonmaterialtoencouragegoodhabits• EncouragepracFceofverbalcommunicaFonasthisisaverydifferentskilltowricen
![Page 27: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/27.jpg)
Conclusions Oralformatbenefits:• Beingabletoguidestudentsontotherighttrack,test“thinkingonfeet”,probeunderstanding• FearisapowerfulmoFvatorforembracingnewlearningapproaches
OralformatlimitaFons:• VariaFonincases–lackofstandardisaFon?• Inter-raterreliability• MassiveamountofFmeandenergy,difficulttoengageexaminers• InternaFonalstudentsdisadvantagedbylanguage,culturallyingrainedbehaviour? • ReallygoodformatforassessingdiagnosFcclinicalreasoning
(oralcommunicaFonskills,professionalism)butdoesgoodoutweighthebad?
![Page 28: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/28.jpg)
Future directions • Reliability–variaFoninmarksacrosscases,examiners,secFonsofexam• InvesFgatewhetherpreparaFonandtakingtheseexamshasalasFnginfluenceonstudents’studyhabitsandapproachtocases• DevelopingmoreefficientmeansofprovidingformaFveassessment• BecerintegraFonwiththeclinicalyearsoftheprogram–evolvingnatureofclinicalreasoningstrategies,rollingoutofseriesofverFcallyintegratedon-linetutorials• ObtainbecerunderstandingofapproachesofweakstudentssoastobecerassisttheirpreparaFon
![Page 29: Development of viva voce exam to assess clinical …bespoke-production.s3.amazonaws.com/fvas/assets/dc/040540b83411e...Development of viva voce exam to assess clinical reasoning in](https://reader035.fdocuments.in/reader035/viewer/2022081722/5b31e4f27f8b9a81728c0db1/html5/thumbnails/29.jpg)
References • Davis,M.,Karunathilake,I.2005.TheplaceoforalexaminaFonsintoday’sassessmentsystems.MedicalTeacher,27;4:294-297.
• Harasymetal.2008CurrentTrendsinDevelopingMedicalStudents'CriFcalThinkingAbiliFes.KaohsiungJMedSci,24:7;341-355.
• Joughin,2010.Ashortguidetooralassessment.LeedsMetropolitanPressinAssociaFonwithUniversityofWollongong.
• Mandin,H.,Harasym,P.,Eagle,C.andWatanabe,M.1995.Developinga“clinicalpresentaFon”curriculumattheUniversityofCalgary.AcademicMedicine.70;3:186-193.
• Mandin,H.,Jones,A.,Woloschuk,W.andHarasym,P.1997.Helpingstudentslearntothinklikeexpertswhensolvingclinicalproblems.AcademicMedicine,72;3:173-179.