Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA...
Transcript of Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA...
![Page 1: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/1.jpg)
Donald J. Sefcik, DO, MBASenior Associate Dean
College of Osteopathic MedicineMichigan State University
![Page 2: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/2.jpg)
• In-Service Examination (ISE) scoresWhat do they represent?How should you interpret them?What should you do with them?
• Techniques used to raise ISE scores• Recommendations
![Page 3: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/3.jpg)
Emergency Medicine• Hern et al (2009); Cheng (2008); Gillen (1997)Family Medicine• Sevensma SC, Navarre G, Richards RK (2008)Internal Medicine• McDonald, Zeger & Kolars (2008)• Garilbaldi et al (2002)OB/Gyne• Withiam-Leitch & Olawaiye (2008)Pediatrics• Aeder, Fogel & Schaeffer (2010)• Langenau, Fogel & Schaeffer (2009)RadiologySurgery• Shellito et al (2010); Subhas et al (2009); Kosir et al (2008)• Ferguson & Warshaw (2006); Virgilio et al (2003); Bull et al (2001)
![Page 4: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/4.jpg)
A 72 year-old patient presents with progressive difficultybreathing, jugular venous distention, orthopnea, bilaterallower extremity edema , and bilateral rales. Which of thefollowing is most important when deciding upontreatment options?
A. minute respirationB. heart rateC. degree of lower extremity edemaD. respiratory rateE. none of the above
![Page 5: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/5.jpg)
A 72 year-old patient presents with progressive difficultybreathing, jugular venous distention, orthopnea, bilaterallower extremity edema , and bilateral rales. Which of thefollowing is most important when deciding upontreatment options?
We don’t treat patients without understanding theetiology of their predicament and the nature of theintervention. Guiding residents’ learning should followsimilar guidelines.
![Page 6: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/6.jpg)
XR = XT + ME
![Page 7: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/7.jpg)
XR = XT + ME
XR = Reported ISE scoreXT = True Score ME = Measurement Errors
![Page 8: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/8.jpg)
XR = XT + ME
XR = Reported ISE Score ScoresPercent = items correct = raw scorePercentile = resident rank compared to peers
![Page 9: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/9.jpg)
The report of the performance of a PGY2 on his ISEreveals a percent correct of 63 and a percentile rank of 67.What should you do?
A. mandate the resident enter a remediation programB. reduce work hours by 25% to increase study timeC. do not promote the resident to the PGY3 yearD. encourage the resident to keep studying E. resign as program director
![Page 10: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/10.jpg)
• What is an “average” percent score?
http://www.eric.vcu.edu/home/curriculum/print/2009Histograms.pdf
![Page 11: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/11.jpg)
• What is an “average” percent score?2009 IM (n = 402 programs)PGY1 = 54%PGY2 = 59%PGY3 = 63%
http://www.eric.vcu.edu/home/curriculum/print/2009Histograms.pdf
![Page 12: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/12.jpg)
• What is an “average” percent score?2009 IM (n = 402 programs)PGY1 = 54%PGY2 = 59%(example was PGY2 = 63%)PGY3 = 63%
http://www.eric.vcu.edu/home/curriculum/print/2009Histograms.pdf
![Page 13: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/13.jpg)
Your PGY2 from the previous example is now a PGY3.His percentile rank while in your program has been PGY1 = 62, PGY2 = 67, PGY3 = 64. What should you do?
A. realize that this is a common pattern of progression B. worry that this resident will fail his certifying examC. do not promote the resident to the PGY4 yearD. terminate this resident from your program E. delegate the situation to your associate PD
![Page 14: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/14.jpg)
Why is percentile important?Increased risk of failing the certifying exam
%F
50th30th 100th
![Page 15: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/15.jpg)
Why is percentile important?Increased risk of failing the certifying exam
%F
50th30th 100th
The lower the percentile rank –
The greater the risk of failing the certifying examination
![Page 16: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/16.jpg)
Percent Correct Percentile
Increases each year
PGY1 PGY2 PGY3 PGY4
![Page 17: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/17.jpg)
Percent Correct Percentile
Potential Problem
PGY1 PGY2 PGY3 PGY4
![Page 18: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/18.jpg)
Percent Correct Percentile
Tends to be stable
PGY1 PGY2 PGY3 PGY4
![Page 19: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/19.jpg)
Percent Correct Percentile
Potential Problem
PGY1 PGY2 PGY3 PGY4
![Page 20: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/20.jpg)
Tracking percentiles• Three different residents • Three years of scores
%F
50th30th
A1B1C1
100th
![Page 21: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/21.jpg)
Tracking percentiles• Three different residents • Three years of scores
%F
50th30th
A1
A2
B1
B2
C1
C2
100th
![Page 22: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/22.jpg)
Tracking percentiles• Three different residents • Three years of scores
%F
50th30th
A1
A3
B1
B3
C1
C3
C2 B2 A2
100th
![Page 23: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/23.jpg)
Tracking percentiles • Three different residents • Three years of scores
%F
50th30th
A1
A3
B1
B3
C1
C3
C2 B2 A2
100th
What can you do to assess the residents between the annual ISE’s?
![Page 24: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/24.jpg)
• Make decisions about • individual’s educational progression• program content areas that are outliers
![Page 25: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/25.jpg)
• Make decisions about • individual’s educational progression• program content areas that are outliers
• Do not make decisions about:• promotion or termination
![Page 26: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/26.jpg)
XR = XT + ME
XT = True Score = two components
![Page 27: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/27.jpg)
XR = XT + ME
XT = True Score = two components
XC = Score based on contentXB = Score based on beyond content
![Page 28: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/28.jpg)
XR = XC + XB + ME
XR = Reported ISE Score XC = Score based on contentXB = Score based on beyond contentME = Measurement Error
![Page 29: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/29.jpg)
• Choosing your intervention options• Content • Beyond Content
![Page 30: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/30.jpg)
XC + XB
Content
Beyond Content
PGY1 PGY2 PGY3 PGY4
![Page 31: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/31.jpg)
• Assigned / Programmed Reading• Conference Attendance
• Didactic lectures • Grand Rounds / M&M
• Practice Questions• Board Review Courses
![Page 32: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/32.jpg)
• Assigned / Programmed Reading• Conference Attendance
• Didactic lectures / Journal Clubs• Grand Rounds / M&M
• Practice Questions• Board Review Courses
![Page 33: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/33.jpg)
• Self-directed Studying• Preparing Lectures for Peers• Reading Fluency and Accuracy• Test Anxiety
![Page 34: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/34.jpg)
• Self-directed Studying• Preparing Lectures for Peers
• Questions from / Clarification by an Attending• Reading Fluency and Accuracy
• Developing theme in repeat failures• Test Anxiety
• Definite impact on memory / recall
![Page 35: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/35.jpg)
• Intervention options • Content = more impact earlier in training
PGY<2 - Programmed Reading / Didactics PGY>3 - Programmed Reading / Didactics
![Page 36: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/36.jpg)
• Beyond Content • Resident Heterogeneity (all years of training)
Studying SkillsTest-taking Skills
![Page 37: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/37.jpg)
• Beyond Content • Resident Heterogeneity
Studying Skills• Memory techniques • Reading and Note-taking • Transfer Test-taking Skills• Reading- Fluency and Accuracy• Test Anxiety• Test-Wiseness
![Page 38: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/38.jpg)
1 / 6
![Page 39: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/39.jpg)
2 / 6
![Page 40: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/40.jpg)
3 / 6
![Page 41: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/41.jpg)
1. Assess the resident’s situationDevelop an individualized planIt’s not just about XC
2. Create a teamResident and AttendingAttending = Coach to Mentor
3. Establish a scheduleRegular meetings
4. Monitor and Refine
4 / 6
![Page 42: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/42.jpg)
1. Assess the resident’s situationDevelop an individualized planIt’s not just about XC
2. Create a teamResident and AttendingAttending = Coach to Mentor
3. Establish a scheduleRegular meetings
4. Monitor and Refine
There are resourcesto help youhelp your resident
5 / 6
![Page 43: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/43.jpg)
6 / 6
We need to transition away from our focus on teaching (faculty-centered) and instead begin to focus on learning (resident-centered).
![Page 44: Donald J. Sefcik, DO, MBA - Statewide Campus System in-service test... · Donald J. Sefcik, DO, MBA Senior Associate Dean College of Osteopathic Medicine Michigan State University](https://reader031.fdocuments.in/reader031/viewer/2022030406/5a8290d37f8b9a571e8e4c9b/html5/thumbnails/44.jpg)
Are there some Questions?
Complete Reference List will be posted (pdf)
on SCS website
Emergency Medicine• Hern et al (2009); Cheng (2008); Gillen (1997)Family Medicine• Sevensma SC, Navarre G, Richards RK (2008)Internal Medicine• McDonald, Zeger & Kolars (2008)• Garilbaldi et al (2002)OB/Gyne• Withiam-Leitch & Olawaiye (2008)Pediatrics• Aeder, Fogel & Schaeffer (2010)• Langenau, Fogel & Schaeffer (2009)RadiologySurgery• Shellito et al (2010); Subhas et al (2009); Kosir et al (2008)• Ferguson & Warshaw (2006); Virgilio et al (2003); Bull et al (2001)