ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK,...
Transcript of ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK,...
![Page 1: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/1.jpg)
DEVELOPMENT OF AN EDUCATION DASHBOARD TO SUPPORT MEDICAL STUDENT LEARNING ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH
TEACHING ENHANCEMENT GRANT, NUS
![Page 2: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/2.jpg)
MEDICAL SCHOOL REQUIREMENTS• Knowledge
• Pre-Clinical Sciences • Anatomy, Biochemistry, Physiology, etc
• Clinical Specialities• Pediatrics, Internal Medicine, Surgery, Psychiatry, etc
• Skills• Clinical Reasoning• Psychomotor
• Suturing, IV placement• Interpersonal Communication
• Attitudes• Professionalism• Empathy
![Page 3: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/3.jpg)
AssessmentPerformance
![Page 4: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/4.jpg)
![Page 5: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/5.jpg)
![Page 6: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/6.jpg)
![Page 7: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/7.jpg)
This image was released to the Public Domain by the National Cancer Institute,
![Page 8: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/8.jpg)
![Page 9: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/9.jpg)
AssessmentPerformance
For the Student and Faculty Mentor to make sense of this:
![Page 10: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/10.jpg)
BACKGROUND
• Medical students are typically assessed in numerous ways and at multiple points in time.
• Students and their advisors are generally unable to combine this assessment data in ways that could provide powerful summary evidence of the students’ overall strengths and weaknesses.
• As a result, they often lack a cohesive strategy to assist them in interpreting this data in ways that can direct their future learning.
![Page 11: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/11.jpg)
STUDENT DASHBOARD
Mission Motorcycle digital dashboard Gramm Bjoern https://www.youtube.com/watch?v=WDCUakUkVvY
![Page 12: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/12.jpg)
OBJECTIVE
Develop an interactive, continuous feedback, technological resource for managing and interpreting assessment data in order to facilitate individualized educational planning.
![Page 13: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/13.jpg)
METHODS: TEG GRANTA secure website (“Dashboard”) for medical students with three functional purposes: 1. To serve as a repository for assessment
reports from various testing services, such as the National Board of Medical Examiner subject-specific exam reports,
2. To serve as a launching pad to third-party test preparation services, and
3. To receive performance reports in real time for continuous feedback and future self-directed learning.
![Page 14: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/14.jpg)
GOOGLE APPS
![Page 15: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/15.jpg)
![Page 16: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/16.jpg)
NUS VERIFICATION ‐ LOGIN
![Page 17: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/17.jpg)
Upload New Students
![Page 18: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/18.jpg)
Rosh Review Invitation Email
![Page 19: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/19.jpg)
Rosh ReviewAll student feedback on one place accessible via single sign-on
Image from ad: https://itunes.apple.com/us/app/rosh-review-mobile/id991352355?mt=8
![Page 20: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/20.jpg)
STUDENT REPOSITORY
![Page 21: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/21.jpg)
RESULTS1. Successfully created a cloud-based website using our design goals.
2. Built Application Programming Interfaces (APIs) with two third-party vendors:
1. USMLE-Rx (www.usmle-rx.com), 2. Rosh Review (www.roshreview.com),
3. The APIs allowed students to launch out to the third-party sites, respond to question banks, and the results were then pushed back to the Dashboard for visualization.
4. Sixteen students piloted the Dashboard.
1. Students rated the access to and the performance output reports from the USMLE-Rx question bank the most valuable elements of the Dashboard.
![Page 22: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/22.jpg)
SINGAPORE PERSONAL DATA PROTECTION ACT• Enacted in July 2014 during the time of this study project.
• Required to shut down Dashboard to comply with Act requirements
• Additional safeguards with the dashboard are required and will incorporate our learning in new web-based learning management system dashboard
![Page 23: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/23.jpg)
CONCLUSIONEarly positive feedback regarding the value of developing individualized education plans.
On a separate but related project, our Predictive Analytics allowed us to achieve a 98% first time pass rate for USMLE Step 1 Examination
Next steps will be to:
1. Ensure that the Dashboard is compliant with the Personal Data Protection Act (PDPA)
2. Investigate additional third-party sites for potential partnerships.
3. Improve the visualization of performance data
4. Evaluate the overall impact on learning provided by this resource.
![Page 24: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/24.jpg)
CONCLUSIONOur education work at DukeNUS Medical School has previously focused on two key areas:
• developing technology to efficiently deliver contentto our students and
• creating a complementary in-class, team based experiences
This project begins to improve the learning that is achieved after the in-class activities. Medical learning can’t stop after the classroom end, instead, students must learn medicine over a lifetime by continually assessing areas of weakness.
As Toffler stated, “unlearning and
relearning” information while acquiring new
knowledge and skills will be necessary.
Acquiring new knowledge and skills is
only possible with accurate and
continuous feedback.
![Page 25: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/25.jpg)
REFERENCESAcknowledgement: The authors would like to thank the support from the National University of Singapore, Teaching Enhancement Grant.
1) Slotnick HB (1999) How doctors learn: physicians' self-directed learning episodes. Acad Med. Oct;74(10):1106-17. 2) Eva KW, Cunnington JPW, Reiter HJ, Keane DR, Norman GR (2004). How Can I Know What I Don’t Know? Poor Self Assessment in a Well-Defined Domain, Advances in Health Sciences Education, 9: 211–224 3) Brown WE, Lovett M, Bajzek, D & Burnette JM (2006). Improving the Feedback Cycle to Improve Learning in Introductory Biology Using the Digital Dashboard. In T. Reeves & S. Yamashita (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2006 (pp. 1030-1035). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). 4) Bajzek D, Brown W, Lovett M & Rule G (2007). Inventing the Digital Dashboard for Learning. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 1084-1092). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). 5) Corrin L & de Barba P (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In B. Hegarty, J. McDonald, &S.K. Loke (Eds.), Rhetoric and Reality: 7 Critical perspectives on educational technology. Proceedings ascilite Dunedin.NZ 2014 (pp. 629-633).
![Page 26: ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH … Kamei.pdf · ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH ... USMLE-Rx (), ... to achieve a 98% first time pass](https://reader031.fdocuments.in/reader031/viewer/2022020214/5ad23ef47f8b9a665f8c3ad3/html5/thumbnails/26.jpg)
www.duke-nus.edu.sg