Assessment and Intervention Enhances the Acquisition of Procedural Skills in Medicine David W. Musick PhD, Robert G. Carroll PhD, and Luan Lawson-Johnson MD

Office of Medical Education, Division of Academic Affairs, Brody School of Medicine, East Carolina University

Email: musickd@ecu.eduAbstract

Procedural skills education for medical students is increasingly difficult to teach and assess. As education shifts to a competency-driven approach, the need to “re-establish the clinical skill development process” 1 is paramount. This project examined data from internal (M4 medical students) and external (residency program directors) surveys concerning proficiency in a designated set of clinical procedural skills. Skill acquisition by medical students is enhanced by a combination of internal and external assessment procedures.

IntroductionThe education of future physicians involves the development of three distinct components well-described by Bloom et al2: Cognitive, or the knowledge base required to practice medicineAffective, or the attitudes & professionalism expected of physiciansPsychomotor, or the clinical and procedural skills required to practice medicine within a specific specialty (e.g., Surgery, Family Medicine, Pediatrics)

The development of these three components occurs initially during the four years of medical school, wherein students learn what is expected and receive the MD degree. This training then continues within a 3-7 year residency training period in a specialty field, whereupon they complete their formal training and are then granted the opportunity to practice medicine independently.

The teaching and assessment of required procedural skills for medical students has traditionally been unstructured, with students encouraged to seek out opportunities to practice procedures as they presented themselves within the inpatient or outpatient facilities of a teaching institution. This approach contributed to variability in the skill level of students graduating with the MD degree. More recently, the traditional approach of “see one, do one, teach one” has been supplemented by use of standardized patients, simulation technology and computer-based learning. These are now widely proposed as viable and safer methods of allowing students to practice their skills prior to the initial performance of a procedure with an actual patient.

The clinical faculty of the Brody School of Medicine developed a formal curriculum pertaining to procedural skills in 2005-2006 and introduced it during the 2006-2007 academic year. The curriculum consisted of 35 specific procedures that were assigned across the first 3 years of the overall medical school curriculum. The majority of the skills were to be taught and assessed during year 3, which is the core clinical training year. Skills ranged from simple to complex, and were categorized broadly as those which could be performed using actual patients, or those that could be performed using a combination of actual patients and simulated experiences.

The need to develop more explicitly measured and formal curricula for medical student procedural skills has been emphasized by the primary professional organization representing medical schools, the Association of American Medical Colleges 1,3. However, there have been very few published research studies about the effectiveness of such curricula, with only two publications located during the initial time period of this project4,5.

Results

Materials and MethodsWe completed two surveys, one external and one internal. The external survey asked residency program directors of BSOM graduates to rate our graduates on their “basic procedural skills” along with 27 other aspects of clinical medicine. The internal survey asked six consecutive classes of senior medical students to report their experiences with the procedural skills curriculum during their four years of medical school. The external survey was launched first, and contributed important data to our decision to develop the formal skills curriculum. Internal data from 2007-2009 were used to design a new required 4th year course in Emergency Medicine that emphasized procedural skills. Data from the 3 years after implementation of the course were used to assess impact.

Conclusions

ReferencesThe AAMC Project on the Clinical Education of Medical Students. Washington, DC:

Association of American Medical Colleges, 2004. Bloom, Benjamin S. & David R. Krathwohl. Taxonomy of educational objectives: The

Classification of Educational Goals by a Committee of College and University Examiners. New York , Longmans Publishers, 1956.

 Recommendations for Clinical Skills Curricula for Undergraduate Medical Education.

Washington, DC: Association of American Medical Colleges, 2005. Langdale LA; Schaad D; Wipf J; et al. Preparing Graduates for the First Year of Residency: Are

Medical Schools Meeting the Need? Academic Medicine 2003; 78 (1): 39-44. Sanders CW; Edwards JC; Burdenski TK. A Survey of Basic Technical Skills of Medical

Students. Academic Medicine 2004; 79 (9): 873-5.

Engum, SA. Do You Know Your Students’ Basic Clinical Skills Exposure? The American Journal of Surgery 2002; 186: 175-81.

Survey of Senior Medical Students, BSOM (2007-2012)

Experiences with Brody Procedural Skills Curriculum

Year RatingTrend

Compared to Baseline (06)

2006 3.70 n/a2007 3.68 -2008 3.82 +2009 3.63 -2010 3.81 +2011 3.81 +

External Rating of Basic Procedural Skills of BSOM Graduates (2006-2011) by their Residency Program DirectorsSkill Class of 2007 2008 2009 3 Year

Avg New EM Course

2010 2011 2012 3 Year Avg

Trend Change

1 Suction tracheostomy - sterile technique 63 61 45 56 54 38 37 43 - -132 KOH/exam of skin 66 51 55 57 46 44 44 45 - -123 Dressing change 88 94 93 92 100 62 98 87 - -54 Oxygen tank: crack tank, assess flow 61 71 47 60 43 61 64 56 - -45 Urinalysis/dipstick only (not microscopic exam) 80 90 76 82 69 78 88 78 - -46 Biopsy, skin/excision or punch 76 80 72 76 74 78 67 73 - -37 Thoracentesis (pleural tap) 14 16 18 16 6 5 28 13 - -38 Labor and delivery, normal vaginal 92 98 93 94 96 93 94 94 = 09 Strep: rapid test/throat swab 96 96 97 96 94 98 94 96 = 0

10 Hemoccult blood testing 94 94 88 92 89 95 94 93 + 111 Suturing (removal of skin sutures) 98 96 96 97 100 98 96 98 + 112 KOH/vaginal smear/wet mount/pap smear/collect 94 96 94 95 100 96 94 97 + 213 Suturing (removal of skin staples) 98 98 96 97 100 98 100 99 + 214 Suturing (simple) 98 98 90 95 96 98 100 98 + 315 Intubation/observe &perform 58 57 58 58 48 68 70 62 + 416 Paracentesis (abdominal tap) 22 26 12 20 22 20 30 24 + 417 Chest tube/observation 56 69 67 64 67 73 68 69 + 518 Pulse oximetry: apply & validate sensing 74 88 72 78 76 85 92 84 + 619 Catheterization, urethral/Foley 84 84 89 86 91 100 98 96 + 1020 Fingerstick glucose measurement/obtain only 68 80 76 75 74 88 96 86 + 11

21 Arterial blood gas/collect only (not interpret) 38 45 32 38 48 51 51 50 + 12

22 Arthrocentesis 42 35 29 35 33 55 53 47 + 1223 Anesthesia, local infiltration 78 84 71 78 89 89 98 92 + 1424 Metered dose inhaler (MDI)/instructions for use 58 73 51 61 72 73 80 75 + 14

25 Lumbar puncture 44 53 43 47 52 62 77 64 + 1726 Injection: SCIMID 49 47 31 42 44 76 71 64 + 2027 Nasogastric/oralgastric intubation 22 41 26 30 37 67 54 52 + 2228 Venipuncture 37 63 42 47 65 65 80 70 + 2329 Splinting only/not casting 42 55 47 48 67 69 83 73 + 2530 Nebulizer treatment 47 59 45 50 65 79 87 77 + 2731 Abscess incision/drain 68 59 60 62 81 95 96 91 + 2932 Blood culture/collection (adult) 20 20 28 22 37 65 70 57 + 3533 Catheter/IV, peripheral insertion 49 53 48 50 72 88 94 85 + 3534 Electrocardiogram/lead placement 51 47 50 49 78 98 98 91 + 42

•The new curriculum in required procedures increased the numbers of students who reported experience with procedural skills.•Adding a mandatory Emergency Medicine rotation, which emphasized procedural skills, contributed to increased number of student opportunities to learn procedures.•Certain procedures were likely less available during the study period, due to an increased regulatory burden and decreased frequency in the academic setting.•As a result of this study, consideration of further changes to the formal procedural skills curriculum is indicated.

Internal Survey of Graduating Physicians 2007-2012

Average improvement after introduction of required Emergency Medicine rotation: 9.7%

Bars in same sequence as internal survey data table