Wisconsin Medical Journal 2005 • Volume 104, No. 334

WISCONSIN MEDICAL JOURNAL

Evidence-Based Medicine for Medical Students: Introducing EBM

in a Primary Care Rotation

William E Cayley, Jr, MD, MDiv

Doctor Cayley is an assistant professor in the University of Wisconsin Department of Family Medicine, teaching at the Eau Claire Family Medicine Residency and the Augusta Family Medicine Rural Training Site. Preliminary data was presented at UME 21 National Symposium (Baltimore, Maryland, 16 March 2002) and Society of Teachers of Family Medicine Spring Conference (San Francisco, California, 29 April 2002). Please address correspondence to William E Cayley, Jr, MD, MDiv, University of Wisconsin, Eau Claire Family Medicine Residency, 617 West Clairemont Ave, Eau Claire, WI 54701; phone 715.839.5175; fax 715.839.5176; e-mail bcayley@yahoo.com.

ABSTRACTBackground and Objectives: Evidence-based medicine (EBM) seeks to improve patients’ lives by applying the best available evidence to decisions affecting health out-comes. Since medical students often do not appreciate the value of an evidence-based approach to medicine when they enter third-year clinical training, a curricu-lum was developed introducing third-year students to the practice of EBM during a primary care clinical rota-tion.

Methods: Twenty-seven students over 4 rotations par-ticipated in the series of 6 hour-long seminars, and 8 items from a 27-item questionnaire were used to mea-sure the impact on students’ self reported understand-ing and use of EBM.

Results: Responses to questionnaires given before and after completion of the curriculum documented im-proved self-reported understanding of EBM and im-proved self-reported comfort with critical appraisal.

Conclusions: A seminar series introducing EBM in a primary care rotation improved students’ familiarity with and receptivity to use of EBM.

INTRODUCTIONEvidence-based medicine (EBM) advocates using the best available evidence1 to guide medical care toward outcomes that are meaningful to patients2 in a way that has been described as a “paradigm shift” in medi-

cal teaching and practice.3 Until recently, students have had minimal exposure to EBM during preclinical train-ing, thus introduction of EBM during clinical work in the third year is sometimes seen as “over-empha-sis on literature” as opposed to “real-life situations.” Investigations into the teaching of EBM have found students and residents reporting improved attitudes to-ward the use of evidence and increased confidence at critical appraisal, and have found enthusiasm for faculty development teaching applied, clinical use of EBM.4-9 In order to evaluate whether a curricular module during a clinical rotation could improve third year medical stu-dents’ understanding of EBM principles, a curriculum integrating EBM instruction with clinical application was developed using insights from previous work.

METHODSAfter a MEDLINE search identified 132 articles ad-dressing the teaching of EBM, insights from these articles were used to develop a 6-session curricu-lum combining exploration of the basic principles of EBM with application of these principles to real-life cases from students’ clinical experience. A curricu-lum overview, including goals and objectives, is pro-vided in Table 1. Subjects for this investigation were third-year medical students working with primary care preceptors during an 8-week primary care clini-cal rotation at a family medicine residency. Twenty-seven students participated during the study period of May 2001–October 2002. The impact of the cur-riculum was measured using a 28-item questionnaire developed by Slawson and Shaughnessy,8 adminis-tered at the beginning and end of each 8-week ro-tation. Students were informed that completion of the questionnaire was optional, that responses were entirely anonymous, and that non-participation in the questionnaire would not affect rotation grading. Questionnaires did not have individual identifying information. Eight items from the questionnaire,

35

WISCONSIN MEDICAL JOURNAL

Wisconsin Medical Journal 2005 • Volume 104, No. 3

outlined in Table 2, were identified as representing the stated objectives of the curriculum. Responses by students from all 4 groups on each of the 8 relevant items on the pretest were pooled, and compared with the pooled responses for the same item on the post-test using the Mann-Whitney non-parametric test. This project was granted exemption from review by the University of Wisconsin Health Sciences Human Subjects Committee on April 16, 2001.

RESULTSAll 27 primary care students completed pre- and post-rotation questionnaires, for a 100% response rate. When all 4 groups were pooled, items 1 and 4 dem-onstrated statistically significant changes from pretest to posttest measurements while the remaining items demonstrated no significant change. There was signifi-cantly increased agreement with the statements that “I understand the concept of EBM” (item 1) and “I am comfortable in my ability to critically appraise a re-search article” (item 4) on the post-rotation question-

Table 1. EBM Curriculum Overview

Goal: To introduce third year medical students to the clinical application of EBM

Objectives: Students will learn to 1. Develop �well-built� clinical questions about patient-oriented outcomes that matter. 2. Search for patient-oriented evidence that matters. 3. Apply patient-oriented evidence to a specific clinical case.

Instructional SessionsSession 1: Defining patient-oriented outcomesDiscussion of sample cases and the prioritization of clinical issues to be addressed; discussion of personal values in making life deci-sions; discussion of increased quality of life for patients as the appropriate common goal of all medical pursuits; discussion of using the patient-oriented outcomes of morbidity, mortality, and quality of life as the appropriate guides for pursuing medical management that adheres to patients� values; students each assigned to report on 5 clinical encounters and the relevant patient-oriented outcomes

Session 2: Developing clinical questions regarding patient-oriented outcomesDiscussion of patient-oriented outcomes from clinical encounters; review of PICO (Patient, Intervention, Comparison, Outcome) model for framing clinical questions; students each assigned to report on clinical encounters and compose relevant clinical ques-tions

Session 3: Finding evidence and finding POEM�sDiscussion of clinical cases and questions; discussion of potential types of information and levels of evidence; discussion of the POEM (�patient-oriented evidence that matters�) concept; discussion of sources of evidence and review of quality internet-based EBM resources; students each assigned to find and present an answer to one of their clinical questions

Session 4: Practicing Internet searches for informationDiscussion of answers found for clinical questions; discussion of information sources used; small-group practice exploring other Internet sources of EBM information

Session 5: Using basic EBM statisticsDiscussion of differences between diagnosis, treatment, screening and prevention; discussion of sensitivity, specificity, likelihood ratio (LR), number needed to treat or to harm (NNT/NNH) and their application to the interpretation of answers to specific clinical questions; students each assigned to identify a clinical case, compose a clinical question, identify evidence sources, and interpret the evidence using relevant statistics

Session 6: Applying POEM�s using EBM statisticsReview of cases, evidence, statistical interpretation, and application of the information to patient care

Table 2. Questionnaire Items Selected for Evaluating Curriculum

1. I understand the concept of EBM.2. I get overwhelmed when I try to sort through available

medical information.4. I am comfortable in my ability to critically appraise a re-

search article.7. I believe I should base my clinical decisions on valid evi-

dence when such evidence exists.9. My colleagues will respect my clinical decisions based on

evidence more than those based on current trends.

Responses to items 1-9 were on a 5-point scale:1 (Strongly Agree), 2 (Agree), 3 (Neutral), 4 (Disagree), 5 (Strongly Disagree)

14. The degree to which I rely on literature searches to keep up with changes.

16. The degree to which I rely on research journals to keep up with changes.

21. The degree to which I rely on medical computer data-bases to keep up with changes.

Responses to items 14-21 were on a 4-point scale:1 (Never), 2 (Rarely), 3 (Sometimes), 4 (Always)

WISCONSIN MEDICAL JOURNAL

Wisconsin Medical Journal 2005 • Volume 104, No. 336

naires, but no change in the already strong agreement that clinical decisions should be based on valid evi-dence (item 7). There was also a non-significant trend toward less agreement with the statement, “I get over-whelmed when I try to sort through available medical information.” There was a wide range of agreement or disagreement with the statement, “My colleagues will respect my clinical decisions based on evidence,” which remained unchanged. The 3 items evaluating frequency of resource use (14, 16, and 21) found most students reporting on both the pretest and posttest that they “sometimes” or “always” relied on literature searches, research journals, and medical computer databases, with no statistically significant difference between pretest responses and posttest responses. Responses to the 8 relevant questionnaire items are summarized in Table 3.

DISCUSSIONThis project demonstrated that a 6-session curriculum integrating clinical EBM instruction into a third-year primary care rotation was able to improve students’ self-reported understanding of EBM and their comfort with critical appraisal of research articles. There was also a non-significant trend towards feeling less over-whelmed by the volume of available medical informa-tion. Students entered the rotation fairly well convinced that clinical decisions should be based on valid evidence, though somewhat less convinced that these decisions would be respected, and the rotation had no demon-strable effect on these opinions. Students’ reported use of information sources was also not measurably affected by the rotation. Thus, the primary impact of the cur-

riculum was to increase students’ self-reported under-standing of EBM and their self-reported comfort with critical appraisal.

This study does have several limitations. First, the questionnaire only assessed students’ self report of be-haviors, attitudes, and ability. The ultimate measure of success in the introduction of EBM would be documen-tation of subsequent improvements in patient outcomes, but the small sample size, the fact that students’ practice was under the supervision of clinical preceptors with varying medical practices, and the short time-frame of the project made actual documentation of behavior change or improvement in clinical outcomes impracti-cal. A second limitation is that an increasing emphasis on EBM in the preclinical curriculum during the study period may have diminished the impact of a first intro-duction to EBM in the first clinical year. Finally, the use of a small convenience sample, without controlling for prior experience with or interest in EBM, may have al-lowed for both underestimation and overestimation of effects.

Despite the these limitations, this project does demonstrate that a curriculum introducing third-year medical students to EBM can improve their self-re-ported understanding of and comfort with EBM, and may help them feel less overwhelmed by medical in-formation. This corroborates findings of Slawson and Shaughnessy,8 and further demonstrates that the impact of teaching EBM to medical students can be measured. Further progress in this area could be made through efforts to more closely integrate introduction of EBM into clinical curricula across the 4 years of un-dergraduate medical education, through education of

Table 3. Comparison of Pooled Responses to Pre- and Post-Rotation Questionnaires

Item Pretest Mean Posttest Mean Significance*

1. I understand the concept of EBM. 1.90 1.38 0.0019 2. I get overwhelmed when I try to sort through available 2.95 3.43 0.0808 medical information. 4. I am comfortable in my ability to critically appraise 2.76 2.24 0.0179 a research article. 7. I believe I should base my clinical decisions on valid . 1.67 1.57 0.2743 evidence when such evidence exists 9. My colleagues will respect my clinical decisions based 2.14 2.38 0.166 on evidence more than those based on current trends.14. The degree to which I rely on literature searches to keep 3.19 3.29 0.33 up with changes.16. The degree to which I rely on research journals to keep 3.24 3.19 0.4247 up with changes.21. The degree to which I rely on medical computer 3.00 3.14 0.3336 databases to keep up with changes.

* Significance of 1-directional Mann-Whitney U.

37

WISCONSIN MEDICAL JOURNAL

Wisconsin Medical Journal 2005 • Volume 104, No. 3

preceptors in the clinical application and clinical teach-ing of EBM, and through investigation of the degree to which students’ self-reports of facility with EBM cor-respond to actual behavior patterns. An area for con-tinuing research in the broader area of EBM in general is whether instruction of student physicians in EBM can have measurable effects on physician behavior in the future.

REFERENCES1. Sackett D, Rosenberg W, Gray J, Haynes R, Richardson W.

Evidence based medicine: what it is and what it isn�t. BMJ. 1996;312:71-72.

2. Slawson DC, Shaughnessy AF. Becoming an information master - using POEMs to change practice with confidence. J Fam Pract. 2000;49:63-67.

3. Geyman JP. POEMs as a paradigm shift in teaching, learn-ing, and clinical practice. Patient-Oriented Evidence that Matters. J Fam Pract. 1999;48(5):343-344.

4. Schneeweiss R. Morning rounds and the search for evi-

dence-based answers to clinical questions. J Am Board Fam Pract. 1997;10(4):298-300.

5. Neale V, Roth LM, Schwartz KL. Faculty development using evidence-based medicine as an organizing curricular theme. Acad Med. 1999;74(5):611.

6. Greenhalgh T, Douglas HR. Experiences of general practi-tioners and practice nurses of training courses in evidence-based health care: a qualitative study. Br J Gen Pract. 1999;49(444):536-540.

7. Grad R, Macaulay AC, Warner M. Teaching evidence-based medical care: description and evaluation. Fam Med. 2001;33(7):602-606.

8. Slawson DC, Shaughnessy AF. Teaching information mas-tery: creating informed consumers of medical information. J Am Board Fam Pract. 1999;12(6):444-449.

9. Wadland WC, Barry HC, Farquhar L, Holzman C, White A. Training medical students in evidence-based medicine: a community campus approach. Fam Med. 1999;31(10):703-708.

5