Journal of Academic Ophthalmology

S U B S C R I B E TO DAY

The Journal of Academic Ophthalmology (JAO) is a peer-reviewed journal that serves as a national and international forum for the publication and scholarly exchange of ideas and information of interest to academic ophthalmology, including medical education, resident and fellow training, research in health education, policy, and regulation. The JAO has a diverse editorial board consisting of leaders in the field of academic ophthalmology. The JAO is interested in publishing original education research, literature reviews, and, by invitation or solicitation, editorial, historical, or policy perspectives.

Information about article submissions to the Journal of Academic Ophthalmology may be found in the Journal or on www.academic-ophthalmology.com.

If you have any questions or concerns, please contact the Managing Editor at [email protected].

jaoJournal of Academic Ophthalmology

Publish Your Article:

Journal of Academic Ophthalmology

Aims and Scope: The Journal of Academic Ophthalmology (JAO) is a peer-reviewed journal that serves as a national and international forum for the publication and scholarly exchange of ideas and information of interest to academic ophthalmol-ogy, including medical education, resident and fellow training, and research in health education, policy, and regulation. The JAO has a diverse editorial board consisting of leaders in the field of academic ophthalmology. The JAO is interested in publishing original education research, literature reviews, case reports illustrating ACGME competencies, and, by invitation or solicitation, editorial, historical, or policy perspectives.

Editor-in-Chief Andrew G. Lee, MD

Chair Professor of Ophthalmology, Neurology, and Neurosurgery

Department of Ophthalmology The Methodist Hospital and Weill Cornell Medical College

Houston, TX

Editorial Review Board

The Journal of Academic Ophthalmology is published as an op en issue on the in ternet at www.academic-ophthalmology.com and twice yearly by FEP International, Inc., 941 25th Avenue, #101, Coralville, IA 52241, e-mail [email protected]. Advertising and classifieds inquiries: [email protected] Periodicals postage paid at Houston, TX and additional mailing offices. POSTMASTER: Send address changes to FEP International, Inc., 941 25th Avenue, #101, Coralville, IA 52241 or e-mail [email protected]

Anthony Arnold, MD Program Director Jules Stein Eye Institute UCLA Geffen School of Medicine Los Angeles, CA J.P. Dunn, MD Program Director Wilmer Eye Institute Johns Hopkins Medical Institution Baltimore, MD Steven Gedde, MD Program Director Bascom Palmer Eye Institute University of Miami Miami, FL Karl Golnik, MD Program Director Department of Ophthalmology The University of Cincinnati Cincinnati, OH Marko Hawlina, MD, PhD, FEBO Professor of Ophthalmology University Eye Hospital Ljubljana, Slovenia

Linda S.M. Lippa, MD Director of Ophthalmology Education The Gavin Herbert Eye Institute University of California Irvine, CA Eduardo Mayorga, MD Chair and Program Director Department of Ophthalmology Hospital Italiano de Buenos Aires, Argentina Neil Miller, MD Chief, Neuro-Ophthalmology Division Wilmer Eye Institute Johns Hopkins Medical Institution Baltimore, MD Alfredo A. Sadun, MD, PhD Program Director Doheny Eye Institute University of Southern California Los Angeles, CA Ingrid U. Scott, MD, MPH Professor of Ophthalmology Department of Ophthalmology Penn State College of Medicine Hershey, PA

Tara Uhler, MD Program Director Wills Eye Residency Program Jefferson Medical College of Thomas Jefferson University Philadelphia, PA Nicholas Volpe, MDChair Northwestern University Chicago, IL Editorial Staff Andrew Doan, MD, PhD Managing Editor Patricia Duffel Assistant Managing Editor Brooke Strickland Editorial Assistant

ContentsVolume 5, Number 1, 2012

Editorials

Wrong Site Surgery in Ophthalmology 1Nader Moinfar, MD, MPH and Rafik Zarifa, MS

The Sandwich Fellowship: a Two-Year Follow-Up 6Faazil Kassam, MD, Karim F. Damji MD, MBA, Dan Kiage, MBChB, MMed, Chris Carruthers, MD, MBA, K.H. Martin Kollmann, MBChB, MMed, MD, Diploma in Tropical Medicine and Medical Parasitology, MBA, Anil Khamis, MA, MEd, PhD

Original Articles

Ophthalmology in the Undergraduate Medical Curriculum: Do Chief Residents in Ophthalmology Pursue More Academically Oriented Careers than Their Peers?

10

Jonathan B. Kahn, MD and Joel M. Solomon, MD

A Cataract and Intraocular Lens Surgical Simulation (CISS) Curriculum for Teaching and Assessing Surgical Competency

13

Andrew G. Lee, MD, Karl C. Golnik, MD, Thomas A. Oetting, MS, MD, Hilary A. Beaver, MD, George Saleh, MD, Vinod Gauba, MD, Elizabeth Baze, MD, Michael Mahr, MD

Pre-residency Characteristics Associated with Post-residency Academic Productivity in a Cohort of Ophthalmology Residents at an Academic Institution

24

Shannon Jie Cui Shan, MD, Jiangxia Wang, MS, Emily West Gower, PhD, Neil Richard Miller, MD

Do U.S. Medical Licensure Examination Step I Scores Correlate with the American Academy of Ophthalmology In-Training Ophthalmic Knowledge Assessment Program Examination Scores and American Board of Ophthalmology Written Qualifying Examination Performance?

37

Sowmya Kantamneni and Oscar A. Cruz, MD

Assessment of Student Preferences for Teaching in Ophthalmology Using a Conjoint Analysis Approach

43

Anthony J. King, MD, MMedSci, FRCOphth and Alexander J.E. Foss, MD, MMedSci, FRCOphth

This Journal is supported in part by an unrestricted edu-cational grant from:

A Computer-Assisted Approach for Teaching Third-Year Medical Students to Identify Optic Disc and Fundus Abnormalities

48

Janice G. Lee, BS, Lynn K. Gordon, MD, PhD, Michael B. Gorin, MD, PhD, Sebastian Uijtdehaage, PhD, JoAnn A. Giaconi, MD

A Simple Low-Cost Eye Model for Teaching Ophthalmic Laser Procedures

56

Todd W. Altenbernd, MD, Lorna Grant, MD

Case Report

Bilateral optic neuropathy following amiodarone administration: The case for systems based competency in ophthalmology

61

Shazia Ali, Derrick Pau, MD, Andrew G. Lee, MD


Submissions Information about article submissions to the Journal of Aca-demic Ophthalmology may be found on the following page in the Journal or on http://www.academic-ophthalmology.com. If you have any questions or concerns, please contact the Managing Editor at [email protected]. There is no cost to the authors for publishing in the Journal of Academic Ophthalmology. Subscription The subscription rates are as follows: Within North America, the Institutional Rate is U.S.

$209 and the Personal Rate is U.S. $125 per year. Outside North America, the Institutional Rate is U.S.

$268 and the Personal Rate is U.S. $160 per year. Prices include postage and are subject to change without

notice. Subscriptions may be placed online at h t tp : / /www.academic-ophthalmology.com or e-mail at [email protected].

Reproductions & Permissions Single photocopies of individual articles may be made for personal use as allowed by copyright laws. Permission of the publisher and payment of a fee is required for all other repro-ductions, including multiple or systematic copying, copying for marketing purposes, resale, and all forms of document delivery. Special rates are available for educational institu-tions and non-profit organizations. Permissions may be sought directly from FEP International at [email protected]. Electronic Storage or Usage Permission of the publisher is required to archive, store, or use electronically any material contained in this Journal. No part of this Journal may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, elec-tronic, mechanical, photocopying, recording, or with future technologies without written permission of the publisher. Please contact [email protected] for more information.

Disclaimer Notice The statements and opinions expressed in the Journal are those of the author(s) and are not necessarily those of the Editor(s), the Journal, or the Publisher. The Editors(s) and the Publisher assume no responsibility for any injury and/or dam-age to persons or property as a matter of products liability, negligence, or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the Journal. The information provided in the Journal is for educa-tional purposes only. The practice of medicine has associated risks and complications. The author(s), Editor(s), and the Publisher do not intend for this material to replace proper medical and surgical training. Although the published mate-rial has been reviewed by licensed physicians for accuracy at the time of publication, medicine and the standard of care may change quickly. Physicians are reminded, therefore, that guidelines for care can change, and opinions can be contro-versial. Neither the Publisher, the Journal, the author(s), nor Editor(s) are responsible for deletions or inaccuracies in in-formation or for claims of injury resulting from any such de-letions or inaccuracies. We advise physicians to consult the product labeling and primary research literature before imple-menting any new treatments. Advertising material conforms to ethical standards in medi-cine, and the appearance of advertising does not constitute an endorsement or guarantee by the Editor(s), author(s), or the Publisher. Advertising, Classifieds, & Commercial Reprint Inquiries Reprints of articles are available in quantities of 100 or more. Article reprints may be customized for individual needs. Ad-vertising within the Journal, the Journal Classifieds, and packaging the Journal for promotion are available. Inquiries should be addressed to [email protected].

This paper meets the requirements of ANSI Standard Z39.48-1992 (Permanence of Paper).

About the Journal


Instructions for Authors (http://www.academic-ophthalmology.com) Guidelines: Please follow these guidelines carefully to assist the Editorial Staff. Careful preparation of manuscripts reduces the amount of work for our staff and prevents delays in publishing. Manuscripts should present original, unpublished content not being considered for publication in another journal or elsewhere. If accepted, the text, figures, and data shall not be published elsewhere without the consent of the Editors and Publisher of the Journal. There is no limit on the length of the manuscript; however, the manuscripts are expected to be concise. Sections: Original articles, Reviews, Editorials, Perspectives, Pearls and Pitfalls, Residency Program Director Corner, Ask the Experts, Best Practices from the Field, and Competency Corner (case reports with an emphasis on ACGME core competencies). Research Articles: Manuscripts are original, unpublished works in academic ophthalmology. A research article must include these sections: Title Page, Abstract, Introduction, Methods, Results, Discussion, and References. Fig-ures, Tables, and Acknowledgements should be submitted with the manu-script. Literature reviews, Perspectives, and Editorials: The Editors invite com-prehensive reviews of current topics in the field of academic ophthalmology. The Editor ([email protected]) should be contacted by e-mail in ad-vance with an outline proposal for the topic. Reviews and Editorials are still subject to peer review, and publication is contingent upon acceptance by the reviewers and the Editor. Reviews consist of the following sections: Title Page, Abstract, Introduction, Discussion, and References. If the review con-sists of Results, then the manuscript should include a Methods and Results section. Figures, Tables, and Acknowledgements should be submitted with the manuscript. Other Types of Articles: Authors must submit proposals via e-mail to the Editor ([email protected]) before submission. Manuscript Preparation: Authors must clearly label each section of the manuscript (e.g., Abstract, Introduction, Methods, etc.). Include Figure leg-ends and Tables after the References section. If any figures or data were previously published, then written permission from the publisher of the pre-vious material must be included with the manuscript. It is the authors’ sole responsibility to obtain this permission, and the Journal cannot publish a manuscript until authorization has been acquired. General Guidelines: Do not use unnecessary text styles. Italics are used for genus and species names, gene symbols, and the first three letters of restric-tion enzymes. Bold text is applied only for headings. Do not use underlining. Read your manuscript carefully for spelling and grammar errors. Write for a general audience. Use double-spacing and Times New Roman, 12 point font in the manuscript. TITLE PAGE: All manuscripts should include the title, the names of the authors, the authors’ institutional affiliations, and contact information of the corresponding author. Each author may also include an e-mail address or a web address. ABSTRACT: Abstracts should be concise and summarize the work pre-sented. Research Articles must be structured with these subsections: Purpose, Methods, Results, and Conclusions. All other articles will have descriptive abstracts summarizing the topics covered in the paper. INTRODUCTION: Be concise and provide a general introduction discuss-ing the relevance of the manuscript. Do not use subheadings. Use of relevant literature is essential. METHODS: The Methods should be written with sufficient detail to facili-tate replication of the work presented. Non-proprietary names should be used, and the name of the supplier of items used in the research should be denoted along with their full company name and location (city, state/country). Research using animals should include a statement that animal care guidelines comparable to those published by the Institute for Laboratory Animal Research or the US Public Health Service were followed. If human subjects were used, then indicate that an appropriate institutional review board (IRB) has approved the project. If an IRB is not available, then the authors should follow the principles of the Declaration of Helsinki. RESULTS: The research findings should be presented in the Results without interpretation or discussion. In short manuscripts where the interpretation of the data is relatively simple, the data may be discussed in a combined Results and Discussion section.

DISCUSSION: The discussion should consist of interpretation and commen-tary on the data, results, and findings presented within the scope of the inves-tigation presented in the manuscript. ACKNOWLEDGEMENTS: Acknowledgements for individuals making significant non-authorship contributions to the manuscript are stated in this section. Funding support for the research should be listed. Authors should disclose any commercial interest or companies presented in the manuscript. All prior presentations of the work at meetings should be listed. Do not in-clude these presentations as manuscript references. REFERENCES: The citations in the text are identified by numbers in brack-ets (e.g., [1,5, 12, 20-25]). For users of Endnote, the Journal uses reference formatting consistent with the journal Biology of Reproduction. Citations are numbered in the order of appearance in the manuscript. The references are listed in the References with corresponding numbers to the citations in the text. References to unpublished work should be made parenthetically in the body of the text and not listed in the References section. Examples of formatting for references: Gasymov OK, Abduragimov AR, Yusifov TN, Glasgow BJ. Structural changes in human tear lipocalins associated with lipid binding. Biochim Biophys Acta 1998; 1386: 145-156. Yanoff M, Fine BS. Ocular Pathology. 5th ed. St. Louis, MO: Mosby; 2002. p. 311-329. Cason, SA. “There is no right or wrong.” Mommy MD. http://www.mommymd.org/ Posted Nov. 19, 2006. [Accessed Jan. 1, 2008.] FIGURES: Each figure should be numbered with Arabic numerals and correspond to their sequence of appearance in the manuscript, e.g., “Figure 1”. The Figure legends should be included after the References section. Each Figure legend should have a title and caption with sufficient detail to explain the figure without referring to the main text. Labels and abbreviations must be explained in the caption. We encourage the authors to use color figures if this will facilitate better communication of their results. Figures that are composites of several images should have white spacing between the differ-ent images. All patient identifiers must be removed from photographs and figures before submitting to the Journal, or a written release from the patient is submitted with the manuscript. TABLES: Tables may be submitted in word processor or HTML format, and tables should be numbered with Arabic numerals corresponding to their sequence of appearance in the manuscript, e.g., “Table 1”. Each table should have a title and caption. The caption should have sufficient detail to explain the table without referring to the main text. Digital Files: Text should be submitted in Rich Text Format (.rtf) or Micro-soft Word Format (.doc). All manuscripts will be published in the Journal of Academic Ophthalmology style; thus, any special formatting used by the author(s) may be lost. Figures may be submitted in the following digital formats: TIFF (.tif), JPG (.jpg), and Photoshop (.psd). Figures submitted in Microsoft PowerPoint or Word format are unacceptable and will be returned to the authors for reformatting into one of the accepted formats. When pre-paring figures, diagrams, and charts, the resolution should be at least 300 dpi with a minimum size of 4” x 6”. If you use Photoshop format, then you may create additional layers to mark your figures, but do not flatten the images. Submission of Manuscripts: All submissions must be done via e-mail or FTP to our server ([email protected]). Each submission must include a letter from the corresponding author, manuscript, supporting documents/releases, and Transfer of Copyright (http://www.academic-ophthalmology.com). All communications between the authors and the Journal will be handled by the corresponding author. Each part of the manuscript must be submitted in a separate file (e.g., text, figures, and tables). DO NOT embed figures or tables into the main text of the manuscript. Use the last name of the author and the date (month/year) of submission to name the files. The signed Transfer of Copyright forms must be signed by all authors, scanned, and sent electroni-cally via e-mail ([email protected]) or FAXED (443-817-0794). Author’s Letter to the Editor: The letter should detail the clinical and/or scientific importance of the manuscript. The title, authors, and corresponding author should be indicated along with the contact information for the corre-sponding author. The authors may recommend reviewers and may also re-quest particular reviewers to be excluded from the pool of potential review-ers. Please explain briefly why particular individuals should be excluded from the reviewer pool. Manuscripts are subjected to blinded reviews.

Journal of Academic Ophthalmology 2012, Volume 5, Issue 1

1

Accepted for publication May 25, 2012

Journal of Academic Ophthalmology 2012; 4:1-5Available via open-access on the web at http://www.academic-ophthalmology.com

Financial Disclosure(s): The authors have no personal financial interests in any of the products or technologies cited herein.

©2012 Journal of Academic Ophthalmology

EditorialWrong Site Surgery in OphthalmologyNader Moinfar, MD, MPH, Rafik Zarifa, MSUniversity of Central Florida College of Medicine

Wrong site procedures in medicine remain a public health concern. As part of the spectrum of sentinel events (an unexpected occurrence involving death or serious physical or psychological injury), wrong site surgery was focused upon in 1998 by the Joint Commission as part of a Sentinel Event Alert[1]. In 1998, a review of 15 cases of wrong site, wrong procedure and wrong person surgeries had been re-ported to JCAHO (Joint Commission on Accredita-tion of Healthcare Organizations)[1], and by 2001, this figure had risen to 150 reported cases. Of these 150 cases, 126 have root-cause analysis informa-tion[1] with 42% related to orthopedic/podiatric surgery, 20% to general surgery, 14% to neurologi-cal surgery, 11% percent to urologic surgery, and the remaining to dental/oral maxillofacial, cardio-vascular-thoracic, otolaryngology and ophthalmol-ogy[1]. Furthermore, of these cases, 76% involved surgery on the wrong body part or site, 13% in-volved surgery on the wrong patient, and 11% the wrong surgical procedure[1]. While 81% of these cases were self-reported, and despite the significant media attention given to some of the cases, data from the New York Department of Health and the Florida Board of Medicine actually suggests a sig-nificant amount of under-reporting to JCAHO by health care organizations[1]. At the time of its original Alert in 1998, the JCAHO identified several factors that contribute to an in-creased risk of wrong site surgery. The risk factors identified included: (1) emergency cases (19%);

(2) unusual patient characteristics such as a physi-cal deformity or severe obesity, causing difficulty in patient or equipment positioning (16%); (3) mul-tiple surgeons involved in the care of the same pa-tient (13%); (4) unusual time pressures to start or complete a case (13%); (5) unusual equipment or set-up in the operating room (13%); and (6) mul-tiple procedures being conducted on the same pa-tient either simultaneously, or sequentially, during the same trip to the operating room (10%)[2]. Root-cause analysis of the original fifteen reported cases in 1998 identified communication, preopera-tive assessment of the patient, and procedures used to verify the operative site as the most common contributing factors[2]. Three years later, further root-cause analyses by the JCAHO of a growing database of wrong site surgeries more specifically identified breakdowns in communication between surgical team members and the patient and family as the most common cause of wrong site surgery. The second most common cause identified was pol-icy lapses such as not requiring the marking of sur-gical sites, failure to require operative site verifica-tion in the operating room and the performance of a verification checklist, incomplete patient pre-op-erative assessment, staffing issues, distraction fac-tors, availability of pertinent information on the op-erating room, and organizational cultural issue[1]. The original Sentinel Event Alert by the JCAHO in 1998 included several strategies for reducing the risk of wrong-site surgeries; these included the marking of the operative site and engagement of the patient during this process, requiring an oral verifi-cation of the corrective site in the operating room by each member of the surgical team, development of a verification checklist that includes all docu-ments referencing the intended operative procedure and site, and personal involvement of the surgeon in obtaining informed consent[2]. By 2001, in its follow-up report to wrong-site surgery events, the JCAHO reiterated its aforementioned recommen-


2

dations, while also suggesting that surgical teams consider taking a “time out” in the operating room to identify the correct patient, procedure and site along with using active, rather than passive, com-munication techniques[1]. Paralleling the JCAHO’s continued concern over wrong site surgery as a persistent public health concern, individual profes-sional organizations and regulatory bodies adopted individual policy statements and procedures, with growing punitive features. The New York State De-partment of Health, for example, released its own report from its Preoperative Protocols Panel, as a baseline model that should be applied to all New York State hospitals and ambulatory care centers. These guidelines emphasized enhanced communi-cation among surgical team members, three inde-pendent verifications including marking or identi-fying the correct site, and having the surgeon see and speak with the patient while in the preoperative area[1]. Growing public intolerance of wrong-site surgery also forced action through state agencies and regulatory bodies. In June of 2001, for ex-ample, the Board of Medicine of Florida instituted punitive steps against physicians and organizations involved in wrong site surgery, including fines of up to $10,000, five hours of risk management medi-cal education, fifty hours of community service, and a one hour lecture to the medical community on wrong site surgery[1].In 2008, nine years after the JCAHO’s original Sen-tinel Event Alert regarding wrong site surgery, and four years after the launching of its Universal Pro-tocol, the JCAHO described a sustained increase in the number of reported cases of wrong site surgery in the United States[3]. As reported to the JCAHO’s voluntary Sentinel Event Database, wrong site sur-gery grew to its current rate of eight to 10 new cases per month, and still remains the most frequently re-ported sentinel event in its database; this upward trend was corroborated by similar date from states with mandatory reporting systems for medical er-rors that include wrong site surgery[3]. Other stud-ies reported that wrong site surgeries conducted on limbs or organs (other than the spine) occurred in one in every 112,994 operations[4] and that un-der optimal conditions, application of the Univer-sal Protocol would have prevented 62% of these cases[5]. While the JCAHO acknowledges that the growing trend of wrong site surgeries could repre-sent a reflection of expanded reporting, the public health magnitude of this problem had already pro-

mulgated the convening of a second Wrong Site Surgery Summit in 2007 which sought, in part, to identify other potential strategies for eliminating wrong site surgery[3]. The 2007 summit reiterated that the three primary components of the Universal Protocol are effective if properly implemented and consistently followed, but that cultural transforma-tions are also needed, which evolve over decades and multiple generations[3]. As such, wrong site surgery remains a recognized public health problem with both unique and shared key determinants and stakeholders across the various medical specialties and practice settings.Subspecialty analyses correlating wrong site sur-gery with specialty type shows various results. Ag-gregate analyses, for example, of several databas-es, including the National Practitioner Data Bank (NPDB), the Florida Code 15 mandatory reporting system, and the American Society of Anesthesiolo-gists (ASA) Closed Claims Project database suggest the highest numbers of wrong site surgeries occur-ring among orthopaedic and dental procedures[6]. Other studies associate ophthalmology and radiol-ogy with the greatest overall association of adverse events associated with incorrect surgical procedures and orthopedics second only to ophthalmology for number of reported adverse events occurring spe-cifically in the operating room[7]. Further analyses focusing on orthopaedics, for example, suggests that wrong site surgeries involve the wrong side in 59% of cases, another wrong site (for example, the wrong digit but on the correct hand) in 23% of cases, the wrong procedure in 14% of cases, and the wrong patient in 5% of cases; anatomically, the most frequent locations were the knee (35%), fin-gers and/or hand (35%), foot/ankle (15%), distal femur (10%) and the spine (5%)[8]. Interspecialty differences in attitudes towards adoption of the Universal Protocol have also been described; or-thopaedists, ophthalmologists and urologists have various reported attitudes concerning the value of preoperative site markings, with widely variable practice patterns[9]. Furthermore, surgeons across these varying specialties place different values on the importance of preoperative site markings, while some even believe it actually increases the risk of wrong site surgery[9]. In addition to believ-ing that the Universal Protocol adds significantly to operative times, some surgeons report avoiding preoperative markings out of concern for breaches of sterility by the marking pens, despite data that

Wrong Site Surgery in Ophthalmology - Zarifa & Moinfar


3

this concern appears to be unfounded[10]; concern does, however, still remain about the possibility of skin markers being erased with chlorhexidine-based skin preparation solutions[11] and thus contributing to wrong site surgery. The uniform application of preoperative site markings is also impacted by the proportionate representation of different specialties in various practice settings, and the size of a partic-ular surgical team and the delegation of authority in preoperative markings. In academic centers, larger surgical teams such as orthopaedics or neurosur-gery may include different team members such as interns, residents, fellows, and attending surgeons, any one of whom may be expected to perform pre-operative site markings, but may not necessarily be the individual most familiar with the patient. This raises concerns as to who should mark patients in general, as well as marking surgical sites for genital surgery or other private body sites where the indi-vidual marking the site may not be the person most familiar to the patient[12].As a subspecialty, ophthalmology appears to show relatively low adherence to preoperative site mark-ings. Noncompliance among ophthalmologists with site markings has been reported to be only 50%, with poor adherence to preoperative checklist pro-tocol as the most frequent contributing factor[13]. Despite this, ophthalmic procedures have unique inherent features that would appear to reinforce the importance of preoperative site markings and ad-herence to Universal Protocol. A very small surgi-cal field precludes observation of laterality by an operative team, and draping very frequently masks site markings or even distinguishing features of laterality, such as the nose. Also, preoperative ret-robulbar anesthetic blocks are often administered outside of the operating room, where the entire operating team is not yet collectively present and the Universal Protocol not yet invoked. Further-more, some procedures require multiple steps on the same eye, such as extraocular muscle surgeries, which may be performed unilaterally or bilaterally; strabismus surgery is at further risk for wrong site surgery in that each of the eye’s six muscles may be modified differently (recessesed, resected and/or transposed), and that general anesthesia often pro-motes realignment of a patient’s preoperative stra-bismus, further confusing identification of the af-fected eye and operative site at the time of initiating surgery. Root-cause analyses of wrong site surgery in ophthalmology has also implicated preoperative

clinic notes as a key determinant, where left/right transposition errors in laterality are propagated in clinical notes and drawings[14], culminating in transpositional errors on the informed consent, and finally the surgical markings. Despite inherent dif-ferences with other surgical specialties, ophthal-mology would likely enjoy the same benefits from application of the Universal Protocol as with other specialties. Retrospective series from the Ophthal-mic Mutual Insurance Company and the New York State Health Depart-ment involving 106 cases of surgical confusion in ophthalmic procedures implicated wrong eye op-erations in 15 cases, wrong eye anesthetic block in 14 cases, wrong patient or procedure in 8 cases, and wrong intraocular lens implantation in 67 cases; it is believed, however, that using the Universal Pro-tocol would have prevented this confusion in 90 cases (85%)[15]. New initiatives focused on reducing rates of wrong site surgery in ophthalmology must start with find-ing ways to increase compliance of the Universal Protocol. Efforts made at developing facility-wide policies specific for each institution to increase compliance of the Universal Protocol have proven to be very successful. An example is a study con-ducted by a healthcare team at Naval Hospital, Cherry Point which resulted in 100% compliance of the Universal Protocol within 4 months of new policy implementation and can serve as a blueprint for other healthcare teams[16]. New methods to de-crease wrong site surgeries have been introduced by various organizations in new studies to supplement JCAHO’s Universal Protocols; many of these meth-ods may have a significant impact on the field of ophthalmology in particular. With the many unique inherent features that ophthalmic procedures pres-ent, some of these new methods may help address the features that increase the risk of wrong site sur-gery. One of these methods is the application of a recently developed standardized anatomic marking form in conjunction with site markings. The ana-tomic marking form is filled out by the surgeon and the patient at the time the surgical decision is made and is confirmed by the preoperative intake unit (preferably prior to retrobulbar block). This form has been implemented with significant success, re-sulting in no cases of wrong site surgery in over 112,500 surgical procedures when the form was used correctly. The form was also highly accepted



4

by surgeons, staff, and patients[17]. Creation of a form tailored specifically to ophthalmic procedures may yield similar results. Another method is the use of an extended surgical time-out that is supple-mentary to the Universal Protocol’s time out and in-cludes verification of surgical procedure, technical and anesthetic details, administered and available medications, blood product availability, and the need for special equipment. The extended surgical time out has been shown to improve communica-tion among surgical team members and allowed for identification of equipment deficiencies[18]. The extended surgical time out may also be tailored specifically for the field of ophthalmology where it could serve an even greater advantage due to the heavy reliance on highly specialized surgical tools and equipment in its procedures. The extra time af-forded in the extended surgical time out may also allow for increased identification of incorrectly selected intraocular lens implants prior to surgi-cal insertion. Another method is the utilization of medical team training which requires surgeons and all staff involved in surgical patient care to attend learning sessions addressing wrong site surgery with an emphasis on conducting preoperative and postoperative briefings, both guided by a checklist. The Veterans Health Administration implemented medical team training in all of its facilities which lead to a decrease from a rate of 3.21 surgical ad-verse effects per month (2001 to Mid-2006) to 2.40 surgical adverse effects per month (Mid-2006 to 2009)[19]. A medical team training program that is specific to ophthalmology, addresses the inherent unique features of ophthalmic procedures, and cov-ers root-cause analysis of ophthalmic wrong site surgeries may yield similar results if implemented effectively. Wrong site surgeries do not appear to have decreased since their description by the Joint Commission in 1998. Despite evidence-based data supporting the role of the Universal Protocol in reducing the risk of wrong-site surgeries, compli-ance amongst surgeons varies, with relative low adherence amongst ophthalmologists. Ophthalmic procedures possess certain unique features that may place them at higher risk for wrong site errors. These include initiation of procedures (retrobulbar anesthetic blocks) outside of the operating room, relatively small surgical field with obscuration of proportionately smaller site markings and anatomic landmarks, the common use of drawings in ophthal-mic clinic notes and the risk of transposition errors,

and the relative high volume of surgeries performed by a single surgeon at a given time using multiple operating rooms simultaneously. Despite these fea-tures, there does appear to be a reasonable amount of evidence-based data in support of the use of the Universal Protocol and new supplemental protocols in mitigating these features and reducing the inci-dence of wrong site surgery in ophthalmology.

References

1. The Joint Commission, Sentinel Event Alert, “A follow-up of wrong site surgery.” Issue 24, Decem-ber 5, 2001.

2. The Joint Commission, Sentinel Event Alert, “Les-sons learned: wrong site surgery.” Issue 6, August 28, 1998.

3. The Joint Commission, “Statement and persistence of the problem.” November 24, 2008.

4. “Wrong-site surgeries seen as rare, prevent-able.” Healthcare Benchmarks Qual Improv. 2006;13(7):79-81.

5. Kwaan MR, Studdert DM, Zinner MJ, Gawande AA. “Incidence, patterns and prevention of wrong-site surgery.” Arch Surg. 2006;141(4):353-57.

6. Seiden SC, Barach P. “Wrong-side/wrong-site, wrong-procedure, and wrong-patient adverse events: are they preventable?” Arch Surg. 2006;141(9):931-39.

7. Neily J, Eldridge N, Dunn EJ, Samples C, Turner JR, Revere A, DePalma RG, Bagian JP. “Incorrect surgical procedures within and outside of the oper-ating room.” Arch Surg. 2009;144(11):1028-34.

8. Wong DA, Herndon JH, Canale ST, Brooks RL, Hunt TR et al. “Medical errors in orthopaedics. Results of an AAOS member survey.” J Bone Joint Surg Am. 2009;91(3):547-57.

9. Giles SJ, Rhodes P, Clements G et al. “Experiences of wrong site surgery and surgical marking practices among clinicians in the UK.” Qual Saf Health Care. 2006;15(5):363-68.

10. Zhao X, Chen J, Fang XQ, Fan SW. “Surgical site marking will not affect sterility of the surgical field.” Med Hypotheses 2009;73(3):319-20.

11. Mears SC, Davani AB, Belkoff SM. “Does the type of skin marker prevent marking erasure of surgical site markings?” Eplasty. 2009(Sept 9);9:e36.

12. Goldberg AE, Harnish JL, Stegienko S, Urbach DR. “Attitudes of patients and care providers toward a surgical site marking policy.” Surg Innov. 2009 Sep;16(3):249-57.



5

13. White MM, Gupta M, Utman SA, Dhillon B. “Im-portance of side marking in ophthalmic surgery.” Surgeon. 2009;7(2):82-85.

14. Elghrably I, Fraser SG. “An observational study of laterality errors in a sample of clinical records.” Eye. 2008;22(3):340-3.

15. Simon JW, Ngo Y, Khan, S, Strogatz D. “Surgical confusions in ophthalmology.” Arch Ophthalmol. 2007;125(11):1515-22.

16. Ludwick S. Surgical Safety: Addressing the JCA-HO Goals for Reducing Wrong-site, Wrong-patient, Wrong-procedure Events. in Henriksen K, Battles JB, Markes ES, Lewin DI, ed. Advances in Patient Safety: from research to implementation (Volume 3, Implementation Issues). Rockville, MD: Agency for Healthcare Research and Quality, 2005.

17. Knight N, Aucar J. Use of an anatomic marking form as an alternative to the Universal Protocol for Pre-venting Wrong Site, Wrong Procedure and Wrong Person Surgery. Am J Surg. 2010; 200(6):803-807; discussion 807-809.

18. Lee SL. The extended surgical time-out: does it im-prove quality and prevent wrong-site surgery? Perm J. 2010;14(1):19-23.

19. Neily J, Mills PD, Eldridge N, et al. Incorrect Sur-gical Procedures Within and Outside of the Op-erating Room: A Follow-up Report. Arch Surg. 2011;146(11):1235-39.



6





The Sandwich Fellowship is an educational mod-el whereby training occurs in sequential rotations both at the fellow’s home institution and a devel-oped world institution in order to develop subspe-cialty expertise and leadership skills. This takes place in conjunction with institutional capacity building to provide sustainable development and prevent brain drain. Since the publication of the original model, continued support and experience with the program has given new insights into some of the refinements needed in order to improve the quality of the fellowship, but as well opportunities to overcome barriers to implementation. Through a rapidly changing landscape of medical and sur-gical practice, it has become evident that more learner-centered approaches are necessary to train fellows from the developing world. Standards and specific goals of the fellowship should be reflected on in the context of the fellows future geographical practice, while keeping in mind standards defined

EditorialThe Sandwich Fellowship: a Two-Year Follow-UpFaazil Kassam, MD*1, Karim F. Damji MD, MBA2, Dan Kiage, MBChB, MMed3, Chris Carruthers, MD, MBA4, K.H. Martin Kollmann, MBChB, MMed, MD, Di-ploma in Tropical Medicine and Medical Parasitology, MBA5, Anil Khamis, MA, MEd, PhD6

1Division of Ophthalmology, University of Calgary, Calgary, Alberta, Canada; 2Department of Ophthalmology, University of Alberta, Edmonton, Alberta, Canada; 3Ophthalmological Society of Eastern Africa, Ophthalmology, Department of Surgery, Aga Khan University Hospital Nairobi, Kenya; 4The Ottawa Hospital, Ottawa, Ontario, Canada; 5University of Nairobi, Kenya; 6Faculty of Policy and Society, Institute of Education, University of London, England

* Corresponding author e-mail: [email protected]

by guiding bodies used to train subspecialists in the developed world. Significant interest in the model has been garnered from prospective fellows and de-veloping world institutions including a pilot by the International Council of Ophthalmology, however, what remains as the largest challenge is a shortage of preceptors and developed world institutions who are willing to enter into these mutually beneficial partnerships. Innovative solutions are required to advertise and advocate the model, and this could in-clude non-governmental organizations and human resource companies whose exclusive role would be to match interested parties in the developing and developed world in order to reduce administrative burden. The authors strongly believe the model is an effective means of spurring sustainable sub-specialty care in the developing world with great benefit to the developed world institution, but more champions are needed to truly make a long term and global impact.Since the original publication of The Sandwich Fellowship[1], there have been a variety of interna-tional fora to present and discuss the model. This, along with continued experience has given new in-sights into some of the learner centered approaches needed to improve the quality of the fellowship. Additionally, there have been new opportunities for advocacy and to overcome barriers such as admin-istration and funding that would encourage its wide scale adoption.


7

Sandwich Fellowship - Damji et al.

Description of Model

The educational model aims to help resolve human resource constraints as part of the greater Millenni-um Development Goals framework[2]. The unique term ‘Sandwich’ refers to both the fellow receiv-ing training in cumulative layers as well as being situated between diverse geographic locations. This approach develops sustainable subspecialty care in the developing world by focusing on two main components: Firstly, strengthening institutional ca-pacity (i.e. equipment, space, human resources) at the home institution of the fellow in order to be able to practice their acquired skills effectively and ef-ficiently. This fosters an enabling environment of continued professional growth, which is instrumen-tal in retaining individuals and preventing brain drain. Secondly, training in layers with rotations in the developed world and developing world allows preparation in their own environment on relevant pathology. It is important to note that benefit is bidirectional between the two institutions as pre-ceptors who travel to train the fellow experience professional growth, and partnerships are formed that can facilitate the exchange of knowledge and skills. Further, the model can be generalized to sub-specialty training in any area of medicine and fa-cilitates the same benefits of retention and capacity growth. The proposed rotation schedule involves five ro-tations over a period of approximately one year, but the duration of rotations can be tailored to the area of subspecialty and extended if needed. Ro-tation 1 involves the potential preceptor visiting the home institution of the candidate to familiar-ize themselves with the needs of the institution, negotiate details of necessary capacity building for the subspecialty service, as well as assess the abili-ties of the fellow. Rotation 2 (3 to 4 months) oc-curs at the institution in the developed world and focuses on developing clinical and surgical skills as well as management and leadership qualities to better understand the environment needed to de-velop the subspecialty service. During rotation 3 (3 to 4 months), the candidate returns home and con-tinues to build on knowledge and skills acquired, while also collecting patients for a structured visit by the preceptor. This allows the fellow to receive guidance on pathology relevant to their future geo-graphic practice, but also allows the preceptor to meet with key decision makers regarding progress

in equipment, space, and staff infrastructure. Ro-tation 4 (4 to 6 months) of the fellowship returns to the developed world institution where clinical and surgical exposure is consolidated to ensure knowledge and skills are at the level expected for graduation. Finally, the fellow returns home for Rotation 5 to become a leader in their department, while maintaining ties with the training institution through collaboration, mentorship, research, case discussions etc.[1]

Lessons Learned

Medical practice patterns have evolved in their own geographic contexts, adapting to differences in populations, resources, and healthcare systems. It has become evident that more learner centered approaches are needed to ensure objectives are tai-lored to their geographical practice while consid-ering guiding principles from bodies used to train subspecialists in the developed world. This flexibil-ity should not affect the central principle of obtain-ing the highest standards for training. For example, a recent Sandwich graduate in glaucoma at the Uni-versity of Alberta (UOA) would not have benefit-ted from training to perform combined phacotra-beculectomy surgeries when it is not commonly performed at her home in Kenya due to high costs. Further, conditions such as congenital glaucoma are rare in Canada. It was decided to create a sub-rota-tion in India, which was able to provide experience in congenital glaucoma and manual small-incisioncataract surgery. This also strengthened the fel-lowship concept as it encouraged South-South col-laborations in addition to existing North-South col-laborations. These factors should be accounted for during objectives development.An unexpected strength of the model has been the benefit of training in the digital age. As the concept emerges in education of two types of individuals, digital natives (those who developed in the techno-logical revolution think and learn differently) and digital immigrants (developed before the revolu-tion)[3,4], technology should be integrated into all aspects of training. As the proportion of digital natives increases over time around the world, this means integrating entities for fellows such as simu-lation and ongoing exchanges of digital cases. An example has been a previous fellow initiating and a preceptor joining an African Glaucoma Listserve.


8

Suggestions for Improvement

Various platforms have allowed advocacy of the model and it has generally been very well received. Thus far, it has been successfully used to train 5 fel-lows who have since returned to practice full time at their home institutions, and an additional 2 are mid way through their training program (Table 1). One positive aspect of this has been the interest generated from several potential Sandwich Fel-lows in orthopedics and ophthalmology who have contacted the authors. However, what remains challenging is a shortage of preceptors and devel-oped world institutions willing to enter into these mutually beneficial partnerships. Assuming train-ing capacity exists, innovative solutions are needed to overcome the two main barriers: administrative burden and funding. Independent parties such as human resource companies and non-governmental organizations could serve to match interested in-stitutions and catalyze partnerships by reducing the initial administrative burden. However, many of the logistical constraints rest with outside orga-nizations such as governments and college bodies and this unpredictability can make taking a fellow from the developing world less attractive. These challenges were highlighted recently as a Sandwich

Fellow from Ethiopia to start at the University of Alberta was postponed due to licensing delays. The key to overcoming these barriers is to begin pro-cesses early (e.g. a year in advance).Another challenging area is obtaining funding for the fellowship. A multitude of small funding sourc-es is more feasible as the burden may be too large to be placed entirely on one of the two institutions. The recent graduate’s funding included her home institution at the University of Nairobi (UON) as well as the Royal Alexandra Hospital Foundation, but the fellowship was also supported by the East African College of Ophthalmologists, ORBIS, and the International Council of Ophthalmology. Of special mention is the role of the UON providing support for education, but as well putting in place resources including a visual field machine, operat-ing room, laser, and slit-lamp for use upon return. With guidance from the UOA, UON converted in-patient to day surgery, which will increase efficien-cy in performing cases. The success of the model hinges on this institutional capacity component and will allow our graduate to provide glaucoma services for many who desperately need it. Often, Canadian fellows are able to generate revenues from their services, and these are usually in excess of their salary. An innovative way to reduce reli-

Table 1: Past and present ‘Sandwich’ fellows

Fellow Home Institution Developed world Institution

Year of Completion

Specialty/Subspecialty

Dr. Dan Kiage

Aga Khan University Hospital Nairobi, Kenya

University of Ottawa 2009 Ophthalmology/glaucoma

Dr. Wafula Khamala

Aga Khan University Hospital Nairobi, Kenya

University of Ottawa 2009 Orthopedics/total joint surgery

Dr. Abeba Giorgis

Addis Abeba Uni-versity, Ethiopia

Wills Eye Hospital/Thomas Jefferson University

2009 Ophthalmology/glaucoma

Dr. Sheila Marco

University of Nairobi, Kenya

University of Alberta 2011 Ophthalmology/glaucoma

Dr. Olusola Olawoye

University of Ibadan, Nigeria

New York Eye and Ear Infirmary

2012 Ophthalmology/glaucoma

Dr. Fisseha Admassu Ayele

Gondar University, Ethiopia

University of Alberta In-progress Ophthalmology/glaucoma

Dr. Girum W. Gebreal Gessesse

Jimma University, Ethiopia

University of Alberta In-progress Ophthalmology/glaucoma



9

ance on external funding is to dedicate the income generated from Canadian fellows into an endow-ment fund reserved for training future international fellows. Other options for funding include service clubs (e.g. Rotary), industry, non-governmental or-ganizations, and government bodies.

Future Vision

The long-term vision is to lead the uplifting of health care professionals surrounding the newly trained individual, e.g. have a multiplier effect. Upon return, the fellow(s) possesses expertise that over time can be used to train future subspecialists; the hope is for the fellow is to strengthen already existing residency programs. Additionally, con-tributing to undergraduate education will advance primary care leading to earlier detection and treat-ment. It is through this powerful ripple effect that modest investment in subspecialty training can im-prove quality of life for an entire region. In order to evaluate the future success of the model, it will be crucial to develop metrics by which to measure impact on the region in which graduates serve e.g. number of surgeries performed, complications, research publications, influence on residency and medical student curricula, etc. as well as conduct graduate tracer studies.We believe the Sandwich Fellowship is a success-ful model and provides the needed platform to cre-ate sustainable subspecialty services for the devel-oping world. We ask for additional champions to come forward in order to achieve a long term global impact.

Acknowledgements

The authors would like to thank Dr. Sheila Marco, recent Sandwich Fellow graduate in glaucoma at the University of Alberta for her feedback in a structured interview on the model.

References

1. Kassam F, Damji KF, Kiage D, Carruthers C, Koll-mann KH. The Sandwich Fellowship: a subspecialty training model for the developing world. Acad Med. 2009;84(8):1152-60.

2. UN Millennium Project 2005. Investing in Development: A Practical Plan to Achieve the Millennium Development Goals. Overview. Available at: http://www.unmillennium project.org/documents/overviewEngi-1LowRes.pdf.


3. Prensky M. Digital Natives, Digital Immigrants. Available at: http://marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf. Accessed July 25, 2011.

4. Prensky M. Digital Natives, Digital Immigrants, Part II: Do They Really Think Differently? Available at: http://marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part2.pdf. Accessed July 25, 2011.


10

Ophthalmology in the Undergraduate Medical Curriculum: Do Chief Residents in Ophthalmology Pursue More Academically Oriented Careers than Their Peers?Jonathan B. Kahn, MD*1 and Joel M. Solomon, MD1

1Department of Ophthalmology, New York University School of Medicine, New York, NY

*Corresponding author e-mail: [email protected]

Abstract

Purpose: To determine whether serving as chief resident is a predictor of academic accomplishments after residency training.

Methods: A seven question survey was collected from graduates of the New York University and Manhattan Eye, Ear & Throat Hospital ophthalmology residencies from 1989 to 2009. Respondents’ answers were stratified into groups of “chief resident” or “peer resident,” and responses were compared between groups.

Results: Chief residents were more likely to have pursued a post-graduate fellowship (95.7% vs. 87.0%, P = 0.10), have published a journal article (78.3% vs. 68.3%, P = 0.19) and have presented a poster or honorary lecture (73.9% vs. 61.0%, P = 0.15) since completing residency. Chief residents were also more likely to have maintained an academ-ic appointment (91.3% vs. 73.9%, P = 0.03), although their peers were more likely to hold a professional leadership position (39.1% vs. 46.7%, P = 0.28).

Conclusions: Ophthalmologists who were chief residents at a single training program had a greater tendency to pur-sue post graduate fellowships, publish journal articles, present posters or honorary lectures, and maintain academic appointments, although their peers had a greater tendency to hold professional leadership positions. These data were only statistically significant for maintaining an academic appointment. Surveying a larger subject pool may allow ad-ditional data to reach statistical significance.

Accepted for publication July 29, 2011

Journal of Academic Ophthalmology 2012; 4:10-12 Available via open-access on the web at http://www.academic-ophthalmology.com

Financial Disclosure(s): The authors have no personal financial interests in any of the products or technologies cited herein..


Introduction

Many ophthalmologists believe that serving as chief resident has a direct impact on their subsequent ca-reers. Specifically, chief residents are thought to pursue more academically oriented careers as com-pared to their peers. There are a number of explanations for this be-lief. Some feel that the chief resident experience

provides strong preparation for future academic leadership. The qualities that make a good chief resident may also make for a successful academic ophthalmologist. Still others assert that the prestige of the chief residency may itself offer more post-residency academic opportunities.Similar studies that have evaluated the leadership careers of former residents have been published in the pediatric and family practice literature. In one study of pediatric residents, Alpert et al. describe that professional leadership was more associated with residents who were former chief residents and/or fellows, and males[1]. Other studies have report-ed that the chief residency is an excellent prepara-tion for future medical leadership roles[2] and for preparing excellent teachers[3]. We explored the careers of past residents from our ophthalmology training program over the course of


11

Chief Residents in Ophthalmology - Kahn & Solomon

21 years. We compared the academic achievements of those who had been chief residents to those of their peers. This is the first study on chief residents in ophthalmology.

Methods

All graduates from a single residency program (New York University / Manhattan Eye, Ear & Throat Hospitals) were identified via a departmen-tal database. Graduates from years 1989 to 2009 were selected for study. A seven-question survey was collected from each graduate via e-mail, postal mail, facsimile, or telephone. (See Figure 1.) All questions used in analysis were binary “yes / no” type questions. Respondents’ answers were strati-fied into groups of “chief resident” or “peer resi-dent,” and percentages of respondents answering “yes” to each question was compared among the groups. Statistical evaluation of results was per-formed via a single tailed, two-sample unequal, students’ t-test.

Results

One hundred seventeen graduates were contacted. A total of 69 responses were received, yielding a response rate of 59.0%, with a slight response bias toward more recent graduates. Twenty-three respondents (33.3%) served as chief resident. Chief residents were more likely to have pursued a post-graduate fellowship (95.7% vs. 87.0%, P = 0.10), have published a peer-reviewed journal article (78.3% vs. 68.3%, P = 0.19) and have presented a poster or honorary lecture (73.9% vs. 61.0%, P = 0.15) since completing residency. Chief residents were also more likely to have maintained an academic appointment (91.3% vs. 73.9%, P = 0.03), although their peers were more likely to hold a professional leadership position (39.1% vs. 46.7%, P = 0.28).

Discussion

Chief residents of our ophthalmology program were more likely to have pursued post graduate fel-lowships, published peer-reviewed journal articles, presented posters or honorary lectures, and main-tained academic appointments, although their peers were more likely to have held professional leader-ship positions. This study has several important implications.

First, the data support the notion that chief residents tend to pursue more academically-oriented careers as compared to their peers. The assumptions made are that the rubrics measured, namely pursuit of fel-lowships, publishing of journal articles, presenta-tion of posters or honorary lectures, maintenance of academic appointments, and maintenance of professional leadership positions, are a sufficient cross-section of those factors in one’s career that indicate involvement in academia. In fact, there are a myriad of additional measures which may be used to evaluate a physician’s academic career, includ-ing involvement in resident instruction, medical student education, education of allied health per-sonnel, development of new technologies, and clin-ical and bench research, among others. The rubrics chosen for this study were intended to be a cross-sectional representation of those parameters we felt most strongly represented academic involvement. We chose to study a simplified set of measures to ensure sufficient physician participation in the re-search.Second, the data do not demonstrate statistical sig-nificance in most parameters used for comparison. Rather, the data are suggestive of the notion that chief residents tend to pursue careers with a more academic focus as compared to their peers. The strongest correlation was of maintaining an aca-demic appointment, although the small sample size obtained from a single institution limit the statisti-

Figure 1: Seven question survey collected from each study participant.

1. What year did you graduate NYU/MEETH?

2. Were you chief resident at NYU/MEETH?[ y / n ]

3. Did you pursue a sub-specialty fellowship?[ y / n ] What sub-specialty, and where did you study?________________

4. Have you published any journal articles since completing residency?[ y / n ]

5. Have you presented any posters or honorary lectures since completing residency?[ y / n ]

6. Do you currently, or have you in the past, maintained an academic appointment?[ y / n ] Where? _________________________________

7. Do you maintain a professional leadership posi-tion[ y / n ] If so, in what role?_________________________


12

Chief Residents in Ophthalmology - Kahn & Solomon

cal significance of the results. Future studies of this topic could include larger sample sizes by studying careers of ophthalmologists from many different training centers; this may reveal additional statisti-cal significance in the comparison parameters. Finally, the data shown do not indicate a causal re-lationship. Namely, we still have yet to understand why chief residents may pursue different empha-ses in their careers as compared to their peers. Is there an impression that is instilled during chief residency that affects later career choices? Or are chief residents pre-disposed to pursue academics in their careers by the qualities that lead to their selec-tion for the position? Becoming a chief resident may indicate the selection of individuals who are already recognized as being leaders of their resi-dent groups, by virtue of their accomplishments, clinical skills, teaching abilities, and personalities. Their subsequent career pursuits may be reflections of these previously demonstrated qualities. There are several important limitations to our study. This study incorporates a smaller sample size than prior similar studies, although the smaller size of ophthalmology training programs as compared to those of pediatrics and family medicine likely ac-count for this difference. Additionally, a limited number of parameters were used to measure career leadership in academia; further research could fo-cus on other measures of academic achievement (previously described) that were not included in this analysis. Furthermore, it is possible that there may be a self-reporting bias in these data. Those who re-sponded may be more likely to consider themselves academic leaders and this could have resulted in an overstatement of the overall academic leadership role in the study. Similarly, those who had served as chief residents may have also been more likely to have responded, possibly overstating their role as academic leaders in the field. Further, chief resi-dents take on varying roles in different training programs. While chief residents in our program

serve during their third year of residency training, many chief residents serve during a dedicated post-residency year. The functions of our chief residents include teaching, leading, scheduling and coordina-tion; functions of chief residents in other programs may vary.This is the first study of chief residents in ophthal-mology. Further study across a broader sample size and inclusive of additional training programs would enhance the depth of knowledge in this area.

References

1. Alpert JJ, Levenson SM, Osman CJ, James S. Does Being a Chief Resident Predict Leadership in Pedi-atric Careers? Pediatrics 2000; 105:984-88.

2. Mygdal WK, Monteiro M, Hitchcock M, et al. Out-comes of the First Family Practice Chief Residency Leadership Conference. Fam Med. 1991; 23:308-10.

3. Wright SE, Kern DE, Kolodner K, Howard DM, Brancati FL. Attributes of excellent attending-physi-cian role models. N Engl J Med. 1998; 339:2015-17.

Table 1: Comparison of percentages of Chief Residents and Peer Residents answering “yes” to each survey question. Asterisk denotes statistical significance.

Chief Residents Peer Residents P valueFellowship 95.7% 87.0% 0.10

Journal Article 78.3% 68.3% 0.19

Poster / Lecture 73.9% 61.0% 0.15

Academic Appointment 91.3% 73.9% 0.03*

Leadership Position 39.1% 46.7% 0.28


13

A Cataract and Intraocular Lens Surgical Simulation (CISS) Curriculum for Teaching and Assessing Surgical CompetencyAndrew G. Lee, MD,*1,2 Karl C. Golnik, MD,3 Thomas A. Oetting, MS, MD,4 Hilary A. Beaver, MD,1 George Saleh, MD,5 Vinod Gauba, MD,6 Elizabeth Baze, MD,7 Michael Mahr, MD8

1The Methodist Hospital (TMH), Houston, Texas and 2the Departments of Ophthalmology, Neurology, and Neuro-surgery at the Weill Cornell Medical College, 3the University of Cincinnati, 4the University of Iowa Hospital and Clinics and the Veterans Affairs Medical Center, Iowa City, IA (Adjunct Professor: AGL), 5The National Institute for Health Research (NIHR) Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, 6Imperial Healthcare Institute and Dubai Medical College, Dubai Healthcare City, 7the Department of Ophthalmology, Baylor College of Medicine, Houston, Texas (Adjunct Professor: AGL), and 8the Department of Ophthalmology, Mayo Clinic, Rochester, MN


Abstract

Purpose: To describe a cataract and intraocular surgical simulation (CISS) teaching and learning curriculum for eye residents.

Methods: Systematic literature review, selection of “best practices”, and expert derived recommendations for de-veloping a standardized surgical simulation curriculum. We review the capabilities of a single existing commercial simulator (EyeSi).

Results: We recommend the following components for pairing with the existing metrics of the simulator: pre- and post-test of cognitive knowledge base; an objective structured assessment of high fidelity and low fidelity simulation based skills and techniques; development of cognitive and technical failure scenarios, and formative and summative global evaluation forms.

Conclusion: A cataract and intraocular lens surgical simulation (CISS) curriculum may form the basis for success-fully utilizing the existing apprentice based surgical teaching, the traditional wet lab, and the new simulation technolo-gies to both teach and assess aspects of resident surgical competence in intraocular surgery.


Journal of Academic Ophthalmology 2011; 5:13-23

Available via open-access on the web at http://www.academic-ophthalmology.com

Disclosure/disclaimer: Although Dr. Lee is a member of the ACGME Residency Review Committee (RRC) in ophthalmol-ogy, the views represented in this paper are those of the authors and do not represent the opinions or policies of the ACGME or the RRC.Dr. Markus Schill from VRmagic the maker of the EyeSi sur-gical simulator provided consultative and technical advice for this manuscript. None of the authors however have any finan-cial interest in EyeSi or any other simulation technology.


Introduction

In the United States, the Accreditation Council for Graduate Medical Education (ACGME) has man-dated that all residency training programs (includ-ing ophthalmology) teach and assess six general competencies (i.e., medical knowledge, commu-nication and interpersonal skills, patient care, pro-fessionalism, practice based learning, and systems based practice)[1,2]. Other national organizations have discussed the possibility that surgery be split from “patient care” to become the seventh compe-tency. It is our opinion that although “competence” in the practice of medicine can be deconstructed into these ACGME “silos” represented by the general competencies, in reality every single competency


14

CISS Curriculum - Lee et al.

is present in every patient encounter in the clinic, the hospital, or in the operating room. Likewise, the concept of “surgical competence” is not simply about mastering surgical knowledge and technique (i.e., medical knowledge and patient care) but also includes achieving competence in non-technical domains of surgical judgment, professional de-meanor in the operating room (i.e., professional-ism), obtaining appropriate informed consent (i.e., communication and interpersonal skills), keeping abreast of cutting edge techniques, analyzing the evidence base and the surgeon’s own individual surgical outcomes (i.e., practice based learning and improvement), and calling upon and utilizing ap-propriately the resources of the system of care (i.e., systems based practice).The teaching and assessing of resident ocular sur-gical performance in live patients poses special educational, ethical, and patient safety concerns. Obviously the traditional apprenticeship model for teaching ocular surgery, while still valuable, should ideally not be the first or only surgical experience for the novice surgeon in training. Performance of a resident surgery in a live patient can create troubling ethical, medical, cost effectiveness, and potentially medico-legal issues. These include disclosing and managing conflicts of interest and defining in the informed consent the details of faculty involvement or supervision. For these and other reasons the num-ber and quality of surgical cases available to eye residents for surgical training is at risk. Economic realities and time constraints on supervising faculty and operating room times and efficiency (e.g., am-bulatory surgery center) have further reduced the available surgical pool for ideal resident cases and many ophthalmic residency programs now “out-source” a percentage of their cataract surgical train-ing to out of state or out of country experiences[3]. In order to partially address the surgical, technical, medical, ethical, economic, and practical supervi-sion issues of live resident surgery, the Residency Review Committee (RRC) in ophthalmology has mandated a wet lab experience. We previously re-viewed the literature on wet lab teaching, gleaned “best practices” from the evidence base, and pub-lished a potential implementation matrix for a wet lab curriculum[4]. In this manuscript we report on a similar systematic literature review of ocular surgi-cal simulation; a selection of potential “best prac-tices”; and a content-expert derived surgical simu-lation curriculum.

Methods

A multicenter Task Force was formed to develop a list of best practices for a learning curriculum for surgical simulation that might potentially in-clude: pre- and post-testing, a qualitative criterion referenced global evaluation form and behavioral narrative anchored scoring rubric, specific cogni-tive failure scenarios, and quantitative simulation scoring metrics and milestones. Face validation for the learning curriculum content was provided by a consensus of the members of the Task Force which included supervising academic faculty surgeons for cataract surgery at the participating academic insti-tutions (HAB, TO, GS, VG, GP, EM, MM), Depart-ment chairmen (AGL, EM), program directors of ophthalmology (TO, KG), content experts for the ACGME competencies for ophthalmology (AGL, KG) and industry experts (VRmagic, EyeSi). The process and curriculum were also validated with content experts solicited from the Program Direc-tors’ Multicenter Educational Research Group (PD-MERG) via email for public comment. A literature review (PubMed, English language, 1966-2009) was performed to define existing cur-riculum materials for surgical simulation in oph-thalmic surgery. The content experts (AGL, KG) reviewed the titles and reviewed selected articles of relevance to the development of a simulation cur-riculum[1-67]. A pre- and post-test was provided by one manufacturer of an ophthalmic surgical simula-tor (VRmagic and EyeSi) for evaluation. Clinical content experts were asked to assess the instrument for content validity and to define the cognitive and technical skills tested in the simulation. Previously an objective skills assessment tool and global eval-uation form for the surgical experience was also constructed by the content experts and externally validated by the Task Force and international ex-perts in the field of cataract surgery[5].

Results

In previous work we have proposed a core strategy for implementing the ACGME mandate (Table 1) and the same principles were applied for the devel-opment of the cataract and intraocular lens surgi-cal simulation (CISS) curriculum. Our proposed simulation curriculum includes the following rec-ommendations: 1) Use explicit, written learning objectives; 2) Deploy a pre-test (i.e., a needs as-sessment to define the educational gap to identify


15


Table 1: Examples of surgical simulation courses from EyeSi

Cataract CoursesCAT-A Beginners

Instrument Handling

Capsulorhexis

Phaco Concepts

Reliability Course

Challenge Course, appears every 60 min.

CAT-B Advanced

Instrument Handling

Capsulorhexis

Phaco Concepts

Challenge Course

Reliability Course, appears every 40 min.

Table 2: The proposed components of the simulation curriculum

1. Pre- and post-test of cognitive knowledge base

2. An objective structured assessment of high fi-delity and low fidelity simulation based skills and techniques using the existing formative and summative feedback metrics of the existing commercial instrument

3. Development of cognitive and technical failure scenarios for testing with the simulator

4. Use of a modified scoring rubric from existing live surgery formats (e.g, ICO OSCAR)[5]

5. Maintain simulation metrics in a portfolio for self reflection and programmatic analysis of time in simulator and other simulation metrics

6. Formative and summative global evaluation assessments of performance benchmarked against individual and pooled aggregate pro-grammatic data

the entry level for the learner) and a post-test (an assessment of how effective the intervention was at closing the gap), 3) Maintain self-assessments in portfolio; and 4) Document a global evaluation with a criterion based scoring rubric, qualitative and quantitative metrics, and milestones for learner achievement over time. We have emphasized pre-viously and continue to emphasize the importance of both formative and summative feedback in the learning process. Table 2 lists the summary compo-nents of the proposed CISS curriculum.As described in our previous work[4] we rely heav-ily on three models modified for educational use in the development of the CISS curriculum: 1) the Dreyfus expertise acquisition model, 2) Schön self-reflection model, and 3) Ericsson’s “deliber-ate practice” model. Previous work has described the advantages of a criterion referenced scale over traditional “peer-based or norm” referenced scales. The Dreyfus model of expertise has been used pre-viously in industry, education, and other academic applications. Briefly, in the Dreyfus model, learn-ers move through stages of expertise (i.e., novice, beginner, advanced beginner, proficient-competent, and expert)[6-8]. The Dreyfus model provides the framework for developing the ophthalmology spe-cific behaviors that in turn define the quantitative scoring rubric. The rubric allows for objective de-construction of the task at hand and can be used to provide both formative and summative feedback to the resident. The formative component allows for the opportunity for learner improvement, self-re-

flection, and documentation of milestone achieve-ment over time. The specific behavioral narrative anchors in the rubric provide raters with objective benchmarks for comparative purposes and provides the learners with specific targets for behavioral change[6-8]. Likewise the tenants of the Schön re-flection model may have some applicability to oph-thalmic education. We believe that the concepts and philosophy of self-reflection before, during and af-ter each surgical encounter improves learner under-standing and motivation for life-long learning[9]. The model itself incorporates “reflection on action” (e.g., recognizing the need for further knowledge and learning), “reflection in action” (e.g., utiliz-ing practice based learning techniques, review and synthesize available evidence based medicine); and “knowledge in action” (e.g., application of learn-ing in the real world patient context)[9]. The main practical application of the Schön model is portfo-lio based documentation of learner self-reflection and self-assessment typically using a learner initi-ated project. The CISS curriculum should also emphasize the Ericsson deliberate practice model[10,11]. In the Ericsson model, expertise aligned with progres-sion along a milestone based Dreyfus scale is achieved not just by simple practice (e.g., “Go to the wet lab and practice.”) but by deliberate prac-tice (e.g., “Go to the wet lab and practice your rota-tional hand movement technique during continuous


16


Figure 1: Evaluation screen from EyeSi (courtesy of VRmagic)

capsulorhexis.”) In this model, the terms “deliber-ate practice” are defined as purposeful task repeti-tion, internal and external analysis of performance metrics, and formative and summative feedback with individual refinement of surgical technique hopefully with both peers (i.e., residents as teach-ers) and faculty mentors/role models[10,11]. In the EyeSi simulator, learners receive feedback when a “sentinel event” occurs (e.g., touching the retina while performing an ILM peeling). The evaluation screen in the simulator after each training session provides both learner and their teacher with forma-tive and summative feedback on the surgical per-formance of the session: Task scores based upon completion of task objectives and incorporating penalty points for sentinel events (e.g., for produc-ing ocular injuries or for inefficient use of instru-ments). For a structured task evaluation, scores for task objectives and sentinel events are sorted into 5 categories: (1) target achievement, (2) efficiency, (3) instrument handling, (4) microscope handling and (5) tissue treatment. While individual scoring

criteria may be different over different training tasks they are always subsumed under these same categories. (Figure 1). We recommend learner documentation in a portfo-lio of stepwise performance and milestone comple-tion and the use of a narrative anchored, criterion referenced, and scoring rubric using the Dreyfus model. Specific simulation milestones have been developed in existing simulators that align with prescribed specific Dreyfus level benchmarks of skill and expertise. Achieving the predefined score is required before proceeding to the next Drey-fus level or simulation task. The required score is called a “gate.” These gates to the next level can be modified by the faculty mentor according to the level of skill of their learners and the expected pace of the learner. The gate system also allows the faculty trainer to monitor longitudinally the trainee’s performance, to benchmark residents’ performance internally, and to document improve-ment over time. Test-retest conditions called “Re-liability Gates” require trainees to reach a defined


17

CISS Curriculum - Lee et al..

score several times in a row before moving to the next level. These test-retest sessions promote and document high training intensity and are designed to produce reproducible results (i.e., metrics for de-liberate practice). At the beginner level the micro-scope is reset after each task and the trainees’ goal is to familiarize themselves with the handling of the microscope in simulation. Preparatory basic skills training tasks include circular anti-tremor-training paths, and various capsulorhexis setups with for-giving, low-tension capsules. In the instrument handling modules, learners have the opportunity to recognize and use ophthalmic instruments, illumi-nation and the operating microscope. In the basic simulations it is also possible to create focused ex-ercises that are designed to test a single, basic skill, while eliminating other variables and minimizing other distractions that can occur in the live patient setting. For example, in the Anti-Tremor Training, module the trainee practices a circular motion with the instruments, allowing them to concentrate on their hand movement without having to interact with tissue. In the basic mode, surgical skill tasks are like-wise deconstructed and simulation modules are designed to improve psychomotor hand, eye and foot coordination. A basic technical skill example would be recognizing the three dimensional spa-tial orientation inside the eye. These basic simula-tion exercises hopefully lead to improved surgeon reaction times, reduced excess movements of the hands and tremor, a shortened learning curve, and reduced time between conscious surgical intention and actual manual execution of the desired brain-eye-hand coordination. In the Forceps Training module, learners become familiar with handling the ocular forceps in the eye and the tasks include arranging geometric objects while trying to avoid unnecessary motion of the instrument. In the Cap-sulorhexis Training Levels 1 and 2, a well-defined flap is offered, so novices can focus on completing a full circular capsulorhexis only. In the Advanced Course, the manually more demanding cracking & chopping techniques are added to the basic skills training tasks and during capsulorhexis simulation more sensitive, high-tension capsules have to be handled. In the advanced courses, the microscope setting remains unchanged and advanced students are expected to focus on the more demanding as-pects of the surgical procedures. For example, in the Advanced Vitreoretinal Course, the learner has

to use the non-dominant hand. In the Reliability Course, the required scores and levels of difficulty of tasks are set at higher levels to demonstrate “con-sistently high level of performance.” The Advanced Reliability Course does not offer any guidance or immediate feedback to trainees and mistakes are only reported in the evaluation after each task has been completed. Advanced courses also train using the non-dominant hand and bimanual maneuvers (e.g., membrane peeling techniques or cracking and chopping during phacoemulsification surgery). In the Advanced Cataract Course trainees need to save a peripheral capsulorhexis that has started to run out to the periphery. Two hydrodissection and -delineation procedures and two cracking & chop-ping trainings foster an advanced understanding of instrument movements and lens interactions. After cracking a pre-sculpted lens, a complete divide & conquer procedure has to be performed including sculpting, cracking and removing both a soft and a moderately hard lens. High fidelity mocks that look and feel like “real world” surgical machines will likely need to be developed to align the surgical simulation with the actual machine being used at the specific institu-tion. Currently, the EyeSi simulator has a generic phacoemulsification machine interface that is not specific for a particular manufacturer. In the Phaco Training of the Cataract Course, learners explore the settings of the phacoemulsification machine and can test and observe the simulated intraocular effects of changing the machine parameters before proceeding to the next module (e.g., Phaco Divide & Conquer). Hydrodissection and hydrodelineation and two Bimanual Basic Skills Training include a basic understanding of instrument movements and lens interaction. After removing a lens quarter and cracking a pre-sculpted lens, the complete divide & conquer procedure can be performed including sculpting, cracking and removing a “soft” lens. A training history summary provides a more de-tailed list of the recorded parameters for a close analysis of individual training requirements and benchmarking to individual performance over time, to individuals within and across postgraduate years, and to programmatic metrics over several cohorts of learners. The existing metrics from the simulator include completion time, tissue damage, precision and parsimony of movement and remaining in or out of focus. The summative “Training Status Re-port” provides a condensed view on the training


18


status of the trainee by listing up to eight prior sim-ulation task results. The Proficiency Graph shows the development of the course score over training time. Each dot in the graph represents an attempt on a training task of the course. The height of the dot indicates the total training score reached at that time. The slope of the graph indicates the rate of improvement.Solverson et al. evaluated the EyeSi ophthalmic surgical simulator’s ability to differentiate “novice” performance (e.g., residents, interns, and nonmi-crosurgical ophthalmic staff) from “expert” perfor-mance (i.e., practicing ophthalmic microsurgeons) using the basic navigational microdexterity mod-ule[9]. Expert surgeons showed a greater initial fa-cility with all microsurgical tasks but with repeated practice novice surgeons showed sequential im-provement in all performance scores, approaching but not equaling expert performance. These authors concluded that VR simulator performance could be used as a gated, quantifiable performance goal to expert-level benchmarks and that this simula-tor (EyeSi) was a valid part-task training platform that may help develop novice surgeon dexterity to expert surgeon levels. A typical scoring display is seen in Figure 2.Mahr and Hodge compared the performance of the anterior segment forceps and anti-tremor training modules of the EyeSi surgical simulator by resi-dents and experienced attending surgeons using the simulator for the first time at Mayo Clinic[12]. Twelve eye residents and 3 experienced anterior segment surgeons participated. Each participant completed 20 task trials on the EyeSi forceps and anti-tremor level 4 training modules and the 15 par-ticipants completed a total of 300 task trials. For the forceps module, the experienced surgeons achieved better total scores (P = 0.03), with lower total task time (P = 0.007) and instrument-in-eye time (P = 0.006) measurements. For the anti-tremor module, experienced surgeons achieved significantly bet-ter total scores (P = 0.02), with lower task time (P = 0.04) and instrument-in-eye time (P = 0.02) measurements. Experienced surgeons performing the anti-tremor task had 76% more precise surgi-cal outcomes as measured by the out-of-tolerance percentage (P = 0.03). All forceps and anti-tremor-measured parameters indicated significantly lower performance (P < 0.05) for the first 1 or 2 trials, with the exception of anti-tremor module incision stress, out-of-tolerance percentage, and average

tremor values. Experienced surgeons had more consistent (lower variance) total scores on the for-ceps (P = 0.05) and anti-tremor (P = 0.03) training modules. In this study the EyeSi surgical simulator anterior segment forceps and anti-tremor modules showed significant (P < 0.05) construct validity. Saleh et al. assessed motion analysis as a discrimi-nator of ophthalmic plastic surgical skill between surgeons of varying experience[13]. Thirty subjects were divided into 3 groups based on surgical expe-rience: novice (< 5 performed procedures; n = 10), intermediate (5-100 procedures; n = 10), and expert (> 100 procedures; n = 10). Detailed 3-dimensional motion data from surgeons performing 2 oculoplas-tic surgical tasks on a wet laboratory skills board were obtained using the Qualisys motion capture system. The first task was a deep 3-1-1 suture. The second was skin closure with a continuous suture. The main outcome measures were time, overall path length, and total number of movements. Krus-kal-Wallis analysis was performed to evaluate sta-tistical significance. Highly significant differences were found during the skin closure task between all groups for mean time (P = 0.002), overall path length (P = 0.002), and number of movements (P = 0.001). For the deep stitch, highly significant dif-ferences were also found for time (P < 0.001), path length (P < 0.001), and number of movements (P < 0.001). These authors concluded that motion analy-sis was able to differentiate between surgeons of varying experience performing oculoplastic tasks and that the technique showed construct validity. Webster et al. described a technique for simulat-ing the capsulorhexis procedure during cataract surgery on the EyeSi system and demonstrated that the technique could be deconstructed into observ-able and measurable components[14]. Laurel et al. showed that a simulator for phacoemulsification could shorten the learning curve and could improve performance and reduce sentinel events when mov-ing from the dry lab simulation to the wet lab for pars plana vitrectomy[15]. Feudner et al. showed that a well-designed training curriculum with EyeSi improves the capsulorhexis wetlab performance on pigs eyes significantly versus the control group[16]. Gill et al. found a correlation between an objective assessment of intraocular surgical skill (with OA-SIS and GRASIS) and the performance scores us-ing the EyeSi ophthalmic surgical simulator[17].The American Board of Ophthalmology (ABO) and the Accreditation Council for Graduate Medi-


19


Figure 2: Scoring display (courtesy of VRmagic)

cal Education (ACGME) are charged with develop-ing and documenting the systematic assessment of surgical competence of ophthalmology residents at all residency programs. The goals, while integrated, are not the same for these organizations; the ABO must certify individual surgeons as “competent” and the ACGME must accredit programs to train competent surgeons. In the current system, pro-grams (through their program directors and chairs) are asked to “sign off” on individual competence for individual surgeons to the ABO. Clear, explicit, de-fined and validated metrics will be necessary if this

“validation” letter from a program about an individ-ual surgeon is to have true meaning. These metrics can and should be aligned with the achievement of specific milestones that can be benchmarked on an individual basis as well as pooled into aggregate outcome measures for programmatic comparisons for accreditation.The concept of objective metrics for surgical out-comes in live surgery has already been published by others (e.g., Objective Assessment tool of Skills in Intraocular Surgery (OASIS)[18,19] and OASIS can be linked to an adjunctive one-page subjective


20


evaluation form, the Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Likewise a wet lab curriculum, the Iowa OWL curriculum and a full ophthalmic surgical curriculum also from Iowa have been published[4]. The development of these curricular materials for ophthalmic surgi-cal training is the important first step in develop-ing milestones for assessing competence in surgery for accreditation of programs and certification of individuals. The true and ultimately most power-ful outcome of the ACGME outcome project in our opinion however will be alignment of the metrics in the simulation and wet lab with real world metrics that benefit patients. These external, data driven benchmarks could include patient safety, learner and patient satisfaction, reduction of medical error, and improvements in quality of care or cost effec-tiveness. Rogers et al. performed a retrospective review of third-year ophthalmic resident quality-assurance surgical outcomes data at a single resi-dency-training site (the University of Iowa) from 1998 to 2008[8]. The primary outcome measure was defined as a “sentinel event”: posterior capsule tear (with or without vitreous loss) or vitreous loss (from any cause) occurring during a resident-per-formed case. The study population was divided into 2 groups. Group 1 comprised surgical cases of resi-dents trained before the surgical curriculum change (academic years 1998 to 2003) and Group 2, surgi-cal cases of residents trained with the enhanced cur-riculum (academic years 2004 to 2008). In Group 1 (before institution of surgical curriculum), there were 823 cases with 59 sentinel complications. In Group 2 (after institution of surgical curriculum), there were 1009 cases with 38 sentinel complica-tions. There was a statistically significant reduc-tion in the sentinel complication rate, from 7.17% before the curriculum changes to 3.77% with the enhanced curriculum (P = 0.001, unpaired 2-tailed t-test). These authors concluded that implementa-tion of a structured surgical curriculum resulted in a statistically significant reduction in sentinel event complications, even after adjusting for surgical ex-perience.

Discussion

The teaching and assessment of ophthalmic surgi-cal competency is a high stakes, difficult, and de-manding educational task. The “burden of proof” for individual surgical competence in the ABO in-dividual certification process currently resides with

individual program directors and chairs of ophthal-mology programs. It is our opinion however that the accreditation of programs via the ACGME and the ophthalmology RRC does not have sufficient individual assessments or metrics for certification of competence in a specific resident. Certification of individual surgical competence will likely re-quire new methods of establishing minimum surgi-cal competence and documentation of progression through specific Dreyfus based milestones under direct supervision and eventually autonomously in a graduated fashion. If program directors are to be responsible for defining surgical competence indi-viduals for certification purposes then a clear and explicit, milestone-benchmarked surgical training curriculum for training must be implemented and validated across programs. In addition, the current teaching model (i.e., apprenticeship model with live patients) is fraught with ethical, patient safety, technical, and practical problems. Although we be-lieve that the RRC requirement for a wet laboratory experience was an important first step for insuring programmatic competence for teaching purposes the establishment of individual surgeon assessment metrics has yet to occur. Likewise, we believe that simulation technology offers an additional opportu-nity for reducing the risks to live patients by mov-ing some of the training and assessment to the vir-tual environment. Khalifa et al. in a review of the surgical virtual re-ality stated that “current training models are lim-ited by an unstructured curriculum, financial costs, human costs, and time constraints” and that “with the newly mandated resident surgical competen-cy, training programs are struggling to find viable methods of assessing and documenting the surgi-cal skills of trainees[20].” In order for the simula-tion techniques to achieve more widespread use we believe that a milestone based simulation curricu-lum needs to be tested in a multicenter fashion that would include validation of the proposed simula-tion and operating room metrics, use of a narrative anchored, criterion-based scoring rubric, formative and summative global feedback, structured and scheduled (not ad hoc) lab and course work, direct faculty participation, and clear and defined sentinel event markers and benchmarks. The capsulorhexis is considered the most difficult step to master by junior residents. Recent work by Privett et al. suggests a correlation between the EyeSi and real operative experience for capsu-


21


lorhexis[21]. In this study of the EyeSi capsulor-rhexis module experienced surgeons (defined as having performed over 200 cases) but who were naive to the EyeSi performed better on a variety of machine metrics related to capsulorhexis compared to medical students who were also naive to the Eye-Si but not proficient in real surgery. This study pro-vides construct validity to this important module of the EyeSi simulation system[21].We believe that the surgical training curriculum should include a set of resources that will allow progression of the learner. The simulator will be an important addition to this set of resources that include web based resources, video, a porcine or cadaveric wet lab, and most importantly live surgi-cal instruction first with parts and then the entire cases. Early in training the simulator and wet lab will be more important than later when the resident is fine tuning skills in the OR. With the present simulation technology certain steps of the cataract procedure are better suited for the wet lab such as forming an incision or suturing. Other parts such as capsulorhexis and centration of the eye while using both hands seem to be well suited for the current technology of simulators. Of course there is no sub-stitute for the eventual transition to the live patient with an experienced and patient surgical instructor. We recognize the limitations of our work including 1) the preliminary nature of the proposed simula-tion curriculum; 2) the lack of multi-institutional experience; and 3) the lack of documented linkage with and quantitative or qualitative improvement in actual surgical outcomes in real patients. All of these limitations will need to be addressed before simulation can be a prescribed requirement for ophthalmic training. Despite these limitations we believe that simulation is indeed part of the future for ophthalmic training and assessment of surgical competency. The use of a standardized curriculum such as CISS allows for accreditation of teaching programs and certification of learner competence and more importantly can be benchmarked against downstream patient related outcomes in safety, quality, and cost. We believe that in the future demonstration of alignment of simulation metrics with downstream outcome measures needs to be performed to pro-duce a validated structured, simulation curriculum that could be applicable across all eye residencies. We would envision a scoring rubric perhaps modi-

fied from existing assessment tools from live sur-gery that can align the surgical simulation metrics with a real world Dreyfus based scoring rubric[5].

Acknowledgements

Dr. Markus Schill from VRmagic the maker of the EyeSi surgical simulator provided consultative and technical advice for this manuscript. Funding/support: This work was supported in part by an unrestricted grant from Research to Prevent Blindness, Inc., N.Y., N.Y.

References1. Lee AG, Carter KD. Managing the new mandate in

resident education: a blueprint for translating a na-tional mandate into local compliance. Ophthalmol-ogy 2004;111:1807-12.

2. Lee AG. The new competencies and their impact on resident training in ophthalmology. Surv Ophthal-mol 2003;48:651-62.

3. Rowden A, Krishna R. Resident cataract surgical training in United States residency programs. J Cat Refract Surg 2002;28:2202-05.

4. Lee AG, Greenlee E, Oetting TA, Beaver HA, John-son AT, Boldt HC, Abramoff M, Olson R, Carter KD. The Iowa Ophthalmology Wet Laboratory Cur-riculum for Teaching and Assessing Cataract Surgi-cal Competency. Ophthalmology. 2007;114(7):e21-6.

5. Golnik KC, Beaver H, Gauba V, Lee AG, Mayorga E, Palis G, Saleh GM. Cataract surgical skill assess-ment. Ophthalmol 2011;118(2):427.e1-5.

6. Brenner P. Using the Dreyfus model of skill acquisi-tion to describe and interpret skill acquisition and clinical judgment in nursing practice and education. Bull Science Technol Soc 2004;24:188-99.

7. Batalden P, Leach D, Swing S, Dreyfus H, Drey-fus S. General competencies and accreditation in graduate medical education. Health Aff (Millwood) 2002;21:103-11.

8. Rogers GM, Oetting TA, Lee AG, et al. Impact of a structured surgical curriculum on ophthalmic resi-dent cataract surgery complication rates. J Cataract Refract Surg 2009;35:1956-60.

9. Solverson DJ, Mazzoli RA, Raymond WR, et al. Virtual reality simulation in acquiring and differen-tiating basic ophthalmic microsurgical skills. Simul Healthcare 2009;4:98-103.


22


10. Oetting TA. Surgical competency in residents. Curr Opin Ophthalmol 2009;20:56-60.

11. Grodin MH, Johnson TM, Acree JL, Glaser BM. Ophthalmic surgical training:a curriculum to en-hance surgical simulation. Retina 2008;28:1509-14.

12. Mahr MA, Hodge DO. Construct validity of anterior segment anti-tremor and forceps surgical simulator training modules:attending versus resident surgeon per-formance. J Cataract Refract Surg. 2008;34:980-85.

13. Saleh GM, Gauba V, Sim D, Lindfield D, Borhani M, Ghoussayni S. Motion analysis as a tool for the evaluation of oculoplastic surgical skill:evaluation of oculoplastic surgical skill. Arch Ophthalmol. 2008;126(2):213-16.

14. Webster R, Sassani J, Shenk R, Good N. A haptic surgical simulator for the continuous curvilinear capsulorhexis procedure during cataract surgery. Stud Health Technol Inform 2004;98:404-06.

15. Laurell CG, Söderberg P, Nordh L, Skarman E, Nor-dqvist P. Computer-simulated phacoemulsification. Ophthalmology. 2004;111(4):693-98.

16. Feudner EM, Engel C, Neuhann IM et al. Vir-tual reality training improves wet-lab performance of capsulorhexis: results of a randomized, con-trolled study. Graefes Arch Clin Exp Ophthalmol 2009;247:955-63.

17. Gill HS, Sit M, Noble J et al. An assessment of the ability of surgical simulation to predict actual surgical proficiency in ophthalmology. Open Medi-cine 2009;3(3):http://www.openmedicine.ca/article/view/353/269

18. Cremers SL, Lora AN, Ferrufino-Ponce ZK. Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Ophthalmology 2005;112:1655-60.

19. Cremers SL, Ciolino JB, Ferrufino-Ponce ZK, Henderson BA. Objective Assessment of Skills in Intraocular Surgery (OASIS). Ophthalmology 2005;112:1236-41.

20. Khalifa YM, Bogorad D, Gibson V, Peifer J, Nuss-baum J. Virtual reality in ophthalmology training. Surv Ophthalmol 2006;51(3):259-73.

21. Privett B, Greenlee E, Rogers G, Oetting T. Con-struct Validity of the EyeSi Surgical Simulator as a Valid Model for Capsulorhexis Training. J Cataract Refract Surg. 2010;36(11):1835-38.

22. Rowden A, Krishna R. Resident cataract surgical training in United States residency programs. J Cat Refract Surg 2002;28:2202-5.

23. Nielsen PE, Foglia LM, Mandel LS, Chow GE. Objective structured assessment of technical skills for episiotomy repair. Am J Obstet Gynecol

2003;189;1257-60.

24. Goff BA, Lentz GM, Lee D, et al. Development of a bench station objective structured assessment of technical skills. Obstet Gynecol 2001;98:412-16.

25. Goff BA, Nielsen PE, Lentz GM, et al. Surgical skills assessment:a blinded examination of obstet-rics and gynecology residents. Am J Obstet Gynecol 2002;186:613-17.

26. Reznick RK. Teaching and testing technical skills. Am J Surg 1993;165:358-61.

27. Dunnington GL, Wright K, Hoffman K. A pilot experience with competency-based clinical skills assessment in a surgical clerkship. Am J Surg 1994;167:604-6.

28. Cuschieri A, Francis N, Crosby J, Hanna GB. What do master surgeons think of surgical competence and revalidation? Am J Surg 2001;182:110-6.

29. Reznick R, Regehr G, MacRae H, Martin J, Mc-Culloch W. Testing technical skill via an inno-vative “bench station” examination. Am J Surg 1997;173:226-30.

30. Scott DJ, Valentine RJ, Bergen PC, Rege RV, Lay-cock R, Tesfay ST, Jones DB. Evaluating surgical competency with the American Board of Surgery In-Training Examination, skill testing, and intraop-erative assessment. Surgery 2000;128:613-22.

31. Bilimoria K, Abbott D, Wayne J. How to design a pilot system for tracking adverse and near miss events in surgical practice. RAP article at http:www.facs.org/education/rap/bilimoria.html (last accessed on September 22, 2005).

32. McCloy R, Stone R. Science, medicine, and the future. Virtual reality in surgery. BMJ 2001;323:912-15.

33. Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery:results from randomised controlled studies using the MIST-VR laparoscopic simulator. Stud Health Technol Inform 1998;50:124-30.

34. Taffinder N, Smith S, Mair J, Russell R, Darzi A. Can a computer measure surgical precision? Reli-ability, validity and feasibility of the ICSAD. Surg Endosc 1999;13(suppl 1):81.

35. Darzi A, Datta V, Mackay S. The challenge of ob-jective assessment of surgical skill. Am J Surg 2001;181:484-86.

36. Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J. Assessment of technical surgical skill. Eur J Surg 2002;168:139-44.

37. Moorthy K, Munz Y, Dosis A, Bello F, Darzi A. Motion analysis in the training and assessment of


23


laparoscopic surgery. Min Invas Ther Allied Technol 2003;12:37-42.

38. Moorthy K, Munz Y, Sarker SK, Darzi A. Objec-tive assessment of technical skills in surgery. BMJ 2003;327:1032-37.

39. Kopta JA. An approach to the evaluation of opera-tive skills. Surgery 1971;70:297-303.

40. Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg 1999;177:28-32.

41. Ross DG, Harris CA, Jones DJ. A comparison of operative experience for basic surgical trainees in 1992 and 2000. Br J Surg 2002;89(suppl 1):60.

42. Skidmore FD. Junior surgeons are becoming deskilled as result of Calman proposals. BMJ 1997;314:1281.

43. Darzi A, Smith S, Taffinder N. Assessing opera-tive skill. Needs to become more objective. BMJ 1999;318:887-88.

44. Martin JA, Regehr G, Reznick R, MacRae H, Mur-naghan J, Hutchison C, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273-78.

45. Bridgewater B, Grayson AD, Jackson M, Brooks N, Grotte GJ, Keenan DJ, et al. Surgeon specific mor-tality in adult cardiac surgery:comparison between crude and risk stratified data. BMJ 2003;327:13-17.

46. Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of check-lists and global rating scales for assessing perfor-mance on an OSCE-format examination. Acad Med 1998;73:993-97.

47. Datta V, Chang A, Mackay S, Darzi A. The relation-ship between motion analysis and surgical technical assessments. Am J Surg 2002;184:70-73.

48. Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to ob-jectively measure open surgical skill in the labora-tory-based model. J Am Coll Surg 2001;193:479-85.

49. Smith SG, Torkington J, Brown TJ, Taffinder NJ, Darzi A. Motion analysis. Surg Endosc 2002;16:640-45.

50. Francis NK, Hanna GB, Cuschieri A. The perfor-mance of master surgeons on the advanced Dundee endoscopic psychomotor tester:contrast validity study. Arch Surg 2002;137:841-44.

51. Emam TA, Hanna GB, Kimber C, Cuschieri A. Dif-ferences between experts and trainees in the motion pattern of the dominant upper limb during intracorpo-real endoscopic knotting. Dig Surg 2000;17:120-23.

52. Satava RM, Cuschieri A, Hamdorf J. Metrics for objective assessment. Surg Endosc 2003;17:220-26.

53. Szalay D, MacRae H, Regehr G, Reznick R. Using operative outcome to assess technical skill. Am J Surg 2000;180:234-37.

54. Datta V, Mandalia M, Mackay S, Chang A, Cheshire N, Darzi A. Relationship between skill and outcome in the laboratory-based model. Surgery 2002;131:318-23.

55. Hanna GB, Frank TG, Cuschieri A. Objective as-sessment of endoscopic knot quality. Am J Surg 1997;174:410-13.

56. Darzi A, Datta V, Mackay S. The challenge of ob-jective assessment of technical skill. Am J Surg 2001;181:484-86.

57. Spencer CF. Teaching and measuring surgical tech-niques-the technical evaluation of competence. Bull Am Coll S 1978; 63:9-12.

58. Evans AW. Assessing competence in surgical den-tistry. Br Dent J 2001;190:1-6.

59. Williams RG. The 10 commandments of perfor-mance appraisal provide a foundation for the sur-gical training process. At http:www.facs.org/educa-tion/rap/williams.html (last accessed on September 22, 2005).

60. Kohls-Gatzoulis JA, Regeher G, Hutchinson C. Teaching cognitive skills improves learning in sur-gical skills courses:a blinded, prospective, random-ized study. Can J Surg 2004;47:277-83.

61. Anastakis DJ, Wanzel KR, Brown MH, et al. Evalu-ating the effectiveness of a 2-year curriculum in a surgical skills center. Am J Surg 2003;185:378-85.

62. Feldman BH, Ake JM, Geist CE. Virtual reality sim-ulation. Ophthalmology 2007;114:828.

63. Henderson BA, Neaman A, Kim BH, Loew-enstein J. Virtual training tool. Ophthalmology 2006;113(6):1058-59.

64. Webster R, Sassani J, Shenk R et al. Simulating the continuous curvilinear capsulorhexis procedure during cataract surgery on the EyeSi system. Stud Health Technol Inform 2005;111:592-95.

65. Jonas JB, Rabethge S, Bender HJ. Computer-assist-ed training system for pars plana vitrectomy. Acta Ophthalmol Scand 2003;81(6):600-4.

66. Wagner C, Schill M, Hennen M, Männer R, Jen-dritza B, Knorz MC, Bender HJ. Virtual real-ity in ophthalmological education. Ophthalmologe 2001;98(4):409-13.

67. Hikichi T, Yoshida A, Igarashi S, Mukai N, Harada M, Muroi K, Terada T. Vitreous surgery simulator. Arch Ophthalmol 2000;118(12):1679-81.


24

Pre-residency Characteristics Associated with Post-residency Academic Productivity in a Cohort of Ophthalmology Residents at an Academic Institution Shannon Jie Cui Shan, MD1, Jiangxia Wang, MS2, Emily West Gower, PhD1, Neil Richard Miller, MD*1 1The Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland; 2 De-partment of Biostatistics, The Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland.

*Corresponding author email: [email protected]

Accepted for publication July 24, 2011Submitted May 12, 2011




Introduction

Academic medicine is in a time of profound change. Recent decades have witnessed a steady decline in the number of physician-scientists throughout med-icine[1-3]. The field of ophthalmology has been no exception. Academic ophthalmology in the United Kingdom is already experiencing critical workforce shortage[4]. In the United States, certain subspe-cialties such as neuro-ophthalmology and pediatric ophthalmology are also witnessing fewer physicians entering the field[5,6]. The need for a steady supply of future generations of academic ophthalmologists is paramount, as they often play an integral role in sustaining and advancing the field through research and education. Although the majority of residency programs in the country aim to train comprehen-sive ophthalmologists, a number of programs have also made training academic ophthalmologists one of their priorities. A primary goal for the resident

selection committee at these academically oriented programs is to identify the residency applicants most likely to enter academic career tracks and lead academically productive careers. A recent survey by Nallasamy et al. found that the factors deemed most important in resident selec-tion include interview performance, clinical course grades, letters of recommendation, and board scores[7]. Of these, both clinical course grades and board scores are measures of cognitive abili-ties, which often also form the basis for the letters of recommendation. Cognitive measures tradition-ally have been used in the resident selection pro-cess, yet the validity of this practice has been un-der scrutiny[8,9]. Studies in various medical fields, including one in ophthalmology in 1989 by Lee et al., have shown that the traditional selection crite-ria, especially the cognitive metrics such as grades and honors, as well as pre-residency research ex-perience and publications, may not reliably predict resident performance[7,8,10-15]. Interestingly, however, several studies in pediatrics, neurology, urology, and otolaryngology have shown that these same characteristics have predictive value for post-residency academic productivity[16-20]. No study has been published in ophthalmology to address this issue. To this end, we conducted a retrospec-tive study on a cohort of ophthalmology residents graduating from the Wilmer Eye Institute, a pro-gram whose mission is to train the future leaders in ophthalmology, to identify pre-residency char-


25

Pre-residency characteristics - Shan et al.

acteristics that are associated with post-residency academic productivity.

Methods

Electronic survey and review of curricula vitae Fifty-seven individuals graduated from the Wilmer Eye Institute ophthalmology residency program between 1990 and 1999. Contact information was available for 53 graduates (93%). Each individual was emailed a 16-question electronic survey that contained both multiple choice and type-in poten-tial answers. The survey contained two sections. The first section pertained to pre-residency scho-lastic and personal characteristics that are typically available to the resident selection committees at the time of interview. These include Alpha Omega Alpha (AOA) membership, United States Medi-cal Licensing Examination (USMLE) I or Federal Licensing Exam (FLEX) percentile score, medical school class rank (divided into thirds), clerkship grades for medicine, surgery, and ophthalmology, age at the start of residency, prior completion of PhD or Masters degree, and having an influential ophthalmologist and/or physician-scientist in their lives up to the time of their residency application. The second section of the survey pertained to post-residency academic productivity with questions re-lating to faculty status (i.e., currently on full-time faculty, once was on full-time faculty, never was on full-time faculty), fraction of time dedicated to different activities (clinical, research, teaching, and administration), and number of active grants and grants in submission. The graduates were instructed to return the survey along with their most updated curricula vitae (CV). We asked the graduates to ensure that their CVs included a complete record of their formal educa-tion and post-residency employment, as well as a list of all peer-reviewed publications. Validation of the count of publications as reported in the CVs was performed against PubMed for a subset of the graduates. Information regarding editorial board memberships, reviewer activities, national and in-ternational lectures given, and mentoring activities also were ascertained from the CV. Non-respond-ers were emailed again 2 months later, and a third time 4 months after the initial contact. Phone calls were made by the investigators to continual non-responders to request the graduates to participate. All data from the results of the survey and the CVs

were collected by January 2009 and were compiled into a single database. Data on post-residency in-formation were entered while masked to the pre-residency characteristics. All data were de-identi-fied and analyzed in aggregate only. This research study was approved by the Johns Hopkins Medical Institutional Review Board.Outcome measures of academic productivity We used three parameters to measure academic productivity. First, we determined the academic ca-reer choice made by the graduates by documenting their faculty status and classifying them into three categories: currently on full-time faculty; previous-ly on full-time faculty; never on full-time faculty. Faculty rank was not used as an independent out-come measure due to the complexity and variabil-ity inherent in the promotion process. Instead, the ranks of those on full-time faculty were taken into consideration in the productivity score calculations described below in this section. The second outcome measure for academic produc-tivity was the total number of articles published as first, second, third, or corresponding author after residency. For those who completed fellowship, this total count includes articles published during and after fellowship. Peer-reviewed publication rate has been used in several previous studies as a metric of academic productivity and is also used as a basis for promotion in many academic institu-tions[14,18,19,21]. Given that this study population was on average 13 years out of residency, the ma-jority (68%) had not published any articles as the corresponding author. Therefore, we used the total count of articles published as first, second, third, and corresponding author.Lastly, we computed a productivity score for each graduate using a point system that was designed jointly by the four co-authors after reviewing the literature for previously published point systems used by academic departments to assess academic productivity for compensation purposes[22-23]. The purpose of this composite score was to expand the definition of academic productivity beyond faculty status and publications to include other ac-tivities important to successful physician-scientists, such as applying for and holding grants; contrib-uting to the academic community as a manuscript reviewer, an editorial board member for journals, or a national or international lecturer; and mentoring future physicians.


26

Pre-residency characteristics - Shan et al

Statistical analysisWe conducted both univariate and multivariate re-gression analyses to estimate the associations be-tween pre-residency characteristics and the three academic productivity outcome measures. For the univariate analysis, we divided the graduates into three groups based on faculty status. We then deter-mined if the distribution of any of the pre-residency and post-residency variables was different among the three groups. One-way analysis of variance (ANOVA) was performed for the normally distrib-

uted continuous variables: age at the time of survey, age at the start of residency, and years since resi-dency completion. Nonparametric tests were per-formed for continuous variables with skewed distri-butions, such as the number of publications before and after residency and the productivity scores. Chi-square tests or Fisher’s exact tests were used to compare categorical variables including gen-der, AOA membership, having another advanced degree, and pre-residency research experience in ophthalmology.

Table 1: Pre-residency characteristics by current faculty status (N=52)

Characteristics

Mean or frequency

Total (N=52)

P value1Never on full-time faculty

(N=13)

Once was on full-time fac-ulty (N=16)

Currently on full-time fac-ulty (N=23)

Female gender, n (%) 3 (23) 3 (19) 7 (30) 13 (25) 0.70

Have a Masters degree, PhD, or both, n (%) 7 (54) 11 (69) 14 (61) 32 (62) 0.71

USMLE1 or FLEX above 90 percentile, n (%)2 4 (57) 4(44) 10 (53) 18 (51) 0.87

AOA member, n (%)3 3 (25) 10 (63) 9 (43) 22 (45) 0.14

Total number of first- and second-author publica-tions 0.38

Median (IQR) 1 (3) 1 (2.5) 2 (3) 2 (3)

Mean (SD) 2 (2.4) 1.6 (1.4) 2.7 (2.6) 2.2 (2.3)

Range 0 - 7 0 - 4 0 - 11 0 - 11

Pre-residency research in ophthalmology, n (%) 8 (62) 11 (69) 17 (74) 36 (69) 0.74

Had Influential person in ophthalmology or aca-demics, n (%)

10 (77) 11 (69) 14 (61) 35 (67)0.61

Age at the time of survey4 0.16

Mean (SD) 43 (5) 44 (4) 41 (4) 43 (4)

Range 36 - 52 38 – 50 35 - 47 35 – 50

Age at the start of residency 0.21

Mean (SD) 31 (4) 29 (3) 29 (3) 29 (3)

Range 27 - 37 25 - 34 25 - 34 25 - 37

SD = Standard deviation; IQR = Interquartile range

1. P-values are from Chi-Square test (or Fisher’s exact test when expected numbers were small) for categorical variables, and ANOVA (or Kruskal-Wallis tests when the distribution is not normal) for continuous variables.

2. There are missing values for graduate degree: six individuals from the never on full-time faculty group, seven from the once was on full-time faculty group and four from the currently on full-time faculty group.

3. There are missing values for AOA: one individual from the never on full-time faculty group and two from the currently on full-time faculty group.4. Two individuals from the currently on full-time faculty group had missing information for age.


27

Pre-residency characteristics - Shan et al.

Table 2: Post-residency academic characteristics by current faculty status (N=52)

CharacteristicsMedian, mean or frequency

Total (N=52) P value1Never on full-time faculty

(N=13)

Once was on full-time faculty

(N=16)

Currently on full-time

faculty (N=23)

Ove

rall

Years between end of residency and survey completion 0.01

Median (IQR) 10 (5) 16 (4) 12 (4) 13 (6)

Mean (SD) 12 (3) 15 (2) 12 (3) 13 (3)

Range 9 -17 11 - 18 9 - 18 9 - 18

Academic rank, n (%)2 0.23

Full professor 0 0 2 (9) 2 (5)

Associate professor 13 (100) 2 (13) 6 (26) 9 (23)

Assistant professor 0 13 (87) 14 (61) 27 (69)

Other 0 0 1 (4) 1 (3)

Percent time currently spent on research4 0.0001

Median (IQR) 0 (0) 0 (7.5) 20 (60) 4.8 (17.4)

Mean (SD) 1 (3) 4 (6) 37 (32) 16 (26)

Range 0 - 10 0 - 20 0 - 90 0 - 90

Prod

uctiv

ity sc

ores

Total Productivity score 0.0001

Median (IQR) 10 (7) 25.8 (27.5) 64.5 (53) 37.8 (49.6)

Mean (SD) 14.2(12.7) 41.8 (43) 71.3 (34) 47.9 (40.5)

Range 4 - 52 6 - 167.5 27.5 - 154.5 4 -167.5

Current faculty status productivity score -

Median (IQR) 0 (0) 5 (0) 20 (0) 5 (17.5)

Mean (SD) 0 (0) 5 (0) 20 (0) 10.4 (8.8)

Range 0 - 0 5 - 5 20 - 20 0 - 20

Academic rank productivity score 0.0001

Median (IQR) 0 (0) 3 (0) 3 (1) 3 (2)

Mean (SD) 0.3 (1.1) 2.9 (0.9) 3.4 (0.7) 2.5 (1.5)

Range 0 - 4 0 - 4 2 - 5 0 - 5

Percent time spent in research productivity score 0.0001

Median (IQR) 0 (0) 0 (1) 2 (7) 0 (2)

Mean (SD) 0.2 (0.4) 0.4 (0.6) 3.2 (3.2) 1.6 (2.6)

Range 0 - 1 0 - 2 0 - 9 0 - 9

Publication productivity score 0.02

Median (IQR) 10 (9) 15.8 (24.3) 22 (24) 17.8 (19.8)

Mean (SD) 12.5 (9.8) 26.6 (28.7) 28.2 (19.7) 23.8 (21.8)

Range 2 - 38 1 – 101.5 3.5 – 84.5 1 – 101.5

Grants productivity score 0.01

Median (IQR) 0 (0) 0 (0) 5 (5) 0 (5)

Mean (SD) 0.8 (1.9) 0.3 (1.3) 2.8 (2.5) 1.5 (2.3)

Range 0 - 5 0 - 5 0 - 5 0 - 5

(table continued, next page)


28

Table 2 (continued): Post-residency academic characteristics by current faculty status (N=52)

Characteristics Median, mean or frequency Total (N=52) P value1

Never on full-time faculty

(N=13)


(N=16)


faculty (N=23)

Prod

uctiv

ity sc

ores

(con

tinue

d)

Editorial board memberships productivity score 0.02

Median (IQR) 0 (0) 0 (0) 0 (1.5) 0 (0)

Mean (SD) 0 (0) 0.3 (1.0) 1.1 (2.4) 0.6 (1.7)

Range 0 - 0 0 - 4 0 - 11 0 - 11

Reviewer activities productivity score 0.05

Median (IQR) 0 (0) 0 (1.5) 0 (7) 0 (1.5)

Mean (SD) 0.1 (0.3) 1.5 (2.9) 3.5 (5.8) 2.0 (4.4)

Range 0 - 1 0 - 9 0 - 25 0 - 25

Teaching activities productivity score 0.02

Median (IQR) 0 (0) 0 (2.1) 2 (14.3) 0 (4)

Mean (SD) 0.5 (1.2) 4.6 (15.4) 9.1 (15.1) 5.6 (13.5)

Range 0 - 4 0 - 62 0 - 60 0 - 62

Publ

icati

ons a

fter r

esid

ency

Number of first-author publications 0.04

Median (IQR) 4 (2) 5 (11) 6 (7) 5 (6.5)

Mean (SD) 4 (3) 9 (10) 9 (7) 8 (7)

Range 0 - 9 0 - 36 0 - 26 0 - 36

Number of corresponding author publications 0.004

Median (IQR) 0 (0) 0 (0) 3 (20) 0 (2.5)

Mean (SD) 0 (0.3) 1 (2) 9 (13) 4 (10)

Range 0 - 1 0 - 8 0 - 40 0 - 40

Number of second-author publications 0.32

Median (IQR) 1 (2) 1 (2.5) 2 (5) 1 (2.5)

Mean (SD) 1 (1) 3 (5) 4 (5) 3 (5)

Range 0 - 4 0 - 18 0 - 23 0 - 23

Number of third-author publications 0.006

Median (IQR) 2 (4) 2 (4) 7 (11) 3 (7)

Mean (SD) 4 (9) 4 (8) 8 (6) 6 (7)

Range 0 - 34 0 - 31 1 - 23 0 - 34

Total number of first, second, third and corresponding author publications 0.003

Median (IQR) 4(12) 9.5 (18.5) 19 (47) 14 (22)

Mean (SD) 9 (11) 17 (20) 29 (25) 20 (22)

Range 1 - 45 1 - 70 1 - 82 0 - 82

Number of first-author book chapters 0.08

Median (IQR) 1 (1) 1 (3) 2 (3) 1 (3)

Mean (SD) 1 (1) 3 (4) 3 (4) 2 (3)

Range 0 - 4 0 - 14 0 - 13 0 - 14

(table continued, next page)



29

Characteristics Median, mean or frequency Total (N=52) P value1

Never on full-time faculty

(N=13)


(N=16)


faculty (N=23)

Table 2 (continued): Post-residency academic characteristics by current faculty status (N=52)G

rant

s

Have active grants, n (%) 2 (15) 1 (6) 13 (57) 16 (31) 0.001

Number of active grants 0.004

Median (IQR) 0 (0) 0 (0) 1 (3) 0 (1.5)

Mean (SD) 0.5 (1) 0.4 (1.5) 2 (2) 1 (2)

Range 0 - 4 0 - 6 0 - 10 0 - 10

Edito

rial/

revi

ewer

acti

vity

Number of present editorial memberships 0.03

Median (IQR) 0 (0) 0 (0) 1 (1) 0 (0)

Mean (SD) 0 (0) 0.3 (1) 1 (2) 0.5 (2)

Range 0 - 0 0 - 4 0 - 11 0 - 11

Past editorial memberships, n (%) 0 1 (6) 4 (17) 5 (10) 0.36

Present journal reviewer activities 0.02

Median (IQR) 0 (0) 0 (1) 0 (7) 0 (1.5)

Mean (SD) 0 (0) 1 (3) 3 (6) 2 (4)

Range 0 - 0 0 - 9 0 - 25 0 - 25

Past journal reviewer activities, n (%) 1 (8) 1 (4) 1 (6) 3 (6) 1.00

Educ

ation

Number of international or national named lectures 0.12

Median (IQR) 0 (0) 0 (0.5) 0 (4) 0 (1)

Mean (SD) 0.2 (0.6) 2 (8) 4 (7) 2 (6)

Range 0 - 2 0 - 31 0 - 30 0 - 31

Number of trainees advised

Median (IQR) 0 (0) 0 (0) 0 (8) 0 (0) 0.003

Mean (SD) 0 (0) 0.6 (2) 8 (16) 4 (11)

Range 0 - 0 0 - 9 0 - 65 0 - 65

SD = Standard deviation; IQR = Interquartile range 1. P-values are from Fisher’s exact test for categorical variables, and Kruskal-Wallis tests for continuous variables.2. Twelve from the never on full-time faculty group and one from the once was on full-time faculty group either did not have applicable information

or had missing information about the professor type.3. One individual has never been on full-time faculty, but has been on part-time faculty as an associate professor.

4. Four individuals from the currently on full-time faculty group had missing information for percent time spent in research.



30


Regression methods appropriate for the different data distributions were used to determine indepen-dent associations between pre-residency character-istics and each of the three outcome measures. Spe-cifically, multinomial logistic regression was used to examine the association between pre-residency characteristics and current faculty status. The num-ber of post-residency publications was assessed us-

ing a negative binomial regression model. Lastly, multivariate log-linear regression analysis was used for the productivity score. Backward stepwise re-gressions and simple regression models were used in facilitating the selection of independent vari-ables. All multivariate models were adjusted for the graduates’ age at the start of residency and years since graduation from residency. The estimated rate

Table 3: Unadjusted and adjusted relative risk ratios for current faculty status

Faculty Category Characteristics

Unadjusted1 Adjusted2

Relative risk ratio

95% Confidence

IntervalP value Relative

risk ratio

95% Confidence

IntervalP value

Currently on full-

time faculty vs. Never on full-time faculty

Years from the end of residency to the time of survey 1.06 0.8, 1.4 0.67 0.96 0.7, 1.3 0.75

Age at the start of residency 0.86 0.7, 1.1 0.19 0.85 0.7, 1.1 0.19

Male vs. female 0.69 0.1, 3.3 0.64 - - -

With a graduate degree vs. without any graduate degree 0.75 0.2, 3.0 0.68 - - -

AOA vs. not AOA 2.25 0.5, 10.8 0.31 2.7 0.5, 13.9 0.23

USMLE1 or FLEX above 90 percentile vs. be-low 90 0.53 0.2, 1.3 0.15 - - -

Total number of first- and second-author publications 1.16 0.8, 1.6 0.38 - - -

Pre-residency research was in ophthalmol-ogy vs. not 1.77 0.4, 7.6 0.44 - - -

Had Influential person in ophthalmology or academics 0.47 0.1, 2.2 0.33 - - -

Previously was on

full-time faculty vs. Never on full-time faculty

Years from the end of residency to the time of survey 1.46 1.1, 2.0 0.01 1.45 1.0, 2.0 0.03

Age at the start of residency 0.81 0.6, 1.0 0.10 0.82 0.6, 1.1 0.17

Male vs. female 1.3 0.2, 7.9 0.78 - - -

With a Masters degree, or PhD, or both vs. without 0.53 0.1, 2.4 0.41 - - -

AOA vs. not AOA 5.0 1.0, 26.1 0.06 4.8 0.8, 30.0 0.09

USMLE1 or FLEX above 90 percentile vs. be-low 90 1.05 0.4, 2.6 0.91 - - -

Total number of first- and second-author publications 0.90 0.6, 1.3 0.59 - - -

Pre-residency research was in ophthalmol-ogy vs. not 1.38 0.3, 6.4 0.67 - - -

Had Influential person in ophthalmology or academics 0.66 0.1, 3.5 0.63 - - -

1. The unadjusted estimates are from simple multinomial logistic regressions that only have one predictor.2. The adjusted estimates are from multivariate multinomial logistic adjusted for years from the end of residency to the time of

survey, age at the start of residency, and AOA.


31


publication, and 52% had at least one third-author publication. Sixty-two percent had another ad-vanced degree. In addition, 67% had an influential person in their lives who was an ophthalmologist or a physician-scientist. As of January 2009, an average of 13 years after completion of residency, 44% of the cohort held a full-time academic posi-tion, 31% were previously on full-time faculty, and 25% had never entered a full-time academic career track. Of the individuals who were full-time faculty members at the time of the survey, 70% were men, and the vast majority (87%) were assistant or as-sociate professors. As noted above, the mean age at the start of resi-dency was 29 years for the entire study population, with a standard deviation of 3 years. There were 10 individuals at one or more standard deviations above the mean. Four of them were full-time facul-ty at the time they answered the questionnaire, and

ratios with 95% confidence intervals (CIs) from the models are presented. A P value <0.05 was consid-ered statistically significant, and a P value between 0.05 and 0.1 was considered to indicate a trend to-wards statistical significance. All data were ana-lyzed using STATA software version 11 (StatCorp, College Station, TX).

Results

We contacted 53 individuals; 52 (98%) returned the surveys and their CV. Seventy-five percent were men. The average age at the start of residency was 29 years (range 25 to 37 years). Forty-five percent had been elected to AOA. Prior to residency, 69% had conducted research in ophthalmology and 81% had at least one pre-residency scientific publication, with 65% having at least one first-author publica-tion. Forty percent had at least one second-author

Table 4: Regression results for the total number of first- author, second- author, third-author and corresponding author publications after residency

Characteristics Incidence-rate ratio

95% Confidence Interval P

Years from end of residency to present 1.05 1.0, 1.1 0.15

Age at start of residency 0.91 0.8, 1.0 0.02

Male vs. female 1.69 0.9, 3.0 0.08

With a graduate degree vs. without any graduate degree 1.53 0.8, 2.9 0.18

AOA vs. not AOA 2.12 1.3, 3.4 0.003

Total number of first- and second-author publications 1.14 1.0, 1.3 0.08

Table 5: Log-linear regression results for productivity score

Characteristics Ratio1 95% Confidence Interval P

Years from end of residency to present 1.02 0.9, 1.1 0.71

Age at start of residency 0.89 0.8, 1.0 0.01

Male vs. female 1.27 0.7, 2.4 0.45

With a graduate degree vs. without any graduate degree 1.35 0.7, 2.8 0.40

AOA vs. not AOA 1.65 1.0, 2.8 0.07

Number of pre-residency first- and second-author publications 1.14 1.0, 1.3 0.08

1Ratio represents the exponentiated coefficient from the log-linear regression model.


32


two once held full-time faculty positions but sub-sequently left for private practice. The remaining four participants had never been on the full-time or part-time faculty and were in private practice. For the six individuals who indicated that they were not currently on full-time faculty, we did not ask if they worked full-time or part-time. In addition, of these six participants, none had any current or past edito-rial board memberships, were involved in any jour-nal review activities, or had mentored any residents or fellows. Categorization of all participants by faculty status showed similar distributions with respect to all pre-residency characteristics (Table 1); however, the number of years since residency completion dif-fered among the three groups (Table 2). Individuals currently on full-time faculty and those who were never on full-time faculty were a median of 10 years and 12 years out of residency, respectively, whereas those who once were on full-time faculty but had since left academics were a median of 15 years out of residency. Not surprisingly, individuals currently on full-time faculty had higher scores for nearly all individual academic success measures (Table 2) after controlling for age and years since gradua-tion and for the composite productivity score when compared with the entire study population (median productivity score 64.5 vs. 37.8; P = 0.0001).Current Faculty Status The associations between post-residency produc-tivity characteristics and current faculty status are presented in Table 3. When comparing those cur-rently on full-time faculty and those who were nev-er on full-time faculty, none of the characteristics showed significant associations. Comparing those who were once on full-time faculty with those who were never on full-time faculty, we observed the trend that graduates with AOA memberships were almost five times more likely to be in the group that was on full-time faculty at some point in time (rate ratio = 4.8, P = 0.09). In addition, graduates with a greater number of years between residency comple-tion and time of survey were more likely to be in the group that once was on full-time faculty but had since left academics (ratio = 1.45, P = 0.03). Of the 16 individuals in this latter group, the major-ity (75%) had left academics for private practice after an average of 3.8 years (range 1-8 years) as assistant professors. One individual had retired for medical reasons, and three others did not provide any explanation regarding their decision to leave

academics. Number of post-residency publications The number of articles published since residency completion varied widely across individuals, with 2% having published no articles and 42% publish-ing more than 10 articles. Since the completion of residency, this cohort had published a median of five articles as first author (range 0 to 36), one ar-ticle as second author (range 0 to 23), 3 articles as third author (range 0 to 34), and no articles as cor-responding author (range 0 to 40). We found AOA membership to be significantly associated with a higher number of post-residency publications (ra-tio = 2.12, P = 0.003) after adjusting for age, num-ber of years since residency completion, and some other characteristics (Table 4). Older age at the start of residency had an inverse association with the number of post-residency publications (ratio = 0.91 P = 0.02); each 1-year increase in age was associ-ated with a 9% reduction in the number of publica-tions after residency, after controlling for number of years since residency completion. Lastly, there was a trend toward a positive correlation between the number of pre-residency publications and post-residency publications (rate ratio = 1.14, P = 0.08).Productivity scoreThe mean productivity score for this cohort was 47.9 (range 4-167.5). On average, approximately 27% (mean: 13 points) of the total productivity score was derived from faculty status and academic rank, 50% from publications, and 19% (mean: 11 points) from other professional activities. Older age at the start of residency was the only statistically significant predictor of productivity score; a 1-year increase in age at the start of residency was associ-ated with an 11% reduction in productivity score (ratio = 0.89, P = 0.01), after adjusting for years since residency completion (Table 5). Finally, al-though the findings did not reach statistical signifi-cance, the analysis also showed that AOA members were more likely to have higher productivity scores (1.65 times) than non-AOA members (P = 0.07) and that individuals with more pre-residency pub-lications also attained higher productivity scores (ratio = 1.14, P = 0.08).

Discussion

To our knowledge, this is the first retrospective co-hort study to evaluate pre-residency characteristics associated with post-residency academic produc-


33


tivity in ophthalmology. The results of this study show that pre-residency scholastic excellence and experience with research and the manuscript pub-lication process are associated with post-residency academic productivity. AOA membership was the only pre-residency characteristic positively asso-ciated with all three outcome measures. We also observed a trend toward statistical significance of a positive correlation between the number of pre-residency publications and both productivity scores and post-residency publications. Lastly, an inverse association with all three measures of academic productivity was observed with older age at the start of residency. The relationship between AOA membership and academic productivity was strong and significant in this study. We found AOA members to be more likely to have been on full-time faculty at some point after residency, to have a greater number of post-residency publications, and to attain higher productivity scores. These results are consistent with the findings of prior studies showing that AOA membership is predictive of holding an academic appointment and generating more publications af-ter residency[16,17,20]. Interestingly, however, in studies in which performance during residency based on faculty evaluation and in-training exami-nation scores are used as the outcome, AOA mem-bership seems to have only a weak or no predictive value[10,12,15]. One possible explanation for this observation is that individuals who are academi-cally productive by our measures may be more research-oriented and thus expend more time and efforts in research activities than in clinical activi-ties or studying for in-service examinations. An-other possible explanation is that if the individuals were elected to AOA in their junior year of medical school, the election would be based mostly on pre-clinical grades and thus not necessarily predictive of clinical performance[9]. Therefore, we cannot assume that pre-residency characteristics that are associated with post-residency academic produc-tivity would also predict performance during resi-dency. Future studies should evaluate for the pre-dictors for the latter. The strong association between AOA membership and academic productivity is not surprising. Even though the exact selection process may differ at each AOA chapter, each must comply with the mis-sion of AOA to recognize “medical students who have excelled academically, demonstrated profes-

sionalism, and have shown promise of becoming leaders in the profession”[24]. Hence, both cogni-tive and non-cognitive attributes should have been taken into consideration, especially when the in-dividuals are elected in their senior year of medi-cal school. This may underlie the observation that although AOA membership was associated with all three outcome measures in our study, no signifi-cance was found for individual cognitive variables such as USMLE score, clinical course grades, and class rank. Taken together, the cognitive and non-cognitive attributes that led the AOA members to excel during medical school likely lay the ground-work for productivity in their future careers, form-ing part of a positive feedback loop through which success begets success. While not quite reaching statistical significance, the number of articles published before residency also showed a trend toward a positive correlation with productivity score and the number of post-residen-cy publications. The association between prior re-search experience and the decision to pursue an aca-demic career, as well as between pre-residency and post-residency publication rates, has been observed in some studies[16,20,25]. Prior research experi-ence likely benefits an individual regardless of the field of research, as the variable “pre-residency re-search was in ophthalmology” was not significantly correlated with academic productivity. Experience in research and the manuscript publication process introduces students to the principles of scientific in-quiry, sparks or increases their interest in academic medicine, equips them with scientific and writing skills, provides them with the confidence to carry a project through to completion, and conceivably builds a foundation for future productivity. The only pre-residency characteristic that was not-ed to have an inverse association with academic productivity was older age at the start of residency. This relationship remains significant after adjusting for other potentially confounding variables such as having an additional graduate degree. Graduates who never entered academics were 1-2 years older at the start of residency than their colleagues who were on full-time faculty at some point after resi-dency. Moreover, graduates who were older at the start of residency also tended to have fewer post-res-idency publications and lower productivity scores, even after adjusting for number of years since resi-dency completion. A similar age effect was noted by Brancati et al. in their study that showed that


34


physicians who graduated before 26 years of age performed slightly better in attained rank and cita-tions than their older colleagues[16]. One hypoth-esis that may account for this observation is that age is correlated to a certain extent with stage of life, and different stages are associated with differ-ent roles, responsibilities, and priorities pertaining to marriage, family, and financial situations such as debt burden. Each of these factors plays a role in a resident’s career decision-making process and post-residency productivity. Therefore, the relationship among age, career choice, and productivity is com-plex, dynamic, and often unpredictable. Further longitudinal studies spanning longer time periods are necessary to generate more information to help us understand this finding.Our study has several limitations. The first pertains to the nature of the study population. We studied graduates of a single residency training program that has traditionally focused on training future leaders in ophthalmology. Therefore, our study may be more generalizable to other residency programs in the country with a similar mission rather than to residency programs whose mission is to train com-prehensive community clinicians. Wilmer residents are trained in a highly academic environment and may feel more inclined or even expected to enter academics, and this is made clear to them during the interview process. More career opportunities may be provided to them in academics than in the private sector. This likely explains our striking ob-servation that 75% of the cohort initially entered academics. Interestingly, however, even within this seemingly homogeneous cohort, career choices and academic productivity varied. Therefore, it remains informative to study the factors that would predict this difference in academic productivity. Future studies should combine data from several diverse residency programs throughout the country to ob-tain a more heterogeneous cohort. Our analysis was also constrained by the relatively small sample size. Hence, several of our observa-tions can only be considered trends that need to be confirmed with larger studies. Having a larger co-hort would increase the power to conduct an analy-sis of more variables simultaneously and to observe smaller differences that are statistically significant. Having said this, we chose a cohort that graduated between 1990 and 1999 to ensure that 1) they had spent a reasonable number of years establishing

their careers after residency, and 2) they had gradu-ated recently enough that we had the most updated contact information for them. Yet, given the chal-lenges often faced by young investigators in estab-lishing their careers, this short time period may not have allowed us to observe the true productivity potential of some graduates. Future investigators should consider studying a larger cohort that has spent more time out of residency or conducting pro-spective longitudinal studies that follow residents through their careers, as these studies could yield more information regarding long-term academic productivity. Another limitation of this study relates to the poten-tial recall bias inherent to survey studies. Some of the survey questions enquired about activities be-fore residency, and as this was more than 20 years ago for some individuals, their answers may have been subject to recall bias. In addition to the sur-veys, we obtained data from the graduates’ curricu-la vitae. The CVs were reviewed for the number of publications both before and after starting residency and for information pertaining to other professional activities such as reviewer activities, editorial board memberships, national and international lectures, and trainees mentored. Although we were able to validate the count of publications as reported in the CVs against PubMed and found a close match, we did not have the means to validate the other pro-fessional activities. Therefore, even though we asked the graduates to provide their complete CV including the aforementioned information, it was difficult to ensure that this was what we received from each graduate, hence subjecting our results to respondent bias. Individuals currently in academics may be more likely to provide a complete record of their publications and activities. Conversely, gradu-ates who have been in private practice for a while may not have had a reason to keep this information on their most updated CVs nor the incentive to add this information to their CV for our study. In the fu-ture, longitudinal prospective studies could be con-ducted, which would ensure that all pre-residency information is collected as new residents start each year. Moreover, public and verifiable sources such as departmental annual reports at academic institu-tions could be used to extract more objective and reliable information of the graduates’ professional activities. Citation index can be used to assess both quantity and quality of the graduates’ publications. There are a few noteworthy points to be made about


35


our choices of outcome measures in this study. First, we recognize that academic productivity is much more complex than can be captured by any linear combination of outcome measures. Never-theless, we have endeavored to be as comprehen-sive and objective as feasible in measuring aca-demic productivity. Faculty status and publication rate have been validated as measures of academic productivity in numerous studies across medical fields[14,16-20,25,26]. We also devised the pro-ductivity score to incorporate other professional activities relevant to academic productivity. As no prior externally validated metrics for academic pro-ductivity exist, we devised our productivity score system through discussions among the co-authors as well as a review of several previously published departmental point systems used for compensation purposes in fields such as orthopedic surgery and family medicine[22,23]. Even though the point as-signment may be somewhat subjective, the factors included are mostly objective and quantifiable. Fu-ture investigators could continue to seek out more comprehensive, objective, and externally validated methods of measuring academic productivity and incorporate them into their studies. Second, con-stricted by a small sample size, we focused our study on only one aspect of post-residency suc-cess: academic productivity. Nevertheless, we fully acknowledge the importance of selecting for applicants who not only will be productive aca-demic ophthalmologists, but also will be excellent residents, caring clinicians, and effective educators. Future studies with larger cohorts should analyze more of these outcome variables simultaneously. Despite the aforementioned limitations, the present study provides valuable information on potential predictors of academic productivity in ophthalmol-ogy. Although our study may have limited general-izability to all residency programs in the country, the shortage of academic ophthalmologists would impact every residency program. Therefore, it is in the common interest to determine how best to ensure a steady supply of future academic ophthal-mologists. We conclude that pre-residency scho-lastic excellence as indicated by AOA membership and research experience resulting in publications are associated with academic productivity after res-idency. The age effect we observed warrants further investigation. Interestingly, individual variables in-cluding USMLE score, clinical course grades, class rank, and having additional graduate degrees such

as a Ph.D. were not significant predictors of post-residency academic productivity. The aim of this study was neither to generate a standardized method for residency selection nor to identify predictors of academic productivity to be used exclusively or prejudicially. Rather, this was an attempt to take a scientific approach to under-standing the residency selection process. We hope our study will serve as a springboard for ideas and discussions for medical schools and residency pro-grams that share a common goal of sustaining aca-demic medicine. Medical schools should continue to encourage students to strive for academic and clinical excellence and provide students with more opportunities and guidance to conduct scientific re-search and to publish. For residency selection com-mittees, our study lends evidence to the validity of using AOA membership and pre-residency publica-tion records to guide the resident selection process insofar as the program’s mission is to train future academic ophthalmologists. Future studies should assess the association between non-cognitive pre-residency characteristics such as humanism, altru-ism, and professionalism and performance during and after residency.

Acknowledgements

Funding/support: Dr. Emily Gower is the recipi-ent of the Ernest and Elizabeth Althouse Special Scholars’ Award from Research to Prevent Blind-ness. Ms. Jiangxia Wang is funded by the Wilmer Eye Institute Biostatistics Core Grant EY01765.

References

1. Healy B. Innovators for the 21st century: will we face a crisis in biomedical-research brain-power? N Engl J Med 1988;319:1058-64.

2. Rosenberg L. Physician-scientists--endangered and essential. Science 1999;283:331-32.

3. Wyngaarden JB. The clinical investigator as an endangered species. N Engl J Med 1979;301:1254-9.

4. Sparrow JM. British academic ophthalmology in crisis. Br J Ophthalmol 2006;90:404-05.

5. Frohman LP. The human resource crisis in neuro-ophthalmology. J Neuroophthalmol 2008;28:231-34.

6. Simon J, Koller HP. Manpower issues in pedi-


36

atric ophthalmology. J Pediatr Ophthalmol Stra-bismus 2008;45:197-99.

7. Nallasamy S, Uhler T, Nallasamy N, et al. Oph-thalmology resident selection: current trends in selection criteria and improving the process. Ophthalmology 2010;117:1041-47.

8. Lee AG, Golnik KC, Oetting TA, et al. Re-en-gineering the resident applicant selection pro-cess in ophthalmology: a literature review and recommendations for improvement. Surv Oph-thalmol 2008;53:164-76.

9. Sadun AA, Dunn JP, Lee AG. The resident se-lection process in ophthalmology: which crite-ria should we use? J Acad Ophthalmol 2011;4:1-10.

10. Boyse TD, Patterson SK, Cohan RH, et al. Does medical school performance predict ra-diology resident performance? Acad Radiol 2002;9:437-45.

11. Erlandson EE, Calhoun JG, Barrack FM, et al. Resident selection: applicant selection cri-teria compared with performance. Surgery 1982;92:270-75.

12. Lee DA, Hepler RS, Wheeler NC, Straatsma BR. A retrospective study on the selection cri-teria for ophthalmology residents. Am J Oph-thalmol 1989;108:326-27.

13. McCollister RJ. The use of Part I National Board scores in the selection of residents in ophthalmology and otolaryngology. JAMA 1988;259:240-42.

14. Patterson SK, Fitzgerald JT, Boyse TD, Cohan RH. Is past academic productivity predictive of radiology resident academic productivity? Acad Radiol 2002;9:211-16.

15. Wood PS, Smith WL, Altmaier EM, et al. A prospective study of cognitive and noncogni-tive selection criteria as predictors of resident performance. Invest Radiol 1990;25:855-59.

16. Brancati FL, Mead LA, Levine DM, et al. Early predictors of career achievement in academic medicine. JAMA 1992;267:1372-76.

17. Daly KA, Levine SC, Adams GL. Predictors for resident success in otolaryngology. J Am Coll Surg 2006;202:649-54.

18. Dorsey ER, Raphael BA, Balcer LJ, Galetta SL. Predictors of future publication record and academic rank in a cohort of neurology resi-dents. Neurology 2006;67:1335-37.

19. Hellenthal NJ, Ramirez ML, Yap SA, Kurz-rock EA. Manuscript publication by urol-ogy residents and predictive factors. J Urol 2009;181:281-86.

20. Ledley FD, Lovejoy FH, Jr. Factors influenc-ing the interests, career paths, and research activities of recent graduates from an aca-demic, pediatric residency program. Pediatrics 1993;92:436-41.

21. Bhattacharyya N, Shapiro NL. Academic oto-laryngology in the new millennium: we are falling behind. Otolaryngol Head Neck Surg 2007;137:535-38.

22. Willis DR, Kelton GM, Saywell RM, Jr, Kiovsky RD. An incentive compensation sys-tem that rewards individual and corporate pro-ductivity. Fam Med 2004; 36:270-78.

23. Emery SE, Gregory C. Physician incentives for academic productivity. An analysis of or-thopaedic department compensation strategies. J Bone Joint Surg Am; 88:2049-56.

24. Constitution of Alpha Omega Alpha. 2010. Alpha Omega Alpha Honor Medical Society.

25. Sanders AB, Fulginiti JV, Witzke DB. Factors influencing resident career choices in emergen-cy medicine. Ann Emerg Med 1992;21:47-52.

26. Lynch PJ, Harrell ER. Factors in the choice of an academic career. Results of a questionnaire. Arch Dermatol 1971;103:328.



37

Do U.S. Medical Licensure Examination Step I Scores Correlate with the American Academy of Ophthalmology In-Training Ophthalmic Knowledge Assessment Program Examination Scores and American Board of Ophthalmology Written Qualifying Examination Performance?Sowmya Kantamneni1 and Oscar A. Cruz, MD*2

1Saint Louis University School of Medicine, Saint Louis, MO, 2Department of Ophthalmology, Saint Louis University, Saint Louis, MO


Abstract

Purpose: To find if there is any association between performing well on the USMLE Step I and/or the annual Ophthal-mic Knowledge Assessment Program (OKAP) examinations and passing the Written Qualifying Examination(WQE) and oral examination of the American Board of Ophthalmology (ABO) board certification process. Also, to examine the success of residents who performed poorly on the OKAP by looking at their likelihood to be given awards during residency. Lastly, to see if any correlation can be seen with residents who fail a portion of the ABO exams and their post-graduation paths.

Methods: In this retrospective study, data from ophthalmology residents graduating from Saint Louis University from the years 1998 to 2009 was examined.

Results: Forty-six residents were examined. Five residents did not pass a portion of the ABO exams. A 0.355 correla-tion (P<0.015, P≤0.05) was found between USMLE Step I performance and passing the ABO exams. The positive correlation indicates that the higher the USMLE Step I score, the more likely the individual was to pass the ABO ex-ams. Correlations between each OKAP and passing the ABO are as follows: OKAP Year 1 Scaled r=0.369, P<0.011, P≤0.05; OKAP Year 2 Scaled r=0.434, P<0.008, P≤0.01; OKAP Year 3 Scaled r=0.446, P<0.004, P≤0.01.

Conclusions: Performance on the USMLE Step I and the OKAPs are positively correlated with passing the ABO ex-ams. No correlations could be made about the likelihood of residents who did poorly on the OKAP receiving awards. No correlations could be made about the post-graduate plans of residents who failed a portion of the ABO exam.





Introduction

Increasingly, competitive fields of medicine have found it difficult to find fair and effective standards with which to judge applicants for residency. Oph-thalmology is no different as residency directors struggle to pinpoint a better method by which they can compare applicants on a level playing field[1]. While it has never been considered to be an abso-lute measure of future success in any one field, the U.S. Medical Licensure Examination (USMLE) is increasingly utilized to screen potential residency candidates. The USMLE seems to be a relatively simple method to equalize students from a wide


38

range of schools across the country where grades, awards, and other accolades may not hold the same value. Since every student has to take this national-ly regulated exam, it seems reasonable to use it as a way to compare students applying to the same resi-dency. However, it remains to be seen whether or not USMLEs are in fact a good predictive factor in how well a resident will perform during residency and ultimately successfully pass a licensing exam.Board certification in ophthalmology beyond com-pleting residency includes taking two examina-tions. After completing residency, the physicians are eligible to sit for the written qualifying exam (WQE) for board certification. Upon passing the WQE, the oral examination may be taken[2]. Be-ing able to predict success on American Board of Ophthalmology (ABO) certification examinations is important for residency directors. It would be im-portant to see if residents passing and doing well on the USMLE and Ophthalmic Knowledge Assess-ment Program (OKAP) exams consistently perform better and pass the ABO examinations. The OKAP is an annual in-training ophthalmology examina-tion taken during residency. This exam is meant to serve as a self-assessment for the resident and as a way for residency programs to assess progress of the resident and overall program effectiveness[3]. In ophthalmology, it would be useful to see whether high performances on the USMLE correlate with the yearly OKAP taken during residency. Similar studies have been done in other fields such as obstetrics and gynecology as well as emergency medicine[4,5]. While there have been studies in ophthalmology that have looked at criteria used by residency chairs when considering students for resi-dency positions[6], only one has looked specifically at USMLE scores predicting success in the yearly OKAP and final ABO certification exam[7]. This study, performed at Case Western Reserve Univer-sity, did not find a significant correlation between USMLE Step I scores and ABO exam performance. However, the same study did find a significant posi-tive correlation between OKAP performance and passing the ABO examinations on the first attempt. It would be valuable to see if residents at another institution would produce similar findings. Addi-tionally, no studies have looked at whether there are any associations between a resident’s perfor-mance on the OKAP examinations and his or her likelihood of receiving accolades and recognitions

during their time in residency. Prior studies have shown correlations between standardized exam performances. In other words, students who have performed well on previous standardized tests also tend to perform in a similar manner on future stan-dardized exams[6,8]. It would be interesting to see if this relationship can be seen specifically in the field of ophthalmology. Due to previous studies and the current stress placed on board scores, we hypothesize the following. High USMLE Step 1 scores will correlate with do-ing well on the yearly OKAPs as well as passing the ABO exam. Additionally, higher performances on the OKAPs will also relate to successfully passing the ABO. The objective of the study is to assess the relationship, if any, between USMLE Step I scores and OKAP exams as well as passing the ABO exam during the first attempt.

Methods

Ophthalmology resident files from Saint Louis Uni-versity from graduation years 1992 until 2009 were collected initially. However, residents graduating prior to 1998 consistently took the National Board of Medical Examiners Step I instead of the USMLE Step I examination. Although the USMLE was in-troduced in 1992, it was not uniformly utilized until 1998. Since the USMLE examination was a vital component of what was being assessed in this study, residents who did not have these scores could not be considered for this study. Therefore, 1998 served as the starting point of the final study data collection, while the 2009 files are the last com-pleted files. Every resident graduating from 1998 onwards was included in the study. One resident was missing a USMLE Step I score and the post-graduate paths of two residents could not be col-lected, but these individuals were all still included in the study for the correlations that could be drawn from their other information.Criteria used for analysis included USMLE Step I scores, OKAP year 1 scaled and percentile scores, OKAP year 2 scaled and percentile scores, OKAP year 3 scaled and percentile scores, awards during residency, results of the ABO WQE exam, and post-graduate career path pursued. Although the OKAP does not set parameters that allow an examinee to “pass” or “fail,” percentile scores above 25% were considered successful, while scores below this were considered “failing” for the purposes of this study.

USMLE Step I vs. OKAP Scores - Kantamneni and Cruz


39

correlation is considered significant at the 0.05 lev-el of the two-tailed test and the p-value was found to be 0.015 as can be seen in Table 1.For the 47 residents with ABO exam results, the OKAP year one scaled score had a 0.369 Pearson correlation with a p-value of 0.011 (Significance at P≤0.05, two-tailed). The OKAP year one percentile score had a 0.385 correlation and p-value of 0.008 (Significance at P≤0.01, two-tailed). The OKAP year two scaled score and the ABO exam results had a 0.434 correlation and p-value of 0.002 (Sig-nificance at P≤0.01, two-tailed). The OKAP year two percentile and ABO exam results had a 0.403 correlation and p-value of 0.005 (Significance at P≤0.01, two-tailed).The OKAP year three scaled score and ABO exam result had a 0.446 correlation and p-value of 0.002 (Significance at P≤0.01, two-tailed). The OKAP year three percentile score and ABO exam result had a 0.411 correlation and P-value of 0.004 (Significance at P≤0.01, two-tailed). These results may all be seen in Table 2.Although there are no established passing or fail-ing scores on the OKAPs, we have set scores above 25% successful and those below as not, or “fail-ing,” for our purposes. Residents who passed all of their OKAP exams had far greater likelihood of performing well on the ABO exam and passing than residents who failed one or more of the OKAPs. Likewise, residents who failed all of their OKAPs were more likely to fail the ABO exam than those who passed at least one OKAP.

The awards during residency that were recorded were whether or not the resident was the chief resident, the Excellence in Patient care award, and Outstanding Resident award. Results of the ABO exam categories included residents who passed on the first attempt, the number of times a resident failed the WQE, if no attempt was made to take the ABO exam, and if the resident failed the oral exam. When correlations between USMLE Step I scores and passing the ABO examination were comput-ed, residents were considered to “pass” if both the written and oral portions of the exam were success-fully completed on the first attempt and those who failed either portion of the exam on the first attempt were noted as “failing”. Post-graduate career path detailed a resident’s plans to go to general private practice, non-clinical research, general academic practice, government employment, and fellowship. Fellowships pursued included Glaucoma, Cornea, Pediatric, Plastics, and Retina.Data was analyzed through statistical analy-sis program, SPSS 15.0 as well as a combination of descriptive and non-parametric statistics and ANOVA. Pearson correlation as well as two-tailed significance was performed to determine correla-tions between USMLE Step I and the ABO exam result.

Results

The residency program at SLU has a maximum of four residents per year. Over a span of 12 years, that would lead to a sample size of 48. However, a resi-dent in the 1998 graduating class and another resi-dent in the 2000 graduating class did not take the USMLE Step I because it was not required by their states. Additionally, a resident’s file, who graduated in 1998, was missing an ABO exam result. As a re-sult, the total number of residents analyzed for any USMLE Step I score was 46, while the total num-ber of residents analyzed for any ABO exam result was 47. Of the 47 residents in the study, 5 individu-als failed at least one portion of the ABO exam at least one time. Three of these residents failed the written portion of the ABO exam on their first at-tempt, while one resident failed the oral portion on the first attempt, and one resident failed the written exam twice.For the 46 residents who had records of both their USMLE Step I scores as well as their ABO exam results, the Pearson correlation was r=0.355. The


Table 1. Correlation between American Board of Oph-thalmology (ABO) exam results and residents’ USMLE Part I exam scores taken during medical school.

ABO exam result

USMLE Part I

ABO exam result:

Pearson Correlation

Sig (2-tailed)

N

1

47

0.355 (*)

0.015

46

USMLE Part I:

Pearson Correlation

Sig. (2-tailed)

N

0.355(*)

0.015

46

1

46

* Correlation is significant at the 0.05 level (2-tailed).


40

None of the four residents with an average OKAP score of 33% or below was awarded Chief resident. One of the four residents who averaged less than a 33% on the three OKAP exams was awarded the Excellence in Patient Care Award. None of the four residents who, on average, scored a 33% or lower on the OKAP exams was awarded an Outstanding Resident award.Post-graduate plans were recorded from each resi-dent, except for one. This individual also failed the WQE for board certification once and no further in-formation was found. Only one resident who failed a portion of the ABO exam went on to do a fellow-ship. Others went into a practice of some kind of went into non-clinical research. This data can all be found in Table 3.

Conclusions

Our study shows evidence that there is a positive correlation between USMLE Step I scores and the ABO exam results at the end of residency. Higher USMLE Step I scores are associated with a higher likelihood of passing the ABO exam. No separate correlations were drawn for the individuals who failed the ABO exam. This is in disagreement with a previous study that was unable to show any di-rect correlation between USMLE scores and pass-ing the written portion of the ABO exam on the first attempt[7]. Performing well on the OKAPs throughout residency was also positively related to passing the ABO. It is interesting to note that the correlation between the OKAP and passing of the ABO examination was stronger with each passing year. Doing well on the OKAP during the second

year had a higher correlation with passing the ABO exam than did doing well on the first year OKAP. Likewise, Performing well on the OKAP during the third year had the greatest positive correlation with passing the ABO exam than other years. The prior study did find a positive correlation between perfor-mance on the OKAP examinations and passing the ABO written qualifying exam[7]. With the findings from our study, it does seem reasonable for residen-cy programs to predict an applicant’s future success on standardized examinations, such as the OKAPs and ABO exams using the USMLE Step I score.

Table 2: Correlation between American Board of Ophthalmology (ABO) exam and the Ophthalmic Knowledge Assessment Program (OKAP) examinations taken after each year of residency.

ABO Exam Result

OKAP Year 1 Scaled Pearson Correlation .369(*) Sig.(2-tailed) .011OKAP Year 1 Percentile Pearson Correlation .385(*) Sig.(2-tailed) .008OKAP Year 2 Scaled Pearson Correlation .434(**) Sig.(2-tailed) .002OKAP Year 2 Percentile Pearson Correlation .403(**) Sig.(2-tailed) .005OKAP Year 3 Scaled Pearson Correlation .446(**) Sig.(2-tailed) .002OKAP Year 3 Percentile Pearson Correlation .411(**) Sig.(2-tailed) .004

N=47* Correlation is significant at the 0.05 level (2-tailed).** Correlation is significant at the 0.01 level (2-tailed).

Table 3: Post-graduate plans of each resident along with whether or not they passed the American Board of Ophthalmology (ABO) exam.

Post Graduate Plans Passed Failed

Fellowship-Glaucoma 4Fellowship-Cornea 2Fellowship-Pediatric 6 1*Fellowship-Plastics 2Fellowship-Retina 7Fellowship: Do Not Know 2General Private Practice 18 1**VA: Government Employment 1***General Academic Practice 1Research (non-clinical) 1*Do Not Know 1*

N=47

*Resident failed WQE for board certification once

**Resident failed oral portion once

***Resident failed WQE for board certification twice



41

As was expected, residents who consistently passed all of the OKAPs had a much greater likelihood than those who failed one or more. Similarly failing all of the OKAPs had a high correlation with also failing the ABO exam during the first attempt than those who had passed at least one of the annual ex-ams. A previous study completed in ophthalmology found a similar relationship between the OKAP and the passage of the WQE portion of the ABO certifi-cation exam during the first attempt[9]. These find-ings may prove important for residency programs who may now be further encouraged to provide in-dividuals performing poorly on the OKAP exams with additional educational support in an effort to increase their chances of success on the ABO ex-aminations during their first attempt.Only one of the residents who failed a portion of the ABO exam continued with further educational training and did a fellowship. Others who failed the ABO exam went on to either work in a private practice, non-clinical research, or government em-ployment. No real associations between failing the ABO exams and post-graduate plans can be made because so few subjects failed. It is important to note that of the five residents who failed any part of the ABO exam, two failed the WQE for board certification on the first attempt, two failed the WQE for board certification twice, and one failed the Oral Certifying Exam once. There are a number of possible weaknesses in our study, most importantly being the small sample size. Though the data collected and analyzed was from a 12-year time frame, with four residents per graduating class, the sample size remains small. Additionally, residents who graduated earlier than 1998 almost all took the National Board of Medical Examiners step I examination. Since these scores were not comparable to the USMLE Step I scores, we could not include the residents from these earlier years to expand the sample size of our study. While residents who averaged a 33% or below on their OKAPs did not receive any awards, except for one receiving the Excellence in Patient Care award, no real correlation can be drawn between performing poorly on the OKAPs and not receiving any awards just because it is such a small sample size. Four residents are simply not enough to find a significant correlation of any kind. Additionally, the number of residents who failed the ABO exam was so small that no real accurate correlations can be made using them. For the same reason, no correlations can be

made about them and their post-graduate paths. It would be valuable if future studies looked at if similar correlations can be drawn between USMLE Step II scores and OKAP score and ABO exams performances. During the twelve year period from which the sample size for this study was gathered, Step II scores were not consistently collected, due to the fact that not all applicants supplied these scores. Only 8 of the 47 residents included in this study had recorded Step II scores, making it unreasonable to include this data in the study. If similar findings were found with Step II scores as were found with Step I scores in this study, it would further encour-age residency programs to use this as additional information to assess future residency candidates.

References

1. McCollister, RJ. The Use of Part I National Board Scores in the Selection of Residents in Ophthalmol-ogy and Otolaryngology. JAMA 1988;259:240-42.

2. “Become Board Certified.” American Board of Ophthalmology. http://www.abop.org 2010.[Ac-cessed June 15, 2010.]

3. “OKAP Exam.” American Academy of Ophthal-mology. http://www.aao.org/ 2010.[Accessed June 15,2010.]

4. Armstrong A, Alvero R, Nielsen P, Deering S, Rob-inson R, Fratarelli J, Sarber K, Duff P, Ernest J. Do U.S. Medical Licensure Examination Step I Scores Correlate with Council on Resident Education in Obstetrics and Gynecology In-Training Examina-tion Scores and American Board Of Obstetrics and Gynecology Written Examination Performance? Military Medicine 2007;172:640-43.

5. Thundiyil JG, Modica RF, Silvestri S, Papa L. Do United States Medical Licensing Examination (USMLE) Scores Predict In-Training Test Perfor-mance for Emergency Medicine Residents? Journal of Emergency Medicine 2010;3:65-69.

6. Nallasamy S, Uhler T, Nallasamy N, Tapino PJ, Volpe NJ. Ophthalmology Resident Selection: Cur-rent Trends in Selection Criteria and Improving the Process. Ophthalmology 2010;117:1041-47.

7. Johnson GA, Bloom JN, Szczotka-Flynn L, Zauner D, Tomsak RL. A Comparative Study of Resident Performance on Standardized Training Examinations and the American Board of Oph-thalmology Written Examination. Ophthalmology 2010;117:2435-39.

8. Borowitz SM, Saulsbury FT, Wilson WG. Informa-tion Collected During Residency Match Process



42


Does Not Predict Clinical Performance. Archives of Pediatrics and Adolescent Medicine 2000;154:256-60.

9. Chen PP, Bhandari A. Resident Performance on the Ophthalmic Knowledge Assessment Program and the Written Qualifying Examination for Board Certification. Journal of Academic Ophthalmology 2010;3:11-14.

10. Naylor RA, Reisch JS, Valentine RJ. Factors Related to Attrition in Surgery Residency Based on Applica-tion Data. Archives of Surgery 2008;143:647-52.

11. Pankratz MJ, Helveston EM. Ophthalmology: The Resident’s Perspective. Archives of Ophthalmology 1992;110:37-43.

12. “Services for Students and Residents.” National Board of Medical Examiners. http://www.nbme.org/ 2010.[Accessed June 15, 2010.]


43

Assessment of Student Preferences for Teaching in Ophthalmology Using a Conjoint Analysis ApproachAnthony J. King, MD, MMedSci, FRCOphth* and Alexander J.E. Foss, MD, MMedSci, FRCOphthDepartment of Ophthalmology, Eye & Ear Building, Queens Medical Centre Campus, Nottingham University NHS Trust, Nottingham NG 7 2UH, UK

*Corresponding author email: [email protected]

Abstract

Purpose: To determine using a conjoint analysis approach what aspects of an ophthalmology teaching module are valued by undergraduate medical students.

Methods: A prospective assessment of medical student’s preferences for delivery of an undergraduate ophthalmol-ogy module in a single centre tertiary care teaching hospital. One hundred consecutive medical students undertaking small group teaching as part of their ophthalmology modular teaching undertook the conjoint analysis exercise. A ranking exercise of scenario cards was undertaken. The scenario cards contained information regarding four aspects (attributes) of teaching, each attribute had several options (levels) available. The attributes considered were: (1) Grade of teacher (3 levels), (2) Size of teaching group (3 levels), (3) Number of hours of teaching per week (3 levels) and (4) Whether patients were present at the teaching (2 levels). The percentage importance of each of the attributes to the students was determined and for each attribute level a utility value was estimated.

Results: 98 students successfully completed the task. The percentage importance values were 33.4%, 30.69%, 17.69% and 18.21% for size of the group, grade of teacher, hours of teaching and whether or not patients were present respec-tively. The utilities generated indicated that students preferred a junior doctor, a small group size, the maximum hours of exposure and a patient to be present in the teaching session.

Conclusion: Undergraduate medical students have a clear preference for high quality teaching rather than quantity and clearly prefer to be taught by junior doctors.





Introduction

Medical students are adult learners and as such should follow the principals[1] for adult learning developed by Knowles. This suggests that adult learners are self directed, seek to acquire learning relevant to their immediate needs and aim to estab-lish an effective learning climate.

Medical students are regularly asked to provide feedback on the teaching they receive. This feed-back usually requires students to comment on the content and delivery of teaching and the quality of the teaching and teachers. Feedback may be used to identify positive and negative aspects of a course’s delivery and inform future planning and course development. The aim of such feedback is to un-derstand what student’s value. Although feedback is regularly undertaken, it is normally undertaken in a unidimensional manner asking for feedback on specific aspects of a course’s delivery as mentioned above. This information is gathered on these differ-ent aspects independently of each other and once gathered used to undertake course modification. However gathering feedback in this manner does not allow those delivering the feedback to provide


44

an integrated method of weighting the importance of each of the components assessed in the feedback process. Thus inappropriate weighting of aspects of course delivery may be assigned using this form of feedback. A more appropriate form of feedback would be a process in which weighting of the im-portance of aspects of course delivery was inherent in the feedback process.There are a number of ways of assessing prefer-ences including one-to-one interviews, didactic interviews, case study analyses, Delphi technique and complaints procedures. Group-based methods include focus groups, concept mapping, citizen ju-ries, consensus panels, public meetings and nomi-nal group techniques. All these techniques have distinct strengths and weaknesses. One particular technique - conjoint analysis performs well when judged on the basis of validity, reliability, general-izability, objectivity, acceptability to respondents, and costs[2]. Much of what people value happens at an intuitive level and this can not be accessed by direct ques-tioning and has to be inferred. Conjoint analysis performs this by asking people to select or rank sce-narios in order of desirability. They are not asked to justify or explain their preferences. It generates a quantitative outcome that allows ranking of the importance of the factors identified. This tech-nique has been widely used in marketing research and is being increasingly applied in medicine for determining patient preferences for service devel-opment[3,4] and in ophthalmology to determine patients preferences for treatment of various oph-thalmic conditions[5-7]. In the full concept (also known as full-profile) method the respondent is asked to rank a set of profiles (package scenarios) according to their pref-erence. All the factors (attributes) of interest in a potential package are represented at different lev-els on each profile so that a complete description (or full concept) of a potential package is provided. This approach allows the relative importance of different attributes to be assessed and shows what features individuals are prepared to trade to obtain what they think is most important. Importance val-ues (percentage importance) are generated for each attribute and utility scores are generated for each of the attributes levels and thus indicate the respon-dent’s value of each attribute level. In this prospective study we wished to ascertain

whether conjoint analysis could be undertaken suc-cessfully in a medical student population and to es-tablish what preferences students would express for delivery of an ophthalmic module. To our knowledge this is a novel approach in evalu-ating medical students preferences for delivery of teaching.

Methods

Ethic committee approval was granted by the Not-tingham Medical School Ethics Committee. As this study was deemed by the ethics committee to be a method of feedback, no specific consenting process for the students participating in this study was re-quired by the committee.A questionnaire based survey was performed to de-termine the opinions of medical students undertak-ing the ophthalmology module, regarding the deliv-ery of the current module in ophthalmology. From this survey a list of potential attributes for a teach-ing package which were thought to be important to medical students was compiled. Four factors were identified as most important by students and these factors formed the basis for the conjoint analysis (CA) scenarios. These factors were:1. Grade of teacher. (3 levels) 2. Size of teaching group (3 levels)3. Number of hours of teaching per week.

(3 levels) 4. Whether patients were present at the teaching.

(2 levels)The factors and their associated factor levels are shown in table 1. With these factors and factor lev-els, it is possible to generate 54 different scenarios (3 x 3 x 3 x 2 = 52) and rather than ask respon-dents to rank all 54 scenarios we used a factor de-sign (termed “orthoplan” in SPSS[8]) to randomly generate an “orthogonal array” of 9 scenarios. In performing this, it is assumed that interactions be-tween the different factors are negligible. An ex-ample of one scenario is shown in figure 1. In ad-dition two holdout scenarios were generated. They were ranked along with the other nine scenarios, but were not included in the main analysis. Instead, they were used for internal validity checks whereby the position that the participants ranked those pack-ages were compared with the position predicted by the conjoint analysis.

Student preferences - King and Foss


45


InterviewsInclusion criteria were all students undergoing the ophthalmology teaching module. There were no ex-clusion criteria. One hundred consecutive students attending ophthalmology tutorials were asked to rank the generated scenarios, all students agreed to undertake this process. All ranking sessions were supervised by one of the authors (AJK). The primary outcome was the percentage value that students placed on each of the factors included in the conjoint analysis model.In addition students undertaking the scenario rank-ing were asked to provide the following additional information: age, gender and whether they pos-sessed a previous degree.Data Entry and AnalysisThe demographic and ranking data were entered into an Access (Microsoft Corp., Redmond, Wash-ington) database and then transferred to SPSS. The conjoint analysis was performed using the “con-joint” procedure in SPSS categories. This proce-dure takes the ranking of the different scenarios for each participant, and through a series of linear regressions, generates utility scores for each factor level. The variables were initially coded as linear, but this resulted in a large number of reversals for the grade of teacher. A reversal is when a perceived less fa-vorable grade is valued more highly than a more favorable grade (or vice versa). This can occur in three situations; the task was not understood by the participants, the variable in question was unimport-ant and therefore was not significantly influencing the choices, or the specification of the variable as linear was inappropriate. We had assumed that a more qualified teacher would be more highly val-

ued than a less qualified teacher and had specified this as linear and ordered in the sequences allied health care professional, junior doctor, senior doc-tor (consultant) in ascending order of preference. Exploration of the data showed that this ranking was not being followed with a clear preference for the junior doctor as teacher over the other two grades. Accordingly, this variable was re-entered as discrete, which makes no assumptions about the or-der of each and it is the results of this analysis that is presented.SPSS calculates a regression coefficient for each factor and the utility scores are the product of the coefficients times the factor level. The relative im-portance of each factor can also be expressed in per-centage terms. This is done by taking the range of utility scores for any factor (highest minus lowest) and dividing it by the sum of all the utility ranges and multiplying by a hundred. The utility scores were saved and used in regression analyses against age, gender and having a previous higher degree.

Results

A hundred students participated in the study. Two participants failed to complete the task and the analyses were performed on a sample of 98 stu-dents comprising 42 men and 56 women. Their ages ranged from 20 to 39 years with mean age of 23 and median 22 years.

Table 1: Results of the conjoint analysis

FACTOR Factor Level Mean Utility

Mean Impor-tance

Grade of Teacher

Senior

Junior

Allied Medical

-0.4491

0.6944

-0.2454

30.69%

Group Size

1 - 2

5 – 10

> 20

-1.3387

-2.6574

-3.9861

33.40%

Hours of Teaching

3 hours

6 hours

12 hours

0.25

0.50

0.75

17.69%

Patients present

Yes

No

0.7917

-1.583318.21%

Profile Number 7

Group size 1-2

Grade of teacher Junior doctor

Hours of teaching 6 hours

Patients present NoFigure 1: example of a scenario card option


46

The most important factor to the students ques-tioned was the size of the group (33.4% impor-tance), followed closely by the grade of teacher (30.69% importance). There was no material dif-ference from the students perspective between the number of hours of teaching (17.69% importance) and whether or not patients were present (18.21% importance).The utilities generated for each of the factor lev-els assigned to the factors showed that for grade of teacher student preferred a junior doctor followed by an allied health care professional followed by a senior doctor. For group size students clearly pre-ferred a small group size and disliked most a large group size. For hours of teaching they preferred the maximum hours of exposure and they preferred a patient to be present in the teaching session.Analysis showed eleven reversals, all were for size of group. This time, inspection showed that for these eleven people, group size was relatively un-important (less than 15%, compared to 33% for the whole group) suggesting that these eleven were not basing their decision on this factor.Internal validity checks showed that the observed rankings for the 9 scenarios and the utility scores from the conjoint analysis were highly correlated (Pearson’s R = 0.948 P<0.00005) and the two hold-outs showed good correlation with the ranking pre-dicted by the conjoint analysis (Kendall’s tau = 1.0; P <0.000005), showing good internal validity. Linear regression of the importance against age, gender and previous degree failed to show predic-tive value for any of the utility scores.

Discussion

These results indicate that students are most in-terested in the size or the group they are taught in and the grade of person carrying out the teaching. These two factors accounted for over 64% of the importance. This suggests that students are con-cerned about the quality and not the quantity of their teaching.Group sizeA small pupil-to-teacher ratio is considered more effective form of teaching and thus will be one of the factors that influences the quality of the teach-ing received. Students may feel more “comfort-able” in a small group environment and may feel that they are in a better position to influence the fo-

cus of teaching in a direction of their choosing and provides them with opportunities to make enquiries that would be difficult in a larger group environ-ment. Thus small group size confers more control on teaching. Control of direction of the teaching will improve quality. In addition there may be an intuitive feeling on the part of the student that be-ing taught in a small group is more likely to result in patient contact as generally this is the situation in which patient exposure is more likely to occur. This may explain why exposure to patients was ranked as only the third most important attribute in this study.Grade of teacher The grade of teacher as the second most important factor indicates that student’s value the individual delivering the teaching as well as the content and environment in which teaching is being deliv-ered. The utility scores revealed for the grades of teacher, add an additional dimension with those surveyed preferring a junior doctor, followed by an allied health care professional and lastly by a senior doctor. It is understandable that medical students would feel more comfortable with junior doctor teaching as they are more likely to be their peers of similar ages and generally would be considered more approachable with less of a seniority gap. It is also likely in the context of ophthalmology that the knowledge base of a junior doctor would be more than sufficient to answer any medical student ques-tions. The choice of health care professional as the next most desirable teacher is again not surprising as in ophthalmology there are two specific allied health professional groups. Orthoptists and opti-cians who in our unit are involved in teaching some of the basic and essential clinical skills required for making ophthalmic assessments. Once again these individuals are often younger and nearer the age of the medical students. Finally, the choice of the se-nior doctor as the last choice as teacher may reflect the seniority gap and a discomfort on the medical students behalf of interaction with a senior doctor. Another explanation is that unsatisfactory teaching from senior doctors compared with the other groups influenced the students choice of teacher and so reflects upon the actual personnel delivering the teaching and not necessarily their grade.Patient exposureThe position of this factor as third most important to students was surprising to the authors as one re-



47

curring theme when talking to students is their de-sire for greater patient exposure. However in the context of this CA it has been traded off into third place. There may be several reasons for this. It is possible that students perceived that teaching in very small groups (their most important factor) is associated with patient exposure and consequently may have assumed that this was inherent in that group size without having to specifically state it. Similarly it is possible that students felt that small group teaching allowed them to exert better control over the teaching delivery and this may be more important in their opinion than exposure to patients. Finally they may have felt that through their con-trol of teaching within the small group environment they were more likely to be able to ensure patient exposure.Amount of teachingThis factor was of least importance in the overall analysis even though the utility values did suggest that students valued more rather than less teaching. Its position as least important suggests that students value time of exposure less than quality and believe that compromising on exposure time while maxi-mizing quality during that reduced time is the best form of teaching delivery.Another point that may influence students deci-sions are their generic preferences based on their experience of teaching in medical school up to this point and the experiences they have had in other teaching modules or attachments. Even though this study was carried out in the context of ophthalmol-ogy these generic influences could have influenced preference decisions and so the results may be more representative of students’ global attitudes to teach-ing rather than attitudes to ophthalmology alone.Conjoint analysis is one of several techniques de-veloped to assess people’s preferences. One of its major advantages is that it is a quantitative tech-nique that allows comparison of the relative impor-tance of identified factors. It is a reliable, objective, valid and cost efficient way of assessing preferenc-es in a manner which has good acceptability. In conclusion this study suggests that undergradu-ate medical students have a clear preference for high quality teaching rather than quantity and clear-ly prefer to be taught by junior doctors.

References

1. Knowles MS, et al. Andragogy in Action: applying modern principals of adult learning. San Francisco. CA: Jossey-Bass; 1984.

2. Ryan M, Scott DA, Reeves C, Bate A, van Teijlingen ER, Russell EM, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess. 2001;5(5):1-186.

3. Ryan M. Using conjoint analysis to take account of patient preferences and go beyond health outcomes: an application to in vitro fertilisation. Soc Sci Med. 1999;48(4):535-46.

4. Parker B, Srinivasan V. A consumer preference ap-proach to planning of rural health-care facilities. Operations Research. 1976;24:991-1025.

5. Ross MA, Avery AJ, Foss AJ. Views of older people on cataract surgery options: an assessment of prefer-ences by conjoint analysis. Qual Saf Health Care. 2003;12(1):13-17.

6. Bhargava JS, Patel B, Foss AJ, Avery AJ, King AJ. Views of glaucoma patients on aspects of their treatment: an assessment of patient preference by conjoint analysis. Invest Ophthalmol Vis Sci. 2006;47(7):2885-88.

7. Bhargava JS, Bhan-Bhargava A, Foss AJ, King AJ. Views of glaucoma patients on provision of follow-up care; an assessment of patient preferences by con-joint analysis. Br J Ophthalmol. 2008;92(12):1601-605.

8. Cranton P. Understanding and Promoting Transfor-mative Learning: a guide for educators of adults. San Fransisco, CA.: Jossey-Bass; 1994.



48

A Computer-Assisted Approach for Teaching Third-Year Medical Students to Identify Optic Disc and Fundus AbnormalitiesJanice G. Lee, BS1, Lynn K. Gordon, MD, PhD2,3, Michael B. Gorin, MD, PhD2, Sebastian Uijtdehaage, PhD4, JoAnn A. Giaconi, MD*2,3

1Albert Einstein College of Medicine, Bronx, NY.; 2Jules Stein Eye Institute, David Geffen School of Medicine, Uni-versity of California, Los Angeles, California; 3VA Healthcare System of Greater Los Angeles; 4 Center for Education Development and Research, David Geffen School of Medicine, University of California, Los Angeles, California.

*Corresponding author e-mail ([email protected]).

Abstract

Objective: To prospectively evaluate a teaching intervention’s effect on fundus pathology recognition by third-year medical students (MSIII).

Methods: During their one-week ophthalmology rotation, third-year medical students were tested for recognition of fundus photographs. Group (A) students were examined on Friday of the rotation. Groups (B) and (C) students were tested on Monday and Friday of the rotation. Group C students were instructed to listen to an online instructional lecture between exams. Correct answers were scored and analyzed in aggregate and for individual types of pathology.

Results: Data for 117 individuals was analyzed. The aggregate accurate Friday scores were 38.1%, 39.3%, and 44.5%, for groups A (n=24), B (n=23), and C (n=31), respectively (P=0.25). Scores for individual students improved between the first and second test in both groups B and C (P<0.0001) but the lecture intervention did not yield a difference in average aggregate individual scores in group C as compared to group B (P=0.89). However, recognition of a number of specific pathologic fundus findings (diabetic retinopathy, vein occlusion, and optic atrophy/pallor) improved sig-nificantly in the intervention group C as compared to group B (P=0.02 to P=0.03)

Conclusions: Recognition of important fundus pathology by MSIIIs improved during a one-week ophthalmology ro-tation. A focused online didactic session increased identification of specific important fundus pathologies. Yet, overall recognition of both normal fundi and pathology remains poor despite this intervention. This study justifies the need for future educational interventions designed to achieve recognition of important, sight-threatening and life-threatening ocular pathology by general practitioners.





Introduction

Ophthalmic problems make up a significant propor-tion (3-19%) of presenting complaints to general practices and emergency departments[1,2]. As the aging population grows, keeping the limited num-

ber of U.S. ophthalmologists busy, it will become more important for the generalist to have confi-dence in basic eye care skills and to make appropri-ate referrals. Ninety percent of residency program directors in family practice, internal medicine, and pediatrics surveyed in 1995 felt that <50% of their entering interns had adequate basic vision/eye examination skills[3], and over 85% of them voiced that these skills should be acquired in medical school[3]. Un-fortunately, exposure to subspecialties, and oph-thalmology in particular, has been declining. (Invest Ophthalmol Vis Sci 45:e-abstract 1404, 2004. http://abstracts.iovs.org/cgi/content/abstract/45/5/1404) Curricular time devoted to ophthalmology is 2-3%


49

at most schools[2,4,5]. This time does not appear to adequately train students, as 47% reported minimal to no confidence in funduscopic examination[6]. Correct fundus interpretation is an important skill in patient care. Best practices for teaching inter-pretation are not yet well defined. On a traditional introductory ophthalmology clerkship, students re-ceive lectures and observe clinical care. It is unclear if they are practicing the ophthalmic skills taught to them in the first two years of medical school and if they are acquiring the knowledge expected of them by the end of their clerkship. The purpose of this study was to assess ability to interpret ocular fundus findings with a computer-based quiz and to investigate whether a specific computer-based teaching session would improve this ability among third-year medical students.

Method

Approval was granted by the University of Cali-fornia, Los Angeles/South General institutional review board to prospectively evaluate the effect of a teaching intervention for third-year medical students (MSIII) on funduscopic/optic nerve dis-ease recognition. Study instruments were posted on the David Geffen School of Medicine’s (DGSOM) Learning Management System (LMS) website called ANGEL (A New Global Environment for Learning). This password-protected website posts all information related to the individual student’s curriculum. MSIIIs logged in at their leisure on the required days when study instruments were acces-sible for viewing and took as much time as needed to complete study instruments. To view study in-struments outside the specified days required an administrator to specifically unlock the instrument. In addition to the study instruments, all students participated in small group face-to-face lectures on ocular anatomy and pathology, the red eye, and retinal pathology with subspecialist faculty, and they observed clinic patients and surgery at UCLA teaching hospitals.Study population/designMSIIIs from DGSOM classes of 2010 and 2011 rotating through the required 1-week ophthalmol-ogy clerkship were asked to take a test and listen to a lecture. Although the test was mandatory to pass the clerkship, the score achieved on the test did not affect their pass/fail grade for the rotation. At the end of each test session, students could check off a

box to opt out of our study. Each week, the clerkship group of 3 to 4 MSIIIs was randomly assigned to either group A, B, or C. Group A took the test on the last day of the clerk-ship, while groups B and C took it on the first and last day. Hereforth, the first day of clerkship test is referred to as the Monday test and last day test as the Friday test. Additionally, students in Group C were instructed to watch a 20-minute online instructional lecture about the fundus at least once between test days. Group A was designed as a control, in order to assess the exiting knowledge of students undergo-ing the current 1-week curriculum. Group B gave students an opportunity to view test images at the beginning of their elective and assessed whether this exercise alone would improve their ability to identify the same images at the end of the week, either through self-directed study or focused learn-ing during that week. Group C was the intervention group, created in order to demonstrate whether an additional online didactic could augment students’ ability to identify funduscopic images compared to their colleagues. Written instructions were provided at clerkship start to complete necessary tasks, along with ver-bal reminders during the week. Access to the exam and lecture through ANGEL was granted accord-ing to group assignment. Clerkships lasted from 4 to 7 days with most students having a 5-day clerk-ship; 4-day clerkships were due to academic holi-days and 7-day clerkships occurred at the start of an 8-week rotation block where the first week had an additional 2 days. Group assignment was done randomly, in principal. As a high percentage of noncompliance became ev-ident within Group C, more groups were assigned to C in the hope that we would capture enough stu-dents completing all necessary C tasks, in order to balance out the numbers between groups. Study Instrument Both an online test and lecture were created. The test consisted of thirty photographs of diseased and normal optic fundi (and normal variants). Important ocular manifestations of systemic disease and com-mon eye diseases were chosen for the test. As well, fundus findings that might be confused with life or sight-threatening problems we wanted students to recognize were included. The images were present-ed through a “parallel extended matching format,” and were randomly shuffled each time the exam

Computer Assisted Teaching - Lee et al.


50

was taken. The same 12 answer choices (a-l) were presented below each image. Answer choice (a) was normal/normal variant, (b) developmental anomaly or pseudoedema, and (c-l) were abnormals, with a specific named entity: true edema, diabetic retinop-athy, macular degeneration or choroidal neovascu-larization, emboli or retinal artery occlusion, disc hemorrhage or shunt vessels, increased cup-to-disc ratio/glaucoma suspect/or glaucoma, vein occlu-sion, cotton wool spots, peripapillary atrophy, and optic atrophy/pallor. We chose to present digitized fundus images instead of using a simulated model/ophthalmoscope, in order to simply determine if students could recognize disease patterns rather than assess their clinical skills.Images were intended to be at a medium diffi-culty level to allow room for improvement. Dif-ficulty was validated by testing 15 ophthalmology residents (PGY-2 to -4), who scored an average of 88%. For three images, we recognized that a non-ophthalmologist could select one of two possible answers and therefore both responses were counted as correct. A PowerPoint (Microsoft, Seattle, WA) lecture with a slide set was created after the test was designed, and it specifically covered all tested entities. The lecture concentrated on anatomy and pathophysiol-ogy of the fundus to explain the visual manifesta-tions seen in specific diseases. An audio-recording simultaneously played with the slides to point out and discuss significant findings. Students could watch the online lecture as frequently as desired Tuesday through Thursday. The lecture and test used different example photographs in order to avoid simple image memorization. The goal was to help students recognize findings/patterns and then apply that knowledge to different patients. Data analysisStudents entered test answers on ANGEL and re-sponses were exported to a spreadsheet. Noncom-pliant students failing to perform tasks according to assignment were excluded from analysis. Taking the Monday test one day late was allowed, as was taking the Friday test within 4 days. Those who wanted to be excluded from the study or did not take the exam within the allowable time frame were excluded from analysis (see Table 1 for summary). Prism version 5.0 for Mac OS X (GraphPad Soft-ware, Inc., La Jolla, California) was used to per-

form t-tests and 1-way-ANOVA to compare stu-dents’ exam scores individually and between groups. P-value of <0.05 was considered statisti-cally significant.

Results

There were 40 one-week clerkships during data col-lection from March 2009 through January 2010. 132 students participated. Of these students, 15 (11.4%) did not give permission to use their test data for study, leaving 117 (88.6%) for possible analysis. Of the 117, thirty-nine students were excluded be-cause they did not complete required tasks as di-rected (see table 1). Group C students logged onto the lecture 1-3 times during the week (average 1.3 times); it is unknown whether each login represents a viewing of the lecture in its entirety. Mean Monday test scores for groups B and C were 25.5% and 30.2%, respectively, P=0.19, unpaired t-test. Mean Friday test scores were A 38.1%, B 39.3%, and C 44.5% with large standard deviations of 14.4%, 15.1% and 16.1%, respectively (figure 1, panel A). Differences between mean Friday test scores were not statistically significant (P=0.25, 1 way ANOVA). For individual students, average test scores im-proved from Monday to Friday in group B by 13.8% (P<0.0001, figure 1, panel B) and in group C by 14.3% (P<0.0001, figure 1, panel C). Eight students in both groups B (8/23, 34.8%) and C (8/31, 25.8%) at least doubled their score between Monday and Friday (data not shown). Conversely, a number of students’ scores did not change or de-clined: 3 of 23 students (13%) in group B (decline of up to -3.3%) and 8 of 31 students (25.8%) in group C (decline of up to -26.7%). Recognizing NormalsFour fundus photographs were normal/normal vari-ant with no other acceptable answer. On Monday, 37% and 46% of group B and C students correct-ly recognized normal as normal. By Friday, these numbers increased to 51% and 52%, respectively, with 49% of group A students identifying normal correctly on Friday (figure 2, panel B). On aver-age, one additional image was correctly identified as normal on Friday, and this change was statisti-cally significant for both groups (B, P=0.01 and C, P=0.03, figure 2, panel A). The percentage of stu-dents answering normal for an abnormal fundus im-



51

age was low in all 3 groups — ≤7% for all groups on Friday and <9.5% on Monday tests (data not shown). When students misidentified normal, there did not appear to be a single diagnosis commonly chosen. Recognizing Abnormals There were 2 to 4 questions per diagnosis. On Mon-day, a mean score of less than 45% was achieved for each diagnosis. Group C’s mean score for each diagnosis improved between Monday and Friday, while group B had smaller improvements or a decline (data not shown). The difference be-tween mean Friday scores (B vs. C) was statisti-cally significant in diabetic retinopathy (P=0.03, figure 3, panel A), vein occlusion (P=0.02, figure 3, panel B), and optic atrophy/pallor (P=0.03, fig-ure 3, panel C). While the mean Friday scores for optic nerve edema, artery occlusion/emboli, and macular degeneration were not different between

Figure 1. Did the intervention improve ability to cor-rectly interpret fundus images? Friday mean scores were similar for A, B, and C, (panel A) (P=0.25, 1-way ANOVA). Mean scores improved between Monday and Friday for B and C, (panel B and C, respectively) (P<0.0001, 2-tailed paired t test).

Figure 2. Could students identify normal as normal? Did the intervention help? The mean number of normal images correctly identified by groups B and C improved between tests. Changes within groups were statistical-ly significant (panel A) (*P=0.01, **P=0.03, 2-tailed paired t-test), while differences between groups were not (panel B) (P=0.78, unpaired t-test).



52

Figure 3. For which diagnoses did the intervention improve student‘s identification? The difference between B and C’s mean Friday scores for some diagnoses was statistically significant (panel A-C, *P<0.03). While Group C’s mean score improved for most diagnoses, B’s mean score did not improve significantly (panel D-F, *P<0.03) (paired and unpaired t-tests).



53

groups B and C, recognition of these entities im-proved greatly from Monday to Friday in group C (P=0.001 for nerve edema, P=0.03 for artery occlu-sion and P=0.001 for macular degeneration, figure 3, panels D-F). Images of abnormal optic nerves were most fre-quently misidentified as normal. These images had the highest number of students in each of the 3 groups consistently answering normal for a find-ing of hypoplasia (A 8.3%, B 4.4%, C 19.4%), disc hemorrhage (A 16.7%, B 21.7%, C 25.8%), glau-coma (A 11.1%, B 15.9%, C 14.0%), optic atrophy/pallor (A 37.5%, B 26.1%, C 17.7%). No student misidentified central retinal vein occlu-sion, cotton wool spots, or macular degeneration as normal. Diabetic retinopathy images were rarely misidentified as normal.

Discussion

We assessed the ability of MSIIIs to recognize clinically important ocular fundus findings before and after an ophthalmology clerkship rotation and whether the addition of a computer-based didactic teaching session augmented recognition. The inter-vention minimally improved overall recognition of findings, although it did improve recognition of specific diseases. Subspecialty exposure in medical schools, par-ticularly to ophthalmology, has declined over the last 20 years[4,7]. Less than 5% of the entire medical school curriculum is devoted to ophthal-mology at most schools around the world, with the majority of this time spent during the preclini-cal years[1,2,5,8,9]. This begs the question, how does one ensure that students acquire the expected knowledge base and practice skills in the short 4 to 5 days dedicated to clinical ophthalmology at most U.S. schools? Do students acquire adequate abil-ity to recognize important ocular fundus pathology with observational rotations as they are currently designed, does prompting them to what’s important increase their pursuit of those entities and improve recognition, or does a specifically designed teach-ing session do it best? The use of computer-aided learning (CAL) in medi-cal education has grown considerably in the last de-cade due to a number of attractive educational fea-tures. CAL can deliver consistent and standardized material to students, regardless of clerkship site and

preceptor availability[10]. It can be personalized to the learner’s pace, repeating, interrupting, and re-suming at will. It can circumvent geographical and temporal constraints. It is well suited to visually intensive and detail oriented subjects, such as oph-thalmology. Finally, there is an economy of scale in that once it is set up there is little cost of offering it to additional students[11]. A number of CAL studies in different medical spe-cialties exist, although only a small number are ran-domized controlled trials (RCT), and these RCTs are not without methodological flaws. Performance on a written assessment is the most frequently used outcome measure. Results are mixed but generally positive[11]. In ophthalmology, a small handful of studies have assessed the learning impact of digi-tal problem-based learning cases[12,13,14], digi-tally simulated pupil responses[15], and digital 3D animation of neuro-ophthalmologic problems[16]. They have all shown a positive impact of the CAL on test results and positive feedback from students. Our study shows mixed results compared to previ-ous CAL ophthalmic literature. At clerkship start, test scores were relatively low as expected (average correct <30%, range 6% to 66%). Both the study and control groups significantly improved their scores between Monday and Friday. The interven-tion group did better than group B, although this was not statistically significant. While no student had a Friday test score as high as the average resi-dent score of 88%, there were a number of students whose scores doubled over the week in groups B and C. On the other hand, there were two group C students who had large drops in score and a num-ber with smaller drops in both groups B and C. The drop in scores is perplexing. It may be attributed to lack of motivation to do well at the end of the week since scores did not affect final clerkship grade (ungraded tests and clerkships are a common trend at U.S. medical schools). In future studies of this type, motivation and interest should be assessed, as motivation can be a driving factor for how much a student learns and how they test. Because the average score was not discriminating between groups, we looked to see if the interven-tion helped in the recognition of certain fundus entities. In order to recognize abnormality, one must first know what is normal. Disappointingly, only 50% of the normal variants were identified as normals, which in practice may lead to unneces-



54

sary consultations and referrals. Groups B and C did slightly better on this task than group A. In the incorrectly identified normals, there was no single diagnosis used most frequently, which might have suggested a photo to be misleading. Students did tend to pick amongst the following answers: disc hypoplasia, disc hemorrhage, glaucoma, and optic atrophy/pallor, all of which may look like a normal nerve to the untrained eye. All students did fairly well in recognizing the pres-ence of an abnormality, although half the time they did not correctly identify the abnormality. The intervention group showed the biggest improve-ments in recognition of diabetic retinopathy, vein occlusion, optic atrophy/pallor, true edema, ar-tery occlusion/emboli, and macular degeneration. These are all diagnoses that should prompt referral to ophthalmology. Despite the intervention, many students had trouble recognizing glaucoma, which is another diagnosis that should prompt referral in clinical practice. Informally (no data collection), students did com-ment on test difficulty, uncertainty about their an-swers, and that the lecture was very useful and illustrative. It is unclear why there was not a big-ger overall improvement in test scores with the in-tervention. It may be that the rotation, without the computer-based teaching, is already covering many test entities; however, this does not explain why group C did better at identifying certain disease classes, such as diabetic retinopathy, which are very common in the clinics and already part of the teach-ing agenda. As mentioned before, there may have been a lack of motivation or enthusiasm to perform well on the ungraded test. Perhaps, the intervention was not adequately designed for an MSIII’s level. The lecture may have introduced new diseases to students without imparting complete understanding of their clinical patterns. This could be remedied with more disease examples and by making re-peated comparisons between diseases to help them differentiate similar clinical presentations. Based upon our experience with this study, we believe that a computer-based teaching session alone is not ade-quate, but that it needs to be complemented specifi-cally with a preceptor session. One published study on medical teaching commented that confidence but not skill increases with repetition without su-pervision[17]. It is possible that viewing the lecture alone created more questions that could have been clarified in a complementary live preceptor session.

We did not examine if students who watched the video before a Wednesday preceptor session did better on Friday than those who watched the lecture at the last minute before the Friday test. A limitation of the test design is the limited num-ber of test questions representing each diagnosis. A greater number of questions per diagnosis may have increased the discriminatory power to assess changes in ability to recognize fundus findings. However, when designing the test we did not want to create a lengthy test out of proportion to the week-long rotation. Another limitation of the study is that a significant number of students had to be excluded from analysis due to lack of compliance with directions. The relatively small numbers are in keeping with other studies in medicine on edu-cational topics. This study demonstrates that the one-week clinical rotation in ophthalmology does improve recogni-tion of ocular pathology by MSIIIs and the addi-tion of a focused online didactic lecture improves identification of specific types of pathology. How-ever, the level of recognition of normal and pathol-ogy among MSIIIs is poor. Additional efforts are needed to improve performance and refine teach-ing paradigms during such limited exposure peri-ods. This study represents our first efforts to cre-ate a standardized environment to introduce and compare teaching innovations into the curriculum. As greater longitudinal data is acquired, we will be able to prospectively develop and assess the impact of additional innovations in teaching that will lead to improvement in recognition of ocular pathology by medical students.

Acknowledgments

This work was supported in part by a Senior Re-search Fellowship grant from Albert Einstein Col-lege of Medicine of Yeshiva University, Bronx, NY. The grant supported work in the conduct of the study, collection, management, analysis and inter-pretation of the data, preparation, and review of the manuscript. We would like to thank Fei Yu, PhD for statisti-cal help at the beginning of this study. We would also like to thank Zhen Gu for technical support for the online material and Ruby Paz for assistance in directing the medical students. We would like to thank Michelle Li for her help with the preparation of figures.



55


References

1. Fan JC, Sherwin T, McGhee CNJ. Teaching of oph-thalmology in undergraduate curricula: a survey of Australian and Asian medical schools. Clin Experi-ment Ophthalmol. 2007; 35(4):310-317.

2. Vernon SA. Eye care and the medical student: where should the emphasis be placed in undergraduate ophthalmology? J R Soc Med. 1988; 81(6):335-337.

3. Stern GA. Teaching ophthalmology to primary care physicians. Arch Ophthalmol. 1995;113(6):722-724.

4. Jacobs DS. Teaching doctors about the eye: Trends in the education of medical students and primary care residents. Surv Ophthalmol. 1998;42(4):383-389

5. Spivey BE. Ophthalmology in medical student edu-cation. Philosophy, content, and process. Ophthal-mology. 1978;85(12):1299-1308.

6. Gupta RR, Lam WC. Medical students’ self-confi-dence in performing direct ophthalmoscopy in clini-cal training. Can J Ophthalmol, 2006; 41(2):169-174

7. Mottow- Lippa L, Boker JR, Stephens F. A prospec-tive study of the longitudinal effects of an embed-ded specialty curriculum on physical examination skills using an ophthalmology model. Acad Med. 2009;84(11):1622-1630.

8. Crombie AL. Ophthalmology in the undergradu-ate curriculum. Trans Ophthalmol Soc U K. 1976;96(1):33-34.

9. Stark D, Beinssen A, Morrey C. Ophthalmology in the undergraduate curriculum. A review in Queensland. Aust N Z J Ophthalmol. 1992;20(4):297-303

10. Leong SL, Baldwin CD, Adelman AM. Integrating web-based computer cases into a required clerk-ship: development and evaluation. Acad Med. 2003;78(3):295-301.

11. Greenhalgh T. Computer assisted learn-ing in undergraduate medical education. BMJ. 2001;322(7277):40-44.

12. Kong J, Li X, Wang Y, Sun W, Zhang J. Effect of digital problem-based learning cases on student learning outcomes in ophthalmology courses. Arch Ophthalmol. 2009;127(9):1211-1214.

13. Devitt P, Smith JR, Palmer E. Improved student learning in ophthalmology with computer-aided in-struction. Eye. 2001;15(5):635-639.

14. Stahl A, Boeker M, Ehlken C, Agostini H, Reinhard T. Evaluation of an internet-based e-learning oph-thalmology module for medical students. Ophthal-mologe. 2009;106(11):999-1005

15. Kaufman D, Lee S. Formative evaluation of a multi-media CAL program in an ophthalmology clerkship. Med Teach. 1993;15(4):327-340.

16. Glittenberg C, Binder S. Using 3D computer simu-lations to enhance ophthalmic training. Ophthalmic Physiol Opt. 2006;26(1):40-49.

17. Leopold SS, Morgan HD, Kadel NJ, Gardner GC, Schaad DC, Wolf FM. Impact of educational inter-vention on confidence and competence in the per-formance of a simple surgical task. J Bone Joint Surg Am. 2005;87(5):1031-1037.


56

A Simple Low-Cost Eye Model for Teaching Ophthalmic Laser ProceduresTodd W. Altenbernd, MD*, Lorna Grant, MDDepartment of Ophthalmology and Vision Science, University of Arizona, Tucson, AZ*Corresponding author email: [email protected]

Abstract

Purpose: To create an eye model for teaching ophthalmology residents basic ophthalmic laser procedures.

Methods: An eye model was created from a clear bouncy-type rubber ball marked with ink or paint. A modified bar clamp was used to position the eye in front of a laser slit lamp.

Results: Pen ink drawn on the outside of a rubber ball to simulate the trabecular meshwork can be viewed through a standard mirrored contact lens and lasered using a selective laser or an argon laser. When the laser strikes the ink it blanches indicating a successful application. Similarly, retinal pathology can be drawn or painted on the ball, viewed through a lens, and lasered, in a fashion similar to retinal photocoagulation. The eye models can be reused by simply reapplying the ink or paint. Residents found that using the eye models advanced their confidence and ability to perform laser procedures on patients.

Conclusions: To our knowledge this low cost, simple eye model is the first synthetic device of its kind to offer training for laser trabeculoplasty. With more effort the posterior segment can be simulated and photocoagulation added. The eye model is well received by residents and serves as a means of attaining some of the skills needed for laser surgery before progressing to human eyes. The total material costs of several eye models and a slit lamp mounting device is under $15.00 with each eye model and mount being reusable.

Accepted for publication May 23, 2012Journal of Academic Ophthalmology 2012; 5:56-60Available via open-access on the web at http://www.academic-ophthalmology.com



Introduction

Teaching the procedural skills of ophthalmic laser can be difficult for the resident, attending, and pa-tient. Traditionally, the resident studies the basics of the procedure from reading and didactics, observes their mentor performing the procedure, and then operates the laser for the first time on a patient. For many residents the hardest part of the procedure is the hand-eye coordination involved in establishing the view while holding the mirror contact lens in place and applying the laser to correct location with the opposite hand. This is second nature once the

skills are acquired but observing the novice strug-gle with the procedure is what led to the develop-ment of this eye model. Several investigators have proposed the use of enu-cleated human eyes, or animal eyes for practicing laser procedures, however there are challenges as-sociated with these methods[1-4]. The Center for Disease Control guidelines suggests avoiding the use of animal tissue on instruments which subse-quently be used for humans, but for many training programs it is not affordable to purchase and main-tain separate laser equipment for the sole purpose of teaching with animal eyes[5]. Furthermore, pre-served tissue is prone to corneal edema which may obstruct the view. Lenart et al. proposed a method for circumventing this problem by replacing the edematous cornea with a polymethylmethacrylate lens[6]. However, by relying on preserved human or animal tissue for any part of the model one is still limited by the natural time constraints of dealing with tissue, which will eventually spoil and only be used a single time. Therefore, synthetic eyes may be more practical.


57

Here we propose a simple artificial model eye for teaching laser procedures which can be easily made from low cost materials, mounted on any laser slit lamp, and re-used at any time.

Methods

The artificial eye model was created using a clear rubber bouncy-type ball (Party City, Rockaway, NJ) which was marked on the outside with black ink from a regular ball-point pen in a circumferential linear manner 1-2 mm posterior to and parallel with the equator of the ball to represent the trabecular meshwork (Figure 1). Using the schematic (Figure 2) the model holding device was constructed from a 12” ratchet bar clamp (Harbor Freight, Calaba-sas, CA), a block of wood, and assorted hardware, which are available at most local hardware stores. Simple hand tools including a hand chisel, pop rivet device, electric drill, wood saw, side cutters,

Figure 1. 24 mm or 35 mm diameter clear bouncy ball. Ball point pen ink applied 1-2 mm anterior to and paral-lel with the equator simulates the trabecular meshwork. A central cartoon image obscures the posterior pole lim-iting this particular ball to anterior segment laser only.

Figure 2. Color-coded design diagram and materials list for constructing the slit lamp mounted eye model holder. The holder is attached to the distal end of the bar of a bar type ratchet clamp.

Low-cost eye model for teaching - Altenbernd and Grant


58

needle nose pliers and screwdriver were sufficient for the fabrication of this device (Figures 3,4). The eye model was then fitted into the bar clamp hold-ing device (Figure 5) which was clamped to the laser slit lamp chin rest (Figure 6). A 1- or 3-mir-ror Goldmann lens or a Latina lens was apposed to the bouncy-ball eye with methylcellulose as a coupling agent. The laser settings for the Selective Laser Trabeculoplasty (SLT) were 1.0-1.5 mj and for the argon laser trabeculoplasty (ALT) power = 800-1200 mw, duration = 0.1 s, spot size = 100 um. 200-300 applications completed a 360 degree tra-beculoplasty.

Results

An eyeball model system for laser procedures was easily constructed from inexpensive and common parts. The ink line drawn on the model, represent-ing the trabecular meshwork, was viewed through a mirrored contact lens and treated using selective or argon laser. The holding device was secured to the chin rest to simulate a right or left eye. The la-ser dissipated the black ink on the model leaving a clear circular endpoint. The results of laser applica-tion were viewed with the unaided eye or by using a slit lamp. The residual ink was removed with an alcohol pad or soap and water and reapplied with an ink pen to the same eye model and the procedure was repeated multiple times. Similarly, focal and peripheral retinal pathology can be drawn or painted on the posterior aspect of cer-tain clear rubber balls and lasered to simulate reti-nal photocoagulation. The ball shown in our figure had a cartoon image in the center, which blocked the view to the posterior pole, however while com-pletely transparent rubber balls may be used for this purpose they are less commercially available with-out special order. A clear ball with inclusions such as glitter or confetti will often suffice and these are readily available. These procedures can be repeated by reapplying the paint and without significant deg-radation to the model.

Conclusion

This simple low-cost eye model system is a useful tool for teaching ophthalmology residents how to perform basic laser procedures such as ALT, SLT, PRP and focal retinal laser. To our knowledge this is the first artificial eye model developed for this

purpose. This model differs from previous model eyes in that it does not require the use of human or animal tissue. The entire model system was built for less than $15.00 and all of the parts can be pur-chased from readily available retailers and assem-bled with common household tools. This synthetic eye model offers several major ad-vantages over using human or animal eye models to teach laser procedures. First, it may be mounted on a laser slit lamp also used for patients thereby eliminating the problems associated with medical device contact with animal tissue. Additionally, successful laser application may be immediately observed by the student as the ink blanches, and then later assessed by a mentor by simply looking at the outside of the ball. Lastly, these model eyes are inexpensive, clean, reusable, and require mini-mal preparation.There are, however, several considerations to take into account when comparing this eye model to hu-man subjects. The power levels needed to blanch the ink on the eye model and endpoint reactions are different. Also the eye anatomy of the model is greatly simplified. The eye model allowed more movement of the contact lens in the x-y direction and in general it was more challenging to perform the laser procedures. Additionally, the number of laser application needed to complete a 360-degree trabeculoplasty far exceeded the standard 90-110 used in the human eye. Residency programs or teaching courses may find this model useful in teaching beginning residents. At our institution this model is used during the first year ophthalmology glaucoma rotation. Resi-dents have been pleased with the model and report a greater level of confidence prior to operating on human eyes. Future work may involve developing more realistic anatomical landmarks and possibly making a fully assembled eye model commercially available for other ophthalmology teaching pro-grams. Laser Safety: Ophthalmic lasers used with this learning tool include argon, diode, krypton and fre-quency doubled Nd:Yag which are generally class 3b or class 4 lasers operating within the visible light spectrum, 400-700 nm. Photochemical dam-age to the eyes and skin may occur from direct or indirect exposure. OSHA standards demand a well-delineated area of hazard, limiting the occupants in this area to essential personnel, covering reflective



59

Figure 3. Fully assem-bled eye model holder. Unlike most holders this device is clamped to the chinrest and can be moved up and down us-ing the chinrest adjust-ment wheel simulating a real procedure situation.

Figure 4. Bottom birds-eye view. An aluminum pressure plate secures the sliding holder in place anywhere along the bar of the ratchet bar clamp.

Figure 5. Close up birds-eye view of holder with eye model in place.



60

Figure 6. The eye model is secured in a holder, which is clamped to the chin rest of the slit lamp to simulate a left eye. The 3-mirror Goldmann lens is secured to the eye model

surfaces with nonflammable, non-reflective ma-terial and labeling the external entrance(s) of the protective area with warning signs which designate the type and power of the laser and clearly indicate when the laser is in use. Most important is proper eye protection, which must be used by all persons within the hazard zone. Eye protection may be in the form of dropdown filters, which are part of the delivery system, or filtering safety goggles. In the latter case the goggles should be clearly tagged to the corresponding laser(s). Designating the pairing of the laser and goggles is important not only to ensure that the appropriate wavelength is filtered but also the optical density of the filter which is dic-tated by the laser’s maximal power[7].

References

1. Shammas AV, Minckler DS. Autopsy eye model for laser trabeculoplasty and iridectomy. Am J Ophthal-mol. 1986;102(5):664-65.

2. Minckler DS, Gaasterland D, Erickson PJ. Im-proved, reusable, autopsy eye model container for laser trabeculoplasty and iridectomy. Am J Ophthal-mol. 1992;113(3):341-42.

3. Oram O, Gross RL, Severin TD, Feldman RM, Orengo-Nania S. A human cadaver eye model for anterior and posterior segment laser applications. Ophthalmic Surg. 1994;25(7):449-51.

4. Weidenthal DT. The use of a model eye to gain en-dophotocoagulation skills.[letter]. Arch Ophthal-mol. 1987;105:1020.

5. Center for Disease Control Healthcare Infection Control Practice Advisory Committee. Guidelines for Environmental Infection Control in Health-Care Facilities. 2003. 119-125.

6. Lenart DT, et al. A contact lens as an artificial cornea for improved visualization during practice surgery on cadaver eyes. Arch Ophthalmol. 2003;121:16-19.

7. Occupational Safety and Health Administration. OSHA Technical Manual. Section III: Chapter 6. pp 1-26, Effective date 1/20/1999. http://www.osha.gov/dts/osta/otm/otm_iii/otm_iii_6.html#app_iii:6_1



61

Case Report

Introduction

Amiodarone is an important drug used primarily to treat ventricular and supraventricular arrhyth-mias but also is used for other cardiac disorders. It is a class III anti-arrhythmic metabolized by the cytochrome p450 system with class I, II, and IV anti-arrhythmic effects which can produce multiple drug interactions and systemic side effects[1]. Ami-odarone has been implicated in optic neuropathy resulting in either asymptomatic or mild to severe permanent visual loss cases. Classic amiodarone optic neuropathy is characterized by an insidious onset, slow progression of bilateral visual loss, and prolonged disk edema. The ophthalmologist needs

Bilateral optic neuropathy following amiodarone administration: The case for systems based competency in ophthalmologyShazia Ali1, Derrick Pau, MD2, Andrew G. Lee, MD*1,2,3

1Baylor College of Medicine, Houston, Texas; 2The Department of Ophthalmology, The Methodist Hos-pital, Houston, Texas; and the Departments of Ophthalmology, Neurology, and Neurosurgery at the Weill Cornell Medical College (Professor), The Department of Ophthalmology, The University of Iowa Hospitals and Clinics (Adjunct Professor), and The University of Texas Medical Branch, Galveston, TX (Clinical Professor)3; Ms. Ali is a medical student at Baylor College of Medicine (BCM) and Dr. Lee is an adjunct professor of Ophthalmology at BCM.*Corresponding author e-mail: [email protected]

Keywords: amiodarone, optic neuropathy, systems based practice

to be aware of amiodarone optic neuropathy, the systems based issues surrounding its use and po-tential toxicity, and should be prepared to counsel both patients and prescribing physicians about the risks of the medication. The Accreditation Coun-cil for Graduate Medical Education (ACGME) de-fines systems based practice as medicine which can “demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value”[2]. Sys-tems based practice relies on physicians to be able to look beyond the current needs of their patient to coordinate the care of their patient while working within the context of the healthcare system. We present a case of amiodarone optic neuropathy in order to emphasize the specific systems based com-petencies for ophthalmology in the management of patients on amiodarone.

Case Presentation

A 77-year-old Caucasian male has a past medical history of coronary artery disease and hypertension. He was status post multiple coronary angioplas-ties and a coronary artery bypass graft and was on chronic amiodarone therapy. The patient had well






62

controlled hypertension, gout, elevated cholesterol, and hypothyroidism. He had right ankle surgery in 1985 and surgery for sequela of a compartment syndrome after his bypass surgery. Past ocular his-tory was significant for bilateral cataract extraction 4 to 5 years ago. His medications included: amio-darone, amlodipine, allopurinol, isosorbide, clopi-dogrel, simvastatin, levothyroxine, and transdermal nitroglycerin. He presented with a chief complaint of a “dark gray-black spot” in his right temporal field while rubbing his left eye. Prior to presentation, he was given oral clonidine for hypertension with a maxi-mum systolic blood pressure of 210mmHg. The clonidine produced hypotension and was discon-tinued. His vision was 20/200 on the right and 20/20 and asymptomatic on the left. There was a right relative afferent pupillary defect. The motil-ity exam was full. Slit-lamp examination showed bilateral pigmentary vortex keratopathy in the in-ferior cornea consistent with amiodarone keratopa-thy. The patient had no signs or symptoms of giant cell arteritis. The visual fields showed a denser right inferior arcuate field defect and a left inferior arcu-ate defect (Figure 1). The patient had bilateral sec-tor disk edema with mild superimposed sector pal-

lor (Figure 2). Several physicians saw the patient prior to presentation to the neuro-ophthalmology service including the prescribing primary doctors, the cardiologist, and two eye care providers but the medication was not discontinued. Neuroimaging and other laboratory testing was performed on the outside and was negative. The patient was referred to The Methodist Hospital neuro-ophthalmology service where an eventual diagnosis of amiodarone optic neuropathy was made and the drug was dis-continued. The disk edema resolved and the patient remained stable off drug with stable visual function in both eyes.

Discussion

Amiodarone has been approved for anti-arrhythmic use in the United States since 1985 and case reports of optic neuropathy have been reported as early as 1987. Amiodarone therapy has been shown to result in many significant side effects, including corneal microdeposits in more than 90% of patients, optic neuropathy, blue-gray skin discoloration, photosen-sitivity, hypothyroidism, hyperthyroidism, pulmo-nary toxicity, peripheral neuropathy, and hepatotox-icity[3]. Hypothyroidism is seen in 6% of patients on amiodarone and may have been related to our

Figure 1: Humphrey visual field 24-2 reveals A) left inferior arcuate defect and B) denser right inferior arcuate field defect.

Bilateral optic neuropathy - Ali et al.


63

patient’s hypothyroidism[4]. Additionally, amioda-rone has a number of drug interactions. Warfarin, for example, has one of the most potent interactions and is potentiated by amiodarone, resulting in sig-nificant bleeding episodes in patients; reduction of warfarin dosage by 25-45% is necessary depending on amiodarone maintenance dose[5]. The spectrum of side effects and drug interactions implicated with this drug shows the necessity of approaching patient care with a systems based mindset so the overall wellbeing of the patient is better addressed. Although it is possible that the patient’s neuropathy was secondary to his acute hypertensive episode and not due to amiodarone induced optic neuropa-thy, the bilateral nature of the disk edema and the dechallenge results suggest toxicity. Amiodarone has been shown to be a potentially life-saving med-ication and the over diagnosis of amiodarone op-tic neuropathy can place the patient at significant cardiac risk. Therefore, all physicians need to look for characteristic signs and symptoms of amioda-rone optic neuropathy in patients presenting with NAION on amiodarone before withdrawing thera-py so patients are not denied life saving treatment. However, in this patient, the bilateral pigmentary vortex keratopathy, bilateral sector disk edema, and superimposed sector pallor provided enough characteristic evidence to discontinue amiodarone therapy until the diagnosis of amiodarone optic neuropathy is ruled out.

The system based competency issues for ophthal-mologists stem from the several increasingly com-mon and potential alarming practices of ambulatory outpatient ophthalmology. Anecdotally, we have seen more and more patients referred to our neuro-ophthalmology service with medication lists that include less and less information about drug names, dosage, route, or duration. In addition, many oph-thalmologists rely heavily upon ophthalmic tech-nicians or other physician extenders to obtain the medical history including systemic medications. These ophthalmology medication lists, especially in busy ophthalmic practices can sometimes be ap-pallingly vague (e.g., “see list”, “some type of lung medicine”, “some type of cardiac drug”, “some type of blood pressure drug”). In many ambulatory eye patients, the chief complaint, diagnosis, evalu-ation, and management of ophthalmic condition do not depend critically on obtaining a complete and accurate medication history. In some patients (in-cluding amiodarone optic neuropathy) reviewing the medical list has critical medical and potentially medicolegal implications.There have been numerous expensive legal settle-ments associated with amiodarone optic neuropa-thy over the past few years, the largest of which in-volved Wyeth-Ayerst pharmaceuticals in 1997[6]. Previously, the U.S. warning label did not address potential visual loss as a side effect, resulting in a $23.3 million settlement and re-labeling the medi-

Figure 2: Color photograph of the A) right optic nerve which reveals moderate disk edema associated with superior and temporal disk pallor. Appearance of the B) left optic nerve reveals moderate disk edema more pronounced superi-orly associated with flame-shaped hemorrhages. There is an incidental inferior juxtapapillary choroidal nevus seen OS and some retinal pigment epithelial change in the macula OD.



64

cation to place more emphasis on the risk of amio-darone optic neuropathy in both the “warnings” and “adverse reactions” sections[7]. The reported inci-dence of optic neuropathy with amiodarone is as high as 1.79% and the frequency of lawsuits associ-ated with amiodarone shifted the emphasis from a pharmaceutical liability issue to a physician mal-practice issue[8]. Ophthalmologists need to be not only aware of the optic side effects of amiodarone, but also the systemic side effects including cardiac, dermatologic, endocrine, hepatic, neurologic, oph-thalmologic, and pulmonary sequelae[3]. Patients should be assessed for endocrine or hepatic issues, yearly for cardiac or pulmonary complications, and as needed for the dermatologic, neurologic, or oph-thalmic conditions. Although diagnosing amioda-rone associated optic neuropathy is difficult, it is important to always consider this possibility for medical as well as medicolegal reasons.The Physicians’ Desk Reference (PDR) recom-mends the use of amiodarone for frequently recur-ring ventricular fibrillation and hemodynamically unstable ventricular tachycardia unresponsive to other antiarrhythmics. The contraindications are severe sinus-node dysfunction, marked sinus bra-dycardia, second or third degree AV block, and cardiogenic shock. The PDR advises the use of amiodarone only for life-threatening arrhythmias due to its high toxicity and states optic neuropathy/neuritis as a possible side effect. Patients should be advised to look for blind spots in central or para-central vision, Amsler grid changes, flashes of light, light sensitivity, and night blindness. Furthermore, patients need to be counseled on the risk of optic neuropathy and how the risk overlaps with vasculo-pathic risk factors, adverse drug-drug interactions, and to consult a cardiologist whenever a new drug is prescribed[3].In the future, the use of the electronic medical record may reduce the system errors which can worsen iatrogenic medication related side effects including visual loss. Studies have shown that the widespread use of EMR could eventually save $81 billion annually while improving healthcare ef-ficiency and safety by better managing the treat-ment of chronic disease[9]. The potential benefits of EMR are opposed by the relatively small number of physicians adopting this system; only 15-20% of physicians offices and 20-25% of hospitals were us-ing EMR in 2005. In patients like ours with multi-ple diagnoses and medications, EMRs would prove

to be advantageous not only because are the records able to be continuously updated, but also because EMR enhances the ability to coordinate informa-tion exchange between different EMR systems in multiple healthcare facilities. This might allow fast-er and easier communication between a prescribing cardiologist and a screening ophthalmologist re-garding amiodarone administration, potential side effects, and dose changes which may decrease the incidence or lessen the severity of amiodarone op-tic neuropathy. EMR will allow a full integration of patient’s medical history between all the specialists seeing the patient raising awareness of any systems based issues that may arise in the patient.The Accreditation Council for Graduate Medical Education (ACGME) core competencies at the resi-dency level requires residents training at an accred-ited institution to obtain competencies in 6 major areas, one of which is systems based medicine. In addition to effective patient care, clinical medical knowledge, professionalism, and communication, all physicians training need to be fluent in the art of systems based medicine to adequately coordinate the care of their patient. Furthermore, residency programs also must demonstrate effective plans by which residents are assessed in systems based practice mechanisms and provide performance feedback to their residents. By integrating the pri-ority of complete and accurate medication histories during the initial training of physicians, we believe that systems based practice can improve overall pa-tient safety. Studies of 5 different family medicine residency programs have shown that residents were better able to identify safety issues and propose sys-tems based solutions when trained using ACGME core competencies tailored to emphasize patient safety[10]. These residents were better able to find cases of polypharmacy, propose medicine replace-ments, and come up with systems based solutions that better fit the needs of the patient.

Residents and Fellows Perspective

Knowledge of the exact medications a patient has been prescribed is vitally important when obtaining the patient’s history. The faculty member often de-pends entirely upon our medication history. In ad-dition, patients will often report the trade names of medications they are taking and it is the physician’s responsibility to know or at least look up the generic names of the medications if he or she is unfamiliar with the trade names. As this case revealed, the pa-



65

tient was taking amiodarone which has known toxic effects on the eye, particularly optic nerve function. Prescribing physicians should be familiar with the side effects of medications he or she is prescribing and should routinely monitor for these side effects. Systems based practice is sometimes a difficult competency for residents and fellows to fully un-derstand. Patient safety, patient satisfaction, and pa-tient centered care are all important aspects of SBP. Understanding the full importance of the ACGME competencies requires that learners integrate and apply the more theoretical concepts within their own specialties, in their own practices, and in real patients. Detailed and careful history taking as it pertains to a patient’s medications is a vital com-ponent of patient safety. Imparting the significance of and practice in obtaining a comprehensive list of a patient’s medications must be emphasized in the earliest stages of physician training. As all phy-sicians become more vigilant regarding a patient’s medications, our patients will face less undue harm and receive the best care possible.

Conclusions

Amiodarone can produce a bilateral optic neuropa-thy and, although it is uncommon, ophthalmolo-gists need to be aware of this possibility and should include a systems based approach to patients with disk edema. Systems based practice provides en-hanced physician performance through patient-centered care while improving patient safety as well as patient satisfaction. For amiodarone, care-ful analysis of the patient’s medication lists by all physicians involved in the case, review of the known side effects, and enhanced communication among healthcare providers can allow the patient and doctors to make better informed decisions on their care. This case also highlights the importance of risk-benefit analysis in the coordination of pa-tient care as well especially identifying errors and implementing potential solutions. We believe that this care of amiodarone optic neuropathy is but one example that makes the case for systems based ap-proach to medicine in ophthalmology.

References

1. Trivier JM, Libersa C, Belloc C, Lhermitte M. Ami-odarone N-deethylation in human liver microsomes: Involvement of cytochrome P450 3A enzymes. Life Sciences 1993;52(10):PL91-PL96.

2. ACGME: Bulletin, Assessment of the six general competencies. Pg 4. Fall 2007. Outcome Project at http://www.acgme.org/outcome/comp/General-Competencies-Standards21307.pdf. Last accessed on 26 July 2011.

3. Vasallo P, Trochman R. Prescribing Amiodarone: an evidence based review of clinical indications. JAMA 2007; 298(11):1312-22.

4. Biondi B, Fazio S, Carella C, Amato G, Cittadini A, Lupoli G, Sacca L, Bellastella A, Lombardi G. Cardiac effects of long term thyrotropin-suppres-sive therapy with levothyroxine. J Clin Endocrinol Metab 1993;77: 334-38.

5. Sanoski CA, Bauman J. Clinical observations with the amiodarone/warfarin interaction. Chest 2002;121(1):19-23.

6. Chen D, Hedges TR. Amiodarone optic neuropathy – review. Seminars Ophthalmol 2003;18(4):169-73.

7. Mindel JM. Amiodarone and optic neuropathy – a medicolegal issue. Surv Opthalmol 1998;42(4)358-59.

8. Feiner LA, Younge B, Kazmier FJ, et al. Optic neu-ropathy and amiodarone therapy. Mayo Clin Proc. 1987;62:702-17.

9. Hillestad R, Bigelow J, Bower A, Girosi F, Meili R, Scoville R, Taylor R. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Affairs 2005;24(5):1103-17.

10. Singh R, Naughton B, Taylor JS, Koenigsberg MR, Anderson DR, McCausland LL, Wahler RG, Rob-inson A, Singh G. A comprehensive collaborative patient safety residency curriculum to address the ACGME core competencies. Medical Education 2005;39(12):1195-1204.


Acknowledgement

Funding/Suport: This work was supported in part by an unrestricted grant from Research to Prevent Blindnesss, Inc., NY, NY

This collection of audio clips is designed as a supplement for studying ophthalmology and cannot replace our time spent reading the Basic and Clinical Science Course (BCSC) books. I hope these series of questions and answers will trigger your memory to recall various differentials and treatments. This collection of questions and answers are not meant to be exhaustive, but designed to reinforce what you already know. Topics range from cornea to retina. The audio clips can be played on your iPod (or other MP3 player), computer or in your car using an MP3 player. The idea is to help maximize your study time while in traffic, jogging, or simply studying at your desk. I hope this collection will be helpful to you as one of many adjuncts

we use to learn about our great field of ophthalmology.

High-yield review for the OKAP and ophthalmology boards. An audio book of over 850 questions and answers covering the etiology, differential diagnoses, and treatments of eye

diseases.

Listen to Ophthalmology Buzzwords on your iPod, computer or in your car using an MP3 player.

Topics include:Contact Lenses, Cornea, Lens and Cataract

Neuro-Ophthalmology, Optics and Refraction Orbit, Eyelids, and Lacrimal Pediatric Ophthalmology, Retina, Uveitis

Synopsis:

www.medrounds.org/buzzwords/

TM

Journal of Academic Ophthalmology

Documents

Transcript of Journal of Academic Ophthalmology