Strategies to Improve Patient Safety: The Evidence Base Matters

Strategies to Improve Patient Safety: The Evidence Base Matures

In 2000, the Agency for Healthcare Research and Qualitycommissioned a report titled, “Making Health Care

Safer: A Critical Analysis of Patient Safety Practices.” Thereport analyzed and rated nearly 80 patient safety strategies(PSSs) (1). It was heralded by many but also generatedcontroversy about the role of evidence in assessing thevalue of PSSs (2). Since its publication, regulators, accredi-tors, and payers have pushed health care organizations toadopt various “safe practices” and avoid adverse events con-sidered largely preventable (3). Partly as a result, healthcare provider organizations are striving to improve patientsafety as never before.

When “Making Health Care Safer: A Critical Analysisof Patient Safety Practices” was published, the science sup-porting PSSs was immature. There was inadequate evi-dence to recommend interventions and how to implementthem and limited methods to measure the effect of safetyinterventions. In the face of such limitations, several na-tional programs, such as those to prevent wrong-site sur-gery and to implement medication reconciliation, were dis-seminated on the basis of face validity alone.

During the past decade, clinicians, researchers, andpolicymakers gained a greater understanding of the epide-miology of errors and preventable harms. The burden islarger than previously thought. Although we do not knowexactly how many patients experience preventable harm,we know that, for example, 44 000 to 80 000 patients dieeach year in the United States of diagnostic errors, 68 000of decubitus ulcers, and many thousands of teamwork andcommunication errors and failure to receive evidence-basedinterventions (4, 5). We also learned that implementingPSSs aimed at certain targets (for example, reducing healthcare–associated infections and venous thromboembolism)can substantially reduce errors and harm (6, 7).

Unfortunately, recent data indicate that the degree ofsuccess in eradicating preventable harm has not matchedthe investment in effort and financial resources. Studiesthat examined some practices that had tremendous intui-tive appeal, such as reducing resident duty hours and im-plementing rapid-response teams, yielded conflicting re-sults (8, 9). Examples of unintended consequences of PSSsemerged (10), and successful implementation was found insome cases to be highly context-dependent (11). Three re-cent U.S. studies showed continuing high rates of prevent-able harm in hospitals (12–14)—one showed evidence ofno improvement in adverse event rates from 2003 to 2008(12).

Against this backdrop, the Agency for Healthcare Re-search and Quality commissioned a team led by investiga-tors at RAND Health; Stanford University; the Universityof California, San Francisco; and Johns Hopkins Univer-sity to reexamine the evidence behind key PSSs. Reexami-nation involved several systematic reviews that addressed

the effectiveness of particular practices, paying attention tothe importance of implementation, context, and any unin-tended consequences of safety interventions.

In a special supplement that accompanies this issue,we present the evidence reviews underpinning 10 of the 41PSSs studied in the new report. These strategies includeinterventions to reduce diagnostic errors (15), in-facilityfalls (16), pressure ulcers (17), and delirium (18); effortsinitiated in hospitals to improve care transitions (19) andmedication reconciliation (20); interventions in inpatientsettings to promote a patient safety culture or climate (21);implementation of rapid-response systems (22); examina-tion of the effect of nurse–patient staffing on patient out-comes (23); and use of simulation exercises to improvepatient safety (24). The supplement also includes an over-view article that describes the entire reexamination processand identifies 10 strongly encouraged and 12 encouragedPSSs that are ready for adoption now (25).

In reviewing this literature, we found evidence ofprogress in bringing science to the field of patient safety.The evidence base about the effectiveness of interventionsto reduce harms grew steadily. For example, we now havestrong evidence that safety interventions have resulted in anational reduction in 1 type of harm: central line–associatedbloodstream infections in intensive care units (6).

Although still imperfect, measures of harm have im-proved. Guidelines that inform the design and descriptionof patient safety intervention studies are available, and theimportance of context in implementing interventions ismore widely appreciated (11). Researchers recognize thatpatient safety is a legitimate field of scholarly endeavor—worthy of career focus, requiring formal training, and pro-viding a path for academic success—although there is adearth of support for training programs, and applied re-search remains below basic research in the academic peck-ing order. Federal support for patient safety research hasimproved, although it will need to increase even more tomeet needs.

Physicians and other health care professionals, profes-sional societies, medical boards, and accreditation bodieshave focused efforts to reduce preventable harms. Practic-ing clinicians increasingly see patient safety as somethingthat they do rather than something that is done to them. Inthe early years after “Making Health Care Safer: A CriticalAnalysis of Patient Safety Practices” was published, mostPSSs were driven by external forces, such as accreditors orinsurers. Now, many of these efforts are driven by profes-sional norms in which patient harms are viewed as a socialproblem that physicians, working with others, are capableof solving.

Although substantial gaps in the evidence base remain,more than enough evidence exists to prompt decisive ac-tion. For example, strong evidence shows that multicom-

Annals of Internal MedicineEditorial

350 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1) www.annals.org

Downloaded From: https://annals.org/ by Steven Levenson on 03/06/2013

ponent interventions aimed at certain safety targets, such asprevention of falls and pressure ulcers, can significantlyreduce harm. All hospitals should implement checklist-based initiatives to prevent central line infections and haveprograms aimed at improving safety culture. Certainthemes underlie successful PSSs, including the develop-ment of a motivated, trained, and resourced interdisciplin-ary team (16). Such teams can convert desired interven-tions into checklists or other system-based tools thatpromote desired behaviors, focusing on the interventionswith the strongest risk reduction and lowest risks.

Researchers will note the need for additional study ofmany PSSs, such as interventions for care transitions andmedication reconciliation and how best to engage patientsand families in improving safety. Research is also needed todevelop better measures of harm and context. Further-more, we need additional studies to identify the best mod-els for training, organizing a safety program, integratingsystems engineering approaches into clinical environments,and taking full advantage of information technology whileavoiding unintended negative consequences. To accom-plish this goal, patient safety research will need to movefrom interdisciplinary (diverse researchers working on di-verse problems) to multidisciplinary (diverse researchersworking on common problems while maintaining theirown conceptual models) to transdisciplinary (diverse re-searchers working on common problems by using commontheories) methods.

A decade ago, our early enthusiasm for patient safetywas accompanied by a hope, and some magical thinking,that finding solutions to medical errors would be relativelystraightforward. It was believed that by simply adoptingsome techniques drawn from aviation and other “safe in-dustries,” building strong information technology systems,and improving safety culture, patients would immediatelybe safer in hospitals and clinics everywhere. We now ap-preciate the naivety of this point of view. Making patientssafe requires ongoing efforts to improve practices, training,information technology, and culture. It requires that seniorleaders supply resources and leadership while simultane-ously promoting engagement and innovation by frontlineclinicians. It will depend on a strong policy environmentthat creates appropriate incentives for safety while avoidingan overly rigid, prescriptive atmosphere that could sap pro-viders’ enthusiasm and creativity.

Although we have become more sophisticated aboutthe challenges of keeping patients safe over the past decade,the fundamentals have not changed. We need competent,well-trained providers equipped with high-quality evidenceand working with talented, strong leaders using well-designed and integrated technologies and sound policies.We hope that the evidence reviews published in the sup-plement contribute to those efforts by identifying PSSsthat will help keep patients safe.

Robert M. Wachter, MDUniversity of California, San FranciscoSan Francisco, California

Peter J. Pronovost, MD, PhDJohns Hopkins Medicine Patient Safety and QualityBaltimore, Maryland

Paul G. Shekelle, MD, PhDWest Los Angeles Veterans Affairs Medical Center and

RAND CorporationSanta Monica, California

Disclaimer: The authors declare that no competing financial interestsexist. All statements expressed in this work are those of the authors andshould not in any way be construed as official opinions or positions ofthe University of California, San Francisco; Johns Hopkins University;RAND Corporation; U.S. Department of Veterans Affairs; Agency forHealthcare Research and Quality; or U.S. Department of Health andHuman Services.

Financial Support: This work was supported by funding from the Agencyfor Healthcare Research and Quality, U.S. Department of Health and Hu-man Services (contract HHSA-290-2007-10062I).

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2570.

Corresponding Author: Robert M. Wachter, MD, Room M-994, Uni-versity of California, San Francisco, 505 Parnassus Avenue, San Fran-cisco, CA 94143; e-mail, [email protected].

Current author addresses are available at www.annals.org.

Ann Intern Med. 2013;158:350-352.

References1. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.Making health care safer: a critical analysis of patient safety practices. Evid RepTechnol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]2. Leape LL, Berwick DM, Bates DW. What practices will most improve safety?Evidence-based medicine meets patient safety [Editorial]. JAMA. 2002;288:501-7. [PMID: 12132984]3. Wachter RM. Patient safety at ten: unmistakable progress, troubling gaps.Health Aff (Millwood). 2010;29:165-73. [PMID: 19952010]4. U.S. Department of Health and Human Services. The Surgeon General’sCall to Action to Prevent Deep Vein Thrombosis and Pulmonary Embolism.Washington, DC: U.S. Department of Health and Human Services; 2008.5. Newman-Toker DE, Pronovost PJ. Diagnostic errors—the next frontier forpatient safety. JAMA. 2009;301:1060-2. [PMID: 19278949]6. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, etal. An intervention to decrease catheter-related bloodstream infections in theICU. N Engl J Med. 2006;355:2725-32. [PMID: 17192537]7. Streiff MB, Carolan HT, Hobson DB, Kraus PS, Holzmueller CG, Demski R,et al. Lessons from the Johns Hopkins Multi-Disciplinary Venous Thromboembo-lism (VTE) Prevention Collaborative. BMJ. 2012;344:e3935. [PMID: 22718994]8. Shetty KD, Bhattacharya J. Changes in hospital mortality associated withresidency work-hour regulations. Ann Intern Med. 2007;147:73-80. [PMID:17548403]9. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams:a systematic review and meta-analysis. Arch Intern Med. 2010;170:18-26.[PMID: 20065195]

EditorialStrategies to Improve Patient Safety

www.annals.org 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1) 351


10. Sittig DF, Singh H. Defining health information technology-related errors:new developments since to err is human. Arch Intern Med. 2011;171:1281-4.[PMID: 21788544]11. Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al.Advancing the science of patient safety. Ann Intern Med. 2011;154:693-6.[PMID: 21576538]12. Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA,Sharek PJ. Temporal trends in rates of patient harm resulting from medical care.N Engl J Med. 2010;363:2124-34. [PMID: 21105794]13. Classen DC, Resar R, Griffin F, Federico F, Frankel T, Kimmel N, et al.‘Global trigger tool’ shows that adverse events in hospitals may be ten timesgreater than previously measured. Health Aff (Millwood). 2011;30:581-9.[PMID: 21471476]14. Levinson DR; Department of Health and Human Services. Adverse Events inHospitals: National Incidence Among Medicare Beneficiaries. November 2010.15. McDonald KM, Matesic B, Contopoulos-Ioannidis DG, Lonhart J,Schmidt E, Pineda N, et al. Patient safety strategies targeted at diagnostic errors.A systematic review. Ann Intern Med. 2013;158:381-9.16. Miake-Lye IM, Hempel S, Ganz DA, Shekelle PG. Inpatient fall preventionprograms as a patient safety strategy. A systematic review. Ann Intern Med.2013;158:390-6.17. Sullivan N, Schoelles KM. Preventing in-facility pressure ulcers as a patientsafety strategy. A systematic review. Ann Intern Med. 2013;158:410-6.

18. Reston JT, Schoelles KM. In-facility delirium programs as a patient safetystrategy. A systematic review. Ann Intern Med. 2013;158:375-80.19. Rennke S, Nguyen OK, Shoeb MH, Magan Y, Wachter RM, Ranji SR.Hospital-initiated transitional care interventions as a patient safety strategy. Asystematic review. Ann Intern Med. 2013;158:432-40.20. Kwan JL, Lo L, Sampson M, Shojania KG. Medication reconciliation dur-ing transitions of care as a patient safety strategy. A systematic review. Ann InternMed. 2013;158:397-403.21. Weaver SJ, Lubomski LH, Wilson RF, Pfoh ER, Martinez KA, Dy SM.Promoting a culture of safety as a patient safety strategy. A systematic review. AnnIntern Med. 2013;158:369-74.22. Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapid-response systems as a patient safety strategy. A systematic review. Ann InternMed. 2013;158:417-25.23. Shekelle PG. Nurse–patient ratios as a patient safety strategy. A systematicreview. Ann Intern Med. 2013;158:404-9.24. Schmidt E, Goldhaber-Fiebert SN, Ho LA, McDonald KM. Simulationexercises as a patient safety strategy. A systematic review. Ann Intern Med. 2013;158:425-31.25. Shekelle PG, Pronovost PJ, Wachter RM, McDonald KM, Schoelles K, DySM, et al. The top patient safety strategies that can be encouraged for adoptionnow. Ann Intern Med. 2013;158:365-8.

Congratulations to Octavian Toma, MD, DESA, winner of the 2012 Annals Personae prize. Dr. Toma’s pho-togragh was published on the cover of the 4 December 2012 issue (vol. 157, no. 11) and is reprinted below.

For more information on the Annals Personae prize and to view a list of past winners, go to www.annals.org/public/personaephotographyprize.aspx.

Editorial Strategies to Improve Patient Safety



Current Author Addresses: Dr. Wachter: Room M-994, University ofCalifornia, San Francisco, 505 Parnassus Avenue, San Francisco, CA94143.Dr. Pronovost: 600 North Wolfe Street, Meyer 295, Baltimore, MD21287-7294.

Dr. Shekelle: RAND Corporation, 1776 Main Street, PO Box 2138,Santa Monica, CA 90407-2138.

Annals of Internal Medicine

www.annals.org 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 1) W-173


Patient Safety Strategies: A Call for Physician Leadership

The American health care enterprise, by far the mostexpensive in the world, continues to deliver, on aver-

age, a mediocre product (1). At its best, health care in theUnited States can be superb. It is fraught, however, withuneven quality, extremely expensive interventions, and anenvironment that continues to be remarkably unsafe forthe patients it serves (2).

Work highlighted in the supplement on patient safetystrategies that accompanies this issue (3) shows that therehas been some progress since the publication in 2000 ofthe landmark Institute of Medicine report “To Err is Hu-man: Building a Safer Health System” (4). A project team(supported by the Agency of Healthcare Research andQuality, from the RAND Corporation; Stanford Univer-sity; the University of California, San Francisco; JohnsHopkins University; and ECRI Institute) examined the ev-idence base underpinning particular patient safety prac-tices. With the help of an international panel of experts,they created a framework for reviewing studies about pa-tient safety strategies. The group scrutinized 158 patientsafety topics and selected 41 for either an in-depth system-atic review (18 topics) or a brief review (23 topics) thatfocused on emerging data or new insights about imple-menting the strategy. Reviews entailed current literaturesearches; a priori–determined selection criteria; assessmentsof the quality of studies of safety interventions; and evalu-ation of context, implementation, and adoption issues. Onthe basis of the reviews, the expert panel rated the strengthof evidence for each safety strategy: They recommendedthat 10 patient safety strategies should be “strongly encour-aged” for adoption and 12 strategies should be “encour-aged” for adoption.

The methods and findings of this review process arethoughtfully constructed and should be useful in futureevaluations of evidence about strategies. In many cases,further evidence and validation are needed before the pa-tient safety intervention can be accepted as proven.

Regardless, the analysis raises questions about why ithas been so difficult over the past 12 years to obtain mean-ingful safety data and more substantially improve patientsafety. The report reveals important clues and possibleremedies. Physicians have been notoriously unwilling toconsider treatment protocols, checklists, and, more re-cently, “bundles” that describe approaches to patient care(5). These have been criticized and labeled as “cookbookmedicine,” while physicians argued for tailored care, em-phasizing the uniqueness of every patient. The data thatwere reviewed, however, clearly demonstrate that checklistsand bundles can substantially improve patient safety andquality of care. In training and in practice, it is essentialthat health care professionals understand that bundles ofcare provide a touchstone on which reproducible qualitycan be achieved while making the modifications that may

be required for each patient. Some electronic health re-cords have begun to incorporate this kind of informationfor the practitioner.

The enormous variability of care provided is a majorchallenge to the identification of evidence-based, safe care.Increasingly, it is important that physicians in each hospi-tal and practice identify the steps that they will take in thediagnosis and care of the most common clinical problemsthey confront and follow these protocols with appropriateindividual variation. Sometimes, they can build on work ofprofessional societies and other organizations that providesuch guidelines, but there is nothing to prevent the ortho-pedic surgeons in a given hospital from agreeing on thefundamental approaches that they will take in the care of apatient requiring a knee replacement, which will impacteverything from prophylactic antibiotic use to protocols forincreasing physical activity. Agreement among colleagueson a standard approach to community-acquired infection,hospital-acquired infection, or initial treatment of hyper-tension would not only provide a much higher consistencyin patient care but also provide the basis for serious clinicalinvestigation to compare outcomes when variations inthese protocols are considered. In many cases, the evidenceis unclear about the proper protocol, but until existingapproaches are systemized, it will be very difficult to obtainthe evidence.

Another lesson from these reviews is the importance ofthe system of care provided by a health care team. Some ofthe most successful programs have been those that mini-mize the roles of the physicians and maximize those ofnursing or respiratory staff. It is ironic but often the casethat the less the physician is required to do in the course ofmaintaining a bundle of care, the more likely it is thatprotocols will be followed and outcomes improved. Ashealth care insurance coverage increases and the populationgrows with increasing amounts of chronic illness, the rolesof nonphysician health professionals, including nurses andpharmacists, to provide quality and timely care can onlyexpand. They are essential parts of an effective team thatrequires physicians to understand team function and theirown leadership roles. Teams will be central to newer deliv-ery models.

Hand hygiene represents another extraordinary exam-ple of the importance of physician behavior in reducinghospital-acquired infections. Despite all of the evidenceabout the germ theory of infection, many physicians some-how believed that their hands and stethoscopes were im-mune from the transmission of these organisms. Creatingsituations in which physicians demonstrate the key rolethey have as models for other members of the health careteam is critical. Once again, the physician must be a leader.

We are very proud of the health sciences research en-terprise, which has produced remarkable results in improv-

Annals of Internal Medicine Editorial

© 2013 American College of Physicians 353


ing outcomes in many aspects of disease, such as cancerand congestive heart failure. However, it is remarkablethat, despite the loss of life from lapses in quality of careand patient safety, so little money is invested in research inthis area. The project team in the supplement calls forenhanced education and training for those interested inhealth care and patient safety research. Providing adequatefunding for well-designed research in health care deliverywould be an important magnet to bring the best and thebrightest health care professionals and researchers intothese studies. To improve survival from illness throughbiomedical research only to lose patients because of poorquality of care is unacceptable.

The articles in the supplement provide important op-portunities for the future. Success depends on multiplicityof factors, including policies, reimbursement models, healthcare delivery models, systems, education, and financing.Critical to all of them is the role of the individual physicianwho is fully committed to the effort and is adaptable in achanging world in which bundles, protocols, team care,electronic data collection, and evidence-based treatmentparadigms are undertaken. Historically, when physicianshave not responded to the key elements of health care, suchas access, cost, and quality, others have imposed solutionson them. Quality of care remains the arena in which phy-sician leadership can still provide direction, innovation,and enhanced satisfaction for patients and caregivers.

Kenneth I. Shine, MDUniversity of Texas SystemAustin, Texas

Potential Conflicts of Interest: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M13-0111.

Requests for Single Reprints: Kenneth I. Shine, MD, University ofTexas System, 601 Colorado Street, Austin, TX 78701; e-mail,[email protected].

Ann Intern Med. 2013;158:353-354.

References1. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al.The quality of health care delivered to adults in the United States. N Engl J Med.2003;348:2635-45. [PMID: 12826639]2. Mehtsun WT, Ibrahim AM, Diener-West M, Pronovost PJ, Makary MA.Surgical never events in the United States. Surgery. 2012. [PMID: 23257079]3. Shekelle PG, Pronovost PJ, Wachter RM, McDonald KM, Schoelles K, DySM, et al. The top patient safety strategies that can be encouraged for adoptionnow. Ann Intern Med. 2013;158:365-8.4. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building aSafer Health System. Washington, DC: National Academies Pr; 2000.5. Gawande A. The Checklist Manifesto: How to Get Things Right. New York:Metropolitan Books; 2010.

READER’S COMMENTS

Readers may comment on published articles at www.annals.org. Whilethis service is free to Annals subscribers, readers without subscriptionswho wish to comment on articles may purchase temporary access.

Editorial Physician Safety Strategies



The Top Patient Safety Strategies That Can Be Encouraged forAdoption NowPaul G. Shekelle, MD, PhD; Peter J. Pronovost, MD, PhD; Robert M. Wachter, MD; Kathryn M. McDonald, MM; Karen Schoelles, MD, SM;Sydney M. Dy, MD, MSc; Kaveh Shojania, MD; James T. Reston, PhD, MPH; Alyce S. Adams, PhD; Peter B. Angood, MD;David W. Bates, MD, MSc; Leonard Bickman, PhD; Pascale Carayon, PhD; Sir Liam Donaldson, MBChB, MSc, MD; Naihua Duan, PhD;Donna O. Farley, PhD, MPH; Trisha Greenhalgh, BM BCH; John L. Haughom, MD; Eileen Lake, PhD, RN; Richard Lilford, PhD;Kathleen N. Lohr, PhD, MA, MPhil; Gregg S. Meyer, MD, MSc; Marlene R. Miller, MD, MSc; Duncan V. Neuhauser, PhD, MBA, MHA;Gery Ryan, PhD; Sanjay Saint, MD, MPH; Stephen M. Shortell, PhD, MPH, MBA; David P. Stevens, MD; and Kieran Walshe, PhD

Over the past 12 years, since the publication of theInstitute of Medicine’s report, “To Err is Human:

Building a Safer Health System,” improving patient safetyhas been the focus of considerable public and professionalinterest. Although such efforts required changes in policies;education; workforce; and health care financing, organiza-tion, and delivery, the most important gap has arguablybeen in research. Specifically, to improve patient safety weneeded to identify hazards, determine how to measurethem accurately, and identify solutions that work to reducepatient harm. A 2001 report commissioned by the Agencyfor Healthcare Research and Quality, “Making HealthCare Safer: A Critical Analysis of Patient Safety Practices”(1), helped identify some early evidence-based safety prac-tices, but it also highlighted an enormous gap betweenwhat was known and what needed to be known.

For the past 4 years, with support from the Agency forHealthcare Research and Quality, our group (a projectteam from the RAND Corporation; Stanford University;the University of California, San Francisco; Johns HopkinsUniversity; and ECRI Institute) and an international panelof 21 stakeholders and evaluation methods experts con-ducted an evidence-based assessment of patient safety strat-egies (PSSs). Our efforts involved 3 phases. In the firstphase, we developed a framework for reviewing existingstudies and prospectively evaluating new PSS implementa-tion studies (2). This framework identified several keypoints about the importance of theory, context, and imple-mentation (Table 1) (2).

The second phase was a review of current patientsafety strategies. We started with the 79 topics in MakingHealth Care Safer and added practices from the NationalQuality Forum’s 2010 update, the Joint Commission, andthe Leapfrog Group; those we identified in an initial scop-ing search; and those suggested by experts. From this list of158 potential topics, we used several rounds of voting withour stakeholders to narrow the scope to 41 PSSs that theexpert panel judged to be most important to the largestaudience. Given limited time and resources, we prioritizedtopics as needing either a traditional systematic review oronly a “brief review.” The latter generally focused on aspecific aspect of the PSS, such as emerging data or newinsights about implementation.

We chose 18 topics for in-depth reviews. As a first stepfor the reviews, we searched for existing relevant systematicreviews. To assess the potential utility of such reviews, wefollowed procedures proposed by Whitlock and colleagues(3) and asked the following questions: Is the existing re-view sufficiently “on topic” to be of use? Is the review ofsufficient quality to foster confidence in the results? If wedetermined that the existing systematic review was suffi-ciently on topic and of acceptable quality, we took 1 of 2further steps. In some cases, we did an “update” search(that is, we searched databases for all new relevant evidencepublished since the search end date in the existing system-atic review); in others, we conducted searches for “signalsfor updating.” Such searches generally followed the criteriaproposed by Shojania and colleagues (4), which involved asearch of high-yield databases and journals for pivotal stud-ies that could signal that a systematic review is out of date.A pivotal study is one that may call into question the re-sults of a previous systematic review. We added any evi-dence identified in either the update search or signalssearch to the evidence base from the existing systematicreview. Some PSSs had no existing systematic reviews andothers had previous reviews that were not of sufficient rel-evance or quality to be used. In those situations, we con-ducted new searches using existing guidance (5).

Evidence about context, implementation, and adop-tion was a key focus of our reviews. We searched for evi-dence on these aspects of primary studies in 2 ways. First,we sought and extracted data about context, implementa-tion, and unintended harms from articles that evaluatedthe effectiveness of PSSs. Second, we identified “imple-mentation studies” from our literature searches. Thesestudies focus on the implementation processes, particularlyelements demonstrated or hypothesized to be of specialimportance for the success, or lack of success, of the inter-vention. To be eligible, implementation studies needed to

Ann Intern Med. 2013;158:365-368.For author affiliations, see end of text. Annals of Internal Medicine

See also:

Web-OnlyCME quiz (Professional Responsibility Credit)

Annals of Internal Medicine Supplement



either report or be linked to reports of effectivenessoutcomes.

The 23 brief reviews were explicitly designed not to befull systematic reviews or updates. The goals of each briefreview varied by PSSs, according to needs identified bytechnical experts and stakeholders. The brief review couldfocus primarily on information about the effectiveness ofan emerging PSS or implementation of an established PSS.Alternatively, the review could explore whether new evi-dence calls into question the effectiveness of an existingPSS or identifies unintended consequences of safety inter-ventions. In general, a content expert on the topic, workingwith the project team, conducted the brief reviews. Themethods involved focused literature searches for evidencerelevant to the specific need. Typically, the author narra-tively summarized the evidence in a format tailored to theparticular goal of the brief review.

We used standard instruments, such as the CochraneEffective Practice and Organisation of Care criteria (6), theU.S. Preventive Services Task Force criteria (7), and theCochrane Risk of Bias criteria (8), to assess the quality orrisk of bias for individual studies of safety interventions.We developed criteria to evaluate strength of evidenceacross studies of effectiveness (9) that were informed byexisting methods (10, 11) and incorporated criteria aboutthe use of theory and description of implementation.

All of the reviews can be found in the Agency for Health-care Research and Quality evidence report, “Making HealthCare Safer II: An Updated Critical Analysis of the Evidencefor Patient Safety Practices” (9). In this supplement issue, wepresent the reviews for 10 PSSs. In an upcoming issue of BMJ

Quality & Safety, we will present several more. A summary ofthe evidence for all 41 PSSs is available in Table 1 of Chapter44 in that report (9). It categorizes each PSS according to thefollowing: the scope of the underlying problem that the PSSaddresses (its frequency and severity); the strength of evidenceabout the effectiveness of the safety strategy; the evidence orpotential for harmful consequences of the strategy; a roughestimate of the cost of implementing the strategy (low, me-dium, or high); and an assessment of the difficulty of imple-menting the strategy.

In the last phase of our effort, the expert panel explic-itly considered the strength and quality of evidence abouteffectiveness and implementation for each PSS and con-cluded that 22 PSSs are ready to be encouraged for adop-tion by health care providers (Table 2). The first 10 arethose that the expert panel believed should be “stronglyencouraged” for adoption. The remaining 12 are ones they“encouraged” for adoption. Future implementation andevaluation will further our understanding of how best toimplement these 22 practices to make them most effectiveand help health care organizations become learning healthcare systems. In the meantime, our expert panel believesthat providers should not delay adopting these practices,

Table 1. Recommendations for Evaluating the Effectivenessof Patient Safety Strategies and High-Priority Contexts toInclude in Reports of Patient Safety Research*

Recommendations for evaluating the effectiveness of patient safetystrategies

Explicitly describe the theory behind the chosen intervention componentsor an explicit logic model for why this patient safety practice shouldwork

Describe the patient safety practice in sufficient detail so it can bereplicated, including the expected effect on staff roles

Measure high-priority contexts in the 4 domains described belowDetail the implementation process, the actual effects on staff roles, and

how the implementation or intervention changed over timeAssess the effect of the patient safety practice on outcomes and possible

unexpected effects, including data on costs, when availableFor studies with multiple intervention sites, assess the influence of context

on the effectiveness of the intervention and implementation

High-priority contexts to include in reports of patient safety researchExternal factors, such as regulatory requirements, public reporting, or

pay-for-performance, and local sentinel eventsOrganization structural characteristics, such as size, complexity, and

financial status or strengthTeamwork, leadership, and patient safety cultureManagement tools, such as training resources, internal organization

incentives, audit and feedback, and quality improvement consultants

* From reference 2.

Table 2. Patient Safety Strategies Ready for Adoption Now

Strongly encouragedPreoperative checklists and anesthesia checklists to prevent operative and

postoperative eventsBundles that include checklists to prevent central line–associated

bloodstream infectionsInterventions to reduce urinary catheter use, including catheter reminders,

stop orders, or nurse-initiated removal protocolsBundles that include head-of-bed elevation, sedation vacations, oral care

with chlorhexidine, and subglottic suctioning endotracheal tubes toprevent ventilator-associated pneumonia

Hand hygieneThe do-not-use list for hazardous abbreviationsMulticomponent interventions to reduce pressure ulcersBarrier precautions to prevent health care–associated infectionsUse of real-time ultrasonography for central line placementInterventions to improve prophylaxis for venous thromboembolisms

EncouragedMulticomponent interventions to reduce fallsUse of clinical pharmacists to reduce adverse drug eventsDocumentation of patient preferences for life-sustaining treatmentObtaining informed consent to improve patients’ understanding of the

potential risks of proceduresTeam trainingMedication reconciliationPractices to reduce radiation exposure from fluoroscopy and CTThe use of surgical outcome measurements and report cards, such as

those from ACS NSQIPRapid-response systemsUse of complementary methods for detecting adverse events or medical

errors to monitor for patient safety problemsComputerized provider order entryUse of simulation exercises in patient safety efforts

ACS � American College of Surgeons; CT � computed tomography; NSQIP �National Surgical Quality Improvement Program.

Supplement The Top Patient Safety Strategies That Can Be Encouraged for Adoption Now



particularly the strongly encouraged ones. Enough isknown now to permit health care systems to move ahead.

From the RAND Corporation, Santa Monica, Veterans Affairs GreaterLos Angeles Healthcare System, Los Angeles, University of California,San Francisco, San Francisco, Stanford Center for Health Policy andCenter for Primary Care and Outcomes Research, Stanford, Kaiser Per-manente, Oakland, and University of California, Berkeley, Berkeley,California; Johns Hopkins Medicine Patient Safety and Quality, JohnsHopkins University, and John’s Hopkins Children’s Center, Baltimore,Maryland; ECRI Institute, Plymouth Meeting, RAND Corporation,Pittsburgh, and University of Pennsylvania, Philadelphia, Pennsylvania;Centre for Patient Safety, University of Toronto, Ottawa, Ontario, Can-ada; National Quality Forum, Washington, DC; Harvard University,Brigham and Women’s Hospital, Boston, Massachusetts; VanderbiltUniversity’s Peabody College, Nashville, Tennessee; University ofWisconsin-Madison, Madison, Wisconsin; Imperial College London andQueen Mary, University of London, London, University of Birming-ham, Edgbaston, Birmingham, and Manchester Business School, Uni-versity of Manchester, Manchester, United Kingdom; New York StatePsychiatric Institute, New York, New York; PeaceHealth MedicalGroup, Eugene, Oregon; Research Triangle Institute International, Re-search Triangle Park, North Carolina; Dartmouth Institute for HealthPolicy and Clinical Practice, Lebanon, North Hampshire; Case WesternReserve University, Cleveland, Ohio; and Veterans Affairs Ann ArborHealthcare System and University of Michigan, Ann Arbor, Michigan.

Note: The Agency for Healthcare Research and Quality reviewed con-tract deliverables to ensure adherence to contract requirements and qual-ity, and a copyright release was obtained from the Agency for HealthcareResearch and Quality before submission of the manuscript.

Disclaimer: All statements expressed in this work are those of the authorsand should not be construed as official opinions or positions of theorganizations where any of the authors are employed, the Agency forHealthcare Research and Quality, the U.S. Department of Health andHuman Services, or the U.S. Department of Veterans Affairs.

Acknowledgment: The authors thank Aneesa Motala, BA.

Financial Support: From the Agency for Healthcare Research and Qual-ity, U.S. Department of Health and Human Services (contract HHSA-290-2007-10062I). Dr. Lilford was supported by the National Instituteof Health Research Collaborations for Leadership in Applied HealthResearch and Care for Birmingham and the Black Country.

Potential Conflicts of Interest: Dr. Shekelle: Consultancy: ECRI Institute;Employment: Veterans Affairs; Grants/grants pending: Agency for HealthcareResearch and Quality (AHRQ), Veterans Affairs, Centers for Medicare &Medicaid Services, National Institute of Nursing Research, Office of theNational Coordinator; Royalties: UpToDate. Dr. Pronovost: Board member-ship: Cantel Medical Group; Consultancy: Association for Professionals inInfection Control and Epidemiology, Hospitals and Health Care Systems;Grants/grants pending (money to institution): AHRQ, National Institutes ofHealth; Payment for lectures: Leigh Bureau (speaking on quality and safety);Royalties: Penguin Group. Dr. Wachter: Grant, support for travel to meetings,payment for writing or reviewing the manuscript, grants/grants pending (moneyto institution): AHRQ; Board membership: American Board of Internal Med-icine, Salem Hospital; Payment for lectures: More than 100 health care orga-nizations (such as hospitals, health care systems, state medical, and hospitalassociations); Royalties: Lippincott, Williams & Wilkins, McGraw-Hill; Pay-ment for development of educational presentations (money to institution): Quan-tiaMD, In-Patient Consulting—The Hospitalist Company; Stock/stock op-

tions: PatientSafe Solutions, CRISI, EarlySense; Other: John Wiley and Sons,Marc and Lynne Benioff, United States–United Kingdom Fulbright Com-mission. Ms. McDonald: Grant (money to institution): AHRQ. Dr. Schoelles:Support for travel to meetings and support of work on publication of “MakingHealth Care Safer II” (money to institution): RAND Corporation (funded byAHRQ). Dr. Dy: Grant (money to institution): AHRQ. Dr. Reston: Grant(money to institution): AHRQ. Dr. Adams: Support for travel to meetings:RAND Corporation. Dr. Bates: Consulting fee and support for travel to meet-ings: RAND Corporation; Consultancy: PatientSafe Solutions; Royalties:Medicalis; Stock/stock options: Calgary Scientific. Dr. Bickman: Support fortravel to meetings and fees for participation in review activities: RAND Corpo-ration. Dr. Carayon: Support for travel to meetings: RAND Corporation;Employment: University of Wisconsin-Madison; Grants/grants pending:AHRQ, Office of the National Coordinator; Royalties: Taylor & Francis. Dr.Donaldson: Consulting fee and support for travel to meetings: RAND Corpo-ration. Dr. Farley: Grant and support for travel to meetings: AHRQ; Consul-tancy: RAND Corporation, World Health Organization; Employment:RAND Corporation. Dr. Greenhalgh: Consulting fee and support for travel tomeetings: RAND Corporation. Dr. Lake: Consulting fee and support for travelto meetings: RAND Corporation. Dr. Lilford: Grant: National Institute ofHealth Research Collaborations for Leadership in Applied Health Researchand Care for Birmingham and the Black Country; Consulting fee: AHRQ;Support for travel to meetings: AHRQ. Dr. Lohr: Consulting fee: RAND Cor-poration. Dr. Meyer: Grant: RAND Corporation; Support for travel to meet-ings (money to institution): RAND Corporation; Expert testimony: WinstonStraw. Dr. Miller: Consulting fee: RAND Corporation. Dr. Neuhauser: Con-sulting fee and support for travel to meetings: RAND Corporation. Dr. Ryan:Grant, consulting fee, support for travel to meetings, fees for participation ofreview activities, and payment for writing or reviewing the manuscript (money toinstitution): AHRQ. Dr. Saint: Consulting fee and support for travel to meet-ings: RAND Corporation (funded by AHRQ); Payment for lectures: Varioushospitals, academic medical centers, group-purchasing organizations (for ex-ample, Veterans Health Administration and Premier), professional societies(for example, Society of Hospital Medicine), and nonprofit foundations (forexample, Institute for Healthcare Improvement and Michigan Health andHospital Association); Stock/stock options: Doximity. Dr. Shortell: Support fortravel to meetings: AHRQ. Dr. Stevens: Consulting fee and support for travel tomeetings: RAND Corporation (funded by AHRQ). All other authors haveno disclosures. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2931.

Requests for Single Reprints: Paul G. Shekelle, MD, PhD, RANDCorporation, 1776 Main Street, Santa Monica, CA 90401; e-mail,[email protected].

Current author addresses and author contributions are available at www.annals.org.

References1. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.Making health care safer: a critical analysis of patient safety practices. Evid RepTechnol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]2. Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al.Advancing the science of patient safety. Ann Intern Med. 2011;154:693-6.[PMID: 21576538]3. Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existingsystematic reviews in complex systematic reviews. Ann Intern Med. 2008;148:776-82. [PMID: 18490690]4. Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. Howquickly do systematic reviews go out of date? A survival analysis. Ann Intern Med.2007;147:224-33. [PMID: 17638714]5. Agency for Healthcare Research and Quality. Methods Guide for Effective-ness and Comparative Effectiveness Reviews. AHRQ publication no. 10(11)-

SupplementThe Top Patient Safety Strategies That Can Be Encouraged for Adoption Now



EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality;2011. Accessed at http://effectivehealthcare.ahrq.gov/ehc/products/60/318/MethodsGuide_Prepublication-Draft_20120523.pdf on 20 July 2012.6. Cochrane Effective Practice and Organisation of Care Group (EPOC) Re-views. Accessed at http://epoc.cochrane.org/epoc-reviews on 20 July 2012.7. Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM,et al; Methods Work Group, Third US Preventive Services Task Force. Currentmethods of the US Preventive Services Task Force: a review of the process.Am J Prev Med. 2001;20:21-35. [PMID: 11306229]8. Higgins JP, Altman DG, Gøtzsche PC, Juni P, Moher D, Oxman AD, et al;Cochrane Bias Methods Group. The Cochrane Collaboration’s tool for assessingrisk of bias in randomised trials. BMJ. 2011;343:d5928. [PMID: 22008217]

9. Shekelle PG, Wachter RM, Pronovost PJ, Schoelles K, McDonald KM, DySM, et al. Making Health Care Safer II: An Updated Critical Analysis of theEvidence for Patient Safety Practices. (Prepared by the Southern California-RAND Evidence-based Practice Center under contract HHSA290200710062I.)Rockville, MD: Agency for Healthcare Research and Quality; 2013. [Forthcoming].10. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al.AHRQ series paper 5: grading the strength of a body of evidence when comparingmedical interventions—agency for healthcare research and quality and the effectivehealth-care program. J Clin Epidemiol. 2010;63:513-23. [PMID: 19595577]11. Grading of Recommendations Assessment, Development and Evaluation(GRADE) Working Group. Accessed at www.gradeworkinggroup.org on 20 July2012.

Supplement The Top Patient Safety Strategies That Can Be Encouraged for Adoption Now



Current Author Addresses: Dr. Shekelle: Veterans Affairs Greater LosAngeles Healthcare System, 11301 Wilshire Boulevard, Los Angeles, CA90073.Dr. Pronovost: Johns Hopkins University School of Medicine, 1909Thames Street, 2nd Floor, Baltimore, MD 21231.Dr. Wachter: University of California, San Francisco, 505 ParnassusAvenue, San Francisco, CA 94143.Ms. McDonald: Stanford University, 117 Encina Commons, Stanford,CA 94305-6019.Drs. Schoelles and Reston: ECRI Institute, 5200 Butler Pike, PlymouthMeeting, PA 19462-1298.Dr. Dy: Johns Hopkins University, Room 609, 624 North Broadway,Baltimore, MD 21205.Dr. Shojania: Sunnybrook Health Sciences Centre, Room H468, 2075Bayview Avenue, Toronto, Ontario M4N 3M5, Canada.Dr. Adams: Kaiser Permanente, Division of Research, 2000 Broadway,Oakland, CA 94612.Dr. Angood: American College of Physician Executives, 400 North Ash-ley Drive, Suite 400, Tampa, FL 33602.Dr. Bates: American College of Physician Executives, 400 North AshleyDrive, Suite 400, Tampa, FL 33602.Dr. Bickman: Center for Evaluation and Program Improvement, Van-derbilt University’s Peabody College, Peabody #151, 230 AppletonPlace, Nashville, TN 37203.Dr. Carayon: University of Wisconsin-Madison, 3126 Engineering Cen-ters Building, 1550 Engineering Drive, Madison, WI 53706.Dr. Donaldson: Department of Surgery & Cancer, Division of Surgery,Imperial College London, Room 1090a, 10th Floor, QEQM Building,St Mary’s Hospital, Praed Street, London W2 1NY, United Kingdom.Dr. Duan: New York State Psychiatric Institute, 1051 Riverside Drive,Unit 48, New York, NY 10032.Dr. Farley: RAND Corporation, 4570 5th Avenue #600, Pittsburgh, PA15213.Dr. Greenhalgh: Global Health, Policy and Innovation Unit, Centrefor Primary Care and Public Health, Blizard Institute, Barts and TheLondon School of Medicine and Dentistry, Yvonne Carter Building,58 Turner Street, London E1 2AB, United Kingdom.Dr. Haughom: PeaceHealth, 770 East 11th Avenue, Eugene, OR 97401.Dr. Lake: University of Pennsylvania School of Nursing, Room 302Fagin Hall, 418 Curie Boulevard, Philadelphia, PA 19104-4217.Dr. Lilford: University of Birmingham, Room 110, 90 Vincent Drive,Edgbaston, Birmingham B15 2TT, United Kingdom.Dr. Lohr: RTI International, 3040 Cornwallis Road, PO Box 12194,Research Triangle Park, NC 27709-2194.Dr. Meyer: Dartmouth-Hitchcock, One Medical Center Drive, Leba-non, NH 03756.Dr. Miller: Johns Hopkins Children’s Center, 200 North Wolfe Street,Room 2094, Baltimore, MD 21287.Dr. Neuhauser: Case Western Reserve University, 10900 Euclid Avenue,Cleveland, OH 44106-4945.

Dr. Ryan: RAND Corporation, 1776 Main Street, Santa Monica, CA90401.Dr. Saint: Veterans Affairs Ann Arbor Healthcare System, 2215 FullerRoad, Ann Arbor, MI 48105.Dr. Shortell: University of California, Berkeley, 50 University Hall, MC7360, Berkeley, CA 94720-7360.Dr. Stevens: Dartmouth Institute for Health Policy and Clinical Prac-tice, 30 Lafayette Street, Lebanon, NH 03766.Dr. Walshe: University of Manchester, Booth Street West, ManchesterM15 6PB, United Kingdom.

Author Contributions: Conception and design: P.G. Shekelle, P.J. Pro-novost, R.M. Wachter, K.M. McDonald, K. Schoelles, S.M. Dy, K.Shojania, J.T. Reston, A.S. Adams, P.B. Angood, D.W. Bates, L. Bick-man, P. Carayon, L. Donaldson, N. Duan, D.O. Farley, T. Greenhalgh,J.L. Haughom, E. Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller,D.V. Neuhauser, G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K.Walshe.Analysis and interpretation of the data: P.G. Shekelle, P.J. Pronovost,R.M. Wachter, K.M. McDonald, K. Schoelles, S.M. Dy, K. Shojania,J.T. Reston, A.S. Adams, P.B. Angood, D.W. Bates, L. Bickman, P.Carayon, L. Donaldson, N. Duan, D.O. Farley, T. Greenhalgh, J.L.Haughom, E. Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller,D.V. Neuhauser, G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K.Walshe.Drafting of the article: P.G. Shekelle, P.J. Pronovost, R.M. Wachter.Critical revision of the article for important intellectual content: P.G.Shekelle, P.J. Pronovost, R.M. Wachter, K.M. McDonald, K. Schoelles,S.M. Dy, K. Shojania, J.T. Reston, A.S. Adams, P.B. Angood, D.W.Bates, L. Bickman, P. Carayon, L. Donaldson, N. Duan, D.O. Farley, T.Greenhalgh, J.L. Haughom, E. Lake, R. Lilford, K.N. Lohr, G.S. Meyer,M.R. Miller, D.V. Neuhauser, G. Ryan, S. Saint, S.M. Shortell, D.P.Stevens, K. Walshe.Final approval of the article: P.G. Shekelle, P.J. Pronovost, R.M. Wa-chter, K.M. McDonald, K. Schoelles, S.M. Dy, K. Shojania, J.T. Reston,A.S. Adams, P.B. Angood, D.W. Bates, L. Bickman, P. Carayon, L.Donaldson, N. Duan, D.O. Farley, T. Greenhalgh, J.L. Haughom, E.Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller, D.V. Neuhauser,G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K. Walshe.Statistical expertise: K. Shojania, N. Duan, D.V. Neuhauser.Obtaining of funding: P.G. Shekelle, P.J. Pronovost, R.M. Wachter,K.M. McDonald, K. Schoelles.Collection and assembly of data: P.G. Shekelle, P.J. Pronovost, R.M.Wachter, K.M. McDonald, K. Schoelles, S.M. Dy, K. Shojania, J.T.Reston, A.S. Adams, P.B. Angood, D.W. Bates, L. Bickman, P. Carayon,L. Donaldson, N. Duan, D.O. Farley, T. Greenhalgh, J.L. Haughom, E.Lake, R. Lilford, K.N. Lohr, G.S. Meyer, M.R. Miller, D.V. Neuhauser,G. Ryan, S. Saint, S.M. Shortell, D.P. Stevens, K. Walshe.



Promoting a Culture of Safety as a Patient Safety StrategyA Systematic ReviewSallie J. Weaver, PhD; Lisa H. Lubomksi, PhD; Renee F. Wilson, MS; Elizabeth R. Pfoh, MPH; Kathryn A. Martinez, PhD, MPH;and Sydney M. Dy, MD, MSc

Developing a culture of safety is a core element of many efforts toimprove patient safety and care quality. This systematic reviewidentifies and assesses interventions used to promote safety cultureor climate in acute care settings. The authors searched MEDLINE,CINAHL, PsycINFO, Cochrane, and EMBASE to identify relevantEnglish-language studies published from January 2000 to October2012. They selected studies that targeted health care workers prac-ticing in inpatient settings and included data about change in pa-tient safety culture or climate after a targeted intervention. Tworaters independently screened 3679 abstracts (which yielded 33eligible studies in 35 articles), extracted study data, and rated studyquality and strength of evidence. Eight studies included executive

walk rounds or interdisciplinary rounds; 8 evaluated multicompo-nent, unit-based interventions; and 20 included team training orcommunication initiatives. Twenty-nine studies reported some im-provement in safety culture or patient outcomes, but measuredoutcomes were highly heterogeneous. Strength of evidence waslow, and most studies were pre–post evaluations of low to mod-erate quality. Within these limits, evidence suggests that interven-tions can improve perceptions of safety culture and potentiallyreduce patient harm.

Ann Intern Med. 2013;158:369-374. www.annals.orgFor author affiliations, see end of text.

THE PROBLEM

Developing a culture of safety is a core element ofmany efforts to improve patient safety and care quality inacute care settings (1, 2). Several studies show that safetyculture and the related concept of safety climate are relatedto such clinician behaviors as error reporting (3), reduc-tions in adverse events (4, 5), and reduced mortality (6, 7).Accreditation bodies identify leadership standards forsafety culture measurement and improvement (8), and pro-moting a culture of safety is a designated National PatientSafety Foundation Safe Practice (9). A search of the Agencyfor Healthcare Research and Quality (AHRQ) PatientSafety Net (www.psnet.ahrq.gov) yields more than 5665articles, tips, and fact sheets related to improving safetyculture. Although much work has focused on promoting aculture of safety, understanding which approaches are mosteffective and the implementation factors that may influ-ence effectiveness are critical to achieving meaningful im-provement (10).

Drawing on the social, organizational, and safety sci-ences, patient safety culture can be defined as 1 aspect of anorganization’s culture (11, 12). Specifically, it can be per-sonified by the shared values, beliefs, norms, and proce-dures related to patient safety among members of an orga-nization, unit, or team (13, 14). It influences clinician andstaff behaviors, attitudes, and cognitions on the job byproviding cues about the relative priority of patient safetycompared with other goals (for example, throughput orefficiency) (11). Culture also shapes clinician and staff per-ceptions about “normal” behavior related to patient safetyin their work area. It informs perceptions about what ispraiseworthy and what is punishable (either formally bywork area leaders or informally by colleagues and fellowteam members). In this way, culture influences one’s mo-tivation to engage in safe behaviors and the extent to whichthis motivation translates into daily practice.

Patient safety climate is a related term—often inadver-tently used interchangeably with culture—that refers spe-cifically to shared perceptions or attitudes about the norms,policies, and procedures related to patient safety amongmembers of a group (for example, care team, unit, service,department, or organization) (11). Climate provides asnapshot of clinician and staff perceptions about the ob-servable, surface-level aspects of culture during a particularpoint in time (10, 15). It is measured most often using aquestionnaire or survey. Clinicians and staff are askedabout aspects of their team, work area, or hospital, such ascommunication about safety hazards, transparency, team-work, and leadership. Because climate is defined as a char-acteristic of a team or group, individual responses to surveyitems are usually aggregated to form unit-, department-, orhigher-level scores. The difference between culture and cli-mate is often reduced to a difference in methodology.Studies involving surveys of clinicians and staff are catego-rized as studies of safety climate, and ethnographic studiesinvolving detailed, longitudinal observations are catego-rized as studies of safety culture. The terms are often usedinterchangeably in practice, but it is important to remem-ber that there are conceptually meaningful differences intheir scope and depth. For the purpose of this review, stud-ies of both patient safety culture and climate were in-cluded. We use the term patient safety culture in discussiononly to simplify the reporting of results.

Given that safety culture can influence care processesand outcomes, efforts to evaluate patient safety climate

See also:

Web-OnlyCME quiz (Professional Responsibility Credit)Supplement




over time are being widely implemented (16). Measure-ment and feedback are necessary—although likelyinsufficient—means to effectively promote a culture ofsafety. One previous systematic review found strong facevalidity for interventions to promote safety culture inhealth care, but heterogeneity among studies, measures,and settings limited conclusions about intervention effec-tiveness (17). Results suggested possible positive effectsfor leadership walk rounds and multifaceted, unit-basedinterventions on survey measures of safety climate. How-ever, the review did not assess effects on patient outcomesor care processes. Another review done by the CochraneCollaboration (18) examined organizational culture–change interventions designed to improve patient out-comes and quality of care. Only 2 studies were identifiedfor inclusion, both of which evaluated different outcomes,and results were inconclusive. We attempted to addressthese gaps by conducting a systematic review of the peer-reviewed literature to identify interventions used to pro-mote safety culture in health care, assess the evidence fortheir effectiveness in improving both safety culture andpatient outcomes, and describe the context and implemen-tation of these interventions.

PATIENT SAFETY STRATEGIES

Promotion of patient safety culture can best be con-ceptualized as a constellation of interventions rooted in

principles of leadership, teamwork, and behavior change,rather than a specific process, team, or technology. Strate-gies to promote a culture of patient safety may include asingle intervention or several interventions combined intoa multifaceted approach or series. They may also includesystem-level changes, such as those in governance or re-porting structure. For example, team training, interdisci-plinary rounding or executive walk rounds, and unit-basedstrategies that include a series of interventions have all beenlabeled as interventions to promote a culture of safety.Team training refers to a set of structured methods foroptimizing teamwork processes, such as communication,cooperation, collaboration, and leadership (19, 20). Previ-ous reviews show that the term has been applied to a rangeof learning and development strategies, but the critical de-fining element is a focus on attaining the knowledge, skills,or attitudes that underlie effective teamwork (20).

Executive walk rounds is an interventional strategythat engages organizational leadership directly with front-line care providers. Executives or senior leaders visit front-line patient care areas with the goal of observing and dis-cussing current or potential threats to patient safety, as wellas supporting front-line staff in addressing such threats (21,22). Walk rounds aim to show leadership commitment tosafety, foster trust and psychological safety, and providesupport for front-line providers to proactively addressthreats to patient safety. However, walk rounds have beenoperationalized in diverse ways, making comparison acrossstudies difficult (21). For example, not all rounding inter-ventions use a structured format, and time intervals be-tween rounds vary widely across studies.

Improvement strategies that combine several interven-tion techniques have also been used to promote safety cul-ture. For example, the Comprehensive Unit-Based SafetyProgram (CUSP) is a multifaceted strategy for culturechange that pairs adaptive interventions (such as continu-ous learning strategies or team training) with technical in-terventions (such as translation and use of best availableevidence-based clinical care algorithms) to improve patientsafety and quality (23, 24). The CUSP methodology in-cludes elements of executive engagement and team train-ing, along with specific strategies for translating clinicalevidence into practice. Other interventions have combinedunit-based interventions with broader organizationalchanges, including restructuring patient safety governance(25, 26).

REVIEW PROCESSES

This review examines the evidence for interventionsthat articulate improvement in patient safety culture as aprimary outcome and intervention goal. We identified rel-evant articles through searches of 5 databases from 1 Jan-uary 2000 through 31 October 2012: PubMed, CINAHL,Cochrane, EMBASE, and PsycINFO. Key search termsincluded patient safety culture, safety climate, and safety at-

Key Summary Points

Safety culture is foundational to efforts to improve patientsafety and may respond to intervention.

Bundling multiple interventions or tools is a common strat-egy to improve safety culture.

Many programs include a form of team training or imple-mentation of communication tools, executive walk roundsor another form of interdisciplinary rounding, or unit-based improvement strategies that target clinical microsys-tems (for example, teams, units, or service lines) and areowned by front-line clinicians and staff.

Low-quality, heterogeneous evidence derived primarilyfrom pre–post evaluations suggests that bundled, multi-component interventions can improve clinician and staffperceptions of safety culture.

Low-quality, limited evidence derived primarily frompre–post evaluations suggests that multifaceted interven-tions aimed at improving patient safety can also improvecare processes and patient outcomes.

Future research should consider investigation of safety cul-ture as a cross-cutting contextual factor that can moderatethe effectiveness of other patient safety practices.

Supplement Promoting a Culture of Safety as a Patient Safety Strategy



titudes (see the Supplement, available at www.annals.org,for a description of the search strategies, an article flowdiagram, and evidence tables). The searches found 3679records, all of which were independently screened by 2reviewers. One hundred sixty-two articles were identifiedfor full screening. Of these, 33 studies (in 35 articles) wereidentified for final inclusion. Two studies each contributed2 papers to the review (26–29).

Studies were included if they targeted health care pro-fessionals or paraprofessionals practicing in adult or pedi-atric inpatient settings, explicitly indicated that the pur-pose of the intervention was promoting or improving aculture or climate of patient safety, used a psychometricallyvalid measure to assess patient safety culture that had pre-vious evidence of sound psychometric properties publishedin a peer-reviewed outlet (15, 30, 31), assessed culture overat least 2 time points, and included adequate data to assesschange in patient safety culture or climate. Only English-language studies conducted in the United States, theUnited Kingdom, Canada, or Australia were included. Al-though a growing number of studies have translatedEnglish-language surveys of culture into other languages,evidence that their construct validity is comparable acrosssamples remains limited. Studies were excluded if they ex-amined interventions aimed at medical or nursing stu-dents, targeted other aspects or types of culture (for exam-ple, general organizational culture), or were primarilyfocused on survey development or establishing the psycho-metric properties of a culture assessment. Qualitative stud-ies were also excluded. Each article was abstracted by aprimary reviewer and checked by a second reviewer.

Strength of evidence, including risk of bias, was eval-uated by both reviewers using the Grading of Recommen-dations Assessment, Development and Evaluation Work-ing Group criteria adapted by AHRQ (32). Interventionsand reported outcomes were highly heterogeneous, andmeta-analyses were not done. We present results from the-matic analysis and qualitative summaries of individualstudies.

This review was supported by the AHRQ, which hadno role in the selection or review of the evidence or thedecision to submit the manuscript for publication.

BENEFITS AND HARMS

Study CharacteristicsOf the 33 studies reviewed, 24 were pre–post studies;

3 were concurrent control or pre–post with concurrentcontrol studies; 3 were time-series studies; 2 were clusterrandomized, controlled trials (RCTs); and 1 had a quasi-stepped wedge design. The clinical care areas studiedincluded intensive care, perioperative, labor and delivery,radiology, and general medical and surgical floors. Twenty-one studies measured patient safety culture or climate withthe Safety Attitudes Questionnaire (33), 10 studies usedthe AHRQ Hospital Survey on Patient Safety (34), and 2

studies used the Patient Safety Climate in Healthcare Or-ganizations survey (35). Most studies operationalized cul-ture at the level of the hospital unit or work area; that is,individual survey responses from clinicians and staff in agiven work area were aggregated to form group-level pa-tient safety climate scores for each work area surveyed.Survey sample sizes ranged from 5461 persons working in144 units in a single hospital to 28 individuals workingwithin a single hospital unit. The response rate—the num-ber of individuals who complete and return surveys out ofthe total invited to complete the survey—is an importantfactor influencing the validity of survey results. Survey re-sponse rates ranged from 23% to 100%.

Intervention TypesHeterogeneity among interventions was substantial.

Most (19 studies) were multicomponent interventionscombining several improvement strategies under a singleoverarching initiative to promote safety culture. For exam-ple, Blegen and colleagues (36) used a 3-component ap-proach that included team training, unit-based safetyteams, and strategies for engaging patients in daily goalsetting. Thematic analysis identified 3 broad categories ofintervention that emerged across multiple studies: 20 stud-ies explicitly included team training or tools to improveteam communication processes, 8 explicitly included someform of executive walk rounds or interdisciplinary round-ing, and 8 explicitly used CUSP.

BenefitsTeam Training

Twenty studies explicitly examined team training ortools to support team communication as interventions topromote safety culture. Of these, 10 were conducted inperioperative care areas, 5 in labor and delivery or pediat-rics, 2 in medical general floors or intensive care, and 3 inother care areas or a mix of care areas. Seventeen had pre–post or pre–post with concurrent control designs. Onestudy was a quasi-cluster RCT; however, only 3 organiza-tions were randomly assigned to 3 conditions. Sixteen ofthe 20 studies reported statistically significant improve-ment in staff perceptions of safety culture. In addition, 5reported improvements in care processes (for example, de-creased care delays or increased use of structured commu-nication) and 7 reported improvements in patient safetyoutcomes (for example, errors resulting in harm or reduc-tions in adverse outcomes index).

Executive Walk Rounds

Eight studies evaluated walk rounds (either executiveor interdisciplinary), including 1 cluster RCT. All reportedimprovement in staff perceptions of safety culture. Onestudy, however, showed improvement on only 2 of 30 sur-vey items and did not report domain scores (37). Threereported improvements in perceptions of care processes(for example, quality of collaboration) or patient safetyoutcomes (for example, improvement in mean number of

SupplementPromoting a Culture of Safety as a Patient Safety Strategy



days since last event). One study (27, 28) found that ad-justed care costs were $24.01 lower for intervention workareas despite an adjusted length of stay that was 0.19 dayslonger. However, neither of these indices were statisticallysignificantly different from control work areas. The studyincluded only 4 units (2 intervention, 2 control) and wasunderpowered to detect differences in these outcomes.

CUSP

Eight studies specifically evaluated the effects ofCUSP. Most used medium- to larger-sample pre–post de-signs in intensive care unit settings, although 1 used aquasi-stepped wedge design. Overall, 6 of the 8 studiesreported statistically significant improvements in staff per-ceptions of safety culture, including perceptions of team-work. Two studies reported improvements in care pro-cesses, such as second-stage labor care (38) and timelyresolution of safety concerns (39). Two studies reportedimprovements (although statistically nonsignificant or notstatistically tested) in nursing turnover (40, 41), 1 reporteda reduction in length of stay (41), and 1 reported greaterreductions in infection rates (although not statistically sig-nificant) (42). Other studies of CUSP have shown sus-tained improvements in infection rates and mortality afterimplementation (23, 27).

Outcomes

Regarding effectiveness, 23 of 32 reviewed studies re-ported a statistically significant effect of the interventionon the overall safety culture score, the safety climate score,or at least half of reported survey domains or items (ifanalyzed at the item level). Several studies reported im-provements in teamwork climate but did not find similarimprovements in safety culture or safety climate (27, 43).

Additional outcomes included changes in care pro-cesses, patient outcomes (for example, indices of harm),and clinician outcomes (for example, turnover or burnout).Nineteen studies also reported the effect of interventionson such outcomes. Statistically significant improvementswere reported in 6 of 11 studies reporting on patient out-comes. Five studies found reductions in indices of patientharm (25, 26, 43–45), and 1 study reported improvementsin length of stay (41). One study found a decrease (0.56 vs.0.15; P � 0.01) in the rate of reported errors that resultedin patient harm after a multifaceted suite of interventionsthat included both cultural (for example, feedback on er-rors in the form of posters) and system-focused changes(for example, medication management protocols) (43). Acluster RCT that found a marginal increase in teamworkculture (45) also found that the experimental unit’sweighted adverse outcome score (an index of patient harm)decreased by 37% after implementation of a team trainingprogram designed to promote patient safety culture, com-pared with a 43% increase in a control unit (P � 0.05).

Two studies also reported reductions in nurse turnover af-ter interventions to promote safety culture (40, 41).

Overall, the strength of evidence was low. Risk of biaswas generally high because of study design issues; for ex-ample, we identified only 1 true cluster RCT (22). Coreissues affecting risk of bias for reviewed studies includedlow survey response rates and incomplete reporting (notreporting full results for all units or hospitals where inter-ventions were conducted, or not reporting results for alldomains measured as part of culture surveys). Results wereinconsistent, with 56% of studies reporting statistically sig-nificant findings. Regarding directness, or the extent towhich findings generalize to different organizations or pop-ulations, few studies discussed the logic model or concep-tual foundation underlying the intervention design. Only 2studies comparatively evaluated the effects of different in-tervention strategies, and patient safety outcomes were in-frequently and heterogeneously reported. Regarding preci-sion, many survey instruments were used across reviewedstudies and results were often reported differently.

HarmsWe did not identify any data on patient harms.

IMPLEMENTATION CONSIDERATIONS AND COSTS

Studies differed in the characteristics of the organiza-tions in which they were implemented, the level of leader-ship support and engagement reported, and the tools andstrategies used to support implementation into daily careprocesses. Thirteen studies were done in academic hospitalsettings, 4 in community-based hospitals, 6 in a mix ofacademic and community hospitals, and several did notaddress the hospital mix in their sample. One study re-ported that the gain in safety climate scores was larger forfaith-based hospitals (14%) than for non–faith-based hos-pitals (8%) but reported no direct statistical test of thesefindings (46). Only 1 study (28) examined costs of careamong intervention and control work areas. No statisticallysignificant differences in mean care costs between controland intervention work areas at follow-up were found.

DISCUSSION

Our review identified 33 studies in 35 articles thatevaluated interventions to promote safety culture in inpa-tient care settings. Although these interventions variedgreatly and often included multiple components, 3 com-mon types of intervention emerged: team training andteam communication tools, executive walk rounds and in-terdisciplinary rounding, and CUSP. These interventionswere implemented across various care areas in both aca-demic and community hospital settings. Most were evalu-ated in either perioperative or intensive care areas.

Overall, results suggest evidence to support the effec-tiveness of such interventions in improving clinician andstaff perceptions of elements of safety culture (for example,




general perceptions of safety climate and teamwork). A fewstudies provide evidence that interventions aiming to im-prove safety culture may meaningfully improve clinicalcare processes (28, 47–49) and suggest the potential toimprove aggregate indices of patient harm (29, 45). How-ever, these conclusions are tempered by the limitations ofthe current evidence. Although 1 true cluster RCT wasidentified (22), most studies had pre–post designs withrelatively small to moderate samples (particularly at theunit or work area level of analysis) that did not includecontrol participants. In addition, few studies examined po-tential variation in perceptions of safety culture by careprovider type.

Although this review offers a systematic analysis ofstrategies to promote safety culture, clear limitations mustbe considered. Only studies in acute care settings usingestablished survey measures were included. Although qual-itative studies of safety culture may offer insight into nu-ances of implementation, they were outside the scope ofthis review. Because several studies in outpatient settingswere not included, results may not generalize beyond in-patient settings. Relevant studies may also have been inad-vertently excluded despite extensive searches. Publicationbias and selective reporting of positive findings also maylimit conclusions about the effectiveness and generalizabil-ity of the interventions evaluated. Finally, traditional crite-ria for evaluating the effectiveness of clinical interventionsfor individual patients are not well-suited to assessing theeffectiveness of quasi-experimental study designs con-ducted at the unit level of analysis. This may have intro-duced systematic bias into our ratings for strength of evi-dence. As noted by Pizzi and colleagues in the original“Making Health Care Safer” report (50); “the threshold forevidence may need a different yardstick than is typicallyapplied in medicine.”

In summary, this review suggests that evidence to sup-port the potential effectiveness of interventions to promotesafety culture is emerging. In particular, the best evidenceto date seems to include strategies comprising multiplecomponents that incorporate team training and mecha-nisms to support team communication and include execu-tive engagement in front-line safety walk rounds. Organi-zations should consider incorporating these elements intoefforts to promote safety culture but also robustly evaluatesuch efforts across multiple outcomes. Future researchshould also consider thorough investigation of safety cul-ture as a cross-cutting contextual factor that can moderatethe effectiveness of other patient safety practices, such asimplementation of rapid response systems. The strength ofevidence for patient safety culture would be improved iftheoretical models (31, 51, 52) were meaningfully used inthe development of interventions for improvement andthose interventions were robustly evaluated. Finally, workis needed to better understand the contextual role thatsafety culture plays in implementation of other patientsafety practices, as well as how efforts to promote safety

culture can best be implemented to enhance the effective-ness of complementary or supplementary interventions forsafety and care quality.

From Johns Hopkins University, Baltimore, Maryland, and University ofMichigan, Ann Arbor, Michigan.

Note: The AHRQ reviewed contract deliverables to ensure adherence tocontract requirements and quality, and a copyright release was obtainedfrom the AHRQ before submission of the manuscript.

Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof the Johns Hopkins University, the AHRQ, or the U.S. Department ofHealth and Human Services.

Financial Support: From the AHRQ, U.S. Department of Health andHuman Services (contract HHSA-290-2007-10062I).

Potential Conflicts of Interest: Dr. Weaver: Grant (money to institu-tion): AHRQ, U.S. Department of Health and Human Services; Travel/accommodations/meeting expenses unrelated to activities listed (money to au-thor): Improvement Science Research Network. Dr. Lubomski: Grant(money to institution): AHRQ. Ms. Wilson: Grant (money to institution):AHRQ. Ms. Pfoh: Grant (money to institution): AHRQ. Dr. Martinez:None disclosed. Dr. Dy: Grant (money to institution): AHRQ. Disclo-sures can be also viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2567.

Requests for Single Reprints: Sallie J. Weaver, PhD, Johns HopkinsUniversity School of Medicine, Department of Anesthesiology and Crit-ical Care Medicine and Armstrong Institute for Patient Safety and Qual-ity, 750 East Pratt Street, 15th Floor, Room 1544, Baltimore, MD21202; e-mail, [email protected].


References1. Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a SaferHealth System. Washington, DC: National Academies Pr; 2000.2. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.Making health care safer: a critical analysis of patient safety practices. Evid RepTechnol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]3. Braithwaite J, Westbrook MT, Travaglia JF, Hughes C. Cultural and asso-ciated enablers of, and barriers to, adverse incident reporting. Qual Saf HealthCare. 2010;19:229-33. [PMID: 20534716]4. Singer S, Lin S, Falwell A, Gaba D, Baker L. Relationship of safety climateand safety performance in hospitals. Health Serv Res. 2009;44:399-421. [PMID:19178583]5. Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relation-ships between hospital patient safety culture and adverse events. J Patient Saf.2010;6:226-32. [PMID: 21099551]6. Estabrooks CA, Tourangeau AE, Humphrey CK, Hesketh KL, GiovannettiP, Thomson D, et al. Measuring the hospital practice environment: a Canadiancontext. Res Nurs Health. 2002;25:256-68. [PMID: 12124720]7. Sexton JB. A Matter of Life or Death: Social, Psychological, and Organiza-tional Factors Related to Patient Outcomes in the Intensive Care Unit. Austin:Univ of Texas; 2002.8. The Joint Commission. Revisions to LD.03.01.01. Oakbrook Terrace, IL:The Joint Commission; 2012. Accessed at www.jointcommission.org/assets/1/6/Pre-Pubs_LD.03.01.01_HAP.pdf on 8 September 2012.9. National Patient Safety Foundation Safe Practices. Accessed at www.npsf.org/for-healthcare-professionals/resource-center on 8 September 2012.

SupplementPromoting a Culture of Safety as a Patient Safety Strategy



10. Singer SJ, Vogus TJ. Safety climate research: taking stock and looking for-ward. BMJ Qual Saf. 2012. [PMID: 23112287]11. Zohar D, Livne Y, Tenne-Gazit O, Admi H, Donchin Y. Healthcare cli-mate: a framework for measuring and improving patient safety. Crit Care Med.2007;35:1312-7. [PMID: 17414090]12. Flin R. Measuring safety culture in healthcare: A case for accurate diagnosis.Saf Sci. 2007;45:653-67.13. Schein EH. Organizational Culture and Leadership. 4th ed. Hoboken, NJ:Jossey-Bass; 2010.14. Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB,Thompson DA, et al. Creating high reliability in health care organizations.Health Serv Res. 2006;41:1599-617. [PMID: 16898981]15. Jackson J, Sarac C, Flin R. Hospital safety climate surveys: measurementissues. Curr Opin Crit Care. 2010. [PMID: 20827181]16. Sorra J, Famolaro T, Dyer N, Nelson D, Smith SA. Hospital Survey onPatient Safety Culture: 2012 User Comparative Database Report. AHRQpublication no. 12-0017. (Prepared by Westat under contract HHSA290200710024C.) Rockville, MD: AHRQ; 2012.17. Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, Brand C.Strategies for improving patient safety culture in hospitals: a systematic review.BMJ Qual Saf. 2012. [PMID: 22849965]18. Parmelli E, Flodgren G, Beyer F, Baillie N, Schaafsma ME, Eccles MP.The effectiveness of strategies to change organisational culture to improvehealthcare performance: a systematic review. Implement Sci. 2011;6:33. [PMID:21457579]19. Salas E, DiazGranados D, Weaver SJ, King H. Does team training work?Principles for health care. Acad Emerg Med. 2008;15:1002-9. [PMID:18828828]20. Weaver SJ, Lyons R, DiazGranados D, Rosen MA, Salas E, Oglesby J, et al.The anatomy of health care team training and the state of practice: a criticalreview. Acad Med. 2010;85:1746-60. [PMID: 20841989]21. Frankel A, Grillo SP, Pittman M, Thomas EJ, Horowitz L, Page M, et al.Revealing and resolving patient safety defects: the impact of leadership Walk-Rounds on frontline caregiver assessments of patient safety. Health Serv Res.2008;43:2050-66. [PMID: 18671751]22. Thomas EJ, Sexton JB, Neilands TB, Frankel A, Helmreich RL. The effectof executive walk rounds on nurse safety climate attitudes: a randomized trial ofclinical units[ISRCTN85147255] [corrected]. BMC Health Serv Res. 2005;5:28.[PMID: 15823204]23. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S,et al. An intervention to decrease catheter-related bloodstream infections in theICU. N Engl J Med. 2006;355:2725-32. [PMID: 17192537]24. Romig M, Goeschel C, Pronovost P, Berenholtz SM. Integrating CUSPand TRIP to improve patient safety. Hosp Pract (Minneap). 2010;38:114-21.[PMID: 21068535]25. Muething SE, Goudie A, Schoettker PJ, Donnelly LF, Goodfriend MA,Bracke TM, et al. Quality improvement initiative to reduce serious safety eventsand improve patient safety culture. Pediatrics. 2012;130:e423-31. [PMID:22802607]26. Pettker CM, Thung SF, Raab CA, Donohue KP, Copel JA, Lockwood CJ,et al. A comprehensive obstetrics patient safety program improves safety climateand culture. Am J Obstet Gynecol. 2011;204:216.e1-6. [PMID: 21376160]27. O’Leary KJ, Wayne DB, Haviley C, Slade ME, Lee J, Williams MV.Improving teamwork: impact of structured interdisciplinary rounds on a medicalteaching unit. J Gen Intern Med. 2010;25:826-32. [PMID: 20386996]28. O’Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams MV. Im-proving teamwork: impact of structured interdisciplinary rounds on a hospitalistunit. J Hosp Med. 2011;6:88-93. [PMID: 20629015]29. Pettker CM, Thung SF, Norwitz ER, Buhimschi CS, Raab CA, Copel JA,et al. Impact of a comprehensive patient safety strategy on obstetric adverseevents. Am J Obstet Gynecol. 2009;200:492.e1-8. [PMID: 19249729]30. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safetyclimate: a review of surveys. Qual Saf Health Care. 2005;14:364-6. [PMID:16195571]31. Flin R, Burns C, Mearns K, Yule S, Robertson EM. Measuring safetyclimate in health care. Qual Saf Health Care. 2006;15:109-15. [PMID:16585110]32. Schunemann HJ, Schunemann AH, Oxman AD, Brozek J, Glasziou P,Jaeschke R, et al; GRADE Working Group. Grading quality of evidence and

strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336:1106-10. [PMID: 18483053]33. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al.The Safety Attitudes Questionnaire: psychometric properties, benchmarkingdata, and emerging research. BMC Health Serv Res. 2006;6:44. [PMID:16584553]34. Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. AHRQpublication no. 04-0041. Rockville, MD: AHRQ; 2004.35. Singer S, Meterko M, Baker L, Gaba D, Falwell A, Rosen A. Workforceperceptions of hospital safety culture: development and validation of the patientsafety climate in healthcare organizations survey. Health Serv Res. 2007;42:1999-2021. [PMID: 17850530]36. Blegen MA, Sehgal NL, Alldredge BK, Gearhart S, Auerbach AA, WachterRM. Improving safety culture on adult medical units through multidisciplinaryteamwork and communication interventions: the TOPS Project. Qual Saf HealthCare. 2010;19:346-50. [PMID: 20693223]37. Tiessen B. On the journey to a culture of patient safety. Healthc Q. 2008;11:58-63. [PMID: 18818531]38. Simpson KR, Knox GE, Martin M, George C, Watson SR. MichiganHealth & Hospital Association Keystone Obstetrics: a statewide collaborative forperinatal patient safety in Michigan. Jt Comm J Qual Patient Saf. 2011;37:544-52. [PMID: 22235539]39. Saladino L, Pickett LC, Frush K, Mall A, Champagne MT. Evaluation of aNurse-Led Safety Program in a Critical Care Unit. J Nurs Care Qual. 2012.[PMID: 23052353]40. Timmel J, Kent PS, Holzmueller CG, Paine L, Schulick RD, Pronovost PJ.Impact of the Comprehensive Unit-based Safety Program (CUSP) on safety cul-ture in a surgical inpatient unit. Jt Comm J Qual Patient Saf. 2010;36:252-60.[PMID: 20564886]41. Pronovost PJ, Weast B, Rosenstein B, Sexton JB, Holzmueller CG, PaineLA, et al. Implementing and validating a comprehensive unit-based safety pro-gram. J Patient Saf. 2005;1:33-40.42. Vigorito MC, McNicoll L, Adams L, Sexton B. Improving safety cultureresults in Rhode Island ICUs: lessons learned from the development ofaction-oriented plans. Jt Comm J Qual Patient Saf. 2011;37:509-14. [PMID:22132663]43. Abstoss KM, Shaw BE, Owens TA, Juno JL, Commiskey EL, Niedner MF.Increasing medication error reporting rates while reducing harm through simul-taneous cultural and system-level interventions in an intensive care unit. BMJQual Saf. 2011;20:914-22. [PMID: 21690249]44. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP,et al; Safe Surgery Saves Lives Study Group. Changes in safety attitude andrelationship to decreased postoperative morbidity and mortality following imple-mentation of a checklist-based surgical safety intervention. BMJ Qual Saf. 2011;20:102-7. [PMID: 21228082]45. Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R. Didactic andsimulation nontechnical skills team training to improve perinatal patient out-comes in a community hospital. Jt Comm J Qual Patient Saf. 2011;37:357-64.[PMID: 21874971]46. Sexton JB, Berenholtz SM, Goeschel CA, Watson SR, Holzmueller CG,Thompson DA, et al. Assessing and improving safety climate in a large cohort ofintensive care units. Crit Care Med. 2011;39:934-9. [PMID: 21297460]47. McCulloch P, Mishra A, Handa A, Dale T, Hirst G, Catchpole K. Theeffects of aviation-style non-technical skills training on technical performance andoutcome in the operating theatre. Qual Saf Health Care. 2009;18:109-15.[PMID: 19342524]48. Weaver SJ, Rosen MA, DiazGranados D, Lazzara EH, Lyons R, Salas E,et al. Does teamwork improve performance in the operating room? A multilevelevaluation. Jt Comm J Qual Patient Saf. 2010;36:133-42. [PMID: 20235415]49. Wolf FA, Way LW, Stewart L. The efficacy of medical team training: im-proved team performance and decreased operating room delays: a detailed anal-ysis of 4863 cases. Ann Surg. 2010;252:477-83. [PMID: 20739848]50. Pizzi LT, Goldfarb NI, Nash DB. Promoting a culture of safety. In: MakingHealth Care Safer: A Critical Analysis of Patient Safety Practices. Rockville, MD:AHRQ; 2001.51. Reiman T, Pietikainen E, Oedewald P. Multilayered approach to patientsafety culture. Qual Saf Health Care. 2010;19:e20. [PMID: 20724396]52. Kirk S, Parker D, Claridge T, Esmail A, Marshall M. Patient safety culturein primary care: developing a theoretical framework for practical use. Qual SafHealth Care. 2007;16:313-20. [PMID: 17693682]




Current Author Addresses: Drs. Weaver and Lubomski: Johns HopkinsUniversity School of Medicine, Department of Anesthesiology and Crit-ical Care Medicine and Armstrong Institute for Patient Safety and Qual-ity, 750 East Pratt Street, 15th Floor, Baltimore, MD 21202.Ms. Wilson: Johns Hopkins University, 1830 East Monument Street,Room 8061, Baltimore, MD 21287.Ms. Pfoh: Johns Hopkins University, 624 North Broadway, Baltimore,MD 21205.Dr. Martinez: University of Michigan, Department of General Medi-cine, 2800 Plymouth Road, Building 16, 4th Floor, Ann Arbor, MI48109-2800.Dr. Dy: Johns Hopkins University, Health Services Research and Devel-opment Center, 624 Broadway, Room 609, Baltimore, MD 21205-1901.

Author Contributions: Conception and design: S.J. Weaver, L.H.Lubomksi, K.A. Martinez, S.M. Dy.Analysis and interpretation of the data: S.J. Weaver, E.R. Pfoh, K.A.Martinez, S.M. Dy.Drafting of the article: S.J. Weaver, E.R. Pfoh, S.M. Dy.Critical revision of the article for important intellectual content: S.J.Weaver, L.H. Lubomksi, S.M. Dy.Final approval of the article: S.J. Weaver, R.F. Wilson, K.A. Martinez,S.M. Dy.Obtaining of funding: S.M. Dy.Administrative, technical, or logistic support: L.H. Lubomksi, R.F.Wilson, K.A. Martinez.Collection and assembly of data: S.J. Weaver, L.H. Lubomksi, R.F.Wilson, K.A. Martinez, S.M. Dy.


W-176 5 March 2013 Annals of Internal Medicine Volume 158 • Number 5 (Part 2) www.annals.org


In-Facility Delirium Prevention Programs as a Patient Safety StrategyA Systematic ReviewJames T. Reston, PhD, MPH, and Karen M. Schoelles, MD, SM

Delirium, an acute decline in attention and cognition, occurs amonghospitalized patients at rates estimated to range from 14% to 56%and increases the risk for morbidity and mortality. The purpose ofthis systematic review was to evaluate the effectiveness and safetyof in-facility multicomponent delirium prevention programs. Asearch of 6 databases (including MEDLINE, EMBASE, and CINAHL)was conducted through September 2012. Randomized, controlledtrials; controlled clinical trials; interrupted time series; and controlledbefore–after studies with a prospective postintervention portionwere eligible for inclusion. The evidence from 19 studies that met

the inclusion criteria suggests that most multicomponent interven-tions are effective in preventing onset of delirium in at-risk patientsin a hospital setting. Evidence was insufficient to determine thebenefit of such programs in other care settings. Future comparativeeffectiveness studies with standardized protocols are needed toidentify which components in multicomponent interventions aremost effective for delirium prevention.


THE PROBLEM

Delirium (also known as acute confusional state) is anacute decline in attention and cognition that constitutes aserious problem for older hospitalized patients and long-term care residents. Estimated hospital occurrence ratesrange from 14% to 56% and vary depending on the reasonfor hospitalization (for example, urgent surgery, intensivecare, or general medical admission) and the patient’s riskfor the condition (1).

Delirium is associated with an increased risk for death,postoperative complications, longer hospital and intensivecare unit stays, and functional decline (1, 2), and it pres-ents a substantial burden in terms of short- and long-termhealth care costs. A study of 841 patients (aged �70 years)admitted to non–intensive care general medical units overa 3-year period at Yale-New Haven Hospital found thatdaily costs were more than 2.5 times higher for patientswith delirium than for those without it. The total costestimates associated with delirium ranged from $16 303 to$64 421 per patient, which the authors extrapolated tonational costs ranging from $38 billion to $152 billioneach year (1). Because these estimates were based on datafrom 1995 to 1998, the costs would be even higher today.Accordingly, prevention of delirium is extremely importantfor improving patient outcomes and decreasing health carecosts.

Evidence from risk-factor studies suggests that delir-ium has a multifactorial cause (more information on thesestudies appears in the full report, available at the Agencyfor Healthcare Research and Quality [AHRQ] Web site[www.ahrq.gov]). No 2 studies evaluated the same set offactors or found the same combination of significant fac-tors associated with delirium. Age was the most commonlyevaluated factor—58.8% of studies that evaluated agefound it to be significantly associated with occurrence ofdelirium. Some studies may have lacked adequate power tofind statistical significance, although this was clearly notthe case in all studies that did not have a significant find-

ing. Among studies that evaluated cognitive impairment ordementia, 84.6% found a significant association betweenthis factor and incidence of delirium. Depression wasfound to have a significant association with delirium oc-currence in only 40% of the studies that evaluated it as apotential risk factor.

Other patient-specific risk factors that showed a signif-icant association with delirium in more than 1 study in-clude male sex, multiple medications, comorbid conditions(for example, diabetes), pneumonia, various anesthetics,neuropsychiatric drugs (for example, benzodiazepines), anti-cholinergics, blood transfusions, abnormal serum chemis-try (for example, blood urea nitrogen levels or creatininelevels), apolipoprotein E4, atrial fibrillation, heavy alcoholintake, volume depletion (dehydration), hypoxia, compli-cations, restraints (rendering patients immobile), and visualimpairment. Several studies evaluated patients having spe-cific surgical procedures (for example, hip repair or replace-ment or cardiac surgery); some of these studies focused onsurgery-specific risk factors (for example, blood transfu-sions or intraoperative anesthesia) and evaluated few non-surgical factors.

Given the multifactorial nature of delirium, a patientsafety strategy designed to assess and remediate multiplefactors is considered likely to be effective for delirium pre-vention. The purpose of this systematic review was to assessthe benefits and harms of multicomponent interventions,including system-level changes, that are designed to pre-vent delirium in hospitals, palliative care centers, and long-term care facilities.

See also:






Several delirium prevention programs consist of multi-factorial intervention bundles. In general, the componentsof the bundle vary across each published evaluation, andthe same bundle is rarely evaluated in more than 1 appli-cation. Therefore, the best that can be done is to describethe components most commonly included in bundles thathave been found to reduce incident delirium. The mostcommon components of successful bundles are shown inthe Table.

Additional components have been reported in success-ful multifactorial bundles. An intervention used for pa-tients with hip fracture in a Swedish university hospitalincluded increased physiologic monitoring, avoidance ofdelays in transfer through different areas of the hospital,daily delirium screening, and avoidance of polypharmacy(as well as several components from the Table, includingextra nutrition, intravenous fluid supplementation, painmanagement, and perioperative or anesthetic period proto-cols) (3). A multifactorial intervention used for patientswith hip fracture at another Swedish university hospitalincluded treatment of sleep apnea, prevention and treat-ment of decubitus ulcers, and measurement of blood pres-sure along with components from the Table, although it isnot clear that all of these components were specificallydesigned to prevent delirium (4).

The Hospital Elder Life Program (HELP), or modi-fied versions thereof, has been evaluated in 3 studies (5–7).This program typically consists of 6 components:orientation, therapeutic activities, vision and hearing pro-tocols, sleep enhancement, and early mobilization. Twostudies (1 in the United States and 1 in Australia) usedproactive geriatric consultation with targeted recommenda-tions (several from the Table) based on a structured pro-tocol (8, 9).

REVIEW PROCESSES

We conducted a systematic review of 6 databases (in-cluding MEDLINE, EMBASE, and CINAHL) for 1999to September 2012. A total of 673 titles were identified, ofwhich 309 were reviewed in detail. The Supplement (avail-able at www.annals.org) provides a complete description ofthe search strategies, an article flow diagram, and evidencetables. Randomized, controlled trials; controlled clinicaltrials; interrupted time series; and controlled before–afterstudies with a prospective postintervention portion wereeligible for inclusion to address effectiveness and harms.Studies were required to have at least 20 patients per in-tervention group. Methods for assessing risk of bias andstrength of evidence are described in the full report on theAHRQ Web site.

Of 309 studies retrieved from our literature searchesand reviewed in detail, we identified 35 that addressedsingle or multicomponent interventions. Of these, 19 eval-uated the efficacy of multicomponent interventions and arethe subject of this review (see Table 2 of the Supplement).Most of these studies reported the incidence of deliriumafter the intervention compared with a control group ofusual care patients treated concurrently or during a periodimmediately before adoption of the new intervention. Be-cause few studies used the same intervention, comparisongroup, study design, or patient population, meta-analyseswere not done.

This review was supported by the AHRQ, which hadno role in the selection or review of the evidence or in thedecision to submit the manuscript for publication.

BENEFITS AND HARMS

BenefitsHospital Inpatient Care

Two studies used HELP, and a third used a modifica-tion of HELP. One was a controlled before–after studywith a concurrent control group consisting of patientsfrom usual care units (7, 10); this study had a moderaterisk of bias. The remaining 2 studies were before–afterstudies where the usual care group consisted of patientstreated before implementation of HELP (historical con-trol) (5, 6); these studies had a high risk of bias. All 3studies found a substantial reduction in incident deliriumafter implementation of HELP compared with usual care.Although the findings of the studies were consistent, theaverage risk of bias was high, mainly because of lack ofrandomization and blinding.

Two studies used proactive geriatric consultation withtargeted recommendations based on a structured protocolfor patients with hip fracture. One was a single-blind, ran-domized, controlled trial with a usual care control group(8), and the other was a before–after study with a historicalusual care control group (9). Both studies reported a re-duction of incident delirium for the geriatric consultationgroup compared with the usual care group; however, the

Key Summary Points

Because delirium has multiple risk factors, multicomponentinterventions targeting several risk factors represent prom-ising patient safety strategies for delirium prevention.

Most of the evidence suggests that most multicomponentinterventions are effective in preventing onset of deliriumin at-risk patients in a hospital setting. These interventionsdo not seem to have significant associated harms.

The evidence is insufficient to identify which multicompo-nent interventions are most beneficial, and the studies donot address the question of which components within aprogram provide the most benefit for delirium prevention.

The evidence is insufficient to determine the benefit ofdelirium prevention programs in palliative care or long-term care settings.

Supplement In-Facility Delirium Prevention Programs as a Patient Safety Strategy



findings from the randomized, controlled trial were no lon-ger statistically significant after adjustment for baseline im-balances. The risk of bias for the studies was high andmoderate, respectively. A nonrandomized, controlled studyused an inpatient geriatric consultation team that madetargeted recommendations (although a list of potential rec-ommendations was not reported) to prevent delirium inpatients with hip fracture. Although delirium incidencewas lower in the intervention group, the difference in in-cidence rates did not reach statistical significance (28% vs.44%; P � 0.067). This study had a high risk of bias due tolack of randomization and a low adherence rate to inpa-tient geriatric consultation team recommendations (onethird of recommendations was not implemented) (11).

Of the remaining multicomponent studies (3, 4, 12–20), all but 1 reported a statistically significant reduction indelirium by at least 1 measure in the intervention groupversus the control group. The exception was a study of asystem-wide quality improvement project (17). A study ofnurse-facilitated family participation reported substantiallyfewer patients with a diagnosis of delirium (defined as ascore �4 on the Intensive Care Delirium ScreeningChecklist) in the intervention group but also no statisti-cally significant between-group difference in mean scores;this study placed more emphasis on the latter measure(18). Overall, the findings are consistent with those fromstudies of the HELP intervention, although the risk of biaswas high—again, because of lack of randomization andblinding.

Palliative Care

One multicenter controlled trial assessed a multicom-ponent intervention intended to prevent delirium in pa-tients with terminal cancer (21). In this population, delir-ium stems from such risk factors as metastatic brainlesions, high opioid intake, and metabolic disturbances;these are not the typical risk factors found in the generalgeriatric population, which include older age, cognitive im-pairment, visual impairment, and multiple medications.Two centers (with 674 patients) implemented the inter-vention, and the remaining 5 centers (with 842 patients)performed usual care. The intervention involved trainingnurses to orient the patient each day, recognize risk factorsfor delirium, and send this information to physicians sothat preventive actions could be taken (for example, chang-ing medication). The closest family member was also edu-cated by nurses on delirium and its symptoms, as well asAmerican College of Physicians recommendations foravoiding symptoms of confusion in this patient popula-tion. Delirium symptoms were assessed by nurses using theConfusion Rating Scale. During the 3-year study, 49% ofpatients in the intervention group and 44% in the usualcare group developed delirium; after adjustment for con-founding factors, there was no significant between-groupdifference in incident delirium (odds ratio, 0.94; P �

0.66). The risk of bias was high because of lack of random-ization; inadequate blinding; and failure to obtain a sys-tematic, formal diagnosis of delirium.

Long-Term Care

The single study done in a nursing home setting re-ported that homes randomly assigned to use pharmacist-led Geriatric Risk Assessment MedGuide reports and au-tomated medication monitoring plans had a significantreduction in delirium onset among newly admitted resi-dents compared with those randomly assigned to usual care(22). However, it is unclear how much of this is due todelirium prevention or resolution of new-onset delirium.Complications did not differ significantly between thegroups.

HarmsMost trials of multicomponent delirium prevention

programs have not reported any harms. However, it is notclear whether the possibility of harms was explicitly as-sessed in all of these trials. One study based on a structuredquality improvement model reported 4 unexpected minorevents (rectal or feeding tube displacement or removal thatdid not lead to any true complications) but no major com-plications (and no statistically significant difference com-pared with usual care, although the study lacked the statis-tical power to detect meaningful differences) (12). Oneother study reported no statistically significant differencesin total complication rates between intervention (50.4%)and usual care (53%) groups; this study was adequatelypowered to detect a meaningful difference in complicationrates (3).


Structural Organizational CharacteristicsMulticomponent delirium prevention programs have

been successfully implemented in acute care hospitals (17studies), palliative care centers (1 study), and nursinghomes (1 study). Five of the acute care hospital studies

Table. The Most Common Components of SuccessfulDelirium Prevention Programs

Anesthesia protocolsAssessment of bowel/bladder functionsEarly mobilizationExtra nutritionGeriatric consultationHydrationMedication reviewPain managementPrevention and treatment of medical complicationsSleep enhancementStaff educationSupplemental oxygenTherapeutic cognitive activities/orientationVision and hearing protocols

SupplementIn-Facility Delirium Prevention Programs as a Patient Safety Strategy



were conducted in the United States; 3 in the UnitedKingdom; 3 in Sweden; and 1 each in Australia, Spain,Italy, Belgium, Chile, and Taiwan. Twelve studies werefrom academically affiliated urban hospitals, 2 were con-ducted in urban hospitals that were not described as teach-ing hospitals, 2 were set in community hospitals (in 1study, the participating community hospitals were part of alarger health system), and the remaining study was set in anaval hospital. No studies have been reported from ruralhospitals. The single study of palliative care was conductedin Canada, and the study set in nursing homes was done inthe United States.

Existing Infrastructure

Only 1 study reported minimal information on pa-tient safety culture at the organizational level. The authorsstated merely that “SHS [Summa Health System] main-tains a strong commitment to patient safety and quality”(17).

External Factors

External factors or motivators were not mentioned inany delirium study.

Implementation

All multicomponent intervention studies provided atleast minimal information about teamwork or leadership atthe level of the unit where the intervention was imple-mented. Thirteen of 19 studies specifically identified thestudy leaders, and 17 of 19 studies identified the teammembers by job status (for example, nurses and geriatri-cians) or at least stated that all staff in the interventionward or unit was part of the team. All of these studiesreported multidisciplinary teamwork that included clinicalexperts, nurses, and other staff (for example, physical ther-apists or volunteers). One study reported minimal infor-mation on teamwork or leadership at the hospital level(17).

Eight studies described multiprofessional implementa-tion, 1 had the intervention performed by the ward staff, 1involved ward staff plus physical therapists (during homevisits), 1 involved ward staff plus ambulance workers, 1involved unit staff plus volunteers, 2 involved the nursingstaff only, 1 involved nursing staff plus consultant pharma-cists, 2 involved nurses assisting family members with theintervention, and 1 involved elder life specialists plusvolunteers.

Fifteen studies reported on staff education and train-ing if this was part of the intervention, and 9 studies re-ported the persons responsible for implementation. Mostof these studies reported that all staff involved in the im-plementation had some type of education or training.Thirteen studies reported the type of training, and only 4reported the length of training.

Four studies reported a change in the implementationprocess due to local tailoring or an iterative process. Only 1study reported that internal incentives were used to pro-mote implementation (5). Allen and colleagues publishedthe only study that provided a table summarizing an actualimplementation instrument (a scorecard used to track pro-cess and outcome variables) (17).

Eighteen studies outlined the intended interventionand the general sequence in which the components wereimplemented; only 13 studies included enough detail todetermine the roles of the various team members. Moststudies generally described how the intervention was sup-posed to be implemented and did not describe any modi-fications or failures of adherence that might have occurredduring the actual implementation. Only 2 studies actuallymeasured adherence to targeted recommendations (8, 11),respectively reporting adherence rates of 77% and approx-imately 67% for implementation of geriatric consultantrecommendations for patients after hip fracture. Fifteenstudies reported patient characteristics.

Although implementation of multicomponent delir-ium prevention programs has not been well-described inmost studies, a few themes seem sufficiently consistent toreport here. First, engagement of front-line clinical staff inthe design of the intervention helps ensure that it will meshwith existing clinical procedures. Second, a multidisci-plinary team comprising clinical experts, nurses, and addi-tional staff is helpful for implementation of a complex in-tervention. Finally, education and training of clinical staffare necessary to help ensure that compliance does not waneover time.

ContextTwo studies reported on the effect of context on out-

comes. One study of an educational package for medicaland nursing staff reported that it was effective at prevent-ing delirium in hospitalized men but not women (12, 23).A study of proactive geriatric consultation with target rec-ommendations based on a structured protocol for patientswith hip fracture reported a “trend” toward more effective-ness among patients without prefracture dementia or im-pairment in activities of daily living, but the differenceswere not statistically significant (8).

One study assessed the somewhat related concept ofpatient adherence and its effect on outcomes of a multifac-torial intervention (HELP). Based on a composite adher-ence score for the 3 components assigned to all patients(orientation, mobility, and therapeutic activities), increasedadherence scores were associated with a reduction in delir-ium incidence rates (odds ratio, 0.69 [95% CI, 0.56 to0.87]) (7).

CostsTwo studies in the evidence base reported information

on costs or cost savings associated with multicomponentdelirium prevention programs. Rizzo and colleagues (24)calculated the total intervention costs of HELP over a




3-year period (1995–1998) at Yale-New Haven Hospitalas $257 385 (personnel plus equipment). In a cost-effectiveness analysis, they found that the intervention wascost-effective for patients at intermediate risk for deliriumbut not for patients at high risk (lack of effectiveness andhigher overall costs). However, these findings may be dueto inadequate power based on their relatively small samplesize of higher-risk patients, leading to uncertainty in theresults for this subgroup (24). Rubin and colleagues (5)calculated that implementation of HELP at their hospitalled to estimated cost savings of more than $2 million peryear from prevention of delirium cases. In addition, morethan $2.2 million per year of estimated revenue was gen-erated by shorter hospital stays for patients withoutdelirium.

DISCUSSION

Moderate-strength evidence suggests that most multi-component interventions are effective in preventing onsetof delirium in at-risk patients in a hospital setting. Theseinterventions have not been reported to have importantassociated harms, although most studies did not explicitlyassess this possibility. In general, successful delirium pre-vention programs involved a multidisciplinary team ofclinical experts, nurses, and other staff (for example, phys-ical therapists or volunteers) and included protocols forearly mobilization of patients, volume repletion (for hydra-tion and electrolyte balance), and addressing visual or hear-ing deficits; a few programs included elimination of unnec-essary medications. Other components reported in morethan 1 study included staff education, geriatric consulta-tion, therapeutic cognitive activities or orientation, extranutrition, sleep enhancement, pain management, anesthe-sia protocols, supplemental oxygen, assessment of bowel orbladder functions, and prevention and treatment of medi-cal complications.

This review has several limitations, the most notable ofwhich is that most of the included studies were rated ashaving a high risk of bias due to lack of randomization andblinding, as well as other shortcomings. Although a fewstudies were rated as having a moderate risk of bias, noneof the studies was considered to be at low risk of bias. Inaddition, although the findings of benefit were consistentacross most studies, the heterogeneity of multicomponentinterventions and the low number of studies evaluatingeach specific intervention preclude identifying a particularprogram as being the most beneficial, and these studies donot address the question of which particular program com-ponents are most beneficial. Finally, the evidence was in-sufficient to determine the benefit of delirium preventionprograms in palliative care or long-term care settings.

Future comparative effectiveness studies with stan-dardized protocols are needed, particularly to identifywhich components in multicomponent interventions are

most effective for delirium prevention. Identification of themost effective bundle of components might encourage hos-pitals to adopt a more standardized approach to deliriumprevention.

From ECRI Institute, Plymouth Meeting, Pennsylvania.


Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof ECRI Institute, the AHRQ, or the U.S. Department of Health andHuman Services.

Acknowledgment: The authors thank Allison Gross, MS, MLS, for per-forming literature searches and Katherine Donahue and Lydia Dharia forediting and formatting the manuscript.

Financial Support: From AHRQ, U.S. Department of Health and Hu-man Services (contract HHSA-290-2007-10062I).

Potential Conflicts of Interest: Dr. Reston: Grant (money to institution):AHRQ, U.S. Department of Health and Human Services. Dr. Schoelles:Support for travel to meetings for the study or other purposes (money toinstitution): Rand Corporation. Other (money to institution): Rand Cor-poration. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2566.

Requests for Single Reprints: James T. Reston, PhD, MPH, ECRI In-stitute, 5200 Butler Pike, Plymouth Meeting, PA 19462-1298; e-mail,[email protected].


References1. Leslie DL, Marcantonio ER, Zhang Y, Leo-Summers L, Inouye SK. One-year health care costs associated with delirium in the elderly population. ArchIntern Med. 2008;168:27-32. [PMID: 18195192]2. Rudolph JL, Marcantonio ER. Review articles: postoperative delirium: acutechange with long-term implications. Anesth Analg. 2011;112:1202-11. [PMID:21474660]3. Bjorkelund KB, Hommel A, Thorngren KG, Gustafson L, Larsson S, Lund-berg D. Reducing delirium in elderly patients with hip fracture: a multi-factorialintervention study. Acta Anaesthesiol Scand. 2010;54:678-88. [PMID:20236093]4. Lundstrom M, Olofsson B, Stenvall M, Karlsson S, Nyberg L, Englund U,et al. Postoperative delirium in old patients with femoral neck fracture: a ran-domized intervention study. Aging Clin Exp Res. 2007;19:178-86. [PMID:17607084]5. Rubin FH, Neal K, Fenlon K, Hassan S, Inouye SK. Sustainability andscalability of the hospital elder life program at a community hospital. J AmGeriatr Soc. 2011;59:359-65. [PMID: 21314654]6. Chen CC, Lin MT, Tien YW, Yen CJ, Huang GH, Inouye SK. Modifiedhospital elder life program: effects on abdominal surgery patients. J Am Coll Surg.2011;213:245-52. [PMID: 21641835]7. Inouye SK, Bogardus ST Jr, Williams CS, Leo-Summers L, Agostini JV. Therole of adherence on the effectiveness of nonpharmacologic interventions: evi-dence from the delirium prevention trial. Arch Intern Med. 2003;163:958-64.[PMID: 12719206]

SupplementIn-Facility Delirium Prevention Programs as a Patient Safety Strategy



8. Marcantonio ER, Flacker JM, Wright RJ, Resnick NM. Reducing deliriumafter hip fracture: a randomized trial. J Am Geriatr Soc. 2001;49:516-22. [PMID:11380742]9. Wong DM, Niam T, Bruce JJ, Bruce DG. Quality project to prevent delir-ium after hip fracture. Aust J Ageing. 2005;24:174-7.10. Inouye SK, Bogardus ST Jr, Charpentier PA, Leo-Summers L, AcamporaD, Holford TR, et al. A multicomponent intervention to prevent delirium inhospitalized older patients. N Engl J Med. 1999;340:669-76. [PMID:10053175]11. Deschodt M, Braes T, Flamaing J, Detroyer E, Broos P, Haentjens P, et al.Preventing delirium in older adults with recent hip fracture through multidisci-plinary geriatric consultation. J Am Geriatr Soc. 2012;60:733-9. [PMID:22429099]12. Needham DM, Korupolu R, Zanni JM, Pradhan P, Colantuoni E, PalmerJB, et al. Early physical medicine and rehabilitation for patients with acute respi-ratory failure: a quality improvement project. Arch Phys Med Rehabil. 2010;91:536-42. [PMID: 20382284]13. Tabet N, Hudson S, Sweeney V, Sauer J, Bryant C, Macdonald A, et al. Aneducational intervention can prevent delirium on acute medical wards. Age Age-ing. 2005;34:152-6. [PMID: 15713859]14. Vidan MT, Sanchez E, Alonso M, Montero B, Ortiz J, Serra JA. Anintervention integrated into daily clinical practice reduces the incidence of delir-ium during hospitalization in elderly patients. J Am Geriatr Soc. 2009;57:2029-36. [PMID: 19754498]15. Harari D, Hopper A, Dhesi J, Babic-Illman G, Lockwood L, Martin F.Proactive care of older people undergoing surgery (‘POPS’): designing, embed-ding, evaluating and funding a comprehensive geriatric assessment service forolder elective surgical patients. Age Ageing. 2007;36:190-6. [PMID: 17259638]16. Lundstrom M, Edlund A, Karlsson S, Brannstrom B, Bucht G, GustafsonY. A multifactorial intervention program reduces the duration of delirium, length

of hospitalization, and mortality in delirious patients. J Am Geriatr Soc. 2005;53:622-8. [PMID: 15817008]17. Allen KR, Fosnight SM, Wilford R, Benedict LM, Sabo A, Holder C, et al.Implementation of a system-wide quality improvement project to prevent delir-ium in hospitalized patients. J Clin Outcomes Manag. 2011;18:253-8.18. Black P, Boore JR, Parahoo K. The effect of nurse-facilitated family partic-ipation in the psychological care of the critically ill patient. J Adv Nurs. 2011;67:1091-101. [PMID: 21214624]19. Colombo R, Corona A, Praga F, Minari C, Giannotti C, Castelli A, et al.A reorientation strategy for reducing delirium in the critically ill. Results of aninterventional study. Minerva Anestesiol. 2012;78:1026-33. [PMID: 22772860]20. Martinez FT, Tobar C, Beddings CI, Vallejo G, Fuentes P. Preventingdelirium in an acute hospital using a non-pharmacological intervention. Age Age-ing. 2012;41:629-34. [PMID: 22589080]21. Gagnon P, Allard P, Gagnon B, Merette C, Tardif F. Delirium preventionin terminal cancer: assessment of a multicomponent intervention. Psychooncol-ogy. 2012;21:187-94. [PMID: 22271539]22. Lapane KL, Hughes CM, Daiello LA, Cameron KA, Feinberg J. Effect of apharmacist-led multicomponent intervention focusing on the medication moni-toring phase to prevent potential adverse drug events in nursing homes. J AmGeriatr Soc. 2011;59:1238-45. [PMID: 21649623]23. Tabet N, Stewart R, Hudson S, Sweeney V, Sauer J, Bryant C, et al. Malegender influences response to an educational package for delirium preventionamong older people: a stratified analysis. Int J Geriatr Psychiatry. 2006;21:493-7.[PMID: 16676296]24. Rizzo JA, Bogardus ST Jr, Leo-Summers L, Williams CS, Acampora D,Inouye SK. Multicomponent targeted intervention to prevent delirium in hospi-talized older patients: what is the economic value? Med Care. 2001;39:740-52.[PMID: 11458138]




Current Author Addresses: Drs. Reston and Schoelles: ECRI Institute,5200 Butler Pike, Plymouth Meeting, PA 19462-1298.

Author Contributions: Conception and design: J.T. Reston, K.M.Schoelles.Analysis and interpretation of the data: J.T. Reston, K.M. Schoelles.Drafting of the article: J.T. Reston, K.M. Schoelles.

Critical revision of the article for important intellectual content: J.T.Reston, K.M. Schoelles.Final approval of the article: K.M. Schoelles.Obtaining of funding: K.M. Schoelles.Administrative, technical, or logistic support: K.M. Schoelles.Collection and assembly of data: J.T. Reston.




Patient Safety Strategies Targeted at Diagnostic ErrorsA Systematic ReviewKathryn M. McDonald, MM; Brian Matesic, BS; Despina G. Contopoulos-Ioannidis, MD; Julia Lonhart, BS, BA; Eric Schmidt, BA;Noelle Pineda, BA; and John P.A. Ioannidis, MD, DSc

Missed, delayed, or incorrect diagnosis can lead to inappropriatepatient care, poor patient outcomes, and increased cost. This sys-tematic review analyzed evaluations of interventions to preventdiagnostic errors. Searches used MEDLINE (1966 to October 2012),the Agency for Healthcare Research and Quality’s Patient SafetyNetwork, bibliographies, and prior systematic reviews. Studies thatevaluated any intervention to decrease diagnostic errors in anyclinical setting and with any study design were eligible, providedthat they addressed a patient-related outcome. Two independentreviewers extracted study data and rated study quality.

There were 109 studies that addressed 1 or more interventioncategories: personnel changes (n � 6), educational interventions(n � 11), technique (n � 23), structured process changes (n � 27),

technology-based systems interventions (n � 32), and reviewmethods (n � 38). Of 14 randomized trials, which were rated ashaving mostly low to moderate risk of bias, 11 reported interven-tions that reduced diagnostic errors. Evidence seemed strongest fortechnology-based systems (for example, text message alerting) andspecific techniques (for example, testing equipment adaptations).Studies provided no information on harms, cost, or contextualapplication of interventions. Overall, the review showed a growingfield of diagnostic error research and categorized and identifiedpromising interventions that warrant evaluation in large studiesacross diverse settings.


THE PROBLEM

The family of patient safety targets that includes diag-nostic errors has unclear boundaries. An operational defi-nition includes diagnoses that are “unintentionally delayed(sufficient information was available earlier), wrong (an-other diagnosis was made before the correct one), ormissed (no diagnosis was ever made), as judged fromthe eventual appreciation of more definitive information”(1, 2).

Although the definition is a bit fluid, there is no doubtthat the scope of the problem is large. A systematic reviewof 53 series of autopsies reported a median antemortemerror rate of 23.5% (range, 4.1% to 49.8%) for majorerrors (clinically missed diagnoses involving a principal un-derlying disease or primary cause of death) and 9.0%(range, 0% to 20.7%) for incorrect diagnoses that are likelyto have affected patient outcomes (3). Disease-specificstudies show that 2% to 61% of patients experience missedor delayed diagnoses (4). In a survey of pediatricians, 54%admitted making a diagnostic error at least once permonth, and 45% noted making diagnostic errors thatharmed patients at least once per year (5). Lack of perti-nent historical or clinical information and team processes(for example, inadequate care coordination) contributed toerrors (5).

Furthermore, research on variation in patient out-comes related to diagnosis timing suggests that there isroom for improvement for some high-risk conditions. Forexample, early identification of sepsis may decrease mortal-ity in surgical intensive care (6).

Problems in care related to diagnosis are particularlyprevalent among precipitating causes for lawsuits; 25% to59% of malpractice claims are attributable to diagnosticerrors (4, 7, 8). A recent study of 91 082 diagnosis-related

malpractice claims from 1986 to 2005 estimated paymentssumming to $34.5 billion (inflation-adjusted to 2010 U.S.dollars) (9). Among 10 739 malpractice claims from the2005–2009 National Practitioner Data Bank, diagnosis-related problems accounted for 45.9% of paid claims fromoutpatient settings and 21.1% of paid claims from inpa-tient settings (10).

Some authors have asserted that diagnostic errors areboth more likely to result in patient harms and more pre-ventable than treatment-related errors (such as wrong-sitesurgery or incorrect medication dose), making the problemparticularly important to address (11). Given this poten-tial, the purpose of this review is to assess the multitude ofinterventions to prevent diagnostic errors and better under-stand their effectiveness.


There is a broad array of patient safety strategies(PSSs) that could affect diagnostic errors. Approachesmight involve technical, cognitive, and systems-orientedstrategies, usually tailored to specific conditions or settings.

Strategies might address specific types of diagnosticerror, root causes of the error, or particular technologiesthat are available. Strategies might target clinician errorsrelated to assessment (for example, failure or delay in con-sidering an important diagnosis) or laboratory and radiol-ogy testing (including failure to order needed tests, techni-

See also:

Web-OnlyCME quiz (Professional Responsibilty Credit)Supplement




Sibelius

Highlight

Sibelius

Highlight

cal errors in processing specimens or tests, or erroneousreading of tests) (2). Interventions that target such failureareas might include tools that generate differential diagno-sis lists based on algorithms and checklists; electronic mon-itoring of test result follow-up; and redesigned documen-tation systems that efficiently aggregate relevant evidenceand aid cognitive interpretation (2). Broad-based strategiesmight target changes in residency training, board certifica-tion, and even patient and family engagement in diagnosticproblem solving.

Finally, many strategies could incorporate advancesin medical problem solving (including heuristics andmetacognition), decision analytic or normative decisionmaking, and clinical diagnostic decision support(12–14). Strategies in this area—computerized diagnosismanagement—could include computerized physician or-der entry with clinical decision support.

REVIEW PROCESSES

We captured relevant literature for review through 2main mechanisms. First, we identified 2 key systematicreviews that summarized data on system-related interven-tions addressing organizational vulnerabilities to diagnosticerrors (15) and cognitively related interventions that couldaffect diagnosis (16). Then, we used broad search strategiesto identify additional literature. We searched MEDLINE(1966 to October 2012), the Agency for Healthcare Re-search and Quality (AHRQ) Patient Safety Network (www

.psnet.ahrq.gov/), and bibliographies of background arti-cles and previous systematic reviews to identify literature oneffects of interventions targeting diagnostic errors and/or di-agnostic delays. The major Medical Subject Heading termswere “diagnostic errors” and “delayed diagnosis.”

Eligible studies were those that evaluated any interven-tion to decrease diagnostic errors (incorrect diagnoses ormissed diagnoses) in any clinical setting and with any studydesign, provided that they addressed patient-related out-comes (that is, the correct diagnosis was eventually con-firmed through patient follow-up testing, surgery, autopsy,or other means) or proxy measures of patient-related out-comes. We also considered studies that evaluated interven-tions intended to affect the time to correct diagnosis orappropriate clinical action. We excluded studies in whichthere was no intervention or no real patients (for example,simulations), the intervention was not aimed to reduce di-agnostic errors, or there were no patient outcomes or prox-ies thereof.

Two independent investigators screened articles for el-igibility at the title and abstract level, and any discrepanciesabout selection were resolved through discussion with theentire research team. We also screened all of the studiesincluded in the reviews by Singh and colleagues (15) andGraber and associates (16) and identified 23 studies thatwere evaluations of interventions.

In total, we identified 109 articles that met inclusioncriteria. The Supplement (available at www.annals.org)provides a complete description of the search strategies,article flow diagram, and evidence tables.

We used AMSTAR, a tool that addresses such items asthe comprehensiveness of the search, the assessment of thequality of included studies, and the methods for synthesiz-ing the results, to assess the methodological quality of the 2key systematic reviews (17). We used a standard risk of biasassessment to evaluate quality of the randomized trials (Ta-ble 3 of the Supplement) (18). We developed and used acategorization scheme to classify, from an organizationalperspective, interventions that target diagnostic errors(Table). Categories included changes that an organizationmight consider generically to reduce errors. Such changesinclude techniques investment; personnel configurations;additional review steps for higher reliability; structuredprocesses; education of professionals, patients, and families;and information and communications technology–basedenhancements.

This review was supported by the AHRQ, which hadno role in the selection or review of the evidence or thedecision to submit this manuscript for publication.

BENEFITS AND HARMS

BenefitsPrior Systematic Reviews

Singh and colleagues (15) considered 43 diagnosticerror studies of systems interventions related to provider–patient encounters, diagnostic test performance and inter-

Key Summary Points

Missed, delayed, or incorrect diagnosis can lead toinappropriate patient care, poor patient outcomes,and increased cost.

Patient safety strategies targeting diagnostic errors haveonly recently been studied.

Approaches to reduce errors may involve technical, cogni-tive, and systems-oriented strategies tailored to specificconditions or settings.

A framework that organizations might use to classify inter-vention strategies aimed at reducing diagnostic errors in-cludes technique, personnel, education, structured process,technology-based systems, and review methods.

Limited evidence from randomized, controlled trials showsthat some interventions, such as text messaging—atechnology-based systems strategy—can reduce diagnosticerrors in certain situations.

Very few studies of interventions to reduce diagnosticerrors have examined clinical outcomes (for example,morbidity, mortality) or evaluated the utility of engagingpatients and families in prevention of diagnostic errors.

Supplement Patient Safety Strategies Targeted at Diagnostic Errors



pretation, follow-up and tracking, referral-related issues,and patient-related issues. Their high-quality review (scoreof 9 out of 9 relevant AMSTAR criteria) identified only 6evaluations of interventions that met eligibility criteria forour review. Three of the 6 reported diagnostic outcomes,such as incidence of delayed diagnosis of injury, incidenceof missed injuries, or misdiagnosis rates. None providedinformation on patients’ downstream clinical course.

Graber and colleagues (16) summarized 141 articleson improving cognition and human factors affecting diag-nosis. Their high-quality review (score of 9 out of 9 rele-vant AMSTAR criteria) included 42 evaluations of inter-ventions. These investigators classified interventions in 3dimensions. For interventions to increase knowledge andexpertise, only 1 (19) of 7 studies provided information ondiagnostic outcomes and clinical course for actual patients.For interventions to improve intuitive and deliberate con-siderations, none of the 5 identified studies reported effectson documented diagnoses with actual patients duringclinical course of care. In the largest group of studies—interventions on getting help from colleagues, consultants,and tools—16 of the 28 identified studies evaluated diag-nostic outcomes in actual patients (20–35).

Graber and colleagues noted the current scarcity ofevidence for any single intervention targeting cognitive andhuman factors in reducing diagnostic error. They high-lighted potential for interventions that target content-focused training, feedback on performance, simulation-based training, metacognitive training, second opinion orgroup decision making, and the use of decision supporttools and computer-aided technologies.

Studies of PSS Evaluations

We identified 109 studies, including 14 randomizedtrials, of interventions that targeted diagnostic errors andaddressed patient-related outcomes (see Tables 1 to 4 ofthe Supplement). Of the 6 categories of interventions,most studies pertained to interventions in the categories oftechnology-based systems and additional review methods(Figure 1). Figure 2 shows increases over time in availableevidence related to the categories of additional reviewmethods, structured process changes, technique, andtechnology-based systems interventions.

Patient-related outcomes and their proxies can be cat-egorized as diagnostic accuracy outcomes (for example,false-positive and false-negative results), management out-comes (for example, use of further diagnostic tests or ther-apeutic interventions), and direct patient outcomes (forexample, death, disease progression, or deterioration). Anintervention that leads to better diagnosis does not auto-matically change management or improve patient out-comes. Management change depends on treatment optionsand the feasibility of implementing those options. Im-provements in direct patient outcomes depend also on ef-fectiveness of treatment or management. Outcomes that

were assessed in the 109 studies varied markedly, but fewstudies (5 randomized, controlled trials and 8 other de-signs) evaluated direct patient-level clinical outcomes (6,31, 36–46).

Results of Randomized, Controlled Trials

Primary and secondary outcomes that were assessed inthe 14 randomized trials are summarized in Table 2 of theSupplement. Eight trials (9 comparisons) addressed diag-nostic accuracy outcomes, and 3 trials (5 comparisons) ad-dressed outcomes related to further diagnostic test use. Sixtrials (8 comparisons) addressed outcomes related to fur-ther therapeutic management. Five trials (7 comparisons)addressed direct patient-related outcomes. Three trials ad-dressed composite outcomes (diagnostic accuracy and ther-apeutic management, and therapeutic management andpatient outcome). One trial addressed time to correct ther-apeutic management, and another trial addressed time todiagnosis.

Trials evaluated various interventions. The controlgroup used most often was usual care. No trials had highrisk of bias, whereas 9 and 5 trials had moderate and lowrisk of bias, respectively.

Statistically significant improvements were seen for atleast 1 outcome in all but 3 trials. Of the 3 trials withnon–statistically significant improvements, 1 was a nonin-feriority trial that showed no more diagnostic errors oc-curred during work-up of abdominal pain among patientsgiven morphine and those not given morphine (47). Twotrials that involved patients with mental conditions (46,48) reported no beneficial diagnostic error effects fromcomputerized decision-support systems. Only 1 trial (42)reported improvements in direct patient outcomes;whether improvements were related to the comparison

Table. Categories of Organizational Interventions toDecrease Diagnostic Errors

Category Example

Technique Changes in equipment, procedures, and clinicalapproaches that target diagnosticperformance in clinical practice

Personnel changes Introduction of additional health care membersand replacing certain professionals withothers

Educationalinterventions

Implementation of educational strategies,residency training curricula, and maintenanceof certification changes

Structured processchanges

Implementation of feedback loops or additionalstages in the diagnostic pathway

Technology-basedsystem interventions

Implementation at the system level oftechnology-based tools, such as computerassistive diagnostic aids, decision-supportalgorithms, text message alerting, and pageralerts

Additional reviewmethods

Introduction of additional independent reviewsof test results, from reporting throughinterpretation

SupplementPatient Safety Strategies Targeted at Diagnostic Errors



against the randomized concurrent control group or a pre-intervention period was unclear.

Technique

There were 23 studies of interventions related to med-ical techniques (39, 47, 49–69). Most of these studies,including 3 randomized trials (47, 49, 55), found thatthese interventions can enhance diagnosis (for example,visual enhancements via ultrasonography-guided biopsy,changes to number of biopsy cores, and cap-fitted colono-scopy) or not make it worse (for example, medical inter-ventions for pain relief in patients with abdominal pain).

Personnel Changes

Six studies (44, 45, 70–73) compared the effect ondiagnosis of substituting 1 type of professional for another,

or adding another professional to the care team. The 3studies (71–73) in which a specialist was added to examinethe interpretation of a test result reported an increase incase detection, although the studies were quite small andtargeted narrow patient populations. There was only 1 ran-domized trial, showing that emergency nurse practitionersperform better than junior physicians (45).

Educational Interventions

Eleven studies (19, 43, 74–82) used educational in-terventions for various targets: patients, parents, commu-nity doctors, and intensive care unit doctors and nurses.Strategies targeted at professionals produced improve-ments, but the studies were nonrandomized. Two random-ized trials that targeted consumers found that parent edu-cation improved discrimination of serious symptomsnecessitating physician diagnosis and patient education im-proved the performance of breast cancer screening (74,78).

Structured Process Changes

Twenty-seven studies (43, 44, 46, 48, 56–59, 73, 77,79, 83–98) examined interventions that added structure tothe diagnostic process. Structure included, among otherthings, triage protocols, feedback steps, and quality im-provement processes. Most interventions included the ad-dition of a tool, often a checklist or a form (for example, toguide and standardize physical examination of a patient).Some of the studies centered on laboratory processes,whereas others occurred during clinical management, oftenin situations related to trauma patients. Beneficial effectson diagnosis-related outcomes were seen in most nonran-domized studies, but of the 3 randomized trials, 2 did notshow benefit for improving diagnosis of mental illness (46,48) and 1 had mixed results for a protocol for orderingradiography in injured patients (84).

Figure 1. Interventions, by type.

Structured process20%

Technique18%

Education8%

Technology-basedsystems

22%

Personnel4%

Additional reviewmethods

28%

The percentage of studies as categorized by the 6 types of interventions.

Figure 2. Intervention studies, by year.

Stud

ies,

n

Published Year of Study

Additional review methods

Technique

Technology-based systems

Structured process

EducationPersonnel

1970–1975

1976–1980

1981–1985

1986–1990

1991–1995

1996–2000

2001–2005

2006–2011

0

2

4

6

8

10

12

14

Timeline of the included studies categorized by the 6 types of interventions.




Technology-Based Systems Interventions

Thirty-two studies (6, 29–36, 40–42, 44, 46, 60, 71,78, 80, 97, 99–111) included computerized decision sup-port systems and alerting systems (for example, for abnor-mal laboratory results), most of which were associated withimprovements to processes on the diagnostic pathway (forexample, relaying a critical laboratory value to the clinicianin a more timely manner). Some interventions related tospecific symptoms (for example, a computer-aided diagnos-tic tool for abdominal pain interpretation), whereas othersintervened at the level of a particular test (for example, anelectronic medical record alert for a positive result on afecal occult blood screen for cancer). All 4 randomizedtrials (31, 36, 42, 100) reported beneficial diagnostic erroreffects (see Table 2 of the Supplement).

Additional Review Methods

The most common type of intervention that was eval-uated was the introduction of redundancy in interpretingtest results (6, 20–28, 34, 37–39, 72, 73, 76, 78, 79, 81,95, 96, 109, 112–126). Most studies showed that an ad-ditional review step (usually by a separate reader, from thesame specialty or from another specialty) had a positiveeffect on diagnostic performance. However, in some cases,false-positive results also increased. Tradeoffs between sen-sitivity and specificity were reported erratically. Some stud-ies targeted higher-risk patients for enriched review. How-ever, the systems to support such targeting were neitherdescribed nor evaluated. Randomized evidence was weak,based on 1 group of 1 trial showing statistically significantbenefit (no effect size reported) for an audit and feedbackapproach (78).

Studies With Interventions That Corresponded to MultipleCategories

Twenty-four studies (6, 34, 39, 43, 44, 46, 56–60,71–73, 76–80, 95–97, 109, 127) combined approaches ina variety of ways and covered diverse clinical areas, withmixed results. These studies are also included in the cate-gories covered above. Twenty of the 24 studies combined 2categories of intervention in almost every permutation pos-sible (11 of 15 combinations). With only 1 to 4 studies forany combination set, it is not possible to draw conclusionsabout whether benefits are enhanced with more complexinterventions. Moreover, complex approaches may be morecostly, but this information was not reported.

Notifying Patients of Test Results

Another potential grouping of PSSs focuses on theinterface between the system and the patient, such as strat-egies that involve patient notification of test results (128).No studies with comparative designs evaluated this inter-vention. The review by Singh and colleagues (15) identi-fied 7 studies of patient preferences or satisfaction withdifferent options for receipt of test results. They also found

no studies that tested ways to reduce error using an inter-vention that affected test notification.

Casalino and colleagues (129) found a 7.1% rate ofapparent failures to inform patients of an abnormal testresult and identified a positive association between use ofsimple processes by physician practices for managing re-sults and lower failure rates. A systematic review that ex-amined failures to follow up test results with ambulatorycare patients reported that failed follow-up ranged from1.0% to 62.0%, depending on the type of test result, in-cluding failures associated with missed cancer diagnoses(130). None of the studies included in that systematic re-view evaluated patient-oriented interventions.

HarmsNo studies in our review evaluated direct patient

harm. Studies generally did not assess unintended adverseeffects, although some reported false-positive rates.


The context in which a safety strategy is implementeddepends on the specific type of diagnostic error and prac-tice being examined. The studies that we reviewed covereda range of subspecialties, settings, patient populations, andinterventions. Context varied greatly. Most interventionswere not tested in more than 1 site. Many studies weresmall, early proof-of-concept evaluations. No informationwas reported on the cost of implementing the reviewedPSSs; costs would probably vary greatly, depending on theparticular strategy or practice.

DISCUSSION

This review identified over 100 evaluations of inter-ventions to reduce diagnostic errors, many of which had areported positive effect on at least 1 end point, includingstatistically significant improvements in at least 1 end pointin 11 of the 14 randomized trials. Mortality and morbidityend points were seldom reported.

We also identified 2 previous systematic reviews ofcognitive and systems-oriented approaches to improve di-agnostic accuracy that mostly found proof-of-concept strat-egies not yet tested in practice. Our review built on theprevious systematic reviews by grouping PSSs targeting di-agnostic errors from an organizational perspective intochanges that an organization might consider more generi-cally (techniques investment; personnel configurations; ad-ditional review steps for higher reliability; structured pro-cesses; education of professionals, patients, families; andinformation and communications technology–based en-hancements), as opposed to individual clinicians lookingfor ways to improve their own cognitive processing in spe-cific diagnostic contexts. Although many of the PSSs testedthus far target diagnostic pathways for specific symptomsor conditions, grouping interventions into common lever-age points will support future development in this field by




the various stakeholders who seek to reduce diagnosticproblems. Involvement of patients and families has re-ceived minimal attention, with only 2 studies addressingeducation of consumers.

Data synthesis is difficult because few studies haveused randomized designs, comparable outcomes, or similarinterventions packages. The existing literature may be sus-ceptible to reporting biases favoring “positive” results fordifferent interventions. It is expected that with heightenedawareness of the problem, the number of studies in thisfield will increase further in the future, including morerandomized trials and studies testing different approaches:for example, policy-level efforts. However, the range ofoutcomes assessed in the studies that we reviewed high-lights the known lack of tools to routinely measure theeffect of interventions to decrease diagnostic errors. Addi-tional work is needed on appropriate measurements of di-agnostic errors and consequential delays in diagnosis. Afinal limitation, especially for synthesis, is the diversity ofinterventions that are reverse-engineered on the basis ofthe many diagnostic targets; the diverse tailored needs foreach clinical situation (for example, protocols designed forspecific work-up pathways); and the variety of specializedpersonnel, and even patients, receiving educational orcognitive-support approaches.

Evidence is also lacking on the costs of interventionsand implementation, particularly how to reduce diagnosticerrors without producing other diagnostic problems, suchas overuse of tests. Eventually reaching the correct diagno-sis with inefficient testing strategies (for example, somesequences of multiple test ordering) is not the appropriatepathway to improved diagnostic safety. Our review found apaucity of studies that assessed both sensitivity and speci-ficity of interventions addressing diagnostic performance inthe context of mitigating diagnostic errors. Thus, althoughwe found several promising interventions, evaluations needto be strengthened before any specific PSSs are scaled up inthis domain.

In conclusion, our review demonstrates that the nas-cent field of diagnostic error research is growing, with newinterventions being tested that involve technical, cognitive,and systems-oriented strategies. The framework of inter-vention types developed in the review provides a basis forcategorizing and designing new studies, especially random-ized, controlled trials, in these areas.

From Stanford Center for Health Policy/Center for Primary Care andOutcomes Research; Stanford University School of Medicine; StanfordPrevention Research Center; School of Humanities and Sciences, Stan-ford University, Stanford, California; and Palo Alto Medical FoundationResearch Institute, Palo Alto, California.

Note: The AHRQ reviewed contract deliverables to ensure adherence tocontract requirements and quality, and a copyright release was obtainedfrom the AHRQ before the manuscript was submitted for publication.

Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof Stanford University, the AHRQ, or the U.S. Department of Healthand Human Services.

Financial Support: From the AHRQ, U.S. Department of Health andHuman Services (contract HHSA-290-2007-100621).

Potential Conflicts of Interest: Ms. McDonald: Grant (money to insti-tution): AHRQ. Mr. Schmidt: Grant (money to institution): AHRQ. Allother authors had no disclosures to report. Disclosures can also be viewedat www.acponline.org/authors/icmje/ConflictOflnterestForms.do?msNum�M12-2571.

Requests for Single Reprints: Kathryn M. McDonald, MM, StanfordUniversity, 117 Encina Commons, Stanford, CA 94305-6019; e-mail,[email protected].


References1. Graber ML. Next steps: envisioning a research agenda. Adv Health Sci EducTheory Pract. 2009;14 Suppl 1:107-12. [PMID: 19669917]2. Schiff GD, Hasan O, Kim S, Abrams R, Cosby K, Lambert BL, et al.Diagnostic error in medicine: analysis of 583 physician-reported errors. ArchIntern Med. 2009;169:1881-7. [PMID: 19901140]3. Shojania KG, Burton EC, McDonald KM, Goldman L. Changes in rates ofautopsy-detected diagnostic errors over time: a systematic review. JAMA. 2003;289:2849-56. [PMID: 12783916]4. Schiff GD, Kim S, Abrams R, Cosby K, Lambert B, Elstein AS, et al.Diagnosing diagnosis errors: lessons from a multi-institutional collaborative proj-ect. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in PatientSafety: From Research to Implementation. vol 2. Rockville, MD; Agency forHealthcare Research and Quality: 2005.5. Singh H, Thomas EJ, Wilson L, Kelly PA, Pietz K, Elkeeb D, et al. Errors ofdiagnosis in pediatric practice: a multisite survey. Pediatrics. 2010;126:70-9.[PMID: 20566604]6. Moore LJ, Jones SL, Kreiner LA, McKinley B, Sucher JF, Todd SR, et al.Validation of a screening tool for the early identification of sepsis. J Trauma.2009;66:1539-46. [PMID: 19509612]7. Phillips RL Jr, Bartholomew LA, Dovey SM, Fryer GE Jr, Miyoshi TJ,Green LA. Learning from malpractice claims about negligent, adverse events inprimary care in the United States. Qual Saf Health Care. 2004;13:121-6.[PMID: 15744204]8. Selbst SM. Pediatric emergency medicine: legal briefs. Pediatr Emerg Care.2005;21:214-8.9. Tehrani AS, Lee H, Mathews S, Shore A, Frick KD, Makary M, et al.20-year summary of U.S. malpractice claims for diagnostic errors from 1985-2005 [Abstract]. 33rd Annual Meeting of the Society for Medical Decision Mak-ing, Chicago, Ilinois, 22–26 October 2011.10. Bishop TF, Ryan AM, Casalino LP. Paid malpractice claims for adverseevents in inpatient and outpatient settings. JAMA. 2011;305:2427-31. [PMID:21673294]11. Ely JW, Graber ML, Croskerry P. Checklists to reduce diagnostic errors.Acad Med. 2011;86:307-13. [PMID: 21248608]12. Cosby KS. A framework for classifying factors that contribute to error in theemergency department. Ann Emerg Med. 2003;42:815-23. [PMID: 14634609]13. Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases.Science. 1974;185:1124-31. [PMID: 17835457]14. Metcalfe J, Shimamura AP. Metacognition: Knowing About Knowing.Cambridge, MA: MIT Press; 1994.15. Singh H, Graber ML, Kissam SM, Sorensen AV, Lenfestey NF, Tant EM,et al. System-related interventions to reduce diagnostic errors: a narrative review.BMJ Qual Saf. 2012;21:160-70. [PMID: 22129930]




16. Graber ML, Kissam S, Payne VL, Meyer AN, Sorensen A, Lenfestey N,et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJQual Saf. 2012;21:535-57. [PMID: 22543420]17. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.Development of AMSTAR: a measurement tool to assess the methodologicalquality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:17302989]18. Assessing risk of bias in included studies. In: Higgins JP, Green S, eds.Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1. TheCochrane Collaboration; September 2008. Accessed at www.cochrane-handbook.org. on 6 September 2012.19. Fridriksson S, Hillman J, Landtblom AM, Boive J. Education of referringdoctors about sudden onset headache in subarachnoid hemorrhage. A prospectivestudy. Acta Neurol Scand. 2001;103:238-42. [PMID: 11328195]20. Raab SS, Stone CH, Jensen CS, Zarbo RJ, Meier FA, Grzybicki DM, et al.Double slide viewing as a cytology quality improvement initiative. Am J ClinPathol. 2006;125:526-33. [PMID: 16627263]21. Raab SS, Grzybicki DM, Mahood LK, Parwani AV, Kuan SF, Rao UN.Effectiveness of random and focused review in detecting surgical pathology error.Am J Clin Pathol. 2008;130:905-12. [PMID: 19019767]22. Manion E, Cohen MB, Weydert J. Mandatory second opinion in surgicalpathology referral material: clinical consequences of major disagreements.Am J Surg Pathol. 2008;32:732-7. [PMID: 18360282]23. Nordrum I, Johansen M, Amin A, Isaksen V, Ludvigsen JA. Diagnosticaccuracy of second-opinion diagnoses based on still images. Hum Pathol. 2004;35:129-35. [PMID: 14745735]24. Hamady ZZ, Mather N, Lansdown MR, Davidson L, Maclennan KA.Surgical pathological second opinion in thyroid malignancy: impact on patients’management and prognosis. Eur J Surg Oncol. 2005;31:74-7. [PMID:15642429]25. Espinosa JA, Nolan TW. Reducing errors made by emergency physicians ininterpreting radiographs: longitudinal study. BMJ. 2000;320:737-40. [PMID:10720354]26. Duijm LE, Groenewoud JH, Fracheboud J, de Koning HJ. Additionaldouble reading of screening mammograms by radiologic technologists: impact onscreening performance parameters. J Natl Cancer Inst. 2007;99:1162-70.[PMID: 17652282]27. Kwek BH, Lau TN, Ng FC, Gao F. Non-consensual double reading in theSingapore Breast Screening Project: benefits and limitations. Ann Acad MedSingapore. 2003;32:438-41. [PMID: 12968545]28. Canon CL, Smith JK, Morgan DE, Jones BC, Fell SC, Kenney PJ, et al.Double reading of barium enemas: is it necessary? AJR Am J Roentgenol. 2003;181:1607-10. [PMID: 14627582]29. Pozen MW, D’Agostino RB, Selker HP, Sytkowski PA, Hood WB Jr. Apredictive instrument to improve coronary-care-unit admission practices in acuteischemic heart disease. A prospective multicenter clinical trial. N Engl J Med.1984;310:1273-8. [PMID: 6371525]30. Selker HP, Beshansky JR, Griffith JL, Aufderheide TP, Ballin DS, BernardSA, et al. Use of the acute cardiac ischemia time-insensitive predictive instrument(ACI-TIPI) to assist with triage of patients with chest pain or other symptomssuggestive of acute cardiac ischemia. A multicenter, controlled clinical trial. AnnIntern Med. 1998;129:845-55. [PMID: 9867725]31. Bogusevicius A, Maleckas A, Pundzius J, Skaudickas D. Prospective ran-domised trial of computer-aided diagnosis and contrast radiography in acutesmall bowel obstruction. Eur J Surg. 2002;168:78-83. [PMID: 12113275]32. Ramnarayan P, Winrow A, Coren M, Nanduri V, Buchdahl R, Jacobs B,et al. Diagnostic omission errors in acute paediatric practice: impact of a remindersystem on decision-making. BMC Med Inform Decis Mak. 2006;6:37. [PMID:17087835]33. Olsson SE, Ohlsson M, Ohlin H, Dzaferagic S, Nilsson ML, Sandkull P,et al. Decision support for the initial triage of patients with acute coronary syn-dromes. Clin Physiol Funct Imaging. 2006;26:151-6. [PMID: 16640509]34. Peldschus K, Herzog P, Wood SA, Cheema JI, Costello P, Schoepf UJ.Computer-aided diagnosis as a second reader: spectrum of findings in CT studiesof the chest interpreted as normal. Chest. 2005;128:1517-23. [PMID:16162752]35. Kakeda S, Moriya J, Sato H, Aoki T, Watanabe H, Nakata H, et al.Improved detection of lung nodules on chest radiographs using a commercialcomputer-aided diagnosis system. AJR Am J Roentgenol. 2004;182:505-10.[PMID: 14736690]

36. Kuperman GJ, Teich JM, Tanasijevic MJ, Ma’Luf N, Rittenberg E, Jha A,et al. Improving response to critical laboratory results with automation: results ofa randomized controlled trial. J Am Med Inform Assoc. 1999;6:512-22. [PMID:10579608]37. Dudley M, Channer KS. Assessment of the value of technician reporting ofelectrocardiographs in an accident and emergency department. J Accid EmergMed. 1997;14:307-10. [PMID: 9315933]38. Nam YS, Pikarsky AJ, Wexner SD, Singh JJ, Weiss EG, Nogueras JJ, et al.Reproducibility of colonic transit study in patients with chronic constipation. DisColon Rectum. 2001;44:86-92. [PMID: 11805568]39. Beigi B, Uddin JM, McMullan TF, Linardos E. Inaccuracy of diagnosis ina cohort of patients on the waiting list for dacryocystorhinostomy when thediagnosis was made by only syringing the lacrimal system. Eur J Ophthalmol.2007;17:485-9. [PMID: 17671919]40. Major K, Shabot MM, Cunneen S. Wireless clinical alerts and patient out-comes in the surgical intensive care unit. Am Surg. 2002;68:1057-60. [PMID:12516808]41. Etchells E, Adhikari NK, Wu R, Cheung M, Quan S, Mraz R, et al.Real-time automated paging and decision support for critical laboratory abnor-malities. BMJ Qual Saf. 2011;20:924-30. [PMID: 21725046]42. Fitzgerald M, Cameron P, Mackenzie C, Farrow N, Scicluna P, GocentasR, et al. Trauma resuscitation errors and computer-assisted decision support.Arch Surg. 2011;146:218-25. [PMID: 21339436]43. Chern CH, How CK, Wang LM, Lee CH, Graff L. Decreasing clinicallysignificant adverse events using feedback to emergency physicians of telephonefollow-up outcomes. Ann Emerg Med. 2005;45:15-23. [PMID: 15635301]44. Vernon DD, Furnival RA, Hansen KW, Diller EM, Bolte RG, JohnsonDG, et al. Effect of a pediatric trauma response team on emergency departmenttreatment time and mortality of pediatric trauma victims. Pediatrics. 1999;103:20-4. [PMID: 9917434]45. Sakr M, Angus J, Perrin J, Nixon C, Nicholl J, Wardrope J. Care of minorinjuries by emergency nurse practitioners or junior doctors: a randomised con-trolled trial. Lancet. 1999;354:1321-6. [PMID: 10533859]46. Rollman BL, Hanusa BH, Lowe HJ, Gilbert T, Kapoor WN, SchulbergHC. A randomized trial using computerized decision support to improve treat-ment of major depression in primary care. J Gen Intern Med. 2002;17:493-503.[PMID: 12133139]47. Thomas SH, Silen W, Cheema F, Reisner A, Aman S, Goldstein JN, et al.Effects of morphine analgesia on diagnostic accuracy in emergency departmentpatients with abdominal pain: a prospective, randomized trial. J Am Coll Surg.2003;196:18-31. [PMID: 12517545]48. Schriger DL, Gibbons PS, Langone CA, Lee S, Altshuler LL. Enabling thediagnosis of occult psychiatric illness in the emergency department: a randomized,controlled trial of the computerized, self-administered PRIME-MD diagnosticsystem. Ann Emerg Med. 2001;37:132-40. [PMID: 11174229]49. Attard AR, Corlett MJ, Kidner NJ, Leslie AP, Fraser IA. Safety of early painrelief for acute abdominal pain. BMJ. 1992;305:554-6. [PMID: 1393034]50. Resnick NM, Brandeis GH, Baumann MM, DuBeau CE, Yalla SV. Mis-diagnosis of urinary incontinence in nursing home women: prevalence and aproposed solution. Neurourol Urodyn. 1996;15:599-613. [PMID: 8916113]51. Borgstein PJ, Gordijn RV, Eijsbouts QA, Cuesta MA. Acuteappendicitis—a clear-cut case in men, a guessing game in young women. Aprospective study on the role of laparoscopy. Surg Endosc. 1997;11:923-7.[PMID: 9294274]52. Vermeulen B, Morabia A, Unger PF, Goehring C, Grangier C, Skljarov I,et al. Acute appendicitis: influence of early pain relief on the accuracy of clinicaland US findings in the decision to operate—a randomized trial. Radiology. 1999;210:639-43. [PMID: 10207461]53. Prieto VG, Argenyi ZB, Barnhill RL, Duray PH, Elenitsas R, From L, et al.Are en face frozen sections accurate for diagnosing margin status in melanocyticlesions? Am J Clin Pathol. 2003;120:203-8. [PMID: 12931550]54. Kokki H, Lintula H, Vanamo K, Heiskanen M, Eskelinen M. Oxycodonevs placebo in children with undifferentiated abdominal pain: a randomized,double-blind clinical trial of the effect of analgesia on diagnostic accuracy. ArchPediatr Adolesc Med. 2005;159:320-5. [PMID: 15809382]55. Hewett DG, Rex DK. Cap-fitted colonoscopy: a randomized, tandemcolonoscopy study of adenoma miss rates. Gastrointest Endosc. 2010;72:775-81.[PMID: 20579648]56. Brossner C, Madersbacher S, Bayer G, Pycha A, Klingler HC, Maier U.Comparative study of two different TRUS-guided sextant biopsy techniques in




detecting prostate cancer in one biopsy session. Eur Urol. 2000;37:65-71.[PMID: 10671788]57. Naughton CK, Miller DC, Mager DE, Ornstein DK, Catalona WJ. Aprospective randomized trial comparing 6 versus 12 prostate biopsy cores: impacton cancer detection. J Urol. 2000;164:388-92. [PMID: 10893592]58. Presti JC Jr, Chang JJ, Bhargava V, Shinohara K. The optimal systematicprostate biopsy scheme should include 8 rather than 6 biopsies: results of aprospective clinical trial. J Urol. 2000;163:163-6. [PMID: 10604337]59. Ravery V, Goldblatt L, Royer B, Blanc E, Toublanc M, Boccon-Gibod L.Extensive biopsy protocol improves the detection rate of prostate cancer. J Urol.2000;164:393-6. [PMID: 10893593]60. Weatherburn G, Bryan S, Nicholas A, Cocks R. The effect of a picturearchiving and communications system (PACS) on diagnostic performance in theaccident and emergency department. J Accid Emerg Med. 2000;17:180-4.[PMID: 10819379]61. Johnson AJ, Zywiel MG, Stroh A, Marker DR, Mont MA. Serologicalmarkers can lead to false negative diagnoses of periprosthetic infections followingtotal knee arthroplasty. Int Orthop. 2011;35:1621-6. [PMID: 21181540]62. Larson EM, O’Donnell M, Chamblee S, Horsburgh CR Jr, Marsh BJ,Moreland JD, et al. Dual skin tests with Mycobacterium avium sensitin and PPDto detect misdiagnosis of latent tuberculosis infection. Int J Tuberc Lung Dis.2011;15:1504-9, i. [PMID: 22008764]63. Maclean JE, Solomon M, Corey M, Selvadurai H. Cystic fibrosis newbornscreening does not delay the identification of cystic fibrosis in children withnegative results. J Cyst Fibros. 2011;10:333-7. [PMID: 21536503]64. Bachur RG, Hennelly K, Callahan MJ, Chen C, Monuteaux MC. Diag-nostic imaging and negative appendectomy rates in children: effects of age andgender. Pediatrics. 2012;129:877-84. [PMID: 22508920]65. Zheng Y, Hawkins L, Wolff J, Goloubeva O, Goldberg E. Detection oflesions during capsule endoscopy: physician performance is disappointing.Am J Gastroenterol. 2012;107:554-60. [PMID: 22233695]66. Garcia EA, Lopez JR, Meier JL, Swislocki AL, Siegel D. Resistant hyper-tension and undiagnosed primary hyperaldosteronism detected by use of a com-puterized database. J Clin Hypertens (Greenwich). 2011;13:487-91. [PMID:21762361]67. Piliouras P, Allison S, Rosendahl C, Buettner PG, Weedon D. Dermoscopyimproves diagnosis of tinea nigra: a study of 50 cases. Australas J Dermatol.2011;52:191-4. [PMID: 21834814]68. Leufkens AM, DeMarco DC, Rastogi A, Akerman PA, Azzouzi K, Roth-stein RI, et al; Third Eye Retroscope Randomized Clinical Evaluation[TERRACE] Study Group. Effect of a retrograde-viewing device on adenomadetection rate during colonoscopy: the TERRACE study. Gastrointest Endosc.2011;73:480-9. [PMID: 21067735]69. Kline JA, Hogg MM, Courtney DM, Miller CD, Jones AE, Smithline HA.D-dimer threshold increase with pretest probability unlikely for pulmonary em-bolism to decrease unnecessary computerized tomographic pulmonary angiogra-phy. J Thromb Haemost. 2012;10:572-81. [PMID: 22284935]70. de Lacey G, Barker A, Harper J, Wignall B. An assessment of the clinicaleffects of reporting accident and emergency radiographs. Br J Radiol. 1980;53:304-9. [PMID: 7378697]71. Jacobs MJ, Edmondson MJ, Lowry JC. Accuracy of diagnosis of fractures bymaxillofacial and accident and emergency doctors using plain radiography com-pared with a telemedicine system: a prospective study. Br J Oral Maxillofac Surg.2002;40:156-62. [PMID: 12180212]72. Trotter MJ, Bruecks AK. Interpretation of skin biopsies by general patholo-gists: diagnostic discrepancy rate measured by blinded review. Arch Pathol LabMed. 2003;127:1489-92. [PMID: 14567717]73. Tsai JJ, Yeun JY, Kumar VA, Don BR. Comparison and interpretation ofurinalysis performed by a nephrologist versus a hospital-based clinical laboratory.Am J Kidney Dis. 2005;46:820-9. [PMID: 16253721]74. McCarthy PL, Sznajderman SD, Lustman-Findling K, Baron MA, FinkHD, Czarkowski N, et al. Mothers’ clinical judgment: a randomized trial of theAcute Illness Observation Scales. J Pediatr. 1990;116:200-6. [PMID: 2405140]75. Thaler T, Tempelmann V, Maggiorini M, Rudiger A. The frequency ofelectrocardiographic errors due to electrode cable switches: a before and afterstudy. J Electrocardiol. 2010;43:676-81. [PMID: 20591441]76. Seltzer SE, Hessel SJ, Herman PG, Swensson RG, Sheriff CR. Residentfilm interpretations and staff review. AJR Am J Roentgenol. 1981;137:129-33.[PMID: 6787863]

77. Gleadhill DN, Thomson JY, Simms P. Can more efficient use be made ofx ray examinations in the accident and emergency department? Br Med J (ClinRes Ed). 1987;294:943-7. [PMID: 3107669]78. McPhee SJ, Bird JA, Jenkins CN, Fordham D. Promoting cancer screening.A randomized, controlled trial of three interventions. Arch Intern Med. 1989;149:1866-72. [PMID: 2764657]79. Kundel HL, Nodine CF, Krupinski EA. Computer-displayed eye position asa visual aid to pulmonary nodule interpretation. Invest Radiol. 1990;25:890-6.[PMID: 2394571]80. Linver MN, Paster SB, Rosenberg RD, Key CR, Stidley CA, King WV.Improvement in mammography interpretation skills in a community radiologypractice after dedicated teaching courses: 2-year medical audit of 38,633 cases.Radiology. 1992;184:39-43. [PMID: 1609100]81. Thomas HG, Mason AC, Smith RM, Fergusson CM. Value of radiographaudit in an accident service department. Injury. 1992;23:47-50. [PMID:1541500]82. Itri JN, Kang HC, Krishnan S, Nathan D, Scanlon MH. Using focusedmissed-case conferences to reduce discrepancies in musculoskeletal studies inter-preted by residents on call. AJR Am J Roentgenol. 2011;197:W696-705.[PMID: 21940542]83. Enderson BL, Reath DB, Meadors J, Dallas W, DeBoo JM, Maull KI. Thetertiary trauma survey: a prospective study of missed injury. J Trauma. 1990;30:666-9. [PMID: 2352294]84. Klassen TP, Ropp LJ, Sutcliffe T, Blouin R, Dulberg C, Raman S, et al. Arandomized, controlled trial of radiograph ordering for extremity trauma in apediatric emergency department. Ann Emerg Med. 1993;22:1524-9. [PMID:8214829]85. Biffl WL, Harrington DT, Cioffi WG. Implementation of a tertiary traumasurvey decreases missed injuries. J Trauma. 2003;54:38-43. [PMID: 12544897]86. Soundappan SV, Holland AJ, Cass DT. Role of an extended tertiary surveyin detecting missed injuries in children. J Trauma. 2004;57:114-8. [PMID:15284560]87. Perno JF, Schunk JE, Hansen KW, Furnival RA. Significant reduction indelayed diagnosis of injury with implementation of a pediatric trauma service.Pediatr Emerg Care. 2005;21:367-71. [PMID: 15942513]88. Ursprung R, Gray JE, Edwards WH, Horbar JD, Nickerson J, Plsek P,et al. Real time patient safety audits: improving safety every day. Qual Saf HealthCare. 2005;14:284-9. [PMID: 16076794]89. Raab SS, Andrew-Jaja C, Condel JL, Dabbs DJ. Improving Papanicolaoutest quality and reducing medical errors by using Toyota production systemmethods. Am J Obstet Gynecol. 2006;194:57-64. [PMID: 16389010]90. Raab SS, Grzybicki DM, Sudilovsky D, Balassanian R, Janosky JE, VrbinCM. Effectiveness of Toyota process redesign in reducing thyroid gland fine-needle aspiration error. Am J Clin Pathol. 2006;126:585-92. [PMID: 16938657]91. Raab SS, Tworek JA, Souers R, Zarbo RJ. The value of monitoring frozensection-permanent section correlation data over time. Arch Pathol Lab Med.2006;130:337-42. [PMID: 16519561]92. Raab SS, Jones BA, Souers R, Tworek JA. The effect of continuous moni-toring of cytologic-histologic correlation data on cervical cancer screening perfor-mance. Arch Pathol Lab Med. 2008;132:16-22. [PMID: 18181668]93. Mueller CA, Klaassen-Mielke R, Penner E, Junius-Walker U, Hummers-Pradier E, Theile G. Disclosure of new health problems and intervention plan-ning using a geriatric assessment in a primary care setting. Croat Med J. 2010;51:493-500. [PMID: 21162161]94. de Vries EN, Eikens-Jansen MP, Hamersma AM, Smorenburg SM,Gouma DJ, Boermeester MA. Prevention of surgical malpractice claims by use ofa surgical safety checklist. Ann Surg. 2011;253:624-8. [PMID: 21209590]95. Ross PD, Huang C, Karpf D, Lydick E, Coel M, Hirsch L, et al. Blindedreading of radiographs increases the frequency of errors in vertebral fracture de-tection. J Bone Miner Res. 1996;11:1793-800. [PMID: 8915788]96. Goodyear N, Ulness BK, Prentice JL, Cookson BT, Limaye AP. Systematicassessment of culture review as a tool to assess errors in the clinical microbiologylaboratory. Arch Pathol Lab Med. 2008;132:1792-5. [PMID: 18976017]97. Lewis G, Sharp D, Bartholomew J, Pelosi AJ. Computerized assessment ofcommon mental disorders in primary care: effect on clinical outcome. Fam Pract.1996;13:120-6. [PMID: 8732321]98. Meier FA, Varney RC, Zarbo RJ. Study of amended reports to evaluate andimprove surgical pathology processes. Adv Anat Pathol. 2011;18:406-13.[PMID: 21841408]




99. Wexler JR, Swender PT, Tunnessen WW Jr, Oski FA. Impact of a systemof computer-assisted diagnosis. Initial evaluation of the hospitalized patient.Am J Dis Child. 1975;129:203-5. [PMID: 1091140]100. Wellwood J, Johannessen S, Spiegelhalter DJ. How does computer-aideddiagnosis improve the management of acute abdominal pain? Ann R Coll SurgEngl. 1992;74:40-6. [PMID: 1736794]101. Poon EG, Kuperman GJ, Fiskio J, Bates DW. Real-time notification oflaboratory data requested by users through alphanumeric pagers. J Am MedInform Assoc. 2002;9:217-22. [PMID: 11971882]102. Gur D, Sumkin JH, Rockette HE, Ganott M, Hakim C, Hardesty L, et al.Changes in breast cancer detection and mammography recall rates after the in-troduction of a computer-aided detection system. J Natl Cancer Inst. 2004;96:185-90. [PMID: 14759985]103. Cupples TE, Cunningham JE, Reynolds JC. Impact of computer-aideddetection in a regional screening mammography program. AJR Am J Roentgenol.2005;185:944-50. [PMID: 16177413]104. Fenton JJ, Taplin SH, Carney PA, Abraham L, Sickles EA, D’Orsi C,et al. Influence of computer-aided detection on performance of screening mam-mography. N Engl J Med. 2007;356:1399-409. [PMID: 17409321]105. Park HI, Min WK, Lee W, Park H, Park CJ, Chi HS, et al. Evaluating theshort message service alerting system for critical value notification via PDA tele-phones. Ann Clin Lab Sci. 2008;38:149-56. [PMID: 18469361]106. Piva E, Sciacovelli L, Zaninotto M, Laposata M, Plebani M. Evaluation ofeffectiveness of a computerized notification system for reporting critical values.Am J Clin Pathol. 2009;131:432-41. [PMID: 19228648]107. Singh H, Wilson L, Petersen LA, Sawhney MK, Reis B, Espadas D, et al.Improving follow-up of abnormal cancer screens using electronic health records:trust but verify test result communication. BMC Med Inform Decis Mak. 2009;9:49. [PMID: 20003236]108. David CV, Chira S, Eells SJ, Ladrigan M, Papier A, Miller LG, et al.Diagnostic accuracy in patients admitted to hospitals with cellulitis. DermatolOnline J. 2011;17:1. [PMID: 21426867]109. Jiang Y, Nishikawa RM, Schmidt RA, Toledano AY, Doi K. Potential ofcomputer-aided diagnosis to reduce variability in radiologists’ interpretations ofmammograms depicting microcalcifications. Radiology. 2001;220:787-94.[PMID: 11526283]110. Leaper DJ, Horrocks JC, Staniland JR, De Dombal FT. Computer-assisted diagnosis of abdominal pain using “estimates” provided by clinicians.Br Med J. 1972;4:350-4. [PMID: 4629240]111. Nishikawa RM, Schmidt RA, Linver MN, Edwards AV, Papaioannou J,Stull MA. Clinically missed cancer: how effectively can radiologists use computer-aided detection? AJR Am J Roentgenol. 2012;198:708-16. [PMID: 22358014]112. Ciatto S, Del Turco MR, Morrone D, Catarzi S, Ambrogetti D, CariddiA, et al. Independent double reading of screening mammograms. J Med Screen.1995;2:99-101. [PMID: 7497164]113. Howard J, Sundararajan R, Thomas SG, Walsh M, Sundararajan M.Reducing missed injuries at a level II trauma center. J Trauma Nurs. 2006;13:89-95. [PMID: 17052086]114. Singh P, Warnakulasuriya S. The two-week wait cancer initiative on oralcancer; the predictive value of urgent referrals to an oral medicine unit. Br DentJ. 2006;201:717-20. [PMID: 17159958]

115. Bruner JM, Inouye L, Fuller GN, Langford LA. Diagnostic discrepanciesand their clinical impact in a neuropathology referral practice. Cancer. 1997;79:796-803. [PMID: 9024718]116. Carew-McColl M. Radiological interpretation in an accident and emer-gency department. Br J Clin Pract. 1983;37:375-7. [PMID: 6671078]117. Galasko CS, Monahan PR. Value of re-examining x-ray films of outpatientsattending accident services. Br Med J. 1971;1:643-4. [PMID: 5548841]118. Lind AC, Bewtra C, Healy JC, Sims KL. Prospective peer review in surgicalpathology. Am J Clin Pathol. 1995;104:560-6. [PMID: 7572817]119. Lufkin KC, Smith SW, Matticks CA, Brunette DD. Radiologists’ review ofradiographs interpreted confidently by emergency physicians infrequently leads tochanges in patient management. Ann Emerg Med. 1998;31:202-7. [PMID:9472181]120. Murphy R, Slater A, Uberoi R, Bungay H, Ferrett C. Reduction of per-ception error by double reporting of minimal preparation CT colon. Br J Radiol.2010;83:331-5. [PMID: 19651707]121. Parameswaran L, Prihoda TJ, Sharkey FE. Diagnostic efficacy of additionalstep-sections in colorectal biopsies originally diagnosed as normal. Hum Pathol.2008;39:579-83. [PMID: 18289637]122. Robson N, van Benthem PP, Gan R, Dixon AK. Casualty X-ray reporting:a student survey. Clin Radiol. 1985;36:479-81. [PMID: 4075715]123. Thiesse P, Ollivier L, Di Stefano-Louineau D, Negrier S, Savary J, Pig-nard K, et al. Response rate accuracy in oncology trials: reasons for interobservervariability. Groupe Francais d’Immunotherapie of the Federation Nationale desCentres de Lutte Contre le Cancer. J Clin Oncol. 1997;15:3507-14. [PMID:9396404]124. Westra WH, Kronz JD, Eisele DW. The impact of second opinion surgicalpathology on the practice of head and neck surgery: a decade experience at a largereferral hospital. Head Neck. 2002;24:684-93. [PMID: 12112543]125. Buchner AM, Shahid MW, Heckman MG, Diehl NN, McNeil RB,Cleveland P, et al. Trainee participation is associated with increased small ade-noma detection. Gastrointest Endosc. 2011;73:1223-31. [PMID: 21481861]126. Swanson JO, Thapa MM, Iyer RS, Otto RK, Weinberger E. Optimizingpeer review: a year of experience after instituting a real-time comment-enhancedprogram at a children’s hospital. AJR Am J Roentgenol. 2012;198:1121-5.[PMID: 22528902]127. Thomas DC, Spitzer WO, MacFarlane JK. Inter-observer error amongsurgeons and nurses in presymptomatic detection of breast disease. J Chronic Dis.1981;34:617-26. [PMID: 7309826]128. Davis Giardina T, Singh H. Should patients get direct access to theirlaboratory test results? An answer with many questions. JAMA. 2011;306:2502-3. [PMID: 22122864]129. Casalino LP, Dunham D, Chin MH, Bielang R, Kistner EO, KarrisonTG, et al. Frequency of failure to inform patients of clinically significant outpa-tient test results. Arch Intern Med. 2009;169:1123-9. [PMID: 19546413]130. Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test resultsfor ambulatory patients: a systematic review. J Gen Intern Med. 2012;27:1334-48. [PMID: 22183961]




Current Author Addresses: Ms. McDonald, Ms. Lonhart, and Mr.Schmidt: Stanford Center for Health Policy/Center for Primary Careand Outcomes Research, Stanford University, 117 Encina Commons,Stanford, CA 94305-6019.Mr. Matesic and Ms. Pineda: School of Medicine, Stanford University,291 Campus Drive, Stanford, CA 94305.Dr. Contopoulos-Ioannidis: Department of Pediatrics, Division of Infec-tious Diseases, Stanford University School of Medicine, 300 PasteurDrive, G312, Stanford, CA 94305.Dr. Ioannidis: Stanford Prevention Research Center, Department ofMedicine, School of Medicine, Stanford University, 1265 Welch Road,X306, Stanford, CA 94305.

Author Contributions: Conception and design: K.M. McDonald,B. Matesic, D.G. Contopoulos-Ioannidis, J. Lonhart, J.P.A. Ioannidis.Analysis and interpretation of the data: K.M. McDonald, B. Matesic,D.G. Contopoulos-Ioannidis, J. Lonhart, E. Schmidt, J.P.A. Ioannidis.Drafting of the article: K.M. McDonald, B. Matesic, D.G. Contopoulos-Ioannidis, J. Lonhart, E. Schmidt, J.P.A. Ioannidis.Critical revision of the article for important intellectual content: K.M.McDonald, B. Matesic, D.G. Contopoulos-Ioannidis, J.P.A. Ioannidis.Final approval of the article: K.M. McDonald, B. Matesic, D.G.Contopoulos-Ioannidis, J. Lonhart, E. Schmidt, N. Pineda, J.P.A.Ioannidis.Provision of study materials or patients: J. Lonhart.Statistical expertise: D.G. Contopoulos-Ioannidis, J.P.A. Ioannidis.Obtaining of funding: K.M. McDonald.Administrative, technical, or logistic support: K.M. McDonald, B.Matesic, J. Lonhart, E. Schmidt, N. Pineda.Collection and assembly of data: K.M. McDonald, B. Matesic, J. Lon-hart, E. Schmidt, N. Pineda, J.P.A. Ioannidis.




Inpatient Fall Prevention Programs as a Patient Safety StrategyA Systematic ReviewIsomi M. Miake-Lye, BA; Susanne Hempel, PhD; David A. Ganz, MD, PhD; and Paul G. Shekelle, MD, PhD

Falls are common among inpatients. Several reviews, including 4meta-analyses involving 19 studies, show that multicomponent pro-grams to prevent falls among inpatients reduce relative risk for fallsby as much as 30%. The purpose of this updated review is toreassess the benefits and harms of fall prevention programs in acutecare settings and to identify factors associated with successful im-plementation of these programs. We searched for new evidenceusing PubMed from 2005 to September 2012. Two new, large,randomized, controlled trials supported the conclusions of the ex-isting meta-analyses. An optimal bundle of components was notidentified. Harms were not systematically examined, but potentialharms included increased use of restraints and sedating drugs anddecreased efforts to mobilize patients. Eleven studies showed that

the following themes were associated with successful implementa-tion: leadership support, engagement of front-line staff in programdesign, guidance of the prevention program by a multidisciplinarycommittee, pilot-testing interventions, use of information technol-ogy systems to provide data about falls, staff education and train-ing, and changes in nihilistic attitudes about fall prevention. Futureresearch would advance knowledge by identifying optimal bundlesof component interventions for particular patients and by determin-ing whether effectiveness relies more on the mix of the compo-nents or use of certain implementation strategies.


THE PROBLEM

The reported rate of falls in acute care hospitals rangesfrom 1.3 to 8.9 per 1000 bed-days (1). Higher rates arereported in neurology, geriatrics, and rehabilitation wards.Because falls are probably underreported, most estimatesmay be overly conservative (1). Defining a “fall” is a chal-lenge in itself (2, 3). For example, the National Database ofNursing Quality Indicators defines a fall as “an unplanneddescent to the floor with or without injury” (4), whereasthe World Health Organization defines a fall as “an eventwhich results in a person coming to rest inadvertently onthe ground or floor or some lower level” (5).

Regardless of the definition, falls occur frequently andcan have serious physical and psychological consequences.Between 30% and 50% of in-facility falls result in injuries(6, 7). Falls are associated with increased health care use,including increased length of stay and higher rates of dis-charge from hospitals into long-term care facilities. Even afall that does not cause an injury can trigger a fear offalling, anxiety, distress, depression, and reduced physicalactivity. Family members, caregivers, and health care pro-fessionals are susceptible to overly protective or emotionalreactions to falls, which can affect the patient’s indepen-dence and rehabilitation.

A fall is often the result of interactions betweenpatient-specific risk factors and the physical environment.The former risk factors include patient age (particularlyolder than 85 years), history of a recent fall, mobility im-pairment, urinary incontinence or frequency, certain med-

ications, and postural hypotension. The latter include poorlighting; “trip” hazards, such as uneven flooring or smallobjects on the floor; suboptimal chair heights; and limitedstaff availability or skills. Because in-facility falls can beprecipitated by many factors and patients who fall oftenhave several risk factors, multicomponent interventions arebelieved to be necessary for prevention. The purpose of thisupdated review is to reassess the benefits and harms ofmulticomponent inpatient programs for fall preventionand to assess the factors associated with successful imple-mentation of such programs.


All of the multicomponent fall prevention strategies inrecent meta-analyses included an assessment of fall risk (of-ten the Morse Fall Scale [8] or St. Thomas’s Risk Assess-ment Tool in Falling Elderly Inpatients [9] is used). Table 1lists additional components commonly included in multi-component interventions. These typically include staff andpatient education, a bedside risk sign or an alert wristband,attention to footwear, a toileting schedule, medication re-view, and a review after the fall to identify causes. Al-though most in-facility fall prevention programs are multi-component interventions, none of the controlled trialsexplicitly articulated a conceptual framework underpinningits intervention. Individual components of published strat-egies varied in type, intensity, duration, and targeting, andnone of the trials that evaluated multicomponent interven-tions used the same combination of components. Table 1of the Supplement (available at www.annals.org) showsdata about components of fall prevention strategies fromstudies addressed in this review.

REVIEW PROCESSES

We identified 4 recent existing reviews that were rele-vant to the topic of inpatient fall prevention. Reviews of

See also:


Annals of Internal MedicineSupplement



fall prevention in community-based settings were excluded.To identify relevant reviews, we used methods described byWhitlock and colleagues (10). See the Supplement for acomplete description of the search strategies, evidence ta-bles, and a literature flow diagram. These reviews weresupplemented with the results of a search by Hempel andcolleagues (11), which was done for a report that addressedprevention of inpatient falls. Hempel and coworkers used16 existing reviews and reports to identify additional per-tinent sources and searched PubMed, CINAHL, and theWeb of Science for relevant literature not yet covered inreviews. The search included randomized, controlled trials;nonrandomized trials; and before-and-after studies inEnglish-language publications that addressed falls in theacute care hospital setting. Searches were conducted from2005 through September 2012.

Previous Studies and ReviewsThe 4 systematic reviews are a 2008 review from the

Cochrane Collaboration by Cameron and colleagues (12),a 2008 review by Coussement and coworkers (13), a reviewby Oliver and colleagues originally published in 2007 (14)and then updated in 2010 as a narrative review (1), and a2012 review by DiBardino and colleagues (15). All 4 re-views scored well on the assessment of multiple systematicreviews (AMSTAR) criteria for systematic reviews (11 outof 11, 10 out of 11, 10 out of 11, and 8 out of 11,respectively), which evaluates such items as comprehensive-ness of the search, assessment of the quality of includedstudies, and methods for synthesizing the results (16). TheCochrane review searched for randomized trials to assessthe effectiveness of fall reduction interventions for olderadults in nursing care facilities and hospitals (12). Of the41 included trials, 11 were conducted in hospital settings,4 of which addressed multicomponent interventions. Thereview by Coussement and coworkers identified 4 multi-component studies, 2 of which were included in the Co-chrane review (13). The review by Oliver and colleaguesused broader inclusion criteria than the Cochrane review,which led to the inclusion of 43 trials, case–control stud-ies, and observational cohort studies (14). Thirteen of thesestudies were classified as multicomponent inpatient inter-ventions. Oliver and coworkers’ updated narrative reviewfocused directly on hospital fall prevention and discussed17 multicomponent studies spanning from 1999 to 2009,which include the 6 trials in the Cochrane and Cousse-ment and colleagues’ reviews (1, 13). The recent review byDiBardino and coworkers (15) identified 6 primary re-search studies in the acute care inpatient setting, 3 ofwhich were included in the Oliver and colleagues’ 2010update.

Supplemental SearchOur supplemental search started with studies identi-

fied by Hempel and coworkers, focusing on individual andcluster randomized, controlled trials with large sample sizesthat assessed multicomponent interventions in acute care

hospitals, in the general population or older adult popula-tion. We were looking for “pivotal studies,” defined byShojania and colleagues (17) as trials that may call intoquestion the results of an existing review. Studies werescreened by a clinician and nonclinician, each of whomwas experienced in systematic reviews. This search identi-fied 2 new relevant studies, both of which showed statisti-cally significant improvements in intervention groupswhen compared with control groups and which we discussbriefly later. We also describe a third study because of itsunique design. Because Oliver and coworkers’ 2010 updateused Downs and Black (18) to assess the quality of indi-vidual studies, we did the same for the 2 new studies. Weassessed the strength of evidence across studies using aframework developed for the Agency for Healthcare Re-search and Quality patient safety review (19). To identifystudies in which a principal goal was reporting on imple-mentation, we surveyed the results of our updated searchand queried experts for additional studies.

This review was supported by the Agency for Health-care Research and Quality, which had no role in the selec-tion or review of the evidence or the decision to submit themanuscript for publication.

BENEFITS AND HARMS

BenefitsTable 2 presents details about the 21 effectiveness

studies included in previous reviews or our updated search.

Key Summary Points

The rate of falls in acute care hospitals ranges fromapproximately 1 to 9 per 1000 bed-days.

High-quality evidence shows that multicomponent inter-ventions can reduce risk for in-hospital falls by as muchas 30%.

The optimal bundle of components is not established, butcommon components include risk assessments for patients,patient and staff education, bedside signs and wristbandalerts, footwear advice, scheduled and supervised toileting,and a medication review.

Harms of multicomponent interventions are unclear be-cause they have not been studied systematically, but theymay include the potential for increased use of restraintsand sedating drugs and decreased efforts to mobilizepatients.

Evidence about successful implementation of multicompo-nent interventions suggests that the following are impor-tant factors: leadership support, engagement of front-lineclinical staff in the design of the intervention, guidance bya multidisciplinary committee, pilot-testing the interven-tion, and changing nihilistic attitudes about falls.

SupplementInpatient Fall Prevention Programs as a Patient Safety Strategy



The 4 reviews we identified reached similar conclusions.The reviews by Cameron and colleagues (12) and Oliverand coworkers (14) found that multicomponent in-facilityprevention programs result in statistically and clinically sig-nificant reductions in rates of falls. Cameron and col-leagues included 6478 older adults from 4 randomized tri-als in a pooled analysis that found a 31% decrease in therate of falling (pooled rate ratio [RR], 0.69 [95% CI, 0.49to 0.96] and a 27% decrease in the incidence of falls whencompared with usual care among 3 trials involving 4824participants (RR, 0.73 [CI, 0.56 to 0.96]) (12). Oliver andcoworkers (14) included 5 randomized trials and 8 before-and-after studies in a pooled analysis that found an 18%decrease in the rate of falling (RR, 0.82 [CI, 0.68 to 1.00]).Coussement and colleagues (13) included 2 randomizedtrials, 1 before-and-after study, and 1 cohort study andfound a pooled RR similar to that of Oliver and coworkers;however, this effect was not quite statistically significant(RR, 0.82 [CI, 0.65 to 1.03]). DiBardino and colleagues’review (15) pooled data from 6 studies (including 1 ran-domized trial, 1 quasi-experimental study, and 4 before-and-after studies) and found a pooled odds ratio of 0.90(CI, 0.83 to 0.99). The studies included in these reviewsused interventions with 3 to 7 components and comparedthem with control participants who received usual care (forexample, “control ward had no trial intervention” [23] andcontrol participants who “followed conventional routines” [33]).

We rated the first trial identified in our update searchas having a low risk of bias. In this cluster randomized trial,Dykes and coworkers (24) compared the fall rates in 8units at 4 urban U.S. hospitals over a 6-month period.Control units in each hospital received usual care, whichincluded fall risk assessments, signage for high-risk pa-tients, patient education, and manual documentation inpatient records. The intervention units at each hospitaltested the Falls Prevention Tool Kit, which was developedby the study team. This kit is a health information tech-

nology application that includes a risk assessment and tai-lored signage, patient education, and plan-of-care compo-nents. Adjusted fall rates in the intervention units (3.15 per1000 patient days [CI, 2.54 to 3.90]) were lower thanthose of control units (4.18 per 1000 patient days [CI,3.45 to 5.06]), yielding a rate difference of 1.03 (CI, 0.57to 2.01). A particularly strong effect was found in patientsaged 65 years or older (rate difference, 2.08 per 1000 pa-tient days [CI, 0.61 to 3.56]).

In the second study, which we also judged to have lowrisk of bias, Ang and colleagues (20) randomly assignedpatients in 8 medical wards of an acute care hospital inSingapore to a target intervention or usual care. An assess-ment tool was used to match high-risk patients withappropriate interventions, in addition to an educationalsession tailored to patient-specific risk factors, in the inter-vention group. Both groups received usual care, which in-cluded environmental modifications, review of medicationsand fall history, and generic fall prevention advice. Theproportion of patients with at least 1 fall in the interven-tion group was 0.4% (CI, 0.2% to 1.1%), whereas in thecontrol group it was 1.5% (CI, 0.9% to 2.6%), for a rela-tive risk reduction of 0.29 (CI, 0.1 to 0.87).

One other study worth noting, by van Gaal and col-leagues (39, 40), evaluated a program that targeted 3 pa-tient safety practices (pressure ulcers, urinary tract infec-tions, and fall prevention) simultaneously. They found anoverall positive effect on development of any adverse event,a composite measure of pressure ulcers, urinary tract infec-tions, and falls. The study was not powered to assess fallsseparately, but it is worth noting that the point estimate forthe relative risk reduction in falls was 0.69, which is withinthe range of results reported in other studies and meta-analyses. The value of this study is the demonstration ofsimultaneous improvements in several safety intervention tar-gets that may be relevant to the same patient population.

HarmsMost trials of fall prevention programs did not report

any harms, although 1 reported constipation from intakeof vitamin D (13). Whether trials explicitly assessed thepossibility of harms was mostly unclear. Despite little em-pirical evidence, concern exists that some fall preventioninterventions may lead to harms. For example, Oliver andcolleagues (1) detailed many potential harms, includingthose that would result from increased use of restraints orsedating medications.


Structural organizational characteristics, existing qual-ity and safety infrastructure, patient safety culture, team-work, and leadership are believed to be important contextsfor understanding the effectiveness of fall prevention pro-grams (41, 42).

Table 1. Intervention Components in Studies of InpatientFalls Prevention Programs

Component Studies Including This Component, n

Patient education 11Bedside risk sign 10Staff education 9Alert wristband 7Footwear 7Review after fall 7Toileting schedules 7Medication review 6Environment modification 5Movement alarms 5Bedrail review 4Exercise 4Hip protectors 3Urine screening 2Vest, belt, or cuff restraint 1

Supplement Inpatient Fall Prevention Programs as a Patient Safety Strategy



Structural Organizational CharacteristicsStudies evaluating fall prevention programs were done

in various geographic areas and settings, including theUnited States, Australia, the United Kingdom, Sweden,Singapore, France, Switzerland, the Netherlands, and Ger-many (see Table 3 of the Supplement). Several were con-ducted in an academically affiliated or teaching hospital.Sizes of hospitals varied from small (fewer than 100 beds)to large (greater than 500). Some studies encompassed sev-eral hospitals (for example, 4), and others involved multi-ple wards. These data show that fall prevention programshave been implemented in hospitals of varying size, loca-tion, and academic or teaching status. No studies reportedon financial concerns (for example, how patient care or theinterventions were financed), although 1 U.S. study men-tioned the potential effect of reimbursement on the em-phasis on fall prevention (24).

Existing InfrastructureExisting organizational infrastructure was described

rarely, with only 5 of the 21 studies describing this for theirsettings. In 4 studies, this description was limited to theirusual fall prevention care. The fifth study provided a moreexplicit statement, namely, “prior to this study none of thewards carried out specific fall assessments or interven-tions . . . there was no specialist falls clinic or other fallsservice available at this hospital” (28). Two studies re-ported on the presence or absence of information systemsthat could be used in fall prevention programs (24, 26).

Patient Safety Culture, Teamwork, and LeadershipAlthough some studies briefly mentioned patient

safety culture, teamwork, or leadership, only 4 studies pre-sented expanded explanations of those factors. Grenier-Sennelier and colleagues (26) used a framework from Shor-

Table 2. Abridged Evidence Tables*

Study, Year(Reference)

Study Design Setting Participants QualityScore†

Outcome

Ang et al, 2011 (20)‡ RCT 8 medical wards; acute care; Singapore 1822 patients 25 Significantly fewer fallsBarker et al, 2009 (21) Before-and-after Small; acute care; Australia 271 095 patients 16 Significantly fewer injuriesBarry et al, 2001 (22) Before-and-after Small; long-stay and rehabilitation;

IrelandAll patients admitted to

95 beds for 3 y15 Significantly fewer injuries

Brandis, 1999 (7) Before-and-after Acute; Australia All patients admitted to500 beds for 2 y

11 Nonsignificantly fewer falls

Cumming et al,2008 (23)

Cluster RCT 24 wards; acute and rehabilitation;Australia

3999 patients 27 Nonsignificantly fewer falls

Dykes et al,2010 (24)‡

Cluster RCT 8 units; medical; urban United States All patients admitted ortransferred to units over6-mo study period

27 Significantly fewer falls

Fonda et al, 2006 (25) Before-and-after 4 wards; elderly acute andrehabilitation; Australia

3961 patients 20 Significantly fewer falls

Grenier-Sennelier et al,2002 (26)

Before-and-after 400 beds; rehabilitation; France All admitted patients over4 y


Haines et al, 2004 (27) RCT 3 wards; subacute, rehabilitation, andelderly; Australia


Healey et al, 2004 (28) Cluster RCT 8 wards; acute and rehabilitation;3 hospitals; United Kingdom


Koh et al, 2009 (29) Cluster RCT 2 hospitals; acute; Singapore All admissions over 1.5 y 14 Nonsignificantly fewer fallsKrauss et al, 2008 (30) Before-and-after General medicine; acute academic

hospital; United StatesAll admissions over 18 mo 18 Nonsignificantly fewer falls

Mitchell and Jones,1996 (31)

Before-and-after 1 acute and 1 subacute ward; 32 beds;Australia

All patients admitted to32 beds for 6 mo

16 Nonsignificantly fewer falls

Oliver et al, 2002 (32) Before-and-after Elderly medical unit; acute hospital;United Kingdom

3200 patients admittedannually; data over 2 y

8 Nonsignificantly greater falls

Schwendimann et al,2006 (6)

Before-and-after 300 beds; internal medical, geriatric,and surgical; Switzerland

34 972 admissions 15 Nonsignificantly fewer falls

Stenvall et al,2007 (33)

RCT 3 wards; orthogeriatric, geriatric,orthopedic; Sweden


Udén et al, 1999 (34) Before-and-after Geriatric department; acute hospital;Sweden

379 patients 12 Nonsignificantly greater falls

van der Helm et al,2006 (35)

Before-and-after Internal medical and neurology wards;acute hospital; the Netherlands

2670 patients 11 Nonsignificantly greater falls

Vassallo et al,2004 (36)

Cohort 3 wards; rehabilitation; UnitedKingdom


von Renteln-Kruse andKrause, 2007 (37)

Before-and-after Elderly acute and rehabilitation wards;Germany


Williams et al,2007 (38)

Before-and-after 3 medical wards and 1 geriatric unit;Australia

1357 admitted patientsduring 6-mointervention


RCT � randomized, controlled trial.* From reference 1.† Downs and Black Quality Score (18), evaluated by Oliver and colleagues (1)—except for entries in italics, which were evaluated by Ms. Miake-Lye and Dr. Shekelle.‡ New studies added from updated search.




tell and coworkers (43) and Gillies and colleagues (44) toanalyze culture at the unit level, teamwork at both theorganizational and unit levels, and leadership at the orga-nizational and unit levels. Stenvall and colleagues (33) dis-cussed teamwork at the unit level. Koh and coworkers (29)discussed leadership on the organizational and unit levels.van der Helm and colleagues (35) made several observa-tions addressing leadership on both the organizational andunit levels.

ImplementationImplementation details are also considered to be im-

portant in understanding the effectiveness of fall preven-tion programs (41). The most commonly reported im-plementation details in the 21 studies were patientcharacteristics and the initial plan, or the intended inter-vention components. Some studies reported the intendedroles of project staff, or by whom the intended interventioncomponents were to be completed. Most studies reportedthe recipients of any training component, with slightlyfewer reporting the type of training or giving a descriptionof the training and even fewer studies reporting the lengthof training. Thus, the context and duration of trainingneeded to implement fall prevention programs need betterdescriptions.

Several studies provided the materials used in programimplementation, and some reported on adherence or fidel-ity to the designed initiative and how and why the planevolved. Adherence or fidelity was most often characterizedin a qualitative statement. According to Brandis (7): “Thestrategies implemented . . . had high acceptance by staff.”Williams and colleagues (38) found staff involvement cru-cial to fidelity: “[I]nvolving ward staff . . . so that they takeownership of the project and do not perceive it as beingdriven by middle management were important strategies.”Dykes and coworkers (24) provided a strong example ofadherence reporting, in which protocol adherence wasmeasured by completion of components in both control(81%) and intervention (94%) wards. Such quantitativedata on protocol adherence should be encouraged in futureevaluations of fall prevention programs. Measures of adop-tion and reach were usually provided in the form of a flowchart—6 studies presented these data for providers, and 8presented the data for patients.

In addition to the studies previously discussed, weidentified 11 studies that focused primarily on implemen-tation. None were randomized, clinical trials and all studieshad either pre–post or time-series designs. Six studies werepoststudy evaluations of fall prevention implementationsthat reported detail about the potential reasons for effec-tiveness or lack thereof. Nine of the 11 studies assessedimplementation at only 1 or 2 facilities. Four studies re-ported no beneficial effects of the fall prevention programand highlighted potential implementation factors that mayaccount for the lack of success. One study explicitly as-sessed the effect of some contextual factors on intervention

success across 34 facilities (described later) (45). One studyexplicitly assessed sustainability. From these 11 studies, weidentified the following 7 themes about effective imple-mentation: leadership support is critical, both at the facilitylevel (for example, hospital director) and at the unit level(for example, unit director or “clinical champions”); en-gagement of front-line clinical staff in the design of theintervention helps ensure that it will mesh with existingclinical procedures; use of multidisciplinary committees isneeded to guide or oversee the interventions; the interven-tion should be pilot-tested to help identify potential prob-lems with implementation; information systems that arecapable of providing data about falls can facilitate evalua-tion of the causes and adherence to the intervention com-ponents and potentially be a crucial facilitator of the inter-vention; changing the prevailing nihilistic attitude that fallsare “inevitable” and that “nothing can be done” is requiredto get buy-in to the goals of the intervention (46, 47); andeducation and training of clinical staff are necessary to helpensure that adherence does not diminish. Table 5 of theSupplement presents evidence from the 11 studies sup-porting each theme.

CostsThe Cochrane review found no economic evaluations

of the fall prevention programs that met inclusion criteria(12). Oliver and colleagues (1) estimated the cost for spe-cific combinations of components in terms of environmentand equipment and in terms of staff; most costs were lowor inconsequential.

The Effects of Context on EffectivenessWe identified only 1 study that explicitly assessed the

effect of context on effectiveness (45). Across 34 VeteransAffairs health centers (a mix of acute care and long-termcare facilities), leadership support was cited as one of thestrongest factors for success. At 1-year follow-up, high-performing sites reported greater agreement with questionsassessing leadership support, teamwork skills, and usefulinformation systems than low-performing sites.

DISCUSSION

The evidence base indicates that inpatient multicom-ponent programs are effective at reducing falls and thatconsistent themes are associated with successful implemen-tation. However, there is no strong evidence about whichcomponents are most important for success. The effects ofcontext have not been well-studied; however, multicompo-nent interventions have been effective in hospitals that varyin size, location, and teaching status. The cost of imple-menting fall prevention programs has not been rigorouslyassessed but generally does not involve capital expenses orhiring new staff.

Our results about effectiveness are consistent with pre-vious reviews on inpatient fall prevention programs. Ourreview additionally identifies 7 themes associated with suc-




cessful implementation. Some themes, such as education ortraining and leadership support, are often included in gen-eral lists of factors for successful implementation of anyintervention, whereas themes that may be more specific tofall prevention programs include development and guid-ance by a multidisciplinary committee and changing theprevailing attitudes of nihilism with respect to falls.

Our findings that multicomponent fall preventionprograms are effective in inpatient settings may seem atodds with recent U.S. Preventive Services Task Force rec-ommendations not to automatically do a multifactorial fallassessment in community-dwelling adults aged 65 years orolder (48). However, there is no contradiction because,although the goal is to prevent falls in both community-dwelling and hospitalized patients, the settings are differ-ent. The hospital environment is more tightly controlledthan the outpatient setting, where it is more difficult toensure that risk factors for falls are appropriately managed.In fact, as Tinetti and Brach (49) note, community-basedmultifactorial programs achieve greater reduction in fallswhen identified risk factors are actually managed.

Our review has several limitations. Like all reviews, weare limited by the quality and quantity of the original re-search articles. Also, we did not do an exhaustive update ofexisting reviews. With several previous reviews reachingconsistent results, including a total of 19 effectiveness stud-ies, we focused instead on identifying “pivotal studies” thatmay call into question the conclusions of previous reviews.None were found; additional large randomized, controlledtrials supported the conclusions of existing reviews. Ourassessment of implementation themes is novel and deservesprospective evaluation (for example, one that could mea-sure the degree of leadership support or staff attitudesabout fall prevention before and during an intervention).

For multicomponent inpatient fall programs, our re-view provides both evidence that such programs reducefalls and insight into how facilities can successfully imple-ment them. Future research would most effectively ad-vance the field by determining whether an “optimal”bundle of components exists or whether effectiveness isprimarily a function of successful implementation.

From the Veterans Affairs Greater Los Angeles Healthcare System andDavid Geffen School of Medicine at the University of California, LosAngeles, Los Angeles, and the RAND Corporation, Santa Monica,California.

Note: The Agency for Healthcare Research and Quality (AHRQ) re-viewed contract deliverables to ensure adherence to contract require-ments and quality, and a copyright release was obtained from the AHRQbefore submission of the manuscript.

Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof the RAND Corporation; U.S. Department of Veterans Affairs; Uni-versity of California, Los Angeles; the AHRQ; or U.S. Department ofHealth and Human Services.

Acknowledgment: The authors thank Aneesa Motala, BA; Sydne New-berry, PhD; and Roberta Shanman, MLS.

Financial Support: From the AHRQ, U.S. Department of Health andHuman Services (contracts HHSA-290-2007-10062I, HHSA-290-2010-00017I, and HHSA-290-32001T). Dr. Ganz was supported by a CareerDevelopment Award from the Veterans Affairs Health Services Research& Development Service, Veterans Health Administration, U.S. Depart-ment of Veterans Affairs through the Veterans Affairs Greater Los An-geles Health Services Research & Development Center of Excellence(project VA CD2 08-012-1).

Potential Conflicts of Interest: Dr. Hempel: Grant (money to institu-tion): AHRQ. Dr. Ganz: Grant (money to institution): AHRQ, VeteransAffairs Health Services Research and Development Service. Dr. Shekelle:Consultancy: ECRI Institute; Employment: Veterans Affairs; Grants/grantspending: AHRQ, Veterans Affairs, Centers for Medicare & MedicaidServices, National Institute of Nursing Research, Office of the NationalCoordinator; Royalties: UpToDate. Ms. Miake-Lye: None disclosed.Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2569.

Requests for Single Reprints: Paul G. Shekelle, MD, PhD, RANDCorporation, 1776 Main Street, Santa Monica, CA 90401; e-mail,[email protected].

Current author addresses and author contributions are available atwww.annals.org.

References1. Oliver D, Healey F, Haines TP. Preventing falls and fall-related injuries inhospitals. Clin Geriatr Med. 2010;26:645-92. [PMID: 20934615]2. Zecevic AA, Salmoni AW, Speechley M, Vandervoort AA. Defining a fall andreasons for falling: comparisons among the views of seniors, health care providers,and the research literature. Gerontologist. 2006;46:367-76. [PMID: 16731875]3. Schwenk M, Lauenroth A, Stock C, Moreno RR, Oster P, McHugh G, et al.Definitions and methods of measuring and reporting on injurious falls in ran-domised controlled fall prevention trials: a systematic review. BMC Med ResMethodol. 2012;12:50. [PMID: 22510239]4. National Database of Nursing Quality Indicators. Guidelines for Data Col-lection on the American Nurses Association’s National Quality Forum EndorsedMeasures: Nursing Care Hours per Patient Day, Skill Mix, Falls, Falls withInjury. National Center for Nursing Quality; March 2012. Accessed at www.odh.ohio.gov/~/media/ODH/ASSETS/Files/dspc/health%20care%20service/nursestaffingmaterials8-2-2010.ashx on 7 January 2013.5. World Health Organization. Violence and Injury Prevention: Falls. 2012.Accessed at www.who.int/violence_injury_prevention/other_injury/falls/en on 25July 2012.6. Schwendimann R, Buhler H, De Geest S, Milisen K. Falls and consequentinjuries in hospitalized patients: effects of an interdisciplinary falls preventionprogram. BMC Health Serv Res. 2006;6:69. [PMID: 16759386]7. Brandis S. A collaborative occupational therapy and nursing approach to fallsprevention in hospital inpatients. J Qual Clin Pract. 1999;19:215-20. [PMID:10619149]8. Morse JM. Preventing Patient Falls. Thousand Oaks, CA: Sage; 1997.9. Oliver D, Britton M, Seed P, Martin FC, Hopper AH. Development andevaluation of evidence based risk assessment tool (STRATIFY) to predict whichelderly inpatients will fall: case-control and cohort studies. BMJ. 1997;315:1049-53. [PMID: 9366729]10. Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existingsystematic reviews in complex systematic reviews. Ann Intern Med. 2008;148:776-82. [PMID: 18490690]11. Hempel S, Newberry S, Wang Z, Shekelle PG, Shanman RM, Johnsen B,et al. Review of the Evidence on Falls Prevention in Hospitals: Task 4 Final




Report. Santa Monica, CA: RAND Corporation; 2012. Accessed at www.rand.org/pubs/working_papers/WR907.html on 7 January 2013.12. Cameron ID, Murray GR, Gillespie LD, Robertson MC, Hill KD, Cum-ming RG, et al. Interventions for preventing falls in older people in nursing carefacilities and hospitals. Cochrane Database Syst Rev. 2010:CD005465. [PMID:20091578]13. Coussement J, De Paepe L, Schwendimann R, Denhaerynck K, DejaegerE, Milisen K. Interventions for preventing falls in acute- and chronic-care hospi-tals: a systematic review and meta-analysis. J Am Geriatr Soc. 2008;56:29-36.[PMID: 18031484]14. Oliver D, Connelly JB, Victor CR, Shaw FE, Whitehead A, Genc Y, et al.Strategies to prevent falls and fractures in hospitals and care homes and effect ofcognitive impairment: systematic review and meta-analyses. BMJ. 2007;334:82.[PMID: 17158580]15. DiBardino D, Cohen ER, Didwania A. Meta-analysis: multidisciplinary fallprevention strategies in the acute care inpatient population. J Hosp Med. 2012;7:497-503. [PMID: 22371369]16. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.Development of AMSTAR: a measurement tool to assess the methodologicalquality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:17302989]17. Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. Howquickly do systematic reviews go out of date? A survival analysis. Ann Intern Med.2007;147:224-33. [PMID: 17638714]18. Downs SH, Black N. The feasibility of creating a checklist for the assessmentof the methodological quality both of randomised and non-randomised studies ofhealth care interventions. J Epidemiol Community Health. 1998;52:377-84.[PMID: 9764259]19. Shekelle PG, Wachter RM, Pronovost PJ, Schoelles K, McDonald KM, DySM, et al. Making Health Care Safer II: An Updated Critical Analysis of the Evi-dence for Patient Safety Practices. (Prepared by the Southern California-RANDEvidence-based Practice Center under contract HHSA290200710062I.) Rockville,MD: Agency for Healthcare Research and Quality; 2013. [Forthcoming].20. Ang E, Mordiffi SZ, Wong HB. Evaluating the use of a targeted multipleintervention strategy in reducing patient falls in an acute care hospital: a random-ized controlled trial. J Adv Nurs. 2011;67:1984-92. [PMID: 21507049]21. Barker A, Kamar J, Morton A, Berlowitz D. Bridging the gap betweenresearch and practice: review of a targeted hospital inpatient fall prevention pro-gramme. Qual Saf Health Care. 2009;18:467-72. [PMID: 19955459]22. Barry E, Laffoy M, Matthews E, Carey D. Preventing accidental falls amongolder people in long stay units. Ir Med J. 2001;94:172, 174-6. [PMID:11495234]23. Cumming RG, Sherrington C, Lord SR, Simpson JM, Vogler C, CameronID, et al; Prevention of Older People’s Injury Falls Prevention in HospitalsResearch Group. Cluster randomised trial of a targeted multifactorial interven-tion to prevent falls among older people in hospital. BMJ. 2008;336:758-60.[PMID: 18332052]24. Dykes PC, Carroll DL, Hurley A, Lipsitz S, Benoit A, Chang F, et al. Fallprevention in acute care hospitals: a randomized trial. JAMA. 2010;304:1912-8.[PMID: 21045097]25. Fonda D, Cook J, Sandler V, Bailey M. Sustained reduction in seriousfall-related injuries in older people in hospital. Med J Aust. 2006;184:379-82.[PMID: 16618235]26. Grenier-Sennelier C, Lombard I, Jeny-Loeper C, Maillet-Gouret MC, Min-vielle E. Designing adverse event prevention programs using quality managementmethods: the case of falls in hospital. Int J Qual Health Care. 2002;14:419-26.[PMID: 12389808]27. Haines TP, Bennell KL, Osborne RH, Hill KD. Effectiveness of targetedfalls prevention programme in subacute hospital setting: randomised controlledtrial. BMJ. 2004;328:676. [PMID: 15031238]28. Healey F, Monro A, Cockram A, Adams V, Heseltine D. Using targeted riskfactor reduction to prevent falls in older in-patients: a randomised controlled trial.Age Ageing. 2004;33:390-5. [PMID: 15151914]29. Koh SL, Hafizah N, Lee JY, Loo YL, Muthu R. Impact of a fall preventionprogramme in acute hospital settings in Singapore. Singapore Med J. 2009;50:425-32. [PMID: 19421690]

30. Krauss MJ, Tutlam N, Costantinou E, Johnson S, Jackson D, Fraser VJ.Intervention to prevent falls on the medical service in a teaching hospital. InfectControl Hosp Epidemiol. 2008;29:539-45. [PMID: 18476777]31. Mitchell A, Jones N. Striving to prevent falls in an acute care setting—actionto enhance quality. J Clin Nurs. 1996;5:213-20. [PMID: 8718053]32. Oliver D, Martin F, Seed P. Preventing patient falls [Letter]. Age Ageing.2002;31:75-6. [PMID: 11850313]33. Stenvall M, Olofsson B, Lundstrom M, Englund U, Borssen B, SvenssonO, et al. A multidisciplinary, multifactorial intervention program reduces post-operative falls and injuries after femoral neck fracture. Osteoporos Int. 2007;18:167-75. [PMID: 17061151]34. Uden G, Ehnfors M, Sjostrom K. Use of initial risk assessment and record-ing as the main nursing intervention in identifying risk of falls. J Adv Nurs.1999;29:145-52. [PMID: 10064293]35. van der Helm J, Goossens A, Bossuyt P. When implementation fails: thecase of a nursing guideline for fall prevention. Jt Comm J Qual Patient Saf.2006;32:152-60. [PMID: 16617946]36. Vassallo M, Vignaraja R, Sharma JC, Hallam H, Binns K, Briggs R, et al.The effect of changing practice on fall prevention in a rehabilitative hospital: theHospital Injury Prevention Study. J Am Geriatr Soc. 2004;52:335-9. [PMID:14962145]37. von Renteln-Kruse W, Krause T. Incidence of in-hospital falls in geriatric pa-tients before and after the introduction of an interdisciplinary team-based fall-prevention intervention. J Am Geriatr Soc. 2007;55:2068-74. [PMID: 17971140]38. Williams TA, King G, Hill AM, Rajagopal M, Barnes T, Basu A, et al.Evaluation of a falls prevention programme in an acute tertiary care hospital.J Clin Nurs. 2007;16:316-24. [PMID: 17239067]39. van Gaal BG, Schoonhoven L, Hulscher ME, Mintjes JA, Borm GF, Koop-mans RT, et al. The design of the SAFE or SORRY? study: a cluster randomisedtrial on the develpment and testing of an evidence based inpatient safety programfor the prevention of adverse events. BMC Health Serv Res. 2009;9:58. [PMID:19338655]40. van Gaal BG, Schoonhoven L, Mintjes JA, Borm GF, Hulscher ME, De-floor T, et al. Fewer adverse events as a result of the SAFE or SORRY? pro-gramme in hospitals and nursing homes. part i: primary outcome of a clusterrandomised trial. Int J Nurs Stud. 2011;48:1040-8. [PMID: 21419411]41. Shekelle PG, Pronovost P, Wachter R, Taylor S, Dy S, Foy R, et al; PSPTechnical Expert Panel. Assessing the Evidence for Context-Sensitive Effective-ness and Safety of Patient Safety Practices: Developing Criteria. (Prepared undercontract HHSA-290-2009-10001C.) AHRQ publication no. 11-0006-EF.Rockville, MD: Agency for Healthcare Research and Quality; 2010. Accessed atwww.ahrq.gov/qual/contextsensitive on 7 January 2013.42. Shojania KG, Duncan BW, McDonald KM, Wachter RM, Markowitz AJ.Making health care safer: a critical analysis of patient safety practices. Evid RepTechnol Assess (Summ). 2001:i-x, 1-668. [PMID: 11510252]43. Shortell SM, O’Brien JL, Carman JM, Foster RW, Hughes EF, BoerstlerH, et al. Assessing the impact of continuous quality improvement/total qualitymanagement: concept versus implementation. Health Serv Res. 1995;30:377-401. [PMID: 7782222]44. Gillies GL, Reynolds JH, Shortell SM, Hughes EF, Budetti P, Huang CF,et al. Implementing continuous quality improvement. In: Kimberly JR, MinvielleE, eds. The Quality Imperative Measurement and Management of Quality inHealthcare. London: Imperial Coll Pr; 2000.45. Neily J, Howard K, Quigley P, Mills PD. One-year follow-up after a col-laborative breakthrough series on reducing falls and fall-related injuries. Jt CommJ Qual Patient Saf. 2005;31:275-85. [PMID: 15960018]46. Semin-Goossens A, van der Helm JM, Bossuyt PM. A failed model-basedattempt to implement an evidence-based nursing guideline for fall prevention.J Nurs Care Qual. 2003;18:217-25. [PMID: 12856906]47. Dempsey J. Falls prevention revisited: a call for a new approach. J Clin Nurs.2004;13:479-85. [PMID: 15086634]48. Moyer VA; U.S. Preventive Services Task Force. Prevention of falls incommunity-dwelling older adults: U.S. Preventive Services Task Force recom-mendation statement. Ann Intern Med. 2012;157:197-204. [PMID: 22868837]49. Tinetti ME, Brach JS. Translating the fall prevention recommendations intoa covered service: can it be done, and who should do it? [Editorial]. Ann InternMed. 2012;157:213-4. [PMID: 22868841]




Current Author Addresses: Ms. Miake-Lye and Drs. Ganz andShekelle:Veterans Affairs Greater Los Angeles Healthcare System, 11301 WilshireBoulevard, Los Angeles, CA 90073.Dr. Hempel: RAND Corporation, 1776 Main Street, Santa Monica, CA90401.

Author Contributions: Conception and design: P.G. Shekelle.Analysis and interpretation of the data: I.M. Miake-Lye, S. Hempel,D.A. Ganz, P.G. Shekelle.Drafting of the article: I.M. Miake-Lye, P.G. Shekelle.Critical revision of the article for important intellectual content: I.M.Miake-Lye, S. Hempel, D.A. Ganz, P.G. Shekelle.Final approval of the article: I.M. Miake-Lye, S. Hempel, D.A. Ganz,P.G. Shekelle.Provision of study materials or patients: P.G. Shekelle.Obtaining of funding: P.G. Shekelle.Administrative, technical, or logistic support: I.M. Miake-Lye, P.G.Shekelle.Collection and assembly of data: I.M. Miake-Lye, S. Hempel, P.G.Shekelle.

50. Browne JA, Covington BG, Davila Y. Using information technology to assistin redesign of a fall prevention program. J Nurs Care Qual. 2004;19:218-25.[PMID: 15326991]51. Capan K, Lynch B. A hospital fall assessment and intervention project. J ClinOutcomes Manag. 2007;14:155-60.52. Gutierrez F, Smith K. Reducing falls in a Definitive Observation Unit: anevidence-based practice institute consortium project. Crit Care Nurs Q. 2008;31:127-39. [PMID: 18360143]53. Kolin MM, Minnier T, Hale KM, Martin SC, Thompson LE. Fall initia-tives: redesigning best practice. J Nurs Adm. 2010;40:384-91. [PMID:20798621]54. McCollam ME. Evaluation and implementation of a research-based fallsassessment innovation. Nurs Clin North Am. 1995;30:507-14. [PMID:7567575]55. O’Connell B, Myers H. A failed fall prevention study in an acute care setting:lessons from the swamp. Int J Nurs Pract. 2001;7:126-30. [PMID: 11811315]56. Rauch K, Balascio J, Gilbert P. Excellence in action: developing and imple-menting a fall prevention program. J Healthc Qual. 2009;31:36-42. [PMID:19343900]57. Weinberg J, Proske D, Szerszen A, Lefkovic K, Cline C, El-Sayegh S, et al.An inpatient fall prevention initiative in a tertiary care hospital. Jt Comm J QualPatient Saf. 2011;37:317-25. [PMID: 21819030]




Medication Reconciliation During Transitions of Care as a PatientSafety StrategyA Systematic ReviewJanice L. Kwan, MD*; Lisha Lo, MPH*; Margaret Sampson, MLIS, PhD; and Kaveh G. Shojania, MD

Medication reconciliation identifies and resolves unintentional dis-crepancies between patients’ medication lists across transitions incare. The purpose of this review is to summarize evidence aboutthe effectiveness of hospital-based medication reconciliation inter-ventions. Searches encompassed MEDLINE through November2012 and EMBASE and the Cochrane Central Register of ControlledTrials through July 2012. Eligible studies evaluated the effects ofhospital-based medication reconciliation on unintentional discrepan-cies with nontrivial risks for harm to patients or 30-day postdis-charge emergency department visits and readmission. Two review-ers evaluated study eligibility, abstracted data, and assessed studyquality.

Eighteen studies evaluating 20 interventions met the selectioncriteria. Pharmacists performed medication reconciliation in 17 ofthe 20 interventions. Most unintentional discrepancies identifiedhad no clinical significance. Medication reconciliation alone proba-bly does not reduce postdischarge hospital utilization but may doso when bundled with interventions aimed at improving caretransitions.

Ann Intern Med. 2013;158:397-403. www.annals.orgFor author affiliations, see end of text.* Dr. Kwan and Ms. Lo contributed equally to this manuscript.

THE PROBLEM

Transitions in care, such as admission to and dischargefrom the hospital, put patients at risk for errors due to poorcommunication and inadvertent information loss (1–5).Unintentional changes to patients’ medication regimensrepresent 1 well-studied category of such errors (6–9).Medication regimens at hospital discharge often differfrom preadmission medications. Some differences reflectdeliberate changes related to the conditions that led tohospitalization (for example, withholding antihypertensivemedications from patients with septic shock). However,other discrepancies are unintentional and result from in-complete or inaccurate information about current medica-tions and doses.

Up to 67% of patients admitted to the hospital haveunintended medication discrepancies (9), and these dis-crepancies remain common at discharge (7, 10). As inother areas of patient safety, errors are more common thanactual harms. Reported proportions of unintended discrep-ancies with the potential for harm range from 11% to 59%of all discrepancies (9). Of note, approximately 40% to80% of patients have no clinically significant unintendedmedication discrepancies (8, 10–16). Thus, although un-intended medication discrepancies are common, clinicallysignificant discrepancies may affect only a few patients.

Nonetheless, medication reconciliation, the formalprocess for identifying and correcting unintended medica-tion discrepancies across transitions of care, has beenwidely endorsed (17, 18) and is mandated by health careaccreditation bodies in both the United States (19) andCanada (20). One previous systematic review (21) lookedbroadly at the effect of medication reconciliation on vari-ous processes and outcomes related to medication safety.We sought to focus specifically on the effect of medication

reconciliation on unintentional discrepancies with the po-tential for harm (“clinically significant discrepancies”) andhospital utilization after discharge, as assessed by un-planned emergency department visits and readmission tothe hospital within 30 days.

PATIENT SAFETY STRATEGY

The best possible medication history (BPMH) pro-vides the cornerstone for medication reconciliation. Morecomprehensive than a routine primary medication history,the BPMH involves 2 steps: a systematic process for ob-taining a thorough history of all prescribed and nonpre-scribed medications by using a structured patient inter-view, and verification of this information with at least 1other reliable source of information (for example, a govern-ment medication database, medication vials, patient med-ication lists, a community pharmacy, or a primary carephysician) (17, 22) (Figure).

At a minimum, medication reconciliation refers to thecompletion of a BPMH and the act of correcting any unin-tended discrepancies between a patient’s previous medicationregimen and the proposed medication orders at admission(from home or a health care facility, such as a nursing home),inpatient transfer (to or from other services or units, such asthe intensive care unit), or discharge (to home or a health carefacility). More advanced medication reconciliation involves

See also:





Sibelius

Highlight

interprofessional collaboration (for example, a physician andnurse or pharmacist conducting medication reconciliation as ateam), integration into discharge summaries and prescrip-tions, and provision of medication counseling to patients (22).Medication reconciliation has also been bundled with otherinterventions to improve the quality of transitions in care,such as patient counseling about discharge care plans, coordi-nation of follow-up appointments, and postdischarge tele-phone calls (23–26).

Recommendations for medication reconciliation inambulatory settings have begun to appear (27, 28). How-ever, most studies still focus on medication reconciliationacross hospital-based transitions in care, which is the focusof our review.

REVIEW PROCESSES

The Supplement, available at www.annals.org, in-cludes a complete description of the search strategies, sum-mary of evidence search and selection, and evidence tables.

We searched MEDLINE to 5 November 2012,EMBASE between 1980 and July 2012, and the CochraneCentral Register of Controlled Trials to July 2012 forEnglish-language articles (Figure 1 of the Supplement).We also scanned reference lists of all included studies andreview articles and directly communicated with study au-thors as required to obtain details not included in pub-lished reports. We included randomized, controlled trials(RCTs); before-and-after evaluations; and postinterventionstudies.

Eligible studies reported emergency department visitsand hospitalizations within 30 days of discharge or evalu-ated the severity or clinical significance of unintentionaldiscrepancies. For studies reporting unintended discrepan-cies, we required that at least 1 clinician independent from

the medication reconciliation process assess severity or clin-ical significance. Thus, we excluded studies in which theperson conducting medication reconciliation provided thesole assessment of clinical significance for identified dis-crepancies. We also required that studies explicitly distin-guish unintentional discrepancies from other (intentional)medication changes through direct communication withthe medical team.

Although studies varied in their definitions of catego-ries of severity for the potential harm associated with med-ication discrepancies, most reported a category thatamounted to “trivial,” “minor,” or “unlikely to causeharm.” We applied the term “clinically significant” to allunintended discrepancies not labeled as such. This defini-tion of clinically significant unintentional discrepanciescorresponds to the concept of potential adverse drug events(ADEs), although only a few studies explicitly used thisterm (25, 29–31).

Two of 3 reviewers independently screened each cita-tion for inclusion. Information was abstracted about clini-cal setting, study design, number of participants, compo-nents of the intervention, transitions of care targeted, andoutcomes. Disagreements between the 2 reviewers were re-solved by discussion and involved a third reviewer whennecessary to achieve consensus. The full data extractionform (available on request) included questions directed atgeneral methodological features (for example, sample sizeand study design), details about the components of themedication reconciliation intervention (for example, com-ponents of the BPMH and the method for confirming thatmedication discrepancies were unintended), and the pro-cess for assessing the clinical significance of identifieddiscrepancies.

Two reviewers independently applied the CochraneCollaboration’s tool for assessing risk of bias (32) to eachof the 5 included RCTs, assessing patient selection bias,selective reporting, patient attrition, and other biases byusing this standardized tool. Meta-analysis was performedwith Comprehensive Meta-Analysis (Biostat, Englewood,New Jersey). For results from studies of disparate designs,we calculated the median effect and interquartile range byusing Microsoft Excel (Microsoft, Seattle, Washington).This approach was first used in a large review of guidelineimplementation strategies (33) and has since been appliedin other systematic reviews of quality improvement inter-ventions (34–37).


BENEFITS AND HARMS

Overview of StudiesOf 1845 screened citations, 18 studies (reporting 20

medication reconciliation interventions) met the inclusion

Key Summary Points

Medication reconciliation is widely recommended to avoidunintentional discrepancies between patients’ medicationsacross transitions in care.

Clinically significant unintentional discrepancies affect onlya few patients.

Medication reconciliation alone probably does not reducepostdischarge hospital utilization within 30 days but maydo so when bundled with other interventions that improvedischarge coordination.

Pharmacists play a major role in most successfulinterventions.

Commonly used criteria for selecting high-risk patientsdo not consistently improve the effect of medicationreconciliation.

Supplement Medication Reconciliation During Transitions of Care



criteria (Figure 2 of the Supplement). All 18 were fromhospitals in the United States or Canada. Studies aboutmedication reconciliation from other countries met pre-specified exclusion criteria, such as not distinguishing in-tended from unintended medication discrepancies (38–40) or basing the assessment of clinical severity solely onjudgments by the personnel conducting medication recon-ciliation (41, 42).

Five studies (reporting 7 medication reconciliation in-terventions) used randomized, controlled designs (23–25,30, 31). All 5 were assessed as having low risk of bias. Onestudy used a quasi-experimental design (intervention deliv-ered in alternating months) (26), 3 had a before-and-afterdesign, and 9 reported postintervention data only (Appen-dix Table, available at www.annals.org). Seven interven-tions focused on “high-risk patients” based on advancedage, presence of chronic illnesses, or use of multiple med-ications (Appendix Table).

Seven studies compared medication reconciliationwith “usual care” (23, 26, 30, 31, 43–45), whereas 2 stud-ies (24, 25) compared 2 forms of medication reconcilia-tion. All but 2 of the studies (15, 44) were done in aca-demic medical centers, although 1 study involved bothteaching and nonteaching settings (43). Five of the inter-ventions targeted admission to a hospital (8, 11, 14, 16,46), 7 targeted discharge home (10, 23, 26, 29, 31, 43,45), 1 targeted in-hospital transfer (13), and 7 targetedmultiple care transitions (15, 24, 25, 30, 44).

Our 2 outcomes of interest—clinically significant un-intentional discrepancies and 30-day postdischarge hospitalutilization—corresponded to the primary outcome in 9 of18 included studies (15, 23–26, 29, 30, 43, 45). The pri-mary outcome for most of the remaining studies involvedvariations of our outcomes of interest, such as all uninten-tional discrepancies rather than the subset of clinically sig-nificant unintentional discrepancies (8, 14, 16, 46). Only 1study (44) reported a primary outcome substantially differ-ent from our outcomes of interest. This study evaluated thefeasibility of implementing an electronic system for tar-geted pharmacist- and nurse-conducted admission, but itincluded sufficient information to abstract data for ouroutcomes of interest.

BenefitsClinically Significant Unintended Medication Discrepancies

The number of clinically significant unintentional dis-crepancies per patient varied greatly across the 12 includedmedication reconciliation interventions (Table 1 of theSupplement). The median proportion of all unintendeddiscrepancies judged as having clinical significance was34% (interquartile range, 28% to 49%). The median pro-portion of patients with at least 1 clinically significant dis-crepancy was 45% (interquartile range, 31% to 56%).

Two of the interventions that reported clinically sig-nificant unintended discrepancies focused on “high-riskpatients” based on number of medications (8) and medical

complexity (14). One intervention identified 0.36 clini-cally significant discrepancies per patient (8), whereas theother reported a much higher value of 0.91 per patient (14).

Only 2 RCTs (30, 31) evaluated the effect of medica-tion reconciliation on clinically significant unintended dis-crepancies. One trial (31) randomly assigned 178 patientsbeing discharged from the medical service at a teachinghospital in Boston, Massachusetts, to an intervention thatincluded medication reconciliation and counseling by apharmacist, as well as a follow-up telephone call within 5days. For patients in the control group, nurses provideddischarge counseling and pharmacists reviewed medicationorders without performing a formal reconciliation process.Fewer patients in the intervention group experienced pre-ventable ADEs (1% vs. 11%; P � 0.01). Total ADEs didnot differ between the 2 groups.

A subsequent cluster randomized trial from the sameresearch group involved 14 medical teams at 2 teachinghospitals in Boston (30). The intervention included aWeb-based application using the hospital’s electronic med-ical record (which included ambulatory visits) to create apreadmission medication list to facilitate the medicationreconciliation process. This study reported a relative reduc-

Figure. Overview of medication reconciliation in acute care.

Best PossibleMedication History

(BPMH)

Best PossibleMedication

Discharge Plan(BPMDP)

Medications ordered duringadmission and internal transfer

Decision to discharge

Sources of Medication Information

Home

DischargeReconciliation

Health careFacility

AdmissionReconciliation

Home

BPMDP communicated to patientand next provider of care

Patient/Family

Interview

MedicationVials/List

GovernmentMedicationDatabase

ReconciledDischarge

Prescriptions

PhysicianDischargeSummary

PatientMedicationSchedule

PreviousPatientHealthRecords

Adapted, with permission, from Fernandes OA. Medication reconcilia-tion. Pharmacy Practice. 2009;25:26.

SupplementMedication Reconciliation During Transitions of Care



tion in potential ADEs (equal to clinically significant un-intended medication discrepancies) of 0.72 (95% CI, 0.52to 0.99). Of note, the intervention’s effect achieved statis-tical significance at only 1 of the 2 participating hospitals,with an adjusted relative risk for potential ADEs of 0.72(CI, 0.52 to 0.99), but not at the other (0.87 [CI, 0.57 to1.32]).

Emergency Department Visits and Readmission Within 30 Days

Nine interventions reported emergency departmentvisits and readmission within 30 days per patient (Table 2of the Supplement). Of these interventions, 5 applied se-lection criteria for high-risk patients (24, 26, 47, 48). Again,however, focusing on high-risk patients did not consis-tently increase the effect of medication reconciliation.

Across 3 RCTs, readmissions and emergency depart-ment visits were reduced by 23% (CI, 5% to 37%; I2 �24%) (Figure 3 of the Supplement). This pooled result wasdriven by the statistically significant reduction achieved by anintensive intervention (23) that included additional com-ponents beyond medication reconciliation that were specif-ically aimed at reducing readmissions.

One other RCT (47) met inclusion criteria but wasexcluded from meta-analysis because it reported hospitalutilization at 12 months rather than 30 days after dis-charge. This study showed that reconciliation led to a sig-nificant 16% reduction in all visits to the hospital. Theintervention consisted of a fairly intensive medication rec-onciliation strategy in which pharmacists identified drug-related problems beyond unintended discrepancies, coun-seled patients at admission and discharge, and telephonedpatients 2 months after discharge to ensure adequate homemanagement of medications.

HarmsMistakes in the medication reconciliation process may

become “hard-wired” into the patient record. Once medi-cation reconciliation has occurred, clinicians assessing agiven patient may rely exclusively on the documentedmedication history and be less likely to confirm its accuracywith the patient or other sources.

The larger concern with medication reconciliation per-tains to the reliance on pharmacists. Pharmacists haveproven roles in the prevention of ADEs (48–50); however,they are in short supply in most hospitals. Thus, involvingpharmacists in medication reconciliation, as most pub-lished studies have done, risks taking these personnel awayfrom other important activities related to patient safety.


Effect of Context on EffectivenessConceptually, 3 categories of contextual factors prob-

ably affect the impact of medication reconciliation: thedegree to which patients can directly provide up-to-datemedication histories, which reflects patients’ knowledge of

their medications, health literacy, and language; availabilityof medication data sources (for example, electronic medicalrecords in an ambulatory setting and regional prescriptiondatabases) to facilitate the medication reconciliation pro-cess; and possibly the clinical informatics milieu, includ-ing the degree to which medication reconciliation canbe integrated into such applications as computerizedphysician order entry and electronic medical records. Wehad hoped to explore the impacts of these factors on effec-tiveness, but the number of included studies and the stud-ies’ descriptions of context were insufficient to permit suchanalyses.

CostsMedication reconciliation has become mandatory for

hospital accreditation in the United States (19) and Can-ada (20). Thus, it has been implemented in hospitals ofvarying types and sizes and across a broad range of clinicalservices. However, most published studies evaluating theeffect of medication reconciliation come from academicsettings (Appendix Table). Moreover, in routine practice,medication reconciliation is probably done by physiciansand nurses, especially outside of academic centers. By con-trast, pharmacists played a major role in conductingmedication reconciliation in 17 of the 20 interventionsincluded in this review (Appendix Table). Nurses or phy-sicians delivered only 3 interventions (23, 25, 45) withoutsubstantial support from pharmacists, and one of these in-terventions used a nurse discharge advocate assigned todeliver the intervention (23).

A clinical informatics milieu (computerized physicianorder entry or electronic medical record) was noted for 13interventions, but electronic medication reconciliation oc-curred in only 9 interventions. The medication reconcilia-tion process generated new medication orders in only 3interventions (25, 44), 2 of which came from 1 study (25)(Table 3 of the Supplement).

One model-based study (51) considered the cost-effectiveness of 5 pharmacist-led strategies for reducingADEs. Pharmacist-led medication reconciliation carried areasonable probability of cost-effectiveness (compared withno reconciliation) at £10 000 ($16 240 as of 31 December2012) per quality-adjusted life-year. The authors estimatedthe cost for implementing pharmacist-led medicationreconciliation at £1897 ($3200) per 1000 prescription or-ders (51). A systematic review of economic analyses of pa-tient safety strategies (52) judged this study as having ac-ceptable quality features for economic analyses of patientsafety strategies. The main limitation identified was theuncertainty surrounding assumptions about expected re-ductions in ADEs as a result of reductions in potentialADEs.

DISCUSSION

Medication reconciliation addresses the conceptuallyplausible and well-documented problem of unintended




medication discrepancies introduced across transitions incare. This review suggests that only a few unintended dis-crepancies have clinical significance. Furthermore, mostpatients have no unintentional discrepancies. Therefore,the actual effect of medication reconciliation on reducingclinically significant discrepancies in the inpatient settingremains unclear.

Medication reconciliation has attracted interest be-cause of its potential effect on reducing postdischarge uti-lization. The pooled results of 3 RCTs showed that inter-ventions significantly reduced emergency department visitsand readmissions within 30 days of discharge. However,this finding was driven by the results of a single trial—arobust intervention that included several additional facetsaimed at improving the discharge process and coordinatingpostdischarge care (23). The degree to which medicationreconciliation contributed to the result is unclear.

The lack of effect of medication reconciliation aloneon hospital utilization within 30 days of discharge mayreflect the need to consider a longer window of observationto demonstrate benefit. The inadvertent discontinuation ofcholesterol-lowering medications, antiplatelet or anticoag-ulant agents, thyroid hormone replacement, antiresorptivetherapy for osteoporosis, and gastric acid suppressionagents—all commonly encountered examples of unin-tended discrepancies—carry risks for adverse clinical ef-fects that may require hospital utilization in the long termbut not usually within 30 days of discharge. It is thusnoteworthy that a trial of medication reconciliation alone(that is, with no additional discharge coordination inter-ventions) that used a longer postdischarge follow-up (12months) reported a significant reduction in emergency de-partment visits and readmissions (47).

Given limited resources, the paramount issue becomeshow to target medication reconciliation to direct resourcesmost efficiently. This is especially important given thatmost studies involve pharmacists to conduct medicationreconciliation, which requires substantial investment of re-sources beyond usual care. Our review suggests that com-mon selection criteria for high-risk patients showed noconsistent correlation with the prevalence of clinically sig-nificant unintentional discrepancies.

The absence of apparent effect from focusing on high-risk patients could reflect the limited number of studies.However, the high-risk criteria that are used also haveplausible limitations. For example, even though elderly pa-tients and patients with multiple chronic conditions mayreceive many medications, their medication regimens mayremain stable for some time or may be well-known to thepatients or their caregivers. These risk factors for unin-tended medication discrepancies do not account for suchnuances. A more direct risk factor is probably frequent orrecent changes to medication regimens. This risk factorunfortunately cannot be ascertained reliably without con-ducting a thorough medication history, not unlike thatrequired by the BPMH for medication reconciliation.

Our findings have some similarities with a previousreview of hospital-based medication reconciliation (21) inthat we found that most successful interventions reliedheavily on pharmacists and that, on the whole, medicationreconciliation remains a potentially promising interven-tion. The previous review found inconsistent reductions inpostdischarge health care utilization and indicated greatersuccess from targeting high-risk patients. These differencesmay reflect the methodological differences between ourstudies. We explicitly selected for studies that assessed theclinical significance of unintentional discrepancies, re-quired a clear distinction between intentional and uninten-tional medication changes through communication withthe medical team, and required that assessments of clinicalsignificance be performed by at least 1 clinician indepen-dent from the reconciliation process.

Our review has several limitations. Although we con-ducted a comprehensive literature search, we had no way ofidentifying unpublished research. One of our outcomes ofinterest, clinically significant unintentional discrepancies,was not always the primary outcome in included studies.In addition, this outcome is subjective and open to indi-vidual interpretation. Lastly, in most of the included stud-ies, the interventions were described with relatively littledetail and frequently omitted potentially important con-textual features (for example, patients’ understanding oftheir medications and the interprofessional culture at theinstitution).

Hospital-based medication reconciliation at care tran-sitions frequently identifies unintended discrepancies, butmany have no clinical significance. Pharmacists play im-portant roles in most published interventions. Most studieshave assessed patient outcomes during or shortly after hos-pitalization, but the benefits of resolving unintended dis-crepancies may not become apparent for months after dis-charge. Perhaps for this reason, medication reconciliationalone does not seem to reduce emergency department visitsor readmission within 30 days.

Bundling medication reconciliation with other inter-ventions aimed at improving care coordination at hospitaldischarge holds more promise, but the specific effect ofmedication reconciliation in such multifaceted interven-tions may not become apparent until much later than 30days after discharge. Future research should examine theeffect of medication reconciliation on postdischarge hospi-tal utilization at time points extending past the traditional30-day mark and identify patient features that more con-sistently increase the risk for clinically significant unin-tended discrepancies.

From the University of Toronto, Toronto, and Children’s Hospital ofEastern Ontario, Ottawa, Ontario, Canada.





Disclaimer: All statements expressed in this work are those of the authorsand should not be construed as official opinions or positions of theorganizations where any of the authors are employed, the Agency forHealthcare Research and Quality, or the U.S. Department of Health andHuman Services.

Financial Support: From the Agency for Healthcare Research and Qual-ity, U.S. Department of Health and Human Services (contract HHSA-290-2007-10062I).

Potential Conflicts of Interest: Dr. Shojania: Other: Agency for Health-care Research and Quality as a subcontract from the University of Cali-fornia, Los Angeles-RAND Evidence-Based Practice Centre. All otherauthors have no disclosures. Disclosures can also be viewed atwww.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2634.

Requests for Single Reprints: Kaveh G. Shojania, MD, Department ofMedicine, Sunnybrook Health Sciences Centre, Room H468, 2075Bayview Avenue, Toronto, Ontario M4N 3M5, Canada; e-mail,[email protected].


References1. Coleman EA, Berenson RA. Lost in transition: challenges and opportunitiesfor improving the quality of transitional care. Ann Intern Med. 2004;141:533-6.[PMID: 15466770]2. Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N, et al.Adverse events among medical patients after discharge from hospital. CMAJ.2004;170:345-9. [PMID: 14757670]3. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidenceand severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med. 2003;138:161-7. [PMID: 12558354]4. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW.Deficits in communication and information transfer between hospital-based andprimary care physicians: implications for patient safety and continuity of care.JAMA. 2007;297:831-41. [PMID: 17327525]5. van Walraven C, Taljaard M, Etchells E, Bell CM, Stiell IG, Zarnke K, et al.The independent association of provider and information continuity on out-comes after hospital discharge: implications for hospitalists. J Hosp Med. 2010;5:398-405. [PMID: 20845438]6. Bell CM, Brener SS, Gunraj N, Huo C, Bierman AS, Scales DC, et al.Association of ICU or hospital admission with unintentional discontinuation ofmedications for chronic diseases. JAMA. 2011;306:840-7. [PMID: 21862745]7. Coleman EA, Smith JD, Raha D, Min SJ. Posthospital medication discrep-ancies: prevalence and contributing factors. Arch Intern Med. 2005;165:1842-7.[PMID: 16157827]8. Cornish PL, Knowles SR, Marchesano R, Tam V, Shadowitz S, JuurlinkDN, et al. Unintended medication discrepancies at the time of hospital admis-sion. Arch Intern Med. 2005;165:424-9. [PMID: 15738372]9. Tam VC, Knowles SR, Cornish PL, Fine N, Marchesano R, Etchells EE.Frequency, type and clinical importance of medication history errors at admissionto hospital: a systematic review. CMAJ. 2005;173:510-5. [PMID: 16129874]10. Wong JD, Bajcar JM, Wong GG, Alibhai SM, Huh JH, Cesta A, et al.Medication reconciliation at hospital discharge: evaluating discrepancies. AnnPharmacother. 2008;42:1373-9. [PMID: 18780806]11. Coffey M, Mack L, Streitenberger K, Bishara T, De Faveri L, Matlow A.Prevalence and clinical significance of medication discrepancies at pediatric hos-pital admission. Acad Pediatr. 2009;9:360-365. [PMID: 19640822]12. Kwan Y, Fernandes OA, Nagge JJ, Wong GG, Huh JH, Hurn DA, et al.Pharmacist medication assessments in a surgical preadmission clinic. Arch InternMed. 2007;167:1034-40. [PMID: 17533206]13. Lee JY, Leblanc K, Fernandes OA, Huh JH, Wong GG, Hamandi B, et al.Medication reconciliation during internal hospital transfer and impact of com-

puterized prescriber order entry. Ann Pharmacother. 2010;44:1887-95. [PMID:21098753]14. Stone BL, Boehme S, Mundorff MB, Maloney CG, Srivastava R. Hospitaladmission medication reconciliation in medically complex children: an observa-tional study. Arch Dis Child. 2010;95:250-5. [PMID: 19948664]15. Vira T, Colquhoun M, Etchells E. Reconcilable differences: correcting med-ication errors at hospital admission and discharge. Qual Saf Health Care. 2006;15:122-6. [PMID: 16585113]16. Gleason KM, McDaniel MR, Feinglass J, Baker DW, Lindquist L, Liss D,et al. Results of the Medications at Transitions and Clinical Handoffs (MATCH)study: an analysis of medication reconciliation errors and risk factors at hospitaladmission. J Gen Intern Med. 2010;25:441-7. [PMID: 20180158]17. World Health Organization. Action on Patient Safety-High 5s. 2006. Ac-cessed at www.who.int/patientsafety/implementation/solutions/high5s/en/index.html on 13 September 2012.18. Institute for Healthcare Improvement. Overview of the 100,000 Lives Cam-paign. 2006. Accessed at www.ihi.org/offerings/Initiatives/PastStrategicInitiatives/5MillionLivesCampaign/Documents/Overview%20of%20the%20100K%20Campaign.pdf on 13 September 2012.19. The Joint Commission. 2011 National Patient Safety Goals. 2011. Accessedat www.jointcommission.org/standards_information/npsgs.aspx on 13 Septem-ber 2012.20. Accreditation Canada. Required Organizational Practices 2012. Ottawa, On-tario, Canada: Accreditation Canada; 2012. Accessed at www.accreditation.ca/uploadedFiles/ROP%20Handbook.pdf Accessed on 15 September 2012.21. Mueller SK, Sponsler KC, Kripalani S, Schnipper JL. Hospital-based med-ication reconciliation practices: a systematic review. Arch Intern Med. 2012;172:1057-69. [PMID: 22733210]22. Fernandes O. Medication reconciliation in the hospital: what, why, where,when, who and how? Healthc Q. 2012;15 Spec No:42-9. [PMID: 22874446]23. Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, JohnsonAE, et al. A reengineered hospital discharge program to decrease rehospitalization:a randomized trial. Ann Intern Med. 2009;150:178-87. [PMID: 19189907]24. Koehler BE, Richter KM, Youngblood L, Cohen BA, Prengler ID, ChengD, et al. Reduction of 30-day postdischarge hospital readmission or emergencydepartment (ED) visit rates in high-risk elderly medical patients through deliveryof a targeted care bundle. J Hosp Med. 2009;4:211-8. [PMID: 19388074]25. Kripalani S, Roumie CL, Dalal AK, Cawthon C, Businger A, Eden SK,et al; PILL-CVD (Pharmacist Intervention for Low Literacy in CardiovascularDisease) Study Group. Effect of a pharmacist intervention on clinically impor-tant medication errors after hospital discharge: a randomized trial. Ann InternMed. 2012;157:1-10. [PMID: 22751755]26. Walker PC, Bernstein SJ, Jones JN, Piersma J, Kim HW, Regal RE, et al.Impact of a pharmacist-facilitated hospital discharge program: a quasi-experimental study. Arch Intern Med. 2009;169:2003-10. [PMID: 19933963]27. Bayoumi I, Howard M, Holbrook AM, Schabort I. Interventions to im-prove medication reconciliation in primary care. Ann Pharmacother. 2009;43:1667-75. [PMID: 19737997]28. Schnipper JL, Liang CL, Hamann C, Karson AS, Palchuk MB, McCarthyPC, et al. Development of a tool within the electronic medical record to facilitatemedication reconciliation after hospital discharge. J Am Med Inform Assoc.2011;18:309-13. [PMID: 21486889]29. Pippins JR, Gandhi TK, Hamann C, Ndumele CD, Labonville SA,Diedrichsen EK, et al. Classifying and predicting errors of inpatient medicationreconciliation. J Gen Intern Med. 2008;23:1414-22. [PMID: 18563493]30. Schnipper JL, Hamann C, Ndumele CD, Liang CL, Carty MG, KarsonAS, et al. Effect of an electronic medication reconciliation application and processredesign on potential adverse drug events: a cluster-randomized trial. Arch InternMed. 2009;169:771-80. [PMID: 19398689]31. Schnipper JL, Kirwin JL, Cotugno MC, Wahlstrom SA, Brown BA, TarvinE, et al. Role of pharmacist counseling in preventing adverse drug events afterhospitalization. Arch Intern Med. 2006;166:565-71. [PMID: 16534045]32. Higgins JP, Altman DG, Gøtzsche PC, Juni P, Moher D, Oxman AD,et al; Cochrane Bias Methods Group. The Cochrane Collaboration’s tool forassessing risk of bias in randomised trials. BMJ. 2011;343:d5928. [PMID:22008217]33. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L,et al. Effectiveness and efficiency of guideline dissemination and implementationstrategies. Health Technol Assess. 2004;8:iii-iv, 1-72. [PMID: 14960256]




34. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD.Audit and feedback: effects on professional practice and health care outcomes.Cochrane Database Syst Rev. 2006:CD000259. [PMID: 16625533]35. Ranji SR, Steinman MA, Shojania KG, Gonzales R. Interventions to reduceunnecessary antibiotic prescribing: a systematic review and quantitative analysis.Med Care. 2008;46:847-62. [PMID: 18665065]36. Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J.Effect of point-of-care computer reminders on physician behaviour: a systematicreview. CMAJ. 2010;182:E216-25. [PMID: 20212028]37. Steinman MA, Ranji SR, Shojania KG, Gonzales R. Improving antibioticselection: a systematic review and quantitative analysis of quality improvementstrategies. Med Care. 2006;44:617-28. [PMID: 16799356]38. Midlov P, Bahrani L, Seyfali M, Hoglund P, Rickhag E, Eriksson T. Theeffect of medication reconciliation in elderly patients at hospital discharge.Int J Clin Pharm. 2012;34:113-9. [PMID: 22207271]39. Abdel-Qader DH, Harper L, Cantrill JA, Tully MP. Pharmacists’ interven-tions in prescribing errors at hospital discharge: an observational study in thecontext of an electronic prescribing system in a UK teaching hospital. Drug Saf.2010;33:1027-44. [PMID: 20925440]40. Steurbaut S, Leemans L, Leysen T, De Baere E, Cornu P, Mets T, et al.Medication history reconciliation by clinical pharmacists in elderly inpatientsadmitted from home or a nursing home. Ann Pharmacother. 2010;44:1596-603.[PMID: 20736427]41. Norris CM, Thomas V, Calvert PS. An audit to evaluate the acceptability ofa pharmacist electronically prescribing discharge medication and providing infor-mation to GPs. Pharmaceutical Journal. 2001;267:857-9.42. Climente-Martı M, Garcıa-Manon ER, Artero-Mora A, Jimenez-TorresNV. Potential risk of medication discrepancies and reconciliation errors at admis-sion and discharge from an inpatient medical service. Ann Pharmacother. 2010;44:1747-54. [PMID: 20923946]43. Dedhia P, Kravet S, Bulger J, Hinson T, Sridharan A, Kolodner K, et al. Aquality improvement intervention to facilitate the transition of older adults from

three hospitals back to their homes. J Am Geriatr Soc. 2009;57:1540-6. [PMID:19694865]44. Kramer JS, Hopkins PJ, Rosendale JC, Garrelts JC, Hale LS, Nester TM,et al. Implementation of an electronic system for medication reconciliation.Am J Health Syst Pharm. 2007;64:404-22. [PMID: 17299180]45. Showalter JW, Rafferty CM, Swallow NA, Dasilva KO, Chuang CH. Effectof standardized electronic discharge instructions on post-discharge hospital utili-zation. J Gen Intern Med. 2011;26:718-23. [PMID: 21499825]46. Gleason KM, Groszek JM, Sullivan C, Rooney D, Barnard C, Noskin GA.Reconciliation of discrepancies in medication histories and admission orders ofnewly hospitalized patients. Am J Health Syst Pharm. 2004;61:1689-95. [PMID:15540481]47. Gillespie U, Alassaad A, Henrohn D, Garmo H, Hammarlund-Udenaes M,Toss H, et al. A comprehensive pharmacist intervention to reduce morbidity inpatients 80 years or older: a randomized controlled trial. Arch Intern Med. 2009;169:894-900. [PMID: 19433702]48. Kaboli PJ, Hoth AB, McClimon BJ, Schnipper JL. Clinical pharmacists andinpatient medical care: a systematic review. Arch Intern Med. 2006;166:955-64.[PMID: 16682568]49. Kucukarslan SN, Peters M, Mlynarek M, Nafziger DA. Pharmacists onrounding teams reduce preventable adverse drug events in hospital general med-icine units. Arch Intern Med. 2003;163:2014-8. [PMID: 14504113]50. Leape LL, Cullen DJ, Clapp MD, Burdick E, Demonaco HJ, Erickson JI,et al. Pharmacist participation on physician rounds and adverse drug events in theintensive care unit. JAMA. 1999;282:267-70. [PMID: 10422996]51. Karnon J, Campbell F, Czoski-Murray C. Model-based cost-effectivenessanalysis of interventions aimed at preventing medication error at hospital admis-sion (medicines reconciliation). J Eval Clin Pract. 2009;15:299-306. [PMID:19335488]52. Etchells E, Koo M, Daneman N, McDonald A, Baker M, Matlow A, et al.Comparative economic analyses of patient safety improvement strategies in acutecare: a systematic review. BMJ Qual Saf. 2012;21:448-56. [PMID: 22523319]




Current Author Addresses: Dr. Kwan: Department of Medicine,Mount Sinai Hospital, Room 427, 600 University Avenue, Toronto,Ontario M5G 1X5, Canada.Ms. Lo: University of Toronto Centre for Patient Safety, 525 UniversityAvenue, Room 630, Toronto, Ontario M5G 2L3, Canada.Dr. Sampson: Children’s Hospital of Eastern Ontario, 401 Smyth Road,Ottawa, Ontario K1H 8L1, Canada.Dr. Shojania: Department of Medicine, Sunnybrook Health SciencesCentre, Room H468, 2075 Bayview Avenue, Toronto, Ontario M4N3M5, Canada.

Author Contributions: Conception and design: J.L. Kwan, M. Samp-son, K.G. Shojania.Analysis and interpretation of the data: J.L. Kwan, L. Lo, M. Sampson,K.G. Shojania.Drafting of the article: J.L. Kwan, L. Lo, M. Sampson, K.G. Shojania.Critical revision of the article for important intellectual content: J.L.Kwan, L. Lo, K.G. Shojania.Final approval of the article: J.L. Kwan, M. Sampson, K.G. Shojania.Statistical expertise: L. Lo, K.G. Shojania.Obtaining of funding: K.G. Shojania.Administrative, technical, or logistic support: L. Lo, K.G. Shojania.Collection and assembly of data: J.L. Kwan, L. Lo, M. Sampson, K.G.Shojania.




App

endi

xT

able

.St

udie

sof

Med

icat

ion

Rec

onci

liati

on,

Incl

udin

gA

sses

smen

tof

Clin

ical

lySi

gnif

ican

tU

nint

ende

dD

iscr

epan

cies

and

Emer

genc

yD

epar

tmen

tV

isit

san

dH

ospi

taliz

atio

nsW

ithi

n30

Day

sof

Dis

char

ge

Stud

y,Y

ear

(Ref

eren

ce)

Sett

ing

Stud

yD

esig

n(S

ampl

eSi

ze,

n)Se

lect

ion

for

Hig

h-R

isk

Pati

ents

Tran

siti

onof

Car

eTa

rget

edPe

rson

Perf

orm

ing

Med

icat

ion

Rec

onci

liati

onA

ddit

iona

lIn

terv

enti

ons

Bey

ond

Med

icat

ion

Rec

onci

liati

on

Out

com

e

Cof

fey

etal

,20

09(1

1)Pe

diat

ricw

ard

inac

adem

icm

edic

alce

nter

inC

anad

a

Pros

pect

ive

post

inte

rven

tion

stud

y(2

72)

Non

eA

dmis

sion

toho

spita

lPh

arm

acy

stud

ent

Non

eC

linic

ally

sign

ifica

ntun

inte

ntio

nal

disc

repa

ncie

s

Cor

nish

etal

,20

05(8

)M

edic

alw

ard

inac

adem

icm

edic

alce

nter

inC

anad

a

Pros

pect

ive

post

inte

rven

tion

stud

y(1

51)

�4

med

icat

ions

Adm

issi

onto

hosp

ital

Phar

mac

ist,

phar

mac

yst

uden

t,or

med

ical

stud

ent

Non

eC

linic

ally

sign

ifica

ntun

inte

ntio

nal

disc

repa

ncie

s

Ded

hia

etal

,20

09(4

3)M

edic

alw

ards

inU

.S.

acad

emic

med

ical

cent

er,

com

mun

ityte

achi

ngho

spita

l,an

dur

ban

com

mun

ityho

spita

l

Pros

pect

ive

befo

re-a

nd-a

fter

stud

y(1

85)

Age

�65

yD

isch

arge

hom

ePh

ysic

ian

follo

wed

byph

arm

acis

tSa

feST

EPS

inte

rven

tion,

incl

udin

gad

mis

sion

asse

ssm

ent,

com

mun

icat

ion

with

PCP,

and

mul

tidis

cipl

inar

ydi

scha

rge

mee

ting

Emer

genc

yde

part

men

tvi

sits

and

hosp

italiz

atio

nsw

ithin

30d

ofdi

scha

rge

Gle

ason

etal

,20

04(4

6)Su

rgic

alan

dm

edic

alw

ards

inU

.S.

acad

emic

med

ical

cent

er

Post

inte

rven

tion

stud

y(2

04)*

Non

eA

dmis

sion

toho

spita

lPh

arm

acis

tN

one

Clin

ical

lysi

gnifi

cant

unin

tent

iona

ldi

scre

panc

ies

Gle

ason

etal

,20

10(1

6)M

edic

alw

ard

inU

.S.

acad

emic

med

ical

cent

er

Pros

pect

ive

post

inte

rven

tion

stud

y(6

51)

Non

eA

dmis

sion

toho

spita

lPh

arm

acis

tN

one

Clin

ical

lysi

gnifi

cant

unin

tent

iona

ldi

scre

panc

ies

Jack

etal

,20

09(2

3)M

edic

alw

ard

inU

.S.

acad

emic

med

ical

cent

er

RC

T(3

73)

Non

eD

isch

arge

hom

eN

urse

disc

harg

ead

voca

teN

urse

disc

harg

ead

voca

tes

crea

ted

apo

stho

spita

lizat

ion

care

plan

and

post

disc

harg

ete

leph

one

call

Emer

genc

yde

part

men

tvi

sits

and

hosp

italiz

atio

nsw

ithin

30d

ofdi

scha

rge

Koe

hler

etal

,20

09(2

4)M

edic

alw

ard

inU

.S.

acad

emic

med

ical

cent

er

RC

T(2

1)†

Age

�70

y,�

5m

edic

atio

ns,

�3

chro

nic

com

orbi

dco

nditi

ons,

requ

irem

ent

for

assi

stan

cew

ith�

1A

DL

Adm

issi

onto

hosp

ital,

disc

harg

eho

me

Nur

sean

dre

view

edby

phar

mac

ist

Cou

nsel

ing

byre

gist

ered

nurs

eEm

erge

ncy

depa

rtm

ent

visi

tsan

dho

spita

lizat

ions

with

in30

dof

disc

harg

e

Koe

hler

etal

,20

09(2

4)M

edic

alw

ard

inU

.S.

acad

emic

med

ical

cent

er

RC

T(2

0)†

Age

�70

y,�

5m

edic

atio

ns,

�3

chro

nic

com

orbi

dco

nditi

ons,

requ

irem

ent

for

assi

stan

cew

ith�

1A

DL

Adm

issi

onto

hosp

ital,

disc

harg

eho

me

Phar

mac

ist

Supp

lem

enta

leld

erly

care

bund

le(c

ouns

elin

gby

phar

mac

ist,

post

disc

harg

ete

leph

one

call,

and

disc

harg

ele

tter

toPC

P)

Emer

genc

yde

part

men

tvi

sits

and

hosp

italiz

atio

nsw

ithin

30d

ofdi

scha

rge

Kra

mer

etal

,20

07(4

4)M

edic

alw

ard

inU

.S.

com

mun

ityte

achi

ngho

spita

l

Pros

pect

ive

befo

re-a

nd-a

fter

stud

y(1

36)

�1

ofth

efo

llow

ing:

�7

med

icat

ions

,su

bsta

ntia

lcom

orbi

dco

nditi

on,

prev

ious

adm

issi

onfo

rA

DR

,�

4dr

ugal

lerg

ies

Adm

issi

onto

hosp

ital,

disc

harg

eho

me

Phar

mac

ist

and

phys

icia

nN

one

Emer

genc

yde

part

men

tvi

sits

and

hosp

italiz

atio

nsw

ithin

30d

ofdi

scha

rge

Krip

alan

iet

al,

2012

(25)

Med

ical

and

card

iolo

gyw

ards

in2

U.S

.ac

adem

icm

edic

alce

nter

s

RC

T(4

28)

Non

eA

ttim

eof

enro

llmen

tin

stud

y,di

scha

rge

hom

e,an

din

-hos

pita

ltra

nsfe

r

Phys

icia

nan

dnu

rse

Dis

char

geco

unse

ling

Clin

ical

lysi

gnifi

cant

unin

tent

iona

ldi

scre

panc

ies

Krip

alan

iet

al,

2012

(25)

Med

ical

and

card

iolo

gyw

ards

in2

U.S

.ac

adem

icm

edic

alce

nter

s

RC

T(4

23)

Non

eA

ttim

eof

enro

llmen

tin

stud

y,di

scha

rge

hom

e,an

din

-hos

pita

ltra

nsfe

r

Phar

mac

ist

Phar

mac

ist

inte

rven

tion,

incl

udin

gin

patie

ntph

arm

acis

tco

unse

ling,

low

-lite

racy

adhe

renc

eai

ds,

and

post

disc

harg

ete

leph

one

call

Clin

ical

lysi

gnifi

cant

unin

tent

iona

ldi

scre

panc

ies

Lee

etal

,20

10(1

3)In

patie

ntw

ards

and

criti

cal

care

units

in2

acad

emic

med

ical

cent

ers

inC

anad

a

Pros

pect

ive

post

inte

rven

tion

stud

y(1

29)‡

Non

eIn

-hos

pita

ltra

nsfe

rPh

arm

acis

tN

one

Clin

ical

lysi

gnifi

cant

unin

tent

iona

ldi

scre

panc

ies

Pipp

ins

etal

,20

08(2

9)M

edic

alw

ards

in2

U.S

.ac

adem

icm

edic

alce

nter

s

Pros

pect

ive

post

inte

rven

tion

stud

y(1

80)§

Non

eD

isch

arge

hom

ePh

arm

acis

tN

one

Clin

ical

lysi

gnifi

cant

unin

tent

iona

ldi

scre

panc

ies

Schn

ippe

ret

al,

2006

(31)

Med

ical

war

din

U.S

.ac

adem

icm

edic

alce

nter

RC

T(9

2)�

Non

eD

isch

arge

hom

ePh

arm

acis

tN

one

Emer

genc

yde

part

men

tvi

sits

and

hosp

italiz

atio

nsw

ithin

30d

ofdi

scha

rge

Schn

ippe

ret

al,

2009

(30)

Med

ical

war

dsin

2U

.S.

acad

emic

med

ical

cent

ers

RC

T(1

62)§

Non

eA

dmis

sion

toho

spita

l,di

scha

rge

hom

e

Phys

icia

nw

ithco

nfirm

atio

nby

phar

mac

ist

ornu

rse

(enh

ance

dby

prea

dmis

sion

med

icat

ion

list

build

erin

elec

tron

icm

edic

alre

cord

)

Non

eC

linic

ally

sign

ifica

ntun

inte

ntio

nal

disc

repa

ncie

sEm

erge

ncy

depa

rtm

ent

visi

tsan

dho

spita

lizat

ions

with

in30

dof

disc

harg

e

Con

tinue

don

follo

win

gpa

ge



App

endi

xT

able

.—C

onti

nued

Stud

y,Y

ear

(Ref

eren

ce)

Sett

ing

Stud

yD

esig

n(S

ampl

eSi

ze,

n)Se

lect

ion

for

Hig

h-R

isk

Pati

ents

Tran

siti

onof

Car

eTa

rget

edPe

rson

Perf

orm

ing

Med

icat

ion

Rec

onci

liati

onA

ddit

iona

lIn

terv

enti

ons

Bey

ond

Med

icat

ion

Rec

onci

liati

on

Out

com

e

Show

alte

ret

al,

2011

(45)

All

adm

itted

patie

nts

thro

ugh

emer

genc

yde

part

men

tin

U.S

.ac

adem

icm

edic

alce

nter

Ret

rosp

ectiv

ebe

fore

-and

-aft

erst

udy

(17

516)

Non

eD

isch

arge

hom

ePh

ysic

ian

Stan

dard

ized

man

dato

ryel

ectr

onic

disc

harg

ein

stru

ctio

nsdo

cum

ent

with

embe

dded

com

pute

rized

med

icat

ion

reco

ncili

atio

n

Emer

genc

yde

part

men

tvi

sits

and

hosp

italiz

atio

nsw

ithin

30d

ofdi

scha

rge

Ston

eet

al,

2010

(14)

Pedi

atric

war

din

U.S

.ac

adem

icm

edic

alce

nter

Pros

pect

ive

post

inte

rven

tion

stud

y(2

3)¶

Iden

tific

atio

nof

med

ical

lyco

mpl

exco

nditi

ons

base

don

publ

ishe

dgu

idel

ines

Adm

issi

onto

hosp

ital

Phar

mac

ist

Non

eC

linic

ally

sign

ifica

ntun

inte

ntio

nal

disc

repa

ncie

s

Vira

etal

,20

06(1

5)A

cute

care

units

inur

ban

com

mun

ityho

spita

lin

Can

ada

Ret

rosp

ectiv

epo

stin

terv

entio

nst

udy

(60)

Non

eA

dmis

sion

toho

spita

l,di

scha

rge

hom

e

Phar

mac

ist

Non

eC

linic

ally

sign

ifica

ntun

inte

ntio

nal

disc

repa

ncie

s

Wal

ker

etal

,20

09(2

6)M

edic

alw

ard

inU

.S.

acad

emic

cent

erPr

ospe

ctiv

equ

asi-

expe

rimen

tal

stud

y(3

58)*

*�

1of

the

follo

win

g:�

5m

edic

atio

ns,

�1

targ

eted

med

icat

ions

††,

med

icat

ion

requ

iring

mon

itorin

g,�

2ch

ange

sto

regi

men

,de

men

tiaor

conf

usio

n,or

inab

ility

tom

anag

em

edic

atio

ns

Dis

char

geho

me

Phar

mac

ist

Phar

mac

ist-

faci

litat

eddi

scha

rge

prog

ram

,in

clud

ing

coun

selin

g,pr

ovis

ion

ofm

edic

atio

nre

conc

iliat

ion

list

toPC

P,an

dpo

stdi

scha

rge

tele

phon

eca

ll

Emer

genc

yde

part

men

tvi

sits

and

hosp

italiz

atio

nsw

ithin

30d

ofdi

scha

rge

Won

get

al,

2008

(10)

Med

ical

war

din

acad

emic

med

ical

cent

erin

Can

ada

Pros

pect

ive

post

inte

rven

tion

stud

y(1

50)

Non

eD

isch

arge

hom

ePh

arm

acis

tor

phar

mac

yre

side

ntN

one

Clin

ical

lysi

gnifi

cant

unin

tent

iona

ldi

scre

panc

ies

AD

L�

acti

vity

ofda

ilyliv

ing;

AD

R�

adve

rse

drug

reac

tion

;PC

P�

prim

ary

care

phys

icia

n;R

CT

�ra

ndom

ized

,co

ntro

lled

tria

l;Sa

feST

EPS

�Sa

fean

dSu

cces

sful

Tra

nsit

ion

ofE

lder

lyPa

tien

tsSt

udy.

*12

adul

tm

edic

al–s

urgi

cal

unit

s.†

2ho

spit

alm

edic

ine

grou

ps.

‡10

pati

ent

care

unit

s.§

7m

edic

alte

ams.

�4

med

ical

team

s.¶

On

2m

edic

alte

ams.

**2

med

ical

team

san

d1

hosp

ital

ist

serv

ice.

††T

arge

ted

med

icat

ions

incl

uded

digo

xin,

diur

etic

s,an

tico

agul

ants

,sed

ativ

es,o

pioi

ds,a

sthm

aor

chro

nic

obst

ruct

ive

pulm

onar

ydi

seas

em

edic

atio

ns,a

ndan

giot

ensi

n-co

nver

ting

enzy

me

inhi

bito

rsor

angi

oten

sin-

rece

ptor

bloc

kers

.



Nurse–Patient Ratios as a Patient Safety StrategyA Systematic ReviewPaul G. Shekelle, MD, PhD

A small percentage of patients die during hospitalization or shortlythereafter, and it is widely believed that more or better nursing carecould prevent some of these deaths. The author systematicallyreviewed the evidence about nurse staffing ratios and in-hospitaldeath through September 2012. From 550 titles, 87 articles werereviewed and 15 new studies that augmented the 2 existing re-views were selected. The strongest evidence supporting a causalrelationship between higher nurse staffing levels and decreasedinpatient mortality comes from a longitudinal study in a singlehospital that carefully accounted for nurse staffing and patient

comorbid conditions and a meta-analysis that found a “dose–response relationship” in observational studies of nurse staffing anddeath. No studies reported any serious harms associated with anincrease in nurse staffing. Limiting any stronger conclusions is thelack of an evaluation of an intervention to increase nurse staffingratios. The formal costs of increasing the nurse–patient ratio cannotbe calculated because there has been no evaluation of an inten-tional change in nurse staffing to improve patient outcomes.

Ann Intern Med. 2013;158:404-409. www.annals.orgFor author affiliation, see end of text.

THE PROBLEM

A small percentage of hospitalized patients die duringor shortly after hospitalization. Evidence suggests thatsome proportion of these deaths could probably be pre-vented with more nursing care. For example, in 1 earlystudy of 232 342 surgical discharges from several Pennsyl-vania hospitals, 4535 patients (2%) died within 30 days ofhospitalization; the investigators estimated that the differ-ence between 4:1 and 8:1 patient–nurse ratios may be ap-proximately 1000 deaths in a group of this size (1). Otherstudies have produced roughly similar estimates, namelyapproximately 1 to 5 fewer deaths per 1000 inpatient dayswith more nurse staffing per patient (2–4). The rationalefor suggesting that increasing the ratio of registered nurses(RNs) to patients will lead to decreased illness or mortalityrates rests on the belief that improved attention to patientsis the critical factor. This systematic review examined theevidence on the effects of interventions aimed at increasingnurse–patient ratios on patient illness and death.


There has been no evaluation of an intentional changein RN staffing to improve patient outcomes; therefore, thepatient safety strategy referred to in this article remainssomewhat unclear. Most studies have been cross-sectionalor longitudinal assessments of differences in nursing staffvariables, with the most commonly assessed measure beingthe proportion of RN time per some measure of inpatientload and the most commonly assessed outcome being mor-tality. However, many other factors have been proposed asbeing causal with respect to the relationship between nurs-

ing care and reductions in hospital mortality, potentially inaddition to or instead of a simple nurse–patient ratio.These factors include measures of nursing burnout, jobsatisfaction, teamwork, nurse turnover, nursing leadershipin hospitals, and nurse practice environment.

Several research groups have proposed conceptualframeworks to explain why more effective nursing caremay reduce inpatient mortality (5–8). Underlying all ofthese conceptual frameworks is the belief that surveillanceis a critical factor that can be improved with more staff,better-educated staff, or a better working environment (9).A representative framework by Aiken and colleagues (8)posits that nurse–patient ratios, along with staffing skillmix, can lead to better surveillance, which, along withmany other factors, can influence the process of care andlead to better patient outcomes (Figure 1).

REVIEW PROCESSES

Two existing reviews relevant to the topic were iden-tified, by using methods described by Whitlock and col-leagues (10). These reviews were supplemented by search-ing the Web of Science for articles published from 2009(the end date of the search from the most recent review) toSeptember 2012 that cited any of 4 key articles in thisfield, including the older of the 2 reviews, and was limitedto studies published in English. For a complete descriptionof the search strategies, literature flow diagram, and evi-dence tables, see the Supplement (available at www.annals.org). The update search identified 546 titles, and 4 articlescame from reference mining. Titles and abstracts were re-viewed and selected if they reported empirical data on therelationship between nurse staffing ratios and mortality ornursing-sensitive outcomes, such as pressure ulcers and fail-ure to rescue. Because several cross-sectional studies haveassessed this relationship, only 1 additional cross-sectionalstudy was included for detailed review. The exception wasa cross-sectional study that evaluated a quasi-intervention(11). Nine longitudinal studies were identified (12–20).

See also:





Four simulation studies reported on costs, and 1 systematicreview article was included (21–25). Two frameworkswere also included (6, 7). No experimental studies wereidentified.

The assessment of multiple systematic reviews(AMSTAR) criteria was used to assess the quality of thesystematic reviews (26). Only criteria relevant to a partic-ular review were applied; for example, 2 of the 11AMSTAR criteria are only applicable to reviews that in-volve meta-analysis. In addition, the AMSTAR criteria re-quiring a list of all excluded studies were not applied. Newstudies were not formally assessed for study quality, buttheir strengths and limitations are discussed later.

This review was supported by the Agency for Health-care Research and Quality, which had no role in the selec-tion or review of the evidence or the decision to submitthis manuscript for publication.

BENEFITS AND HARMSBenefits

Two recent relevant systematic reviews on this topic, ameta-analysis (27) and a narrative review (28), respectively

scored 10 out of 10 relevant criteria and 7 out of 9 relevantcriteria according to AMSTAR.

The meta-analysis included 28 studies, of which 17were cohort studies, 7 were cross-sectional studies, and 4were case–control studies (no experimental studies wereidentified). Most were U.S. studies, and the average level ofstaffing was 3.0 patients per RN for the intensive care unit(ICU) setting, 4.0 patients per RN in the surgical setting,and 4.4 patients per RN for the medical setting. It found aconsistent relationship between higher RN staffing andlower hospital-related mortality: An increase of 1 RN full-time equivalent (FTE) per patient day was related to a 9%reduction in the odds of death in the ICU, a 16% reduc-tion in the surgical setting, and a 6% reduction in themedical setting. With respect to other outcomes, lowerrates of hospital-acquired pneumonia, pulmonary failure,unplanned extubation, failure to rescue, and nosocomialbloodstream infections were related to higher RN staffingin pooled analyses of several studies. However, severalother outcomes that were presumed to have strong sensi-tivity to nurse staffing levels did not show consistent rela-tionships, including falls, pressure ulcers, and urinary tractinfections.

The authors also conducted an indirect analysis of thepotential for a dose–response relationship. This analysisassessed the effect across studies of additional RNs pershift. In each case, comparisons of quartiles of nurse staff-ing levels showed the expected relationship (Figure 2). Inother words, if the relationship between nurse staffing andmortality is causal, the difference in the risk for deathshould be greater between the first and third quartiles ofnurse staffing than it is between the first and second quar-tiles because the difference in staffing between the first andthird quartiles is greater than that between the first andsecond quartiles.

The authors of the meta-analysis concluded that aconsistent relationship has been shown but identified sev-eral limitations in the literature with respect to establishing

Key Summary Points

Cross-sectional studies, mostly in intensive care unit orpostsurgical settings, support a relationship betweenthe number of nurses staffed per patient and inpatientmortality.

The strongest evidence supporting a causal relationshipbetween higher nurse staffing levels and decreased inpa-tient mortality comes from a longitudinal study in a singlehospital that carefully accounted for nurse staffing levelsand found decreases in mortality of 2% to 7%.

Limiting any stronger conclusions is the lack of an evalua-tion of an intervention to increase nurse staffing ratios.

Figure 1. Hospital organization, nursing organization, and patient outcomes.

Hospital organization

Process of care

Medical staff qualifications

Nurseoutcomes

Patientoutcomes

Organizational support for nursing care

Resource adequacyNurse autonomyNurse controlNurse–physician relations

Nurse–patient ratios/staffing skill mix

Surveillance/earlydetection of complications

From reference 8, with permission.

SupplementNurse–Patient Ratios as a Patient Safety Strategy



that this relationship is causal. The authors ultimately con-cluded that the arguments for a causal relationship are“mixed,” and they called for future research to address therole of nurse staffing and competence on the effectivenessof patient care, “taking greater cognizance of other relevantfactors such as patient and hospital characteristics andquality of medical care” (27).

The narrative review identified literature publishedthrough 2009 and was restricted to studies that usedhospital-related mortality as the outcome; the authorsidentified 17 studies (10 of which were not included in thefirst review and 7 that were published since 2007) (28).Although this review was narrative, the 2 reviews hadbroadly similar results: 14 of 17 studies found a statisticallysignificant relationship between nurse staffing variables andlower mortality rates. In addition, the narrative reviewidentified mixed findings for mortality among 5 studiesassessing the characteristics of the nurse work environmentand work relationships, 3 studies assessing nurses’ re-sponses to work and the work environment (for example,burnout), and 7 studies assessing nurses’ educational prep-aration and experience. Only 1 study, which had a cross-sectional design, assessed nursing process-of-care variables;it found a relationship between the use of care maps andlower hospital-associated mortality, with an estimated ef-fect size of 10 fewer deaths per 1000 acute medicine dis-

charged patients. Like the meta-analysis, the narrative re-view concluded that a strong relationship exists but moreresearch is needed to understand the reasons why this re-lationship between higher nurse staffing and lower hospitalmortality may be causal (that is, they called for a theoret-ical model that explains the relationship in ways that canbe tested and refined).

Thus, these 2 reviews came to broadly similar conclu-sions. Mostly cross-sectional studies consistently reportthat higher RN staffing is related to lower hospital-relatedmortality.

However, many factors can confound the observed re-lationship. In cross-sectional studies, hospitals that are“better” in other ways may also be better staffed with moreRNs. For example, 1 published study of electronic healthrecord implementation showed that hospitals with elec-tronic health records have higher nurse staffing ratios andlower patient mortality (29). If the cross-sectional relation-ship is confounded, then critics worry that adoption offixed nurse–patient ratios will not necessarily lead to betterhealth outcomes, that such a policy is “an inflexible solu-tion that is unlikely to lead to optimal use of resources” (30).

The results of the updated search are as follows. Ninelongitudinal studies and one new systematic review (12–20, 25) were identified. The systematic review includedstudies that assessed nurse staffing ratios and outcomes re-stricted to adult ICU settings (25) and reached conclusionssimilar to the previous reviews: a consistent relationshipbetween increased nurse staffing and better patient out-comes in observational studies, evidence that falls short ofcausality. One longitudinal study narratively reported thatincreased nurse staffing was related to “significantly (P �0.01) decreased rates of decubiti, pneumonia, and sepsis,”but data were not presented (20). The cross-sectional studyaddresses the effect of an “intervention” to change nursestaffing ratios, implemented in response to a 2004 Califor-nia law requiring minimum nurse–patient ratios in acutecare hospitals (11). This legislation mandated patient–nurse staffing levels of 5:1, 4:1, and 2:1 for medical orsurgical units, pediatric units, and ICUs, respectively. TheCalifornia legislative mandate does not require nurse staff-ing to be met with RNs (that is, licensed vocational [prac-tical] nurses can also meet the mandate).

Aiken and colleagues (11) assessed the relationship be-tween nurse staffing and mortality in 2006, 2 years afterthe California mandate, comparing data from Californiawith those of 2 states without mandates, New Jersey andPennsylvania. Data about workloads were drawn from asurvey of RNs in the 3 states (22 336 nurses in total); theresponse rate was 35.4%. Hospital data came from theAmerican Hospital Association, and patient and outcomedata came from state hospital discharge databases.

The authors reported that their survey data showedsubstantial adherence to the California mandate, with 88%of medical or surgical nurses, 85% of pediatric nurses, and85% of ICU nurses reporting that the staffing of their last

Figure 2. Pooled odds ratio of quartiles of nurse staffing levels.

Quartiles of Patients/RN per Shift

All patients

1 vs. 2

1 vs. 3

1 vs. 4

2 vs. 3

2 vs. 4

3 vs. 4

Intensive care units

2 vs. 3

Medical patients

1 vs. 2

Surgical patients

1 vs. 3

1 vs. 4

2 vs. 3

2 vs. 4

3 vs. 4

Odds Ratioof Death*(95% CI)

Odds Ratio of Death*

0.94 (0.92–0.95)

0.76 (0.71–0.81)

0.62 (0.59–0.66)

0.81 (0.76–0.87)

0.66 (0.63–0.70)

0.82 (0.76–0.88)

0.94 (0.92–0.97)

0.94 (0.92–0.95)

0.76 (0.70–0.82)

0.62 (0.58–0.66)

0.80 (0.74–0.87)

0.65 (0.61–0.70)

0.81 (0.75–0.88)

0.5 1

Odds ratios are based on pooled analysis consistent across the studies(heterogeneity not significant). From reference 27, with permission.RN � registered nurse.

Supplement Nurse–Patient Ratios as a Patient Safety Strategy



shift was within the mandated ratio. In logistic regressionanalyses adjusted for many patient characteristics and 3hospital characteristics (such as bed size, teaching status,and technology use), Aiken and colleagues found statisti-cally significant relationships between the estimation of theaverage number of patients per nurse and 2 outcomes: 30-day mortality and failure to rescue (11).

Although the study collected data after implementa-tion of the California staffing mandate, it did not test theeffect of that mandate per se because it had no comparisondata from the period before the mandate went into effect.The possibility that the relationship is causal is blunted bylongitudinal studies that examined measures from beforeand after the California mandate, which showed the ex-pected changes in nurse staffing and proportion of licensedstaff per patient but no improvement in other patient out-comes believed to be nursing-sensitive (such as falls, pres-sure ulcers, and failure to rescue) (16, 17, 19). In fact, anunexpected statistically significant increase in pressure ul-cers was related to a greater number of hours of care for thepatient (which may have been because of greater detec-tion). These studies did not assess mortality.

Five additional longitudinal studies add further infor-mation to this picture. The first is a longitudinal assess-ment of nurse staffing and hospital mortality and failure torescue in 283 California hospitals between 1996 and 2001,which had access to direct measures of nurse staffing (14).In multivariable models that included many hospital mar-ket characteristics as well as risk adjustment using theMedstat Disease Staging methodology to produce a pre-dicted probability for complications or death, the authorsfound that an increase of 1 RN FTE per 1000 inpatientdays was related to a statistically significant decrease inmortality of 4.3%.

The second longitudinal study assessed care at 39Michigan hospitals between 2003 and 2006; it includedadults admitted through the emergency department withacute myocardial infarction, heart failure, stroke, pneumo-nia, hip fracture, or gastrointestinal bleeding (15). Thisstudy simultaneously controlled for 4 factors—high hospi-tal occupancy on hospitalization, weekend hospitalization,seasonal influenza, and nurse staffing levels—each of whichhad a statistically significant effect on in-hospital mortality.Each additional RN FTE per patient day was related to a0.25% decrease in mortality.

The third longitudinal study assessed the effect of amandate in 3 Western Australia public hospitals to imple-ment a new staffing method, the Nursing Hours per Pa-tient Day (12). The study assessed 3 periods: 20 monthsbefore implementation, 7 months of a “transition period,”and 2 months after implementation. The authors foundthat the total nursing hours and RN hours increased dur-ing the observation period. However, the percentage oftotal nursing hours provided by RNs decreased (from 87%to 84%). Also, the article stated that “although the nursinghours increased for all three hospitals (in the post-

implementation period), the changes were not statisticallysignificant” (12). Mortality rates were reduced duringthis period. Among many other outcomes, some improved,others did not, and some changes were inconsistentacross hospitals. Although the study was described as aninterrupted time series, it was analyzed as a before–afterstudy.

The fourth longitudinal study assessed changes innurse staffing over 9 years in 124 Florida hospitals andrelated these to changes in Agency for Healthcare Researchand Quality Patient Safety Indicators (18). The study usedboth initial staffing ratios and changes in staffing ratios.Results were mixed but generally favored better patientsafety outcomes with higher RN staffing levels.

The methodologically strongest longitudinal study isthat of Needleman and colleagues (13). The researchersused data over time from a single hospital to assess therelationship between natural differences in levels of RNstaffing in the same hospital and inpatient mortality. Thestudy is further characterized by a careful matching ofnurse staffing on a shift-by-shift basis with the actual pa-tients cared for during that shift. Knowing the actual pa-tients cared for allowed for more sophisticated adjustmentsof risk for death at the patient level. The study was done ata tertiary academic hospital between 2003 and 2006 andincluded 197 691 hospitalizations and 176 696 nursingshifts across 43 hospital units. The patients themselves av-eraged 60 years of age, and approximately 50% were cov-ered under Medicare. The variable of interest was exposureof the patient to nursing care that was below the targetlevel (for that type of unit) for that shift (that is, the pro-portion of shifts below target level staffing on a per-patientbasis). An additional exposure variable was a “high-turnover” shift (that is, a shift with many hospitalizations,discharges, or transfers). The authors found that exposureto each shift of below-target staffing or high turnover wasrelated to a 2% to 7% increase in mortality, with higherlevels of risk if the high-turnover or below-target shift oc-curred in the first 5 days after hospitalization. For patientswho were not in an ICU, this risk was increased by 12%and 15% during below-target and high-turnover shifts,respectively.

The data from Needleman and colleagues contributeto the “causality” determination because the study is lon-gitudinal in 1 hospital, thus controlling for the “hospitaleffect” potentially present in all cross-sectional studies, andhas detailed measures of exposure and confounding vari-ables. These results and the dose–response analysis fromthe meta-analysis provide the strongest evidence in supportof causality.

HarmsThe survey administered as part of the cross-sectional

study previously described, which collected data 2 yearsafter the California mandate for minimum nurse staffingratios (11), found that some California nurses perceived




that they had less support from the use of licensed voca-tional nurses, unlicensed personnel, and nonnursing sup-port services (such as housekeeping and unit clerks) afterimplementation of the mandate. For example, 25% of RNsreported that they perceived that they had decreased use oflicensed vocational nurses after the mandate, whereas 10%perceived that they had increased use and 56% reportedthat use remained the same.

The longitudinal assessments from California (16–19)and Western Australia (12) reported an increase in pressureulcers related to increased nurse staffing, although this de-velopment may reflect increased detection. Few other stud-ies mentioned an explicit assessment of potential unex-pected adverse outcomes.

IMPLEMENTATION CONSIDERATIONS AND COSTSImplementation Contexts

Because no published studies of an assessment of an“implementation” were found, the contexts in which in-terventions have been implemented cannot be directly as-sessed. However, the cross-sectional and longitudinal stud-ies that have been published and have consistently shown arelationship between staffing levels and patient outcomeshave included a broad array of hospitals, often all or nearlyall of the hospitals (except for very small ones) in a state.Therefore, if the relationship between increased RN staff-ing and inpatient mortality is a causal one, it very likelyapplies to most hospitals and contexts. This strategy ismost likely to be implemented when mandated by state orfederal policy.

As previously noted, the relationship between staffingand mortality that underpins this strategy has been seen invarious hospitals and contexts. The effect, if causal, is prob-ably relatively insensitive to the usual effects of contextsconsidered in other patient safety strategy reviews. Of note,the recent study by Needleman and colleagues was con-ducted in a tertiary medical center that has a lower-than-expected in-hospital mortality rate and a reputation forexcellence. Therefore, the relationship between increasedRN staffing and lower mortality, if causal, is potentiallyapplicable even to high-performing hospitals.

CostsFour simulation studies reported information about

costs. The first used 2003 data from 28 Belgian cardiacsurgery centers to assess the costs and outcomes of increas-ing nurse staffing. Assuming a causal relationship betweenthis staffing increase and an outcome of 5 fewer patientdeaths per 1000 elective hospitalizations, the authors con-cluded that the incremental cost-effectiveness ratio was€26 372 (approximately $35 000) per avoided death and€2639 (approximately $3500) per life-year gained (21).

The second simulation study was conducted by theUniversity of Minnesota Evidence-based Practice Center,which produced the systematic review on nurse staffing(22). It used its own meta-analysis as the basis for estimat-

ing the potential monetary benefits of increased RN staff-ing. Assuming that those relationships were causal and tak-ing a societal perspective, the authors concluded thatincreasing RN staffing by 1 FTE per patient day was re-lated to positive savings–cost ratios across a broad range ofclinical settings. For example, the net cost of adding 1 RNFTE per 1000 hospitalized ICU patients was an estimated$590 000, whereas the net benefit (in terms of life-yearssaved and productivity) was an estimated $1.5 million, fora benefit–cost ratio of 2.51. However, hospitals did notsave money because the net cost of adding an extra RNFTE was not offset by the expected 24% decrease in lengthof stay.

A third simulation study (24) used data from studiesby Aiken and colleagues and Needleman and colleagues toestimate benefits in mortality and length of stay, respec-tively, and estimated an incremental cost-effectiveness ratiobetween $25 000 and $136 000 per life saved as patient–RN staffing ratios decreased from 8:1 to 4:1. The modelwas most sensitive to the estimate of effect on mortality.

Lastly, 1 additional study from Portugal estimated thatincreasing neonatal nurse staffing to “adequate” would in-crease staff costs more than 30% of the current rate (23).

DISCUSSION

Nurse staffing ratios have a relationship with reduc-tions in hospital-related mortality in most published stud-ies. However, lack of a published evaluation of an inten-tional change in RN staffing from some initial value (forexample, 6 patients to 1 RN on general medical wards) tosome lower patient–RN staffing value (such as 5:1 or 4:1)limits conclusions on increasing nurse staffing ratios as apatient safety strategy. All longitudinal published studies todate have assessed natural variations in RN staffing. Theconcern also remains that mortality is not reduced by in-creased nurse staffing but by something the nurses do. De-termining what this is and how it can best be facilitatedshould be the goal of an effective patient safety strategy.

Limitations of this review include those of the originalarticles, such as lack of rigorous evaluations of an inten-tional intervention, low response rates to surveys that col-lect explanatory variables (such as RN staffing), potentiallypoor matching of RN staffing to actual patients cared forand their risk for death, and lack of replication of the 1high-quality longitudinal study that has been published;and the possibility that some relevant evidence was notfound, either because it was not identified during thesearch or because some completed evaluations have notbeen unpublished.

To further advance this field, studies assessing an in-tentional change in nurse staffing ratios are needed. It maybe impractical for such a study to be a randomized, con-trolled trial, but high-quality evidence could come from atime series analysis or a controlled before-and-after study,particularly if it included the necessary process variables to

Supplement Nurse–Patient Ratios as a Patient Safety Strategy



serve as a test of a conceptual framework for how increasedstaffing can influence outcomes.

From the RAND Corporation, Santa Monica, and Veterans AffairsGreater Los Angeles Healthcare System, Los Angeles, California.


Disclaimer: All statements expressed in this work are those of the authorand should not in any way be construed as official opinions or positionsof the RAND Corporation, Veterans Affairs, the Agency for HealthcareResearch and Quality, or the U.S. Department of Health and HumanServices.

Acknowledgment: The author thanks Robert Kane, MD; Eileen Lake,PhD, RN; Aneesa Motala, BA; Sydne Newberry, PhD; and RobertaShanman, MLS.


Potential Conflicts of Interest: Consultancy: ECRI Institute; Employ-ment: Veterans Affairs; Grants/grants pending: Agency for Healthcare Re-search and Quality, Veterans Affairs, Centers for Medicare & MedicaidServices, National Institute of Nursing Research, Office of the NationalCoordinator; Royalties: UpToDate. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2574.

Requests for Single Reprints: Paul G. Shekelle, MD, MPH, RANDCorporation, 1776 Main Street, Santa Monica, CA 90401; e-mail,[email protected].

Author contributions are available at www.annals.org.

References1. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nursestaffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA.2002;288:1987-93. [PMID: 12387650]2. Aiken LH, Clarke SP, Sloane DM, Lake ET, Cheney T. Effects of hospitalcare environment on patient mortality and nurse outcomes. J Nurs Adm. 2008;38:223-9. [PMID: 18469615]3. Tourangeau AE, Giovannetti P, Tu JV, Wood M. Nursing-related determi-nants of 30-day mortality for hospitalized patients. Can J Nurs Res. 2002;33:71-88. [PMID: 11998198]4. Person SD, Allison JJ, Kiefe CI, Weaver MT, Williams OD, Centor RM,et al. Nurse staffing and mortality for Medicare patients with acute myocardialinfarction. Med Care. 2004;42:4-12. [PMID: 14713734]5. Tourangeau AE, Doran DM, McGillis Hall L, O’Brien Pallas L, Pringle D,Tu JV, et al. Impact of hospital nursing care on 30-day mortality for acutemedical patients. J Adv Nurs. 2007;57:32-44. [PMID: 17184372]6. Thornlow DK, Anderson R, Oddone E. Cascade iatrogenesis: factors leadingto the development of adverse events in hospitalized older adults. Int J Nurs Stud.2009;46:1528-35. [PMID: 19643409]7. Despins LA, Scott-Cawiezell J, Rouder JN. Detection of patient risk bynurses: a theoretical framework. J Adv Nurs. 2010;66:465-74. [PMID: 20423428]8. Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organization, and qualityof care: cross-national findings. Nurs Outlook. 2002;50:187-94. [PMID:12386653]

9. Aiken LH, Sochalski J, Lake ET. Studying outcomes of organizational changein health services. Med Care. 1997;35:NS6-18. [PMID: 9366875]10. Whitlock EP, Lin JS, Chou R, Shekelle P, Robinson KA. Using existingsystematic reviews in complex systematic reviews. Ann Intern Med. 2008;148:776-82. [PMID: 18490690]11. Aiken LH, Sloane DM, Cimiotti JP, Clarke SP, Flynn L, Seago JA, et al.Implications of the California nurse staffing mandate for other states. Health ServRes. 2010;45:904-21. [PMID: 20403061]12. Twigg D, Duffield C, Bremner A, Rapley P, Finn J. The impact of thenursing hours per patient day (NHPPD) staffing method on patient outcomes: aretrospective analysis of patient and staffing data. Int J Nurs Stud. 2011;48:540-8. [PMID: 20696429]13. Needleman J, Buerhaus P, Pankratz VS, Leibson CL, Stevens SR, HarrisM. Nurse staffing and inpatient hospital mortality. N Engl J Med. 2011;364:1037-45. [PMID: 21410372]14. Harless DW, Mark BA. Nurse staffing and quality of care with direct mea-surement of inpatient staffing. Med Care. 2010;48:659-63. [PMID: 20548254]15. Schilling PL, Campbell DA Jr, Englesbe MJ, Davis MM. A comparison ofin-hospital mortality risk conferred by high hospital occupancy, differences innurse staffing levels, weekend admission, and seasonal influenza. Med Care.2010;48:224-32. [PMID: 20168260]16. Burnes Bolton L, Aydin CE, Donaldson N, Brown DS, Sandhu M, Frid-man M, et al. Mandated nurse staffing ratios in California: a comparison ofstaffing and nursing-sensitive outcomes pre- and postregulation. Policy Polit NursPract. 2007;8:238-50. [PMID: 18337430]17. Donaldson N, Bolton LB, Aydin C, Brown D, Elashoff JD, Sandhu M.Impact of California’s licensed nurse-patient ratios on unit-level nurse staffingand patient outcomes. Policy Polit Nurs Pract. 2005;6:198-210. [PMID:16443975]18. Unruh LY, Zhang NJ. Nurse staffing and patient safety in hospitals: newvariable and longitudinal approaches. Nurs Res. 2012;61:3-12. [PMID:22166905]19. Cook A, Gaynor M, Stephens M Jr, Taylor L. The effect of a hospital nursestaffing mandate on patient health outcomes: evidence from California’s mini-mum staffing regulation. J Health Econ. 2012;31:340-8. [PMID: 22425767]20. Duffield C, Diers D, O’Brien-Pallas L, Aisbett C, Roche M, King M, et al.Nursing staffing, nursing workload, the work environment and patient outcomes.Appl Nurs Res. 2011;24:244-55. [PMID: 20974086]21. Van den Heede K, Simoens S, Diya L, Lesaffre E, Vleugels A, Sermeus W.Increasing nurse staffing levels in Belgian cardiac surgery centres: a cost-effectivepatient safety intervention? J Adv Nurs. 2010;66:1291-6. [PMID: 20546363]22. Shamliyan TA, Kane RL, Mueller C, Duval S, Wilt TJ. Cost savings asso-ciated with increased RN staffing in acute care hospitals: simulation exercise.Nurs Econ. 2009;27:302-14, 331. [PMID: 19927445]23. Fugulin FMT, Lima AFC, Castilho V, Bochembuzio L, Costa JA, Castro L,et al. Cost of nursing staffing adequacy in a neonatal unit. Revista da Escola deEnfermagem da Usp. 2011;45:1582-8.24. Rothberg MB, Abraham I, Lindenauer PK, Rose DN. Improving nurse-to-patient staffing ratios as a cost-effective safety intervention. Med Care. 2005;43:785-91. [PMID: 16034292]25. McGahan M, Kucharski G, Coyer F; Winner ACCCN Best Nursing Re-view Paper 2011 sponsored by Elsevier. Nurse staffing levels and the incidenceof mortality and morbidity in the adult intensive care unit: a literature review.Aust Crit Care. 2012;25:64-77. [PMID: 22515951]26. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.Development of AMSTAR: a measurement tool to assess the methodologicalquality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:17302989]27. Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association ofregistered nurse staffing levels and patient outcomes: systematic review and meta-analysis. Med Care. 2007;45:1195-204. [PMID: 18007170]28. Tourangeau AE. Mortality rate as a nurse-sensitive outcome. In: Doran DM.Nursing Outcomes: The State of the Science. Sudbury, MA: Jones & Bartlett;2011.29. Furukawa MF, Raghu TS, Shao BB. Electronic medical records, nurse staff-ing, and nurse-sensitive patient outcomes: evidence from California hospitals,1998-2007. Health Serv Res. 2010;45:941-62. [PMID: 20403065]30. Griffiths P. RN�RN�better care? What do we know about the associationbetween the number of nurses and patient outcomes? [Editorial]. Int J Nurs Stud.2009;46:1289-90. [PMID: 19647533]




Author Contributions: Conception and design: P.G. Shekelle.Analysis and interpretation of the data: P.G. Shekelle.Drafting of the article: P.G. Shekelle.Critical revision of the article for important intellectual content:P.G. Shekelle.Final approval of the article: P.G. Shekelle.Obtaining of funding: P.G. Shekelle.Administrative, technical, or logistic support: P.G. Shekelle.Collection and assembly of data: P.G. Shekelle.

31. Stone PW, Pogorzelska M, Kunches L, Hirschhorn LR. Hospital staffingand health care-associated infections: a systematic review of the literature. ClinInfect Dis. 2008;47:937-44. [PMID: 18767987]32. Cummings GG, MacGregor T, Davey M, Lee H, Wong CA, Lo E, et al.Leadership styles and outcome patterns for the nursing workforce and work en-vironment: a systematic review. Int J Nurs Stud. 2010;47:363-85. [PMID:19781702]33. Butler M, Collins R, Drennan J, Halligan P, O’Mathuna DP, Schultz TJ,et al. Hospital nurse staffing models and patient and staff-related outcomes. Co-chrane Database Syst Rev. 2011:CD007019. [PMID: 21735407]34. Flynn M, McKeown M. Nurse staffing levels revisited: a consideration of keyissues in nurse staffing levels and skill mix research. J Nurs Manag. 2009;17:759-66. [PMID: 19694919]35. Cho SH, Hwang JH, Kim J. Nurse staffing and patient mortality in intensivecare units. Nurs Res. 2008;57:322-30. [PMID: 18794716]36. Kiekkas P, Sakellaropoulos GC, Brokalaki H, Manolis E, Samios A, Skart-sani C, et al. Association between nursing workload and mortality of intensivecare unit patients. J Nurs Scholarsh. 2008;40:385-90. [PMID: 19094155]

37. Hamilton KE, Redshaw ME, Tarnow-Mordi W. Nurse staffing in relationto risk-adjusted mortality in neonatal care. Arch Dis Child Fetal Neonatal Ed.2007;92:F99-F103. [PMID: 17088341]38. Mark BA, Harless DW, Berman WF. Nurse staffing and adverse events inhospitalized children. Policy Polit Nurs Pract. 2007;8:83-92. [PMID: 17652626]39. Rafferty AM, Clarke SP, Coles J, Ball J, James P, McKee M, et al. Out-comes of variation in hospital nurse staffing in English hospitals: cross-sectionalanalysis of survey data and discharge records. Int J Nurs Stud. 2007;44:175-82.[PMID: 17064706]40. Stone PW, Mooney-Kane C, Larson EL, Horan T, Glance LG, ZwanzigerJ, et al. Nurse working conditions and patient safety outcomes. Med Care. 2007;45:571-8. [PMID: 17515785]41. Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P.The impact of hospital nursing characteristics on 30-day mortality. Nurs Res.2005;54:74-84. [PMID: 15778649]42. Halm M, Peterson M, Kandels M, Sabo J, Blalock M, Braden R, et al.Hospital nurse staffing and patient mortality, emotional exhaustion, and job dis-satisfaction. Clin Nurse Spec. 2005;19:241-51. [PMID: 16179855]43. Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educationallevels of hospital nurses and surgical patient mortality. JAMA. 2003;290:1617-23. [PMID: 14506121]44. Sasichay-Akkadechanunt T, Scalzi CC, Jawad AF. The relationship betweennurse staffing and patient outcomes. J Nurs Adm. 2003;33:478-85. [PMID:14501564]45. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002;346:1715-22. [PMID: 12037152]46. Tarnow-Mordi WO, Hau C, Warden A, Shearer AJ. Hospital mortality inrelation to staff workload: a 4-year study in an adult intensive-care unit. Lancet.2000;356:185-9. [PMID: 10963195]




Preventing In-Facility Pressure Ulcers as a Patient Safety StrategyA Systematic ReviewNancy Sullivan, BA, and Karen M. Schoelles, MD, SM

Complications from hospital-acquired pressure ulcers cause 60 000deaths and significant morbidity annually in the United States. Theobjective of this systematic review is to review evidence regardingmulticomponent strategies for preventing pressure ulcers and toexamine the importance of contextual aspects of programs that aimto reduce facility-acquired pressure ulcers. CINAHL, the CochraneLibrary, EMBASE, MEDLINE, and PreMEDLINE were searched forarticles published from 2000 to 2012. Studies (any design) thatimplemented multicomponent initiatives to prevent pressure ulcersin adults in U.S. acute and long-term care settings and that re-ported pressure ulcer rates at least 6 months after implementation

were selected. Two reviewers extracted study data and rated qual-ity of evidence. Findings from 26 implementation studies (moderatestrength of evidence) suggested that the integration of several corecomponents improved processes of care and reduced pressure ulcerrates. Key components included the simplification and standardiza-tion of pressure ulcer–specific interventions and documentation,involvement of multidisciplinary teams and leadership, use of des-ignated skin champions, ongoing staff education, and sustainedaudit and feedback.


THE PROBLEM

Pressure ulcers are largely preventable, but pressure ul-cer rates continue to escalate at an alarming rate. Between1995 and 2008, incidence increased by as much as 80%(1). An estimated 2.5 million patients will develop a pres-sure ulcer annually in the United States (2); more than 1million patients are affected annually in U.S. acute andlong-term care settings (3). Because of the forecasted in-crease in populations most at risk for pressure ulcers (forexample, obese, diabetic, and elderly patients), rates arepredicted to continue to increase.

Preventing this problem is important not only to pro-tect patients from harm but also to reduce costs of caringfor them. Morbidity caused by pressure ulcers can lead torequirements for more care and resources and a longerinpatient stay. In some cases, late-stage pressure ulcers caneven lead to life-threatening infections. In fact, 60 000U.S. patients die annually of complications related tohospital-acquired pressure ulcers (2).

The objective of this review is to review the evidenceon implementation of multicomponent strategies for pre-venting pressure ulcers, focusing on the importance of con-textual aspects of programs to reduce the likelihood offacility-acquired pressure ulcers. We focus on implementa-tion of multicomponent initiatives because a patient safetystrategy designed to address multiple factors is believed tobe more effective than single-component initiatives in pre-venting this condition.

PATIENT SAFETY STRATEGIESStrategies aimed at preventing pressure ulcers may

consist of individual or multicomponent interventions ora series of interventions and may include system-levelchanges. A systematic review by Reddy and colleagues (4)included 59 prevention studies that addressed impairedmobility, impaired nutrition, or impaired skin health,mostly in patients in acute care settings. The authors con-cluded that using support surfaces, regularly repositioningthe patient, optimizing nutritional status, and moisturizingsacral skin are appropriate strategies for preventing pressureulcers. Other reviews and guidelines stress the importanceof initial and repeated assessment of patients’ risk, tailoredcare for individuals found to be at increased risk, and reg-ular skin examinations (5–17).

Many organizations endorse the concept of bundlingcare practices (for example, standardized risk assessmentand regular repositioning), which typically include 3 to 5evidence-based practices that “when performed collectivelyand reliably, have been proven to improve patient out-comes” (18). Some recommend having an identifiabletheme (such as “Save Our Skin”) (1, 19). Besides bundlingcare practices, experts recommend that attention be paid toorganizational and care coordination components (1, 20).Organizational components include selecting lead teammembership, establishing policies and procedures, evaluat-ing quality processes, educating staff, using skin champi-ons, and communicating written care plans. Care coordi-nation components include creating a culture of changeand establishing regular meetings to facilitate communica-tion, collegiality, and learning.

REVIEW PROCESSESThis review was done in parallel with another Agency

for Healthcare Research and Quality (AHRQ)–sponsoredsystematic review on specific interventions for preventingpressure ulcers (for example, different kinds of supportsurfaces, heel supports, nutritional supplementation, and

See also:





repositioning). We searched CINAHL, the CochraneLibrary, EMBASE, MEDLINE, and PreMEDLINE for ar-ticles published from 2000 to September 2012 and thegray literature by using keywords related to the concepts ofpressure ulcer prevention efforts, barriers, and settings.Searches were restricted to English-language literature. Weidentified 587 abstracts, from which 95 full-text articleswere reviewed in more detail, yielding 51 articles contrib-uting data to this review. We selected studies of any designthat implemented multicomponent initiatives in acute andlong-term care settings in the United States. Studies wereincluded if they considered multicomponent pressure ulcerpreventive measures (such as evidence-based clinical deci-sion tools combined with training and education), targetedadult populations, and reported pressure ulcer rates 6months after implementation.

Two independent reviewers screened publications forinclusion; 26 studies (18 acute care, 8 long-term care) metinclusion criteria. The reviewers extracted information oncontext, including influence of external factors (such asstate survey deficiencies); descriptions of teamwork, leader-ship, and safety culture; and implementation tools (such asongoing performance monitoring). They detailed descrip-tions of the implementation efforts (such as processes, bar-riers, and sustainability) in the studies and extracted infor-mation about our main (pressure ulcer rates) and secondary(process-of-care measures) outcomes.

We assessed study quality using the 19-item Standardsfor Quality Improvement Reporting Excellence (SQUIRE)guidelines (21). We paid particular attention to a subset ofthe items we thought were important for implementationstudies, such as the following: 1) describes the interventionand its component in sufficient detail that others couldreproduce it, 2) presents data on changes observed in thecare delivery process and changes observed in measures ofpatient outcomes, 3) reports on study limitations, and 4)interprets possible reasons for differences between observedand expected outcomes. Our assessment did not considerother requirements in the SQUIRE guidelines such as in-cluding an abstract, describing the local problem, or re-porting funding. We considered a study to be high qualityif it reported 8 to 10 items, moderate quality if it reported5 to 7 items, and low quality if it reported fewer than 5required items.

The Supplement (available at www.annals.org) com-pletely describes the search strategies, provides an articleflow diagram, and provides evidence tables.

This review was supported by AHRQ, which had norole in the selection or review of the evidence or the deci-sion to submit the manuscript for publication.

BENEFITS AND HARMS

BenefitsTwenty-six studies met inclusion criteria. Eighteen

studies were conducted in acute care settings and 8 in long-

term care settings. Study designs were mostly time seriesassessments of changes before, during, and after implemen-tation of the intervention. Other designs included random-ized, controlled trials (22–24) and a controlled before-and-after (24). Several of the studies were identified from a2011 review of nurse-focused quality improvement inter-ventions in hospitals (25) and a 2012 review of compre-hensive programs for preventing pressure ulcers (5). Of the26 studies, 9 were high-quality, 14 were moderate-quality,and 3 were low-quality.

Nine core components of programs for pressure ulcerprevention, in addition to specific patient care practices,have been associated with a reduction in incidence or prev-alence of pressure ulcers. Appendix Tables 1 and 2 (avail-able at www.annals.org) show which components and pa-tient care practices were used in the 18 studies in acute caresettings and the 8 studies in long-term care settings. Stud-ies showed that most organizations educated and trainedstaff (96%), developed or revised their protocols for assess-ment and documentation of wounds (96%), performedquality audits and provided feedback to staff (81%), ad-opted the Braden Scale for Predicting Pressure Sore Risk(61%), and redesigned documentation processes and re-porting (58%).

In the 18 studies of pressure ulcer prevention pro-grams in U.S. hospitals, study authors described multiplepatient care interventions or cited clinical practice guide-lines or resources that describe specific interventions toreduce patients’ risk for pressure ulcers. The hospital care-givers performed initial and repeated risk assessments(such as the Braden Scale), followed by tailored interven-tions chosen from a menu of options based on a risk cat-egory or specific risk factors. These interventions includedsupport surfaces (for example, specialized mattresses andheel supports), getting patients out of bed or frequentlyrepositioning those who were bed-bound, moisture man-agement (including incontinence interventions and skincare products), mechanical means of reducing friction and

Key Summary Points

Despite being largely preventable, pressure ulcer rates areescalating in the United States.

Moderate-strength evidence suggests that implementingmulticomponent initiatives for pressure ulcer prevention inacute and long-term care settings can improve processesof care and reduce pressure ulcer rates.

Key components of successful implementation efforts in-clude: simplification and standardization of pressure ulcer–specific interventions and documentation, involvement ofmultidisciplinary teams and leadership, designated skinchampions, ongoing staff education, and sustained auditand feedback.

SupplementPreventing In-Facility Pressure Ulcers as a Patient Safety Strategy



shear forces on body areas at greatest risk, nutritional as-sessments or interventions, and hydration. Pressure ulcerprevention programs that were used in the 8 studies inlong-term care facilities typically referenced guidelines orother resources developed by their state’s quality improve-ment organizations.

Twenty-four studies reported at least some improve-ment in pressure ulcer rates. Two additional studies re-ported that process-of-care quality measures improved butthat pressure ulcer rates did not (26, 27). Statistically sig-nificant reductions in pressure ulcer rates were reported in11 (42%) of 26 studies (median reduction, 82% [range67% to 100%]) (24, 28–37). Of the 13 studies with im-provements not reaching statistical significance, 5 reportedimprovements in both pressure ulcer rates and process-of-care measures (19, 38–41).

The implementation of a multicomponent strategy byWalsh and colleagues (2009) reduced pressure ulcer prev-alence (12.8% to 0.6%), increased focused communicationamong patient caregivers, and improved clinician behaviorand clinical processes once other improvements were rec-ognized (38). Young and colleagues streamlined onlinepolicies (from 7 to 1) and reduced time spent documentingskin care, which resulted in “clinically relevant reductions”in development of nosocomial pressure ulcers (19). In 1year, pressure ulcer rates were reduced by 82.8% (from2.8% to 0.48%) at 1 rehabilitation hospital. Lynch andVickery (39) reported that streamlining documentation in-creased timely and accurate completion from 60% to 90%in 90 days. Delmore and colleagues (41) reported a reduc-tion in incidence (from 7.3% to 1.3%) and reduction intime for collection of prevalence and incidence data (from8 hours to 2.5 hours).

In the long-term care setting, implementation of theon-time approach in 10 participating facilities led to reduc-tions in prevalence of pressure ulcers for 7 facilities, reduc-tions in the average number of in-house pressure ulcers (allstages) for 8 facilities, and reductions in the average num-ber of certified nursing assistant documentation forms for10 facilities (35). Another study (37) reported a statisticallysignificant reduction in pressure ulcer incidence (28.3% vs.9.3%) and improvements in identifying patients as “high-risk” (increase from 22.3% and 28.0%). Milne and col-leagues (40) reported reducing prevalence from 41% to4.2% after increased monitoring of patients with nasal can-nulas (pulmonary unit) and increased attentiveness to heeloffloading, support surfaces, and proper positioning (spinalcord injury and trauma unit). Of the 396 charts reviewedafter implementation, fewer than 1% had missing data. Areview of 45 patient charts showed that wound teams con-sistently determined staging and wound cause in more than90% of cases.

HarmsNo harms were reported for the patient safety strate-

gies that were used to prevent pressure ulcers.


Use of a Model or TheoryOf the 26 studies, 6 programs described a model or

theory as the basis of their implementation strategy. Severalquality improvement approaches were described. ThePDSA (Plan, Do, Study, Act) framework used in a 17-hospital-initiative (26) involves 4 improvement cycles: 1)identifying the problem and designing an intervention(Plan), 2) implementing change (Do), 3) evaluating col-lected data (Study), and 4) implementing what was learned(Act). Courtney and colleagues (32) integrated Six Sigmamethods called DMAIC into treatment processes devel-oped for a multisite, not-for-profit facility. Described as adata-driven quality strategy for improving processes,DMAIC consists of 5 interconnected steps: (1) Definingthe problem, (2) Measuring the performance, (3) Analyz-ing the data, (4) Improving the process, and (5) Control-ling change (42). Young and colleagues (19) and Chicanoand Drolshagen (43) empowered staff at the point of care,which “suggests a model of shared governance where deci-sions are made at the point of service” (44). Two studiesdescribed use of failure mode and effects analysis (40) andHavelock’s (1974) model of effective research utilization(24). Of these 6 studies, 2 reported statistically significantreductions in pressure ulcers (24, 32); 1 reported improve-ments in processes of care (26).

External Factors Motivating Attention to PressureUlcer Prevention

Most studies in acute care facilities reported feelingpressure from impending changes in U.S. Centers forMedicare & Medicaid Services reimbursement to imple-ment pressure ulcer prevention strategies. Specifically, sub-sequent to passage of the Deficit Reduction Act of 2005,the Centers for Medicare & Medicaid Services no longerallows higher diagnosis-related group payments for patientswith stage 3 and 4 hospital-acquired pressure ulcers. Addi-tional positive and negative external motivators are de-scribed below.

Positive motivators included a stakeholder’s commit-ment to improve patient outcomes and a goal “to be rec-ognized as a quality provider of patient services” (19). Theemergence of new guidelines from the American NursesAssociation and AHRQ’s “revitalized interest” in prevent-ing and treating pressure ulcers was cited by Courtney andcolleagues (32). One facility, at which prevalence ofhospital-acquired pressure ulcers was lower than nationalnorms, set out to eliminate hospital-acquired pressure ul-cers completely (33).

Negative motivators for 1 cancer hospital included theidentification of 2 stage 4 pressure ulcers and evidence thatpressure ulcer prevalence exceeded the national benchmarkby nearly 50% (31). Two facilities reported influence froma G-level citation (a deficiency judged to cause actual harmto residents) (36) and other citations from the Departmentof Health (37). Two critical incidents (not specified) and

Supplement Preventing In-Facility Pressure Ulcers as a Patient Safety Strategy



inconsistent documentation were listed as external motiva-tors by Dibsie (45). Additionally, “the frequency withwhich concerns and incidents were discussed, but wentunreported within the internal reporting system” was ofconcern (45).

Teamwork/LeadershipA majority of studies used multidisciplinary teams

with skin champions being described as key team mem-bers. Studies typically designated 1 individual (for example,a certified wound ostomy continence nurse) (28, 30, 46) tocoordinate prevention efforts.

Two studies provided detailed descriptions of leader-ship support. Stier and colleagues (34) described supportprovided to multidisciplinary teams at 1 health care sys-tem. Teams consisting of clinical experts from 18 facilitiesconvened to discuss the various risk assessment tools andfacility protocols already in place. Multidisciplinary teamsthen agreed to develop a uniform protocol, skin care for-mulary, and specialty bed contract. “System leadership(e.g., nurse executives, quality management directors, andsenior physicians) provided support to the team at both thesystem and facility level” vis-a -vis “resources, ensured stafforientation and education, maintained quality control pro-grams, and continually assessed actions to improve perfor-mance through system-wide care committee meetings”(34). Dibsie (45) described broadening teamwork fromnursing management to a larger group of managers andclinical specialists after “it became evident that seriousskin-related issues crossed many areas and could be betterhandled by the group together.”

Implementation ToolsMore than 21 initiatives provided examples of unique

tools used for audit and feedback, education and training,and streamlining products and processes. For a completelisting of implementation tools, see the Data Supplement(available at www.annals.org). Audit and feedback (positiveand negative) were mentioned as key elements in 20 (80%)preventive initiatives. Hiser and colleagues (46) reportedthat providing frequent feedback to clinical staff on unitprogress helped engage staff members and “allowed themto take credit for the improved clinical outcomes.” Certif-icates for the most improved units were used as reinforce-ments. While providing feedback to nursing staff in 1study, the certified nurse specialists balanced complimentsfor a job well done with recommendations for improve-ment (47). In 1 long-term care study, facilitators provideddirect feedback to certified nursing assistants regarding datainconsistencies by unit and by shift to help track progress(35). Real-time management feedback in Rosen and col-leagues’ study (37) consisted of a prominently displayedthermometer tracking weekly pressure ulcer incidence andpositive ($10 reward) or negative (termination) reinforce-ment. Weekly informal feedback by nursing supervisors(36), formal weekly walk-rounds (39), and frequent patientpositioning audits were also used during implementation

(36). One rehabilitation hospital posted report cards unit-wide, allowing staff to track progress against other unitsand unit goals (39).

Unique tools used during education and training ses-sions included enrollment of guest speakers to educate phy-sicians about the role of the certified wound ostomy con-tinence nurse and best-practice interventions for woundcare (46). In another study, participants sat on bedpansduring 30-minute mandatory sessions as a reminder thatpressure ulcers can occur in less than 1 hour (19). Thissame study tailored educational content for multilevelstaffing and measured effectiveness of presentations byposttest survey. Finally, Delmore and colleagues describedthe involvement of perioperative services in establishing aneducational newsletter for the facility’s Skin and WoundCare Web site and hosting a Skin Fair Day (41).

Barriers SolvedReported barriers to implementation included unmo-

tivated staff (28, 31, 43), staff turnover (23, 24, 27, 35),staff and physician resistance (19, 26, 27), inconsistentdocumentation (27, 28, 47), difficulties in exporting data(35), and miscommunication between electronic systems(47). Staff disruption of implementation initiatives was themost commonly reported barrier. One study described staffas relatively uninvolved in planning (43), whereas anotherstudy described staff members focusing more on the role ofwound care products and specialty beds than on nursingcare when patients developed in-facility pressure ulcers(31). The launching of monthly to quarterly campaigns(28); perseverance by leadership (43); and use of additionaleducation, mentoring, and support at the unit level (31)were solutions given for motivating staff. Staff reverting topreviously unsuccessful practices (27), staff turnover (24,27, 35), and variations in new staff orientation also slowedprogram momentum. The development of a strong multi-disciplinary team (35), assignment of responsibility forprocesses to multiple nurses (23), and monthly visits by astate quality improvement organization (27) helped addressthese issues.

To address concerns regarding inconsistent reportingand documentation, Horn and colleagues (35) workedwith long-term care facilities to simplify and standardizecertified nursing assistant documentation and translate theinformation into reports that were used in weekly careplanning meetings. Bales and Padwojski (28) responded byrecognizing and awarding nursing units in which patientshad 0 hospital-acquired pressure ulcers. Initiatives werealso challenged by limited resources. Finally, LeMaster (47)indicated that 2 different electronic documentation sys-tems were causing shortfalls in pressure ulcer risk report-ing. Transition to 1 universal electronic record system re-solved this issue (47).

SustainabilitySeveral acute and long-term care facilities reported on

sustainability or long-term maintenance of prevention ef-




forts. Conducting quarterly prevalence studies (33), requir-ing registered nurses and licensed practical nurses to dem-onstrate competency annually (19), and providing monthlyupdates via intranet to staff of product changes (19) werekey to sustaining improvements in 2 studies. McInerney(29) indicated that publicizing improvements in pressureulcer rates kept staff focused on prevention efforts. Onerehabilitation hospital printed quarterly newsletters and at-tached them to paychecks. The newsletters described find-ings, results, and new initiatives in pressure ulcer manage-ment (39). Other studies describe basing staff bonuses onpressure ulcer incidence (32), establishing a wound carecoordinator position (36) and a wound care committee(24), and keeping current regarding “initiatives for im-proved patient safety, changes in regulatory mandates, andchanges in EBP [evidence-based practices]” (38) helpedmaintain gains.

Cost-SavingsFour studies reported on cost-savings. Two studies

(36, 37) referenced a secondary analysis by Xakellis andFrantz (48) that evaluated long-term care and hospitalcosts for healing 45 pressure ulcers from 30 patients. Rosenand colleagues stated that “based on a mean cost of $2,700to treat a single stage II pressure ulcer, reducing the inci-dence of ulcers by approximately 15 over 12 weeks wouldyield savings of approximately $40,000” (37, 48). In 2009,a 151-bed Midwest skilled-nursing facility described cost-savings 4 years after program implementation. After adjust-ing (using the Consumer Price Index) the 1996 mean costof treating a patient with a pressure ulcer ($1115 permonth), the authors estimated their cost-savings at $1617per pressure ulcer per month, $10 187 in total monthlysavings, and greater than $122 000 in yearly savings (36).

Estimated cost-savings in the remaining 2 studies(based on an additional cost per case of approximately$3000) were also significant (29, 32). In 2006, Courtneyand colleagues reported that a reduction of hospital-acquired pressure ulcers by 50% to 5% would reduce over-all costs by $2 438 000 (32). In 2008, a 2-hospital system(548 beds) in Naples, Florida (29), estimated cost-savingsof approximately $11.5 million annually as a result of sta-tistically significant reductions in pressure ulcer prevalence.

Effects of ContextAuthors of studies in long-term care (27) and acute

care (26) settings agreed that the most sustainable interven-tions were those that were institutionalized. For example,interventions that were less dependent on sufficient staffing(for example, changing to pressure-relieving mattresses andusing risk assessment tools) were easier to sustain than in-terventions that were more dependent on sufficient staffing(such as ensuring that every resident is turned every 2hours). Horn and colleagues (35) found that full integra-tion of clinical reports derived from documentation byfront-line staff (certified nursing assistants) was key to suc-cess. Studies also specifically mentioned that nurses taking

ownership (45), as well as promotion and support by lead-ership (28, 43), were significant factors in achieving goals.

DISCUSSION

Moderate-strength evidence from 26 implementationstudies suggests that the integration of a common set ofcomponents in pressure ulcer prevention programs couldlead to reductions in pressure ulcer rates. Key issues werethe simplification and standardization of pressure-ulcer-specific interventions and documentation, involvement ofmultidisciplinary teams and leadership, designated skinchampions, ongoing staff education, and sustained auditand feedback for promoting both accountability and rec-ognizing successes.

Two recent systematic reviews of quality improvementprograms to prevent pressure ulcers found improvementsin process or ulcer outcomes that were similar to our find-ings (5, 25). Nurse-focused initiatives led to improvements“on at least one nursing process or patient health outcomemeasure in the intended direction” in 36 of 39 acute carestudies in a 2011 review by Soban and colleagues (25). Ina 2012 review by Niederhauser and colleagues (5), 17 of 20studies reporting on process-of-care measures and out-comes reported improvements in acute and long-term caresettings. Both reviews included a listing of core compo-nents integrated during implementation. Our review addsto the previous reviews by providing details on implemen-tation of prevention programs, lessons learned (see theSupplement), solutions to barriers, and potential cost-savings.

Neither our review nor those by Soban (25) and Nie-derhauser (5) and their colleagues discussed the effective-ness of individual components included in preventive bun-dles because the included studies did not focus on theeffectiveness of specific intervention components. Never-theless, most studies included certain aspects of direct pa-tient care: initial and repeated risk assessments and skinexaminations; the use of specialized support surfaces (suchas special mattresses and overlays); repositioning or mobil-ity protocols; moisture, friction and shear management;and nutrition and hydration. Most studies cited clinicalpractice guidelines that informed the choice of interven-tions. Additional limitations of our review included theexclusion of non-U.S. studies, possible selective reporting,and no formal evaluation of the possibility of publicationbias. Niederhauser and colleagues (5) speculated that pub-lication bias explains the positive results in most publishedstudies.

All 3 reviews agree on the need for future research todelve deeper into daily care processes to better understandtheir influence on outcomes. Limitations of the evidenceinclude the lack of information on processes of care andtheir measurement. In fact, in this review, only 9 of 26studies included information on both processes and out-come measures. Studies also did not describe study limita-




tions or summarize successes and barriers to implementa-tion, items listed by the SQUIRE guidelines (21) as key toreporting in a discussion.

In 2000, a review of measures to prevent pressure ul-cers in older patients in Making Health Care Safer (49)included a brief discussion on implementation of pressure-relieving devices specifically noting cost, time, and diffi-culty in assessing change in pressure ulcer rates after imple-mentation. Since that time, guidance provided by suchorganizations as the Institute for Healthcare Improvement(6), National Pressure Ulcer Advisory Panel (50), andAHRQ (51) has resulted in successful implementation ofbundled evidence-based practices throughout the UnitedStates. Although we identified 26 implementation studies(published since 2000), we are concerned about the possi-bility of publication bias. To continue to understand theinfluence of context on implementation of strategies toprevent pressure ulcers, we encourage clinicians to reportfindings regardless of success level and to provide detail onthe patient care processes, staff education and training ini-tiatives, and system-level interventions. In addition, futureresearch should report strategies to sustain momentum ofpreventive programs, a topic rarely discussed in the imple-mentation studies we reviewed. Given the persistent signif-icant morbidity and mortality resulting from pressure ul-cers, further study of both system-level and patient careinterventions aimed at preventing pressure ulcers is stillneeded for clinicians and managers to choose the mosteffective and efficient practices.

From ECRI Institute Evidence-based Practice Center, Plymouth Meet-ing, Pennsylvania.


Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof ECRI Institute, the AHRQ, or the U.S. Department of Health andHuman Services.

Acknowledgment: The authors thank Allison Gross, MS, LIS, for per-forming the literature searches; Lydia Dharia and Katherine Donahue forpreparing the manuscript for publication; and Paul G. Shekelle, MD,PhD, for his review and suggestions on earlier versions of the manuscript.

Financial Support: From the AHRQ, U.S. Department of Health andHuman Services (contract HHSA-290-2007-10062I).

Potential Conflicts of Interest: Ms. Sullivan: None disclosed. Dr.Schoelles: Support for travel to meetings for the study or other purposes(money to institution): RAND Corporation; Other (money to institution):work done by several ECRI staff on Making Health Care Safer II: AnUpdated Critical Analysis of the Evidence for Patient Safety Practices forthe AHRQ supported by RAND. Disclosures can also be viewed atwww.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2655.

Requests for Single Reprints: Nancy Sullivan, BA, ECRI InstituteEvidence-based Practice Center, 5200 Butler Pike, Plymouth Meeting,PA 19462-1298; e-mail, [email protected].


References1. Lyder CH, Ayello EA. Annual checkup: the CMS pressure ulcer present-on-admission indicator. Adv Skin Wound Care 2009;22:476-84. [PMID:20026923]2. Lyder CH. The benefits of a multi-disciplinary approach to the prevention andtreatment of pressure ulcers. Infection Control Today. 10 August 2011. Accessedat www.infectioncontroltoday.com/news/2011/08/the-benefits-of-a-multi-disci-plinary-approach-to-the-prevention-and-treatment-of-pressure-ulcers.aspx on 15November 2012.3. Sharkey S, Hudak S, Horn SD, Spector W. Leveraging certified nursingassistant documentation and knowledge to improve clinical decision making: theon-time quality improvement program to prevent pressure ulcers. Adv SkinWound Care. 2011;24:182-8; quiz 188-90. [PMID: 21407045]4. Reddy M, Gill SS, Rochon PA. Preventing pressure ulcers: a systematic re-view. JAMA. 2006;296:974-84. [PMID: 16926357]5. Niederhauser A, VanDeusen Lukas C, Parker V, Ayello EA, Zulkowski K,Berlowitz D. Comprehensive programs for preventing pressure ulcers: a review ofthe literature. Adv Skin Wound Care. 2012;25:167-88. [PMID: 22441049]6. Institute for Healthcare Improvement. How-to Guide: Prevent Pressure Ul-cers. Cambridge, MA: Institute for Healthcare Improvement; 2011.7. Advancing Excellence in America’s Nursing Homes. Implementation Guide:Goal 4: Reducing High Risk Pressure Ulcers. NH Quality Campaign; 2008.8. Institute for Clinical Systems Improvement. Pressure ulcer prevention andtreatment protocol. Health care protocol. Bloomington, MN: Institute for Clin-ical Systems Improvement; 2012.9. Registered Nurses’ Association of Ontario. Risk assessment & prevention ofpressure ulcers 2011 supplement. Toronto, Ontario, Canada: Registered Nurses’Association of Ontario; 2011.10. Wound, Ostomy, and Continence Nurses Society. Guideline for preventionand management of pressure ulcers. Mount Laurel, NJ: Wound, Ostomy, andContinence Nurses Society; 2010.11. Association for the Advancement of Wound Care. Association for the Ad-vancement of Wound Care guideline of pressure ulcer guidelines. Malvern, PA:Association for the Advancement of Wound Care; 2010.12. National Pressure Ulcer Advisory Panel, European Pressure Ulcer AdvisoryPanel. Pressure ulcer prevention recommendations. In: Prevention and treatmentof pressure ulcers: clinical practice guideline. Washington, DC: National PressureUlcer Advisory Panel; 2009.13. Lyder CH, Ayello EA. Pressure ulcers: a patient safety issue. In: Patient Safetyand Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agencyfor Healthcare Research and Quality; 2008.14. American Medical Directors Association. Pressure ulcers in the long-termcare setting. Columbia, MD: American Medical Directors Association; 2008.15. Ayello EA, Sibbald RG. Preventing pressure ulcers and skin tears. In: Cap-ezuti E, Zwicker D, Mezey M, Fulmer T, eds. Evidence-Based Geriatric NursingProtocols for Best Practice. 3rd ed. New York: Springer; 2008.16. Registered Nurses’ Association of Ontario. Risk assessment & prevention ofpressure ulcers. Toronto, Ontario, Canada: Registered Nurses’ Association ofOntario; 2005.17. Pressure Ulcer Guideline Panel, Agency for Health Care Policy and Re-search. Pressure ulcers in adults: prediction and prevention. Rockville, MD: U.S.Department of Health and Human Services, Public Health Service, Agency forHealth Care Policy and Research; 1992.18. Institute for Healthcare Improvement. What is a bundle? 26 April 2011.Accessed at www.ihi.org/knowledge/Pages/ImprovementStories/WhatIsaBundle.aspx on 18 November 2011.19. Young J, Ernsting M, Kehoe A, Holmes K. Results of a clinician-ledevidence-based task force initiative relating to pressure ulcer risk assessment andprevention. J Wound Ostomy Continence Nurs. 2010;37:495-503. [PMID:20736858]




20. Jankowski IM, Nadzam DM. Identifying gaps, barriers, and solutions inimplementing pressure ulcer prevention programs. Jt Comm J Qual Patient Saf.2011;37:253-64. [PMID: 21706985]21. Ogrinc G, Mooney SE, Estrada C, et al. The SQUIRE (Standards forQuality Improvement Reporting Excellence) guidelines for quality improvementreporting: explanation and elaboration. Qual Saf Health Care 2008 Oct;17 Suppl1:i13-32. [PMID: 18836062]22. Rantz MJ, Hicks L, Petroski GF, Madsen RW, Alexander G, Galambos C,et al. Cost, staffing and quality impact of bedside electronic medical record(EMR) in nursing homes. J Am Med Dir Assoc. 2010;11:485-93. [PMID:20816336]23. Rantz MJ, Popejoy L, Petroski GF, Madsen RW, Mehr DR, Zwygart-Stauffacher M, et al. Randomized clinical trial of a quality improvement inter-vention in nursing homes. Gerontologist. 2001;41:525-38. [PMID: 11490051]24. Ryden MB, Snyder M, Gross CR, Savik K, Pearson V, Krichbaum K, et al.Value-added outcomes: the use of advanced practice nurses in long-term carefacilities. Gerontologist. 2000;40:654-62. [PMID: 11131082]25. Soban LM, Hempel S, Munjas BA, Miles J, Rubenstein LV. Preventingpressure ulcers in hospitals: a systematic review of nurse-focused quality improve-ment interventions. Jt Comm J Qual Patient Saf. 2011;37:245-52. [PMID:21706984]26. Lyder CH, Grady J, Mathur D, Petrillo MK, Meehan TP. Preventingpressure ulcers in Connecticut hospitals by using the plan-do-study-act model ofquality improvement. Jt Comm J Qual Saf. 2004;30:205-14. [PMID:15085786]27. Abel RL, Warren K, Bean G, Gabbard B, Lyder CH, Bing M, et al. Qualityimprovement in nursing homes in Texas: results from a pressure ulcer preventionproject. J Am Med Dir Assoc. 2005;6:181-8. [PMID: 15894247]28. Bales I, Padwojski A. Reaching for the moon: achieving zero pressure ulcerprevalence. J Wound Care. 2009;18:137-144. [PMID: 19349933]29. McInerney JA. Reducing hospital-acquired pressure ulcer prevalence througha focused prevention program. Adv Skin Wound Care. 2008;21:75-8. [PMID:18349734]30. Ballard N, McCombs A, Deboor S, Strachan J, Johnson M, Smith MJ,et al. How our ICU decreased the rate of hospital-acquired pressure ulcers. J NursCare Qual. 2008;23:92-6. [PMID: 18281882]31. Catania K, Huang C, James P, Madison M, Moran M, Ohr M. Woundwise: PUPPI: the Pressure Ulcer Prevention Protocol Interventions. Am J Nurs.2007;107:44-52. [PMID: 17413732]32. Courtney BA, Ruppman JB, Cooper HM. Save our skin: initiative cutspressure ulcer incidence in half. Nurs Manage. 2006;37:36, 38, 40 passim.[PMID: 16603946]33. Gibbons W, Shanks HT, Kleinhelter P, Jones P. Eliminating facility-acquired pressure ulcers at Ascension Health. Jt Comm J Qual Patient Saf. 2006;32:488-96. [PMID: 17987872]34. Stier L, Dlugacz YD, O’Connor LJ, Eichorn AM, White M, Fitzpatrick J.Reinforcing organization wide pressure ulcer reduction on high-risk geriatric in-patient units. Outcomes Manag. 2004;8:28-32. [PMID: 14740581]35. Horn SD, Sharkey SS, Hudak S, Gassaway J, James R, Spector W. Pressureulcer prevention in long-term-care facilities: a pilot study implementing standard-

ized nurse aide documentation and feedback reports. Adv Skin Wound Care.2010;23:120-31. [PMID: 20177165]36. Tippet AW. Reducing the incidence of pressure ulcers in nursing homeresidents: a prospective 6-year evaluation. Ostomy Wound Manage. 2009;55:52-8. [PMID: 19934464]37. Rosen J, Mittal V, Degenholtz H, Castle N, Mulsant BH, Hulland S, et al.Ability, incentives, and management feedback: organizational change to reducepressure ulcers in a nursing home. J Am Med Dir Assoc. 2006;7:141-6. [PMID:16503306]38. Walsh NS, Blanck AW, Barrett KL. Pressure ulcer management in the acutecare setting: a response to regulatory mandates. J Wound Ostomy ContinenceNurs. 2009;36:385-8. [PMID: 19609158]39. Lynch S, Vickery P. Steps to reducing hospital-acquired pressure ulcers.Nursing. 2010;40:61-2. [PMID: 20975436]40. Milne CT, Trigilia D, Houle TL, Delong S, Rosenblum D. Reducingpressure ulcer prevalence rates in the long-term acute care setting. OstomyWound Manage. 2009;55:50-9. [PMID: 19387096]41. Delmore B, Lebovits S, Baldock P, Suggs B, Ayello EA. Pressure ulcerprevention program: a journey. J Wound Ostomy Continence Nurs. 2011;38:505-13. [PMID: 21860330]42. Terry K. What is DMAIC? iSixSigma. 13 March 2010. Accessed at www.isixsigma.com/methodology/dmaic-methodology/what-dmaic on 10 November2011.43. Chicano SG, Drolshagen C. Reducing hospital-acquired pressure ulcers.J Wound Ostomy Continence Nurs. 2009;36:45-50. [PMID: 19155823]44. Porter-O’Grady T. Shared governance. J Nurs Adm. 1995;25:8-9. [PMID:7636580]45. Dibsie LG. Implementing evidence-based practice to prevent skin break-down. Crit Care Nurs Q. 2008;31:140-9. [PMID: 18360144]46. Hiser B, Rochette J, Philbin S, Lowerhouse N, Terburgh C, Pietsch C.Implementing a pressure ulcer prevention program and enhancing the role of theCWOCN: impact on outcomes. Ostomy Wound Manage. 2006;52:48-59.[PMID: 16464994]47. LeMaster KM. Reducing incidence and prevalence of hospital-acquired pres-sure ulcers at Genesis Medical Center. Jt Comm J Qual Patient Saf. 2007;33:611-6, 585. [PMID: 18030863]48. Xakellis GC, Frantz R. The cost of healing pressure ulcers across multiplehealth care settings. Adv Wound Care. 1996;9:18-22. [PMID: 9069752]49. Agostini JV, Baker DI, Bogardus ST. Chapter 27: prevention of pressureulcers in older patients. In: Making Health Care Safer: A Critical Analysis ofPatient Safety Practices. AHRQ publication no. 01-E058. Evidence Report/Technology Assessment no. 43. Washington, DC: Agency for Healthcare Re-search and Quality; 2001.50. Pressure ulcer prevention points. Washington, DC: National Pressure UlcerAdvisory Panel; 2007. Accessed at www.npuap.org/wp-content/uploads/2012/03/PU_Prev_Points.pdf on 8 January 2013.51. On-Time Prevention of Pressure Ulcers: Partnering With Quality Improve-ment Organizations. Final Report. Rockville, MD: Agency for Healthcare Re-search and Quality; 2007.




Current Author Addresses: Ms. Sullivan and Dr. Schoelles: ECRI In-stitute Evidence-based Practice Center, 5200 Butler Pike, PlymouthMeeting, PA 19462-1298.

Author Contributions: Conception and design: N. Sullivan, K.M.Schoelles.Analysis and interpretation of the data: N. Sullivan, K.M. Schoelles.Drafting of the article: N. Sullivan.Critical revision of the article for important intellectual content: N.Sullivan, K.M. Schoelles.Final approval of the article: K.M. Schoelles.Obtaining of funding: K.M. Schoelles.Administrative, technical, or logistic support: N. Sullivan, K.M.Schoelles.Collection and assembly of data: N. Sullivan, K.M. Schoelles.

52. Kelleher AD, Moorer A, Makic MF. Peer-to-peer nursing rounds andhospital-acquired pressure ulcer prevalence in a surgical intensive care unit: aquality improvement project. J Wound Ostomy Continence Nurs. 2012;39:152-7. [PMID: 22415120]53. Ackerman CL. ‘Not on my watch:’ treating and preventing pressure ulcers.Medsurg Nurs. 2011;20:86-93. [PMID: 21560959]54. Institute for Healthcare Improvement. Prevent Pressure Ulcers. Cambridge,MA: Institute for Healthcare Improvement; 2007.




Appendix Table 1. Components of Pressure Ulcer Prevention Studies in U.S. Hospitals, 2000–2012

Study MultidisciplinaryTeam

SkinChampion

Education/Training

RiskAssessmentTool

ReviewWoundCareProducts

UpgradeAutomatedSystems(InformationTechnology)

ImplementProtocol

Patient CareInterventions

IntegrateNewReporting

AuditandFeedback

Kelleher et al, 2012 (52) � � � � � � RA; MM;F, S; N;RP; SS

� �

Ackerman 2011 (53)* � � � � � RA; MM;F, S; N;RP; SS

� �

Delmore et al, 2011 (41) � � � � � � RA; SE; MM;F, S; N;SS; PEd

�

Lynch and Vickery,2010 (39)

� � � � RA; SE; MM;RP; SS

� �

Young et al, 2010 (19) � � � � � � � RA; SE; MM;F, S; N;RP; SS

� �

Bales and Padwojski,2009 (28)†‡§

� � � � RA; SE; MM;N; RP; SS

�

Chicano and Droishagen,2009 (43)‡

� � � � � RA; MM;F, S; N;RP; SS

� �

Walsh et al, 2009 (38) � � � � � RA; SE; MM;N; RP; SS

Dibsie, 2008 (45) � � � � MM;SS � �McInerney, 2008 (29)† � � � � � � RA; RP; SS � �Ballard et al. 2008 (30)† � � � � � RA; MM;

F, S; RP;SS

� �

Catania et al,2007 (31)*†

� � � � RA; SE; MM;F, S; N;RP; SS

�

LeMaster, 2007 (47)‡§ � � � � RA; SE; MM;F, S; N;RP; SS

� �

Courtney et al,2006 (32)†

� � � � � RA; MM; N;RP; SS

� �

Gibbons et al,2006 (33)†§

� � � RA; MM; N;RP; SS

�

Hiser et al, 2006 (46) � � � RA; MM;F, S; N;RP; SS

�

Lyder et al, 2004 (26)* � � � RA; SE; N;RP; SS

�

Stier et al, 2004 (34)† � � � � � RA; SE; MM;F, S; RP;SS

�

F, S � interventions to reduce friction and shear on at-risk body areas; MM � moisture management (includes incontinence interventions and skin care products); N �nutrition; PEd � patient and family education; RA � risk assessment (usually Braden scale, typically with repeated assessments during hospital stay); RP � repositioning orincreasing activity/time out of bed when possible; SE � frequent skin examinations; SS � support surfaces (includes specialty beds and heel supports or heel elevation).* Audit only.† Reported a statistically significant reduction in pressure ulcer rates.‡ Reduced prevalence/incidence to 0.§ Describes use of incentives.



Appendix Table 2. Components of Pressure Ulcer Prevention Studies of Long-Term Care, 2000–2012

Study MultidisciplinaryTeam

Use ofOutsideConsultants

SkinChampion

Education/Training

NewAssessmentTool

UpgradeAutomatedSystems(InformationTechnology)

ImplementProtocol

FeaturedPatient CareInterventions

IntegrateNewReporting

AuditandFeedback

Horn et al,2010 (35)*†

� � � � � � � RA; SE; MM;N; RP

� �

Rantz et al,2010 (22)*

� � � RP

Milne et al,2009 (40)‡

� � � � � � � RA; MM; N;RP; SS

� �

Tippet,2009 (36)†

� � � � � � RA; SE; MM;F, S; N; RP;SS

� �

Rosen et al,2006 (37)†

� � � � RA; SE; RP;AHCPR

�

Abel et al,2005 (27)

� � � � � RA; SE; MM;N; RP; SS;PEd

Rantz et al,2001 (23)§

� � � RA; AHCPR � �

Ryden et al,2000 (24)†�

� � � RA; MM; F, S;RP

AHCPR � Agency for Health Care Policy and Research clinical practice guideline on pressure ulcer prediction and prevention; F, S � interventions to reduce friction andshear on at-risk body areas; MM � moisture management (includes incontinence interventions and skin care products); N � nutrition; PEd � patient and family education;RA � risk assessment (usually Braden scale, typically with repeated assessments during hospital stay); RP � repositioning or increasing activity/time out of bed when possible;SE � frequent skin examinations; SS � support surfaces (includes specialty beds and heel supports or heel elevation).* Study focused on improving communication of observations by nursing assistants using electronic documentation tools.† Reported a statistically significant reduction in pressure ulcer rates.‡ Long-term acute care hospital setting.§ Study focused on use of minimum data set–derived quality indicators for quality improvement efforts.� Study focused on involvement of advance practice nurses to improve a variety of quality issues.



Rapid-Response Systems as a Patient Safety StrategyA Systematic ReviewBradford D. Winters, MD, PhD; Sallie J. Weaver, PhD; Elizabeth R. Pfoh, MPH; Ting Yang, PhD; Julius Cuong Pham, MD, PhD;and Sydney M. Dy, MD, MSc

Rapid-response systems (RRSs) are a popular intervention inU.S. hospitals and are supported by accreditors and qualityimprovement organizations. The purpose of this review is toevaluate the effectiveness and implementation of these sys-tems in acute care settings. A literature search was performedbetween 1 January 2000 through 30 October 2012 usingPubMed, PsycINFO, CINAHL, and the Cochrane Central Reg-ister of Controlled Trials. Studies published in any languageevaluating outcome changes that occurred after implementingan RRS and differences between groups using and not usingan RRS (effectiveness) or describing methods used by RRSs(implementation) were reviewed.

A single reviewer (checked by a second reviewer) abstracted dataand rated study quality and strength of evidence. Moderate-strength evidence from a high-quality meta-analysis of 18 studiesand 26 lower-quality before-and-after studies published after thatmeta-analysis showed that RRSs are associated with reduced ratesof cardiorespiratory arrest outside of the intensive care unit andreduced mortality. Eighteen studies examining facilitators of andbarriers to implementation suggested that the rate of use of RRSscould be improved.


THE PROBLEM

Patients in the general ward often experience unrecog-nized deterioration that may progress to cardiorespiratoryarrest. Patients commonly show signs and symptoms ofdeterioration for hours or days before cardiorespiratory ar-rest (median time, 6 hours) (1). Such arrests are associatedwith a poor prognosis (mortality up to 80%).

Almost all cardiorespiratory arrests have a common setof antecedents that are often poorly recognized secondaryto the low sensitivity and fidelity of periodic assessments bystaff. Improving this process should lead to earlier recogni-tion and intervention. Many approaches have been devised(for example, single- and multiple-track and trigger systemsand weighted early warning scoring systems), but none hasbeen shown to have a clear advantage.

Even when recognition of deterioration is prompt, in-tervention may lag because of such barriers as a physician-centric medical culture that discourages speaking up orbypassing the chain of command, and imbalances betweenpatient and clinician needs and resources. Improving rec-ognition and overcoming the barriers to an effective andtimely response should reveal problems before they becomelife-threatening.

PATIENT SAFETY STRATEGY

Rapid-response systems (RRSs) were created to im-prove recognition of and response to deterioration of pa-tients on general hospital wards, with the goal of reducingthe incidence of cardiorespiratory arrest and hospital mor-tality. An RRS generally has 3 components.

1) Criteria and a system for notifying and activating theresponse team (known as an “afferent limb,” the mechanismby which team responses are triggered). Activation criteriausually include vital signs (single-trigger criteria vs. aggre-gate and weighted early warning scoring) or general con-cern expressed by a clinician or family member. The affer-

ent limb defines the variables that indicate deteriorationand democratizes that knowledge to all clinicians. It alsoempowers bedside clinicians to trigger the response team(or “efferent limb,” the team of clinicians that respond toan event) when the clinician has a suspicion that a patientis deteriorating (2). As such, most RRSs rely on cliniciansto proactively identify deteriorating patients rather thansolely on continuous monitoring technology, which iscommon in the intensive care unit (ICU).

2) The response team (efferent limb). The response teammost frequently comprises ICU-trained personnel andequipment. Team composition varies on the basis of localneeds and resources but generally uses one of the followingmodels: medical emergency teams (METs), which includea physician; rapid-response teams, which do not include aphysician; and critical care outreach teams, which followup on patients discharged from an ICU but also respond toall ward patients.

3) An administrative and quality improvement compo-nent. This team collects and analyzes event data and pro-vides feedback, coordinates resources, and ensures im-provement or maintenance over time.

Many hospitals have implemented RRSs to remedythe failure of our current system to adequately monitorpatients in the general ward, recognize the signs and symp-toms of deterioration, rescue deteriorating patients, anddeliver optimal care rapidly through escalation and triage.That RRSs should be able to improve outcomes has strongface validity. Given the rapid pace of RRS literature since the

See also:





last systematic review on the subject, done in 2010, we con-ducted this systematic review to update the current state of theevidence for RRS effectiveness and implementation.

REVIEW PROCESSES

PubMed, PsycINFO, CINAHL, and the CochraneCentral Register of Controlled Trials were searched be-tween January 2000 through October 2012. The Supple-ment (available at www.annals.org) describes the searchstrategies and includes the summary of evidence search andselection. Effectiveness articles were restricted to studiesthat used a MET, rapid-response team, or critical care out-reach team model; had a comparison group; and were pub-lished after November 2008 (the end date for the high-quality systematic review described later) (3).

Studies of RRS implementation selected for inclusioncould be either qualitative (that is, studies using interviews,focus groups, or ethnographic methods) or quantitative(that is, studies examining implementation strategy on useof the RRS or patient outcomes that provided numericaloutcome data). There were no exclusions based on countryor language.

Two reviewers independently screened all abstracts.Full articles identified for inclusion had outcome data ab-stracted by a single reviewer and checked by a second re-viewer. We did not abstract data on nursing satisfaction,which was rarely reported. The strength of evidence, in-cluding risk of bias, was evaluated using the Grading ofRecommendations Assessment, Development and Evalua-tion Working Group criteria adapted by the Agency forHealthcare Research and Quality (4). We evaluated thequality of systematic reviews by using the assessment ofmultiple systematic reviews criteria (5).

Of 2560 abstracts captured by the search strategy,2321 were excluded during screening and 232 articles wereexcluded in 2 rounds of article screening. Forty-three arti-cles met the inclusion criteria (26 studies for effectiveness;17 studies for implementation).

For effectiveness, the main outcome variables wereadult and pediatric non-ICU cardiorespiratory arrest andadult and pediatric total hospital mortality. Studies thatprovided complete raw data (sample sizes and number ofevents both before and after the intervention periods) orrelative risk (RR) estimate and its 95% CI or RR estimatewith its associated accurate P value for any of these mainoutcomes were included. Studies that did not provide suf-ficient quantitative data were excluded.

Using data from each study, we were able to present orcalculate a risk ratio estimate, its logarithm, and the asso-ciated SE. The results are summarized as risk ratios with95% CIs and are shown in Figures 1 to 4, which includethe data from the high-quality systematic review (3) The95% CIs were computed and plotted in the log scale butlabeled in the original scale. We used R, version 2.15.1 (RFoundation for Statistical Computing, Vienna, Austria),for these analyses. For implementation, Table 2 of theSupplement shows qualitative summaries of individualstudies.


BENEFITS AND HARMS

BenefitsWe identified 7 systematic reviews of RRSs; however,

only 1 was rated as high quality and is described in detailhere (3). A second review addressed implementation and isdiscussed in the next section. The highest-quality system-atic review (3) (assessment of multiple systematic reviewscriteria score, 10 of 11) identified 16 studies (6–21)through November 2008 involving nearly 1.3 million hos-pital admissions.

The meta-analysis concluded that, among adults, im-plementation of an RRS was associated with a statisticalreduction in non-ICU cardiorespiratory arrest (RR, 0.66[95% CI, 0.54 to 0.80]) but not with lower total hospitalmortality (RR, 0.96 [CI, 0.84 to 1.09]). In children, im-plementation of an RRS was associated with statistical re-ductions in both non-ICU cardiorespiratory arrest (RR,0.62 [CI, 0.46 to 0.84]) and total hospital mortality (RR,0.79 [CI, 0.63 to 0.98]).

The review rated studies as high quality if they ad-justed for confounders and for time trends by using eitherconcurrent control groups or an interrupted time seriesdesign. Studies were rated as fair quality if they adjustedonly for confounding. Five of the 18 studies were rated ashigh quality, 2 as fair quality, and the rest as low quality.

Key Summary Points

Many hospitals have implemented rapid-response systems(RRSs) over the past 15 years to improve recognition ofand response to deteriorating patients in the general ward.

Moderate-strength evidence suggests that RRSs are associ-ated with reduced rates of cardiorespiratory arrest andmortality.

Important components of successful RRSs include criteriaand a system for notifying and activating the responseteam; a response team; and an administrative and qualityimprovement component to train staff, collect and analyzeevent data, provide feedback, coordinate resources, andensure improvement or maintenance over time.

Implementation issues are critical in RRSs because rates ofuse are often suboptimal secondary to various barriers thatcould be improved.

Supplement Rapid-Response Systems



In this updated review, we identified 26 additionaleffectiveness studies (22–47) that met our inclusion crite-ria and were published since the previous high-quality sys-tematic review. None used randomization in its methodol-ogy or had a concurrent control group; 2 studies includedmultiple centers. Three studies were done in pediatric hos-pitals. Most took place in the United States, Australia, orCanada, with only a few in Europe or Asia. Most studieswere conducted in teaching hospitals.

Almost no studies included any information on con-text, and no studies reported a theoretical or logic model.The number of included hospital admissions or dischargesin the studies ranged from 1920 to 277 717. All were ratedas low or moderate quality.

Most studies reported the main outcomes of non-ICUcardiorespiratory arrest and total hospital mortality; somestudies also reported variations on these outcomes, such asunexpected cardiorespiratory arrest or unexpected mortal-ity. From our included studies, Figures 1 to 4 show studiesproviding adequate data (29–47) along with the studiesincluded in the high-quality review.

Figure 1 shows studies describing adult non-ICU car-diorespiratory arrest. Nineteen of 20 studies reporting thisoutcome had point estimates favoring the intervention, 12

of which reached statistical significance. Figure 2 showsadult total hospital mortality. Eighteen of 23 studiesshowed favorable point estimates, 7 of which weresignificant.

Figure 3 shows pediatric non-ICU cardiorespiratoryarrest; all point estimates favored the RRS, and 3 of 7 weresignificant. Figure 4 shows pediatric total hospital mortal-ity, where all but 1 study (21) had point estimates favoringRRSs; however, only 2 of these findings were significant.

More recent studies more often showed positive re-sults. Although outcomes were heterogeneous—number ofhospital discharges during the study period, type of hospi-tal (size and teaching vs. nonteaching status), and RRSmodel—there was no clear correlation between interven-tion effectiveness and these characteristics (data notshown).

The overall strength of evidence was moderate whenthe high-quality systematic review (3) was included, butstrength of evidence in the additional studies identifiedsince 2008 was low to moderate. Risk of bias was high forall additional studies because of the before-and-after de-sign. Only a few studies accounted for differences in pa-tient populations over time or reported characteristics ofproviders in the 2 periods.

Figure 1. Studies that reported the outcome of non–intensive care unit adult cardiorespiratory arrest.

Study, Year (Reference)

Bristow et al, 2000 (6) (first hospitalization)

Bristow et al, 2000 (6) (second hospitalization)

Buist et al, 2002 (7)

Bellomo et al, 2003 (8)

DeVita et al, 2004 (9)

Kenward et al, 2004 (10)

Hillman et al, 2005 (19)

Jones et al, 2005 (11)

Dacey et al, 2007 (13)

Jolley et al, 2007 (29)

Offner et al, 2007 (30)

Thomas et al, 2007 (31)

Baxter et al, 2008 (14)

Bosch and de Jager, 2008 (47)

Chan et al, 2008 (20)

Hatler et al, 2009 (37)

Gerdik et al, 2010 (35)

Beitler et al, 2011 (33)

Sarani et al, 2011 (44) (adjusted)

Shah et al, 2011 (45)

Risk Ratio (95% CI)

0.88 (0.62–1.24)

1.00 (0.73–1.37)

0.50 (0.35–0.72)

0.35 (0.22–0.56)

0.81 (0.71–0.93)

0.92 (0.72–1.17)

0.94 (0.79–1.12)

0.47 (0.35–0.63)

0.39 (0.26–0.58)

0.86 (0.69–1.08)

0.32 (0.16–0.63)

0.52 (0.12–2.28)

0.61 (0.40–0.94)

0.50 (0.41–0.61)

0.59 (0.40–0.88)

0.68 (0.36–1.28)

0.26 (0.12–0.57)

0.49 (0.40–0.61)

0.57 (0.47–0.69)

0.97 (0.72–1.31)

Risk Ratio

0.05 0.10 0.25 4.000.50 1.00 2.00

SupplementRapid-Response Systems



Few studies attempted to control for secular trendsover time that could have affected outcomes. The 1 studythat accounted for such trends found that benefits in mor-tality and cardiorespiratory arrest rate remained after theywere adjusted for (33). No studies reported on or ac-

counted for other safety initiatives that may have influ-enced the outcomes.

No studies conducted blinded outcome assessment.Although mortality is an objective outcome, the other keyoutcome, incidence of cardiorespiratory arrest, can be de-

Figure 2. Studies that reported the outcome of total hospital adult mortality.


Bristow et al, 2000 (6) (first hospitalization)

Bristow et al, 2000 (6) (second hospitalization)

Buist et al, 2002 (7)

Bellomo et al, 2003 (8)

Kenward et al, 2004 (10)

Priestley et al, 2004 (12)

Hillman et al, 2005 (19)

Jones et al, 2005 (11)

Dacey et al, 2007 (13)

Jolley et al, 2007 (29)

Baxter et al, 2008 (14)

Chan et al, 2008 (20) (adjusted)

Campello et al, 2009 (34) (adjusted)

Konrad et al, 2010 (39) (adjusted)

Lighthall et al, 2010 (42)

Santamaria et al, 2010 (43)

Beitler et al, 2011 (33) (adjusted)

Laurens and Dwyer, 2011 (41)

Sarani et al, 2011 (44) (medicine)

Sarani et al, 2011 (44) (surgery)

Shah et al, 2011 (45)

Howell et al, 2012 (38) (adjusted)

Tobin and Santamaria, 2012 (46) (adjusted)

Risk Ratio (95% CI)

0.93 (0.77–1.12)

1.20 (1.00–1.43)

0.87 (0.76–1.01)

0.74 (0.70–0.79)

0.99 (0.91–1.07)

0.52 (0.32–0.85)

1.03 (0.83–1.27)

1.18 (1.10–1.27)

1.07 (0.87–1.31)

1.00 (0.74–1.36)

0.99 (0.85–1.15)

0.95 (0.81–1.11)

0.83 (0.64–1.07)

0.90 (0.84–0.97)

0.82 (0.62–1.09)

0.48 (0.41–0.56)

0.89 (0.82–0.97)

0.76 (0.66–0.87)

0.74 (0.68–0.80)

0.92 (0.80–1.05)

0.89 (0.79–1.00)

0.91 (0.82–1.01)

0.90 (0.88–0.92)

Risk Ratio

0.05 0.25 4.000.50 1.00 2.00

Figure 3. Studies that reported the outcome of non–intensive care unit pediatric cardiorespiratory arrest.


Brilli et al, 2007 (15)

Sharek et al, 2007 (16)

Zenker et al, 2007 (21)

Hunt et al, 2008 (18)

Hanson et al, 2009 (36)

Tibballs and Kinney, 2009 (17)

Anwar-ul-Haque et al, 2010 (32)

Risk Ratio (95% CI)

0.41 (0.20–0.86)

0.29 (0.13–0.65)

0.64 (0.47–0.87)

0.49 (0.20–1.20)

0.35 (0.10–1.24)

0.91 (0.50–1.64)

0.52 (0.12–2.26)

Risk Ratio

0.05 0.25 4.000.50 1.00




fined in numerous ways (for example, calling the codeteam vs. documented use of cardiopulmonary resuscita-tion) and is subject to bias, as are some of the otheroutcome variations reported (for example, unexpectedmortality).

Studies ideally should not report rates for the ICU andemergency departments because these patient populationsare rarely part of the exposure group; however, hospital-wide rates were often reported. One study (20) includedICU cardiorespiratory arrest in the analysis (total hospitalcardiorespiratory arrest); this study concluded that there wasno effect, although data presented on the non-ICU cardio-respiratory arrest rate showed a statistical improvement.

Such arrest rates are also affected by changes in patientcase-mix over time, frequency of do-not-resuscitate orders,and terminal illness. However, most studies did not ac-count for these potential confounders. Other outcomes re-ported, such as unanticipated ICU admissions, are indirectoutcomes. In addition, no studies compared RRSs withother interventions that may affect these outcomes, such asenhanced nurse–patient ratios or hospitalists. An unex-pected beneficial consequence was improved frequency andquality of end-of-life discussions with patients and theirfamilies.

HarmsPotential harms included “deskilling” of ward staff be-

cause of dependence on the RRS, inappropriate patientcare for other patients (decreased responsiveness of theusual team), staff conflict, diversion of ICU staff fromusual care in the ICU, and communication errors causedby introducing additional providers (2). Despite several ar-ticles discussing potential harms and unexpected conse-quences, neither the high-quality systematic review nor anyof the additional studies reported any quantitative data forthese variables.


Seventeen studies (10, 48–63) met our inclusion cri-teria for studies of the implementation processes surround-

ing RRSs. Eleven of these used quantitative methods pri-marily for evaluating the effect of a change in theimplementation process for an RRS program and 7 usedprimarily qualitative methods. Most implementation stud-ies were conducted in academic hospitals; however, severalstudies specifically detailed implementation in communityhospitals (10, 13, 14, 23).

Rapid-response systems have been implemented inseveral contexts and vary in composition, activation crite-ria, and implementation process. Strong external factorshave driven the implementation of RRSs in U.S. hospitals:The 2009 Joint Commission’s National Patient SafetyGoal 16 (64) and the Institute for Healthcare Improve-ment (65), as well as numerous other organizations, havecreated toolkits to help implement RRS interventions. De-spite these attempts to reduce variability in the implemen-tation process, our review found that implementation pro-cesses differed widely and that local needs and resourcestended to dominate the processes.

Education and promotion of the new service was oftena factor in preparing for implementation. For staff trainingand education, several studies introduced new staff, such asa nurse educator. Most studies indicated that implementa-tion processes explicitly included educational activities;however, such activities varied in the degree to which theywere strictly information-based or included dedicatedtraining and practice opportunities for RRS members orstaff. Such activities as simulation education and trainingwere uncommon. Most studies explicitly noted that cogni-tive aids, such as posters listing activation criteria, wereincluded.

During development of the afferent limb, various ob-jective criteria were used for calling the team and manyinterventions depended on nurses’ clinical judgment toactivate the team on the basis of subjective “worry” or“general concern” (58, 59). One study found that METhospitals in the MERIT (Medical Emergency Response In-tervention and Therapy) trial were 35 times more likely toactivate their emergency response team based on the “wor-

Figure 4. Studies that reported the outcome of total hospital pediatric mortality.


Brilli et al, 2007 (15)

Sharek et al, 2007 (16)

Zenker et al, 2007 (21)

Hanson et al, 2009 (36)

Tibballs and Kinney, 2009 (17)

Kotsakis et al, 2011 (40)

Risk Ratio (95% CI)

0.55 (0.14–2.10)

0.82 (0.71–0.95)

1.05 (0.74–1.50)

0.08 (0.01–1.03)

0.65 (0.56–0.75)

0.97 (0.84–1.12)

Risk Ratio

0.05 0.25 4.000.50 1.00




ried” criteria than control (activated by vital signs) hospi-tals (P � 0.001) (58).

A few studies also included systems for family or pa-tient initiation of the RRS team. One study showed im-proved outcomes only after family-initiated activation wasimplemented (35). No studies reported specific use of tech-nology (such as computerized alerts) to enhance RRS im-plementation. Some studies used single vital sign triggers,whereas others used early warning scoring systems. In mostreports, activation of the efferent limb was voluntary, al-though 1 study changed its program to mandatory activa-tion on the basis of alert criteria (59).

Most studies used METs as the efferent limb model,but several studies examined rapid-response teams or crit-ical care outreach teams (23); however, none were directlycompared. One of the very few studies to compare thesemodels studied a resident-led team and an attending-ledMET and found no difference in outcomes (24).

Many RRS implementation efforts have low utiliza-tion rates; that is, ward staff do not activate the team de-spite criteria for activation being met. Although the sys-tematic reviews that we identified did not address thisissue, several did so individually. Jones and colleagues (66)examined the barriers and facilitators that affecting nurses’activation of the RRS. The following 5 major themesemerged: adequate education on the RRSs’ purpose androle, clinical expertise, support by medical and nursingstaff, nurses’ familiarity with and advocacy for the patient,and nurses’ workload.

Other studies found changes in culture (that is, devel-opment of strong support for calling for help and lack ofcriticism or punishment for activating the team), knowl-edge of activation criteria, communication, teamwork, andperceptions about the team’s helpfulness to nurses and pa-tients to be important influences on utilization. Anotherfactor affecting utilization is the time since initial imple-mentation or duration of the RRS. For example, 1 study(67) specifically examined RRS processes over time andfound that the proportion of patients with delayed RRSactivation decreased as the RRS matured (40.3% vs.22.0%; P � 0.001). Other programs have tried variousstrategies to improve utilization (education, mandatory ac-tivation, and changing the activation criteria) (59–63).

Team structure may also influence utilization. For ex-ample, 1 study reported the effects of separating the overallemergency response system into 2 teams with different ac-tivation criteria and processes. Utilization increased sharply(15.7 to 24.7 activations per 1000 admissions; P � 0.001)after the changes were implemented (61).

Patient populations may also benefit differently fromRRSs. The high-quality systematic review concluded thatRRSs were associated with significantly reduced hospitalmortality in pediatric patients but not in adults (3). Onestudy that included 2 separate RRS teams showed an effectin a medical but not a surgical population (44). Most stud-ies to date were conducted in academic centers, although

nonacademic hospitals also frequently reported RRS suc-cess. Earlier studies that the high-quality systematic reviewreported were mainly from Australia and the United States;2 were in England, and 1 was in Canada (3). Since then,the number of countries reporting effectiveness data forRRSs has increased, but how differing national medicalcultures affect implementation and effectiveness of the in-tervention is unclear.

Finally, cost was not evaluated in the high-quality re-view (3) or in the additional articles that we reviewed.

DISCUSSION

The previous high-quality systematic review and meta-analysis found that, although RRSs were associated with astatistical reduction in rates of cardiorespiratory arrest out-side of the ICU among pediatric and adult patients, totalhospital mortality was not reduced in adults (3). Our up-date supports the previous conclusions, although the mostrecent studies were more likely to show positive results.

The high-quality review found the opposite in its cu-mulative analysis: Early studies tended to have more posi-tive results. In fact, in 7 sequentially published studies,starting with Kenward and associates’ study (10) in 2004and continuing to Chan and coworkers’ study (20) in2008, the point estimate of effect did not decrease below0.95.

After Chan and coworkers’ study (20) in 2008, allpoint estimates were less than 0.95. Potential explanationsfor this result include maturation of the intervention andimproved implementation strategies that may have led toimproved results in and across institutions. In addition,secular trends of total hospital mortality may have de-creased over time unrelated to the intervention, and fewstudies controlled for this, although 1 study that did foundit to not be the case (33).

Although the beneficial effects of RRSs are becomingclearer as the intervention is more universally applied, notall RRS programs realize these benefits. There are severalpotential explanations for this. An archaic model of patientmonitoring on general wards limits the afferent limb (the lowsensitivity of periodic visits by clinicians to identify deterio-rating patients) (1). Automating the identification of a de-teriorating patient through continuous monitoring and adirectly activated response team potentially would both im-prove sensitivity and fidelity and mitigate cultural barriers.

Optimal team composition and structure are un-known. Restricted financial resources may also affect theRRS’s ability to self-audit, evaluate events, and improvesystematically. Utilization rates are often reported to below. Creativity and maturation of the intervention are nec-essary to achieve ultimate long-term goals. Other factorsmay affect commonly measured outcomes, and severalmetrics (that is, total cardiorespiratory arrest) count pa-tients who are not exposed to the intervention. Staff andeducation themes mainly focus on information rather than




training. Barriers to effective recognition and response in-grained in the culture of medicine persist.

Given that 80% of patients who have an inpatientcardiorespiratory arrest die, several potential reasons mayexplain why results for mortality are less robust than thosefor arrests. For example, the measurement of cardiorespi-ratory arrest can be subjective. In addition, many arrestsoccur in terminally ill patients, and some studies of RRSshave found evidence for increased rates of do-not-resuscitate orders after RRS implementation. Some studieshave tried to account for this factor by measuring rates ofcardiorespiratory arrests that are “unexpected” or that oc-cur in patients without do-not-resuscitate orders, but accu-rately defining terminally ill patients is challenging andmay be subject to bias or measurement error.

Our review and the literature have limitations. Theupdated literature since 2008 includes low- to moderate-quality studies, and several studies have inconsistent find-ings across outcomes. The elements of the RRS, samplesize, and reporting of outcomes varied among these studies.All of the most recent studies have used a before-and-afterhistorically controlled design, which needs to be consideredcarefully because a recent evaluation of a multifaceted pa-tient safety program in the United Kingdom found statis-tical improvements in the before-and-after comparison butnot in the concurrent cohort controlled comparison (68),as the MERIT study (19) did.

In addition, we reviewed only “effectiveness” studiesthat reported raw data for mortality and cardiorespiratoryarrest. Also, the possibility of selective reporting and pub-lication bias cannot be excluded. For implementation stud-ies, few used formal qualitative methods, and these alsoaddressed various RRS types and study populations. Fi-nally, the relative effectiveness of RRSs compared withother interventions to identify and treat deteriorating pa-tients is unknown.

In summary, we found moderate strength of evidencethat RRSs improve outcomes from both a high-quality sys-tematic review through November 2008 and the additionalliterature published through October 2012. Our reviewalso identified key barriers and facilitators of effective RRSimplementation, which included staff acceptance and lead-ership of the RRS, rates of calling the RRS, and triggermechanisms.

Rapid-response systems have been described as a“band-aid” for a failed model of managing patients in thegeneral ward in hospitals (69). Although this interventionis beginning to help many hospitals increase recognition ofpatient deterioration and reduce preventable deaths, theyare unlikely to more universally improve these outcomesuntil we address the culture and system defects that con-tribute to the root of the problem. For now, RRSs seem tobe the best option.

From Johns Hopkins School of Medicine and Johns Hopkins University,Baltimore, Maryland.


Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof the Johns Hopkins University, Agency for Healthcare Research andQuality, or U.S. Department of Health and Human Services.

Financial Support: From the Agency for Healthcare Research and Qual-ity, U.S. Department of Health and Human Services (contract HHSA-290- 2007-10062I).

Potential Conflicts of Interest: Dr. Winters: Grant (money to institu-tion): Agency for Healthcare Research and Quality; Employment: JohnsHopkins University; Expert testimony: several law firms; Payment for lec-tures including service on speakers bureaus: 3M; Royalties: Lippincott. Dr.Weaver: Grant (money to institution): Agency for Healthcare Researchand Quality; Travel/accommodations/meeting expenses unrelated to activitieslisted: Improvement Science Research Network. Ms. Pfoh: Grant (moneyto institution): Agency for Healthcare Research and Quality. Dr. Dy:Grant (money to institution): Agency for Healthcare Research and Quality.All other authors have no disclosures. Disclosures can also be viewed atwww.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2568.

Requests for Single Reprints: Bradford D. Winters, MD, PhD, De-partment of Anesthesiology and Critical Care Medicine and ArmstrongInstitute for Patient Safety and Quality, Johns Hopkins UniversitySchool of Medicine, Zayed 9127, 1800 Orleans Street, Baltimore, MD21287; e-mail, [email protected].


References1. Buist MD, Jarmolowski E, Burton PR, Bernard SA, Waxman BP, AndersonJ. Recognising clinical instability in hospital patients before cardiac arrest or un-planned admission to intensive care. A pilot study in a tertiary-care hospital. MedJ Aust. 1999;171:22-5. [PMID: 10451667]2. Jones DA, DeVita MA, Bellomo R. Rapid-response teams. N Engl J Med.2011;365:139-46. [PMID: 21751906]3. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid ResponseTeams: A Systematic Review and Meta-analysis. Arch Intern Med. 2010;170:18-26. [PMID: 20065195]4. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, et al.AHRQ series paper 5: grading the strength of a body of evidence when compar-ing medical interventions—Agency for Healthcare Research and Quality and theEffective Health-Care Program. J Clin Epidemiol. 2010;63:513-23. [PMID:19595577]5. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al.Development of AMSTAR: a measurement tool to assess the methodologicalquality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [PMID:17302989]6. Bristow PJ, Hillman KM, Chey T, Daffurn K, Jacques TC, Norman SL,et al. Rates of in-hospital arrests, deaths and intensive care admissions: the effectof a medical emergency team. Med J Aust. 2000;173:236-40. [PMID:11130346]7. Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, NguyenTV. Effects of a medical emergency team on reduction of incidence of andmortality from unexpected cardiac arrests in hospital: preliminary study. BMJ.2002;324:387-90. [PMID: 11850367]




8. Bellomo R, Goldsmith D, Uchino S, Buckmaster J, Hart GK, Opdam H,et al. A prospective before-and-after trial of a medical emergency team.Med J Aust. 2003;179:283-7. [PMID: 12964909]9. DeVita MA, Braithwaite RS, Mahidhara R, Stuart S, Foraida M, SimmonsRL; Medical Emergency Response Improvement Team (MERIT). Use of med-ical emergency team responses to reduce hospital cardiopulmonary arrests. QualSaf Health Care. 2004;13:251-4. [PMID: 15289626]10. Kenward G, Castle N, Hodgetts T, Shaikh L. Evaluation of a medicalemergency team one year after implementation. Resuscitation. 2004;61:257-63.[PMID: 15172703]11. Jones D, Bellomo R, Bates S, Warrillow S, Goldsmith D, Hart G, et al.Long term effect of a medical emergency team on cardiac arrests in a teachinghospital. Crit Care. 2005;9:R808-15. [PMID: 16356230]12. Priestley G, Watson W, Rashidian A, Mozley C, Russell D, Wilson J, et al.Introducing Critical Care Outreach: a ward-randomised trial of phased introduc-tion in a general hospital. Intensive Care Med. 2004;30:1398-404. [PMID:15112033]13. Dacey MJ, Mirza ER, Wilcox V, Doherty M, Mello J, Boyer A, et al. Theeffect of a rapid response team on major clinical outcome measures in a commu-nity hospital. Crit Care Med. 2007;35:2076-82. [PMID: 17855821]14. Baxter AD, Cardinal P, Hooper J, Patel R. Medical emergency teams at TheOttawa Hospital: the first two years. Can J Anaesth. 2008;55:223-31. [PMID:18378967]15. Brilli RJ, Gibson R, Luria JW, Wheeler TA, Shaw J, Linam M, et al.Implementation of a medical emergency team in a large pediatric teaching hos-pital prevents respiratory and cardiopulmonary arrests outside the intensive careunit. Pediatr Crit Care Med. 2007;8:236-46. [PMID: 17417113]16. Sharek PJ, Parast LM, Leong K, Coombs J, Earnest K, Sullivan J, et al.Effect of a rapid response team on hospital-wide mortality and code rates outsidethe ICU in a children’s hospital. JAMA. 2007;298:2267-74. [PMID: 18029830]17. Tibballs J, Kinney S. Reduction of hospital mortality and of preventablecardiac arrest and death on introduction of a pediatric medical emergency team.Pediatr Crit Care Med. 2009;10:306-12. [PMID: 19307806]18. Hunt EA, Zimmer KP, Rinke ML, Shilkofski NA, Matlin C, Garger C,et al. Transition from a traditional code team to a medical emergency team andcategorization of cardiopulmonary arrests in a children’s center. Arch PediatrAdolesc Med. 2008;162:117-22. [PMID: 18250234]19. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al;MERIT study investigators. Introduction of the medical emergency team(MET) system: a cluster-randomised controlled trial. Lancet. 2005;365:2091-7.[PMID: 15964445]20. Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA.Hospital-wide code rates and mortality before and after implementation of arapid response team. JAMA. 2008;300:2506-13. [PMID: 19050194]21. Zenker P, Schlesinger A, Hauck M, Spencer S, Hellmich T, Finkelstein M,et al. Implementation and impact of a rapid response team in a children’s hospi-tal. Jt Comm J Qual Patient Saf. 2007;33:418-25. [PMID: 17711144]22. Benson L, Mitchell C, Link M, Carlson G, Fisher J. Using an advancedpractice nursing model for a rapid response team. Jt Comm J Qual Patient Saf.2008;34:743-7. [PMID: 19119728]23. Bader MK, Neal B, Johnson L, Pyle K, Brewer J, Luna M, et al. Rescue me:saving the vulnerable non-ICU patient population. Jt Comm J Qual Patient Saf.2009;35:199-205. [PMID: 19435159]24. Karvellas CJ, de Souza IA, Gibney RT, Bagshaw SM. Association betweenimplementation of an intensivist-led medical emergency team and mortality.BMJ Qual Saf. 2012;21:152-9. [PMID: 22190540]25. Medina-Rivera B, Campos-Santiago Z, Palacios AT, Rodriguez-CintronW. The effect of the medical emergency team on unexpected cardiac arrest anddeath at the VA Caribbean Healthcare System: a retrospective study. Critical Careand Shock. 2010;13:98-105.26. Rothberg MB, Belforti R, Fitzgerald J, Friderici J, Keyes M. Four years’experience with a hospitalist-led medical emergency team: an interrupted timeseries. J Hosp Med. 2012;7:98-103. [PMID: 21998088]27. Scherr K, Wilson DM, Wagner J, Haughian M. Evaluating a new rapidresponse team: NP-led versus intensivist-led comparisons. AACN Adv Crit Care.2012;23:32-42. [PMID: 22290088]28. Scott SS, Elliott S. Implementation of a rapid response team: a success story.Crit Care Nurse. 2009;29:66-75. [PMID: 19487782]

29. Jolley J, Bendyk H, Holaday B, Lombardozzi KA, Harmon C. Rapid re-sponse teams: do they make a difference? Dimens Crit Care Nurs. 2007;26:253-60. [PMID: 18090145]30. Offner PJ, Heit J, Roberts R. Implementation of a rapid response teamdecreases cardiac arrest outside of the intensive care unit. J Trauma. 2007;62:1223-7. [PMID: 17495728]31. Thomas K, VanOyen Force M, Rasmussen D, Dodd D, Whildin S. Rapidresponse team: challenges, solutions, benefits. Crit Care Nurse. 2007;27:20-7.[PMID: 17244856]32. Anwar-ul-Haque, Saleem AF, Zaidi S, Haider SR. Experience of pediatricrapid response team in a tertiary care hospital in Pakistan. Indian J Pediatr.2010;77:273-6. [PMID: 20177830]33. Beitler JR, Link N, Bails DB, Hurdle K, Chong DH. Reduction in hospital-wide mortality after implementation of a rapid response team: a long-term cohortstudy. Crit Care. 2011;15:R269. [PMID: 22085785]34. Campello G, Granja C, Carvalho F, Dias C, Azevedo LF, Costa-Pereira A.Immediate and long-term impact of medical emergency teams on cardiac arrestprevalence and mortality: a plea for periodic basic life-support training programs.Crit Care Med. 2009;37:3054-61. [PMID: 19770754]35. Gerdik C, Vallish RO, Miles K, Godwin SA, Wludyka PS, Panni MK.Successful implementation of a family and patient activated rapid response teamin an adult level 1 trauma center. Resuscitation. 2010;81:1676-81. [PMID:20655645]36. Hanson CC, Randolph GD, Erickson JA, Mayer CM, Bruckel JT, HarrisBD, et al. A reduction in cardiac arrests and duration of clinical instability afterimplementation of a paediatric rapid response system. Qual Saf Health Care.2009;18:500-4. [PMID: 19955465]37. Hatler C, Mast D, Bedker D, Johnson R, Corderella J, Torres J, et al. Imple-menting a rapid response team to decrease emergencies outside the ICU: one hospi-tal’s experience. Medsurg Nurs. 2009;18:84-90, 126. [PMID: 19489205]38. Howell MD, Ngo L, Folcarelli P, Yang J, Mottley L, Marcantonio ER,et al. Sustained effectiveness of a primary-team-based rapid response system. CritCare Med. 2012;40:2562-8. [PMID: 22732285]39. Konrad D, Jaderling G, Bell M, Granath F, Ekbom A, Martling CR.Reducing in-hospital cardiac arrests and hospital mortality by introducing a med-ical emergency team. Intensive Care Med. 2010;36:100-6. [PMID: 19760206]40. Kotsakis A, Lobos AT, Parshuram C, Gilleland J, Gaiteiro R, Mohseni-BodH, et al; Ontario Pediatric Critical Care Response Team Collaborative. Imple-mentation of a multicenter rapid response system in pediatric academic hospitalsis effective. Pediatrics. 2011;128:72-8. [PMID: 21690113]41. Laurens N, Dwyer T. The impact of medical emergency teams on ICUadmission rates, cardiopulmonary arrests and mortality in a regional hospital.Resuscitation. 2011;82:707-12. [PMID: 21411218]42. Lighthall GK, Parast LM, Rapoport L, Wagner TH. Introduction of a rapidresponse system at a United States veterans affairs hospital reduced cardiac arrests.Anesth Analg. 2010;111:679-86. [PMID: 20624835]43. Santamaria J, Tobin A, Holmes J. Changing cardiac arrest and hospitalmortality rates through a medical emergency team takes time and constant re-view. Crit Care Med. 2010;38:445-50. [PMID: 20029341]44. Sarani B, Palilonis E, Sonnad S, Bergey M, Sims C, Pascual JL, et al.Clinical emergencies and outcomes in patients admitted to a surgical versus med-ical service. Resuscitation. 2011;82:415-8. [PMID: 21242020]45. Shah SK, Cardenas VJ Jr, Kuo YF, Sharma G. Rapid response team in anacademic institution: does it make a difference? Chest. 2011;139:1361-7.[PMID: 20864618]46. Tobin AE, Santamaria JD. Medical emergency teams are associated withreduced mortality across a major metropolitan health network after two yearsservice: a retrospective study using government administrative data. Crit Care.2012;16:R210. [PMID: 23107123]47. Bosch FH, de Jager CPC. Number of resuscitations for in-hospital cardio-pulmonary arrests decreases after introduction of a medical emergency team.“The Arnhem experience.” Netherlands Journal of Critical Care. 2008;12:256-9.48. Calzavacca P, Licari E, Tee A, Egi M, Downey A, Quach J, et al. Theimpact of Rapid Response System on delayed emergency team activation patientcharacteristics and outcomes—a follow-up study. Resuscitation. 2010;81:31-5.[PMID: 19854557]49. Williams DJ, Newman A, Jones C, Woodard B. Nurses’ perceptions of howrapid response teams affect the nurse, team, and system. J Nurs Care Qual.2011;26:265-72. [PMID: 21248644]




50. Shapiro SE, Donaldson NE, Scott MB. Rapid response teams seen through theeyes of the nurse. Am J Nurs. 2010;110:28-34. [PMID: 20505459]51. Donaldson N, Shapiro S, Scott M, Foley M, Spetz J. Leading successfulrapid response teams: A multisite implementation evaluation. J Nurs Adm. 2009;39:176-81. [PMID: 19359889]52. Adelstein BA, Piza MA, Nayyar V, Mudaliar Y, Klineberg PL, Rubin G.Rapid response systems: a prospective study of response times. J Crit Care. 2011;26:635.e11-8. [PMID: 21703813]53. Cretikos MA, Chen J, Hillman KM, Bellomo R, Finfer SR, Flabouris A;MERIT Study Investigators. The effectiveness of implementation of the medicalemergency team (MET) system and factors associated with use during theMERIT study. Crit Care Resusc. 2007;9:206-12. [PMID: 17536993]54. Mackintosh N, Rainey H, Sandall J. Understanding how rapid responsesystems may improve safety for the acutely ill patient: learning from the frontline.BMJ Qual Saf. 2012;21:135-44. [PMID: 21972419]55. Shearer B, Marshall S, Buist MD, Finnigan M, Kitto S, Hore T, et al.What stops hospital clinical staff from following protocols? An analysis of theincidence and factors behind the failure of bedside clinical staff to activate therapid response system in a multi-campus Australian metropolitan healthcare ser-vice. BMJ Qual Saf. 2012;21:569-75. [PMID: 22626737]56. Soo S, Berta W, Baker GR. Role of champions in the implementation ofpatient safety practice change. Healthc Q. 2009;12 Spec No Patient:123-8.[PMID: 19667789]57. Chen J, Bellomo R, Hillman K, Flabouris A, Finfer S; MERIT StudyInvestigators for the Simpson Centre and the ANZICS Clinical Trials Group.Triggers for emergency team activation: a multicenter assessment. J Crit Care.2010;25:359.e1-7. [PMID: 20189754]58. Genardi ME, Cronin SN, Thomas L. Revitalizing an established rapid re-sponse team. Dimens Crit Care Nurs. 2008;27:104-9. [PMID: 18434864]59. Jones CM, Bleyer AJ, Petree B. Evolution of a rapid response system fromvoluntary to mandatory activation. Jt Comm J Qual Patient Saf. 2010;36:266-70, 241. [PMID: 20564888]

60. Peebles E, Subbe CP, Hughes P, Gemmell L. Timing and teamwork—anobservational pilot study of patients referred to a rapid response team with theaim of identifying factors amenable to re-design of a rapid response system. Re-suscitation. 2012;83:782-7. [PMID: 22209834]61. Jones DA, Mitra B, Barbetti J, Choate K, Leong T, Bellomo R. Increasingthe use of an existing medical emergency team in a teaching hospital. AnaesthIntensive Care. 2006;34:731-5. [PMID: 17183890]62. Foraida MI, DeVita MA, Braithwaite RS, Stuart SA, Brooks MM,Simmons RL. Improving the utilization of medical crisis teams (Condi-tion C) at an urban tertiary care hospital. J Crit Care. 2003;18:87-94.[PMID: 12800118]63. Jones D, Bates S, Warrillow S, Goldsmith D, Kattula A, Way M, et al.Effect of an education programme on the utilization of a medical emergency teamin a teaching hospital. Intern Med J. 2006;36:231-6. [PMID: 16640740]64. Joint Commission on Accreditation of Healthcare Organizations.2008 National Patient Safety Goals. Joint Commission Perspectives. 2007;27:10-22.65. Institute for Healthcare Improvement. 5 Million Lives Campaign: Over-view. Accessed at www.ihi.org/offerings/Initiatives/PastStrategicInitiatives/5MillionLivesCampaign/Pages/default.aspx on 28 November 2012.66. Jones L, King L, Wilson C. A literature review: factors that impact on nurses’effective use of the Medical Emergency Team (MET). J Clin Nurs. 2009;18:3379-90. [PMID: 20487489]67. Buist M, Harrison J, Abaloz E, Van Dyke S. Six year audit of cardiac arrestsand medical emergency team calls in an Australian outer metropolitan teachinghospital. BMJ. 2007;335:1210-2. [PMID: 18048504]68. Benning A, Dixon-Woods M, Nwulu U, Ghaleb M, Dawson J, Barber N,et al. Multiple component patient safety intervention in English hospitals: con-trolled evaluation of second phase. BMJ. 2011;342:d199. [PMID: 21292720]69. Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304:1375-6. [PMID: 20858881]




Current Author Addresses: Drs. Winters, Weaver, Yang, and Pham:Department of Anesthesiology and Critical Care Medicine and Arm-strong Institute for Patient Safety and Quality, Johns Hopkins Univer-sity School of Medicine, 750 East Pratt Street, 15th Floor, Baltimore,MD 21231.Ms. Pfoh and Dr. Dy: Department of Health Policy and Management,Johns Hopkins University, Hampton House, Room 609, 624 NorthBroadway, Baltimore, MD 21205.

Author Contributions: Conception and design: B.D. Winters, S.J.Weaver, J.C. Pham, S.M. Dy.Analysis and interpretation of the data: B.D. Winters, T. Yang, J.C.Pham, S.M. Dy.

Drafting of the article: B.D. Winters, S.J. Weaver, E.R. Pfoh, J.C. Pham,S.M. Dy.Critical revision of the article for important intellectual content: B.D.Winters, J.C. Pham, S.M. Dy.Final approval of the article: B.D. Winters, S.J. Weaver, E.R. Pfoh, J.C.Pham, S.M. Dy.Provision of study materials or patients: B.D. Winters,Statistical expertise: T. Yang, J.C. Pham.Obtaining of funding: S.M. Dy.Administrative, technical, or logistic support: E.R. Pfoh.Collection and assembly of data: B.D. Winters, S.J. Weaver, E.R. Pfoh,S.M. Dy.




Simulation Exercises as a Patient Safety StrategyA Systematic ReviewEric Schmidt, BA; Sara N. Goldhaber-Fiebert, MD; Lawrence A. Ho, MD; and Kathryn M. McDonald, MM

Simulation is a versatile technique used in a variety of health caresettings for a variety of purposes, but the extent to which simula-tion may improve patient safety remains unknown. This systematicreview examined evidence on the effects of simulation techniqueson patient safety outcomes. PubMed and the Cochrane Librarywere searched from their beginning to 31 October 2012 to identifyrelevant studies. A single reviewer screened 913 abstracts and se-lected and abstracted data from 38 studies that reported outcomesduring care of real patients after patient-, team-, or system-levelsimulation interventions. Studies varied widely in the quality of

methodological design and description of simulation activities, butin general, simulation interventions improved the technical perfor-mance of individual clinicians and teams during critical events andcomplex procedures. Limited evidence suggested improvements inpatient outcomes attributable to simulation exercises at the healthsystem level. Future studies would benefit from standardized re-porting of simulation components and identification of robust pa-tient safety targets.


THE PROBLEM

It is well-known, both in medical practice and in otherprofessions, that error rates decrease with experience (1).Yet, an important challenge is how to train physicians andensure that they maintain competency while minimizingthe potential for patient harm. Many medical educatorsnow regard the traditional medical training model “seeone, do one, teach one” as unstructured and inadequate(2). In contrast, simulation exercises allow patient-safe cli-nician training. Although all physicians must eventuallyperform procedures on and manage critical events for anactual patient for the first time, simulation can make initialinteractions with patients safer.

Clinical expertise and mastery within a specialty donot increase simply as a function of experience (3), andlikewise, patient safety issues are not likely to decrease sim-ply as a function of more practice hours. Deliberate prac-tice, or practice that includes reflection on performance,increases mastery (4), and simulation exercises offer oppor-tunities for deliberate practice with flexibility to adjust pro-cedure complexity and provide regular practice for raretreatments. Experienced clinicians also must maintain pro-ficiency in a wide array of skills, most of which are knownto deteriorate over time without practice (5). Simulationcan serve to maintain clinical skills and may be part ofmaintenance of board certification, as is the case for theAmerican Board of Anesthesiology (6).

The versatility of simulation techniques affords manypotential benefits to those working to improve patientsafety. First, simulation is designed to match user needsand has been associated with increased technical perfor-

mance (7) and knowledge acquisition and clinical reason-ing (8). Second, simulation can replicate rare, complex, orhigh-stakes scenarios known to affect individual and teamperformance (9, 10). Third, mistakes are not only allowedin simulation, they enhance learning through reflectionand debriefing (11, 12). Fourth, new technologies or pro-cedures may be tested in simulation before implementationin real time with real patients (13). Finally, teams can sim-ulate patient care flow in situ for critical events (14) oradequacy of new facilities and equipment (15).

However, studies evaluating the relationship betweenthese benefits and patient safety outcomes, including po-tential harms, have not been thoroughly evaluated. Thepurpose of this systematic review is to examine evidence onthe benefits and harms of using simulation to improvepatient safety in medicine.


Research demonstrating the benefits of simulationcomes from studies about simulation as well as studies us-ing simulation (16). Research about simulation directly ex-amines the effect of a simulation technique as an interven-tion on behaviors and actions at the health professional orteam level that could directly improve patient safety if thattraining were widely implemented. In contrast, studies us-ing simulation harness these techniques as a laboratory toinvestigate new technologies and human performance forinsights into potential causal pathways to improve safety.

Simulation is used along the translational pathwayfrom health care provider actions in simulated “laboratory”contexts to similar actions in clinical settings to patientoutcomes (17). As such, simulation is considered a tech-nique rather than any 1 specific technology (18).

Simulation to enhance patient safety has 4 generalpurposes: education (for example, in transitioning traineesfrom content knowledge to experiential practice, and incontinuing education); assessment (for example, in qualitycontrol or quality improvement, or usability testing); re-

See also:





search (for example, regarding clinician behaviors) andhealth system integration (for example, team processes)(19). These purposes are not mutually exclusive, and eachmay span a range of complexity. A classic low-fidelity ex-ample of partial task training is simulation of intramuscu-lar medication administration by inserting a needle into anorange. Individual dynamic medical management exercisesmay include high-fidelity simulations that utilize anatomi-cally accurate mannequins and vital sign monitors. Patientsafety may also be enhanced through full scenario teammanagement, in which a human patient simulator and afully simulated care environment, such as entire operatingrooms or emergency department bays, are utilized.

On a basic level, simulations improve patient safety byallowing physicians to become better trained without put-ting patients at risk and, importantly, by providing pro-tected time for reflection and debriefing—where most ofthe learning takes place (11, 12). The challenge is match-ing the best simulation method to the desired learningobjectives while recognizing the costs of each method (18).Because simulation is a broad technique, faculty trainingand time are often a more important investment than arespecific expensive simulation equipment. Practitionersmust be appropriately trained to effectively use simulationtechniques, as well as any specific technologies, to accom-plish the relevant training, assessment, or systems probinggoals.

The simulation needs to feel real enough for partici-pants to be able to suspend disbelief, enabling them to feel,think, and act much as they would in a real scenario (12,19). If the learning objective is mainly to practice cognitiveskills for diagnosis or treatment, a verbal simulation, suchas, “What would you do if . . . ,” may be sufficient. Incontrast, if development of management skills, such as sit-uational awareness or team communication, is the focus, amore accurate replication of the actions and team presencebecome important for the simulation experience.

REVIEW PROCESSES

Methodology to capture literature in this review in-volved 3 mechanisms. First, we used structured searchstrategies to search PubMed and the Cochrane Libraryfrom their beginning to 31 October 2012. These searcheswere limited to meta-analyses; systematic reviews; and ran-domized, controlled trials (RCTs) or observational studiespublished in English. Second, practitioners with expertisein simulation provided recommendations on key articles,including issues in implementation and empirical researchon simulation and patient safety. Finally, the reference listsof articles captured by using the first 2 methods werescanned for relevant literature. Abstracts of references cap-tured by these searches (n � 913) were screened by a singlereviewer.

Studies, including systematic reviews and meta-analyses, were included in the review if they reported eval-

uative results of patient outcomes or changes in clinicianactions in patient care. Studies that only providedlaboratory-based results were excluded.

Data from the 38 studies that met the inclusion crite-ria were abstracted by a single investigator. Given the var-ied nature of the included studies and the broad area ofsimulation, quality assessment of individual studies waslimited to reporting study design. Selected studies are de-scribed with narrative synthesis. The Supplement (avail-able at www.annals.org) provides a complete description ofthe search strategies, article flow diagram, and evidencetables.


BENEFITS AND HARMS

Of the 38 included studies, 22 were RCTs, 11 wereprospective observational studies, and 5 were retrospectiveanalyses of previous simulation interventions. Table 1shows the distribution of study methodology and aspects ofsimulation interventions by targeted areas for improvementin patient safety. Thirty-four studies reported patient out-comes from care provided by trainees at varied levels ofeducation or specialties; postgraduate residents and fellowswere highly represented. Of the 27 studies that specified asetting for the simulation, academic medical settings pre-dominated (n � 23).

Diagnostic ProceduresFive RCTs and 1 prospective before–after study on

training for colonoscopy and upper gastrointestinal endos-copy found better initial performance in actual patientswhen physicians received simulation-based training (20–25). Studies generally reported a similar training periodrequirement to ultimately reach desirable levels of proce-dure mastery (20–25). Safety outcomes focused primarily

Key Summary Points

Simulation is a versatile technique that may be applied inpatient safety strategies across a variety of factors thatcontribute to patient harm.

Heterogeneous evidence across multiple topic areas showsthat training with simulation-based exercises increasestechnical and procedural performance.

Heterogeneous evidence shows that simulation-basedexercises can improve team performance and interper-sonal dynamics.

Limited evidence suggests that improvements in patientoutcomes attributable to simulation exercises can occurat the health system level.

SupplementSimulation Exercises as a Patient Safety Strategy



on patient discomfort (for example, insufflation). Simula-tion training was associated with less discomfort in 1 study(20), no difference in another (21), and greater patientdiscomfort in a third study (24). No critical patient safetyevents or major complications were reported. A systematicreview (26) that addressed virtual reality–based simulationfor endoscopy also found no studies that reported majorcomplications or critical patient safety events.

In a prospective randomized mixed-methods study onsimulation training for bronchoscopy, there was no ob-served difference in procedure time between participantswho did and did not have simulation training (27). An-other study showed that training for thoracentesis was as-sociated with fewer pneumothoraces and procedures ad-vancing to thoracostomy when coupled with simulation(28). Finally, cordocentesis procedure time was shorter andsuccess rate was higher with simulation training, althoughthere were no statistically significant differences inprocedure-related fetal loss or overall fetal loss (29).

Surgical ProceduresA meta-analysis of laparoscopic training with virtual

reality simulators reported that procedure time was nofaster but was more accurate among simulation-trained cli-nicians than traditional video-trained clinicians (standard-ized mean difference, 0.68 [95% CI, 0.05 to 1.31]) (30).Simulation training for laparoscopic cholecystectomy wasassociated with improved performance, 3-fold fewer errors,

an 8-fold decreased variation in error making (31), andincreased “respect for tissue” during the procedure (32–34). Laparoscopic simulation practice improved globalscores on the Objective Structured Assessment of Techni-cal Skills (OSATS) during cholecystectomies (35). Simula-tion training for extraperitoneal hernia repair was associ-ated with increased individual OSATS item scores forknowledge of procedure, knowledge of instruments, anduse of assistants, but this association was not significantwhen these individual item scores were aggregated into aglobal OSATS score (36). Cataract surgeries performed byresidents trained with simulation had a lower rate of sen-tinel complications than did surgeries performed by resi-dents who were trained before simulation was imple-mented (37). Finally, faster procedures and improvedperformance during prostate resection was observed amongphysicians trained with simulation (38).

Central Venous CatheterizationA recent meta-analysis of RCTs and observational

studies (39) showed that simulation-based education incentral venous catherization techniques improved learneroutcomes and performance during actual procedures. Forexample, simulation-based education resulted in fewer nee-dle passes (standardized mean difference, �0.58 [CI,�0.95 to �0.20]) and reduced pneumothoraces (relativerisk, 0.62 [CI, 0.40 to 0.97]) (39).

Table 1. Number of Studies, by Study Characteristics and Intervention Components

Study Characteristic Study Category, n TotalStudies, n

DiagnosticProcedures

SurgicalProcedures

Central VenousCatheterization

Other Proceduresand Processes

Team andSystems Studies

Total studies 9 8 12 3 6 38

Study designRCT 5 5 6 3 3 22Prospective* 3 2 5 0 1 11Retrospective* 1 1 1 0 2 5

Primary target of simulation†‡Procedural 9 8 12 2 4 35Medical management 0 0 0 1 5 6Team training 0 1 0 0 5 6

Components of simulation‡§Didactic component 7 6 11 3 4 31Demonstrate technique 5 3 11 3 4 26Deliberate practice 5 7 8 2 4 26Practice time 4 1 2 0 1 8Debriefing 3 4 5 1 5 18Feedback 5 1 4 1 0 11

RCT � randomized, controlled trial.* Includes pre–post, before–after, case–control, and other mixed methods or observational designs.† Categories represent a set of general categories created to represent targets that could be abstracted with confidence from the literature base. Categories are not consideredexhaustive of possible applications of simulation techniques.‡ Characteristics are not mutually exclusive in this category.§ Components of simulation-based interventions were developed post hoc as properties common to all studies included in the review. These are not considered exhaustiveof the possible components of simulation. The Supplement (available at www.annals.org) contains more detailed information on each intervention component category. TheDiscussion section of the article includes recommendations on future reporting of components.

Supplement Simulation Exercises as a Patient Safety Strategy



Several RCTs and observational studies that we re-viewed confirmed that simulation-based education im-proved performance (40–51). Two prospective studies and1 RCT reported that simulation training decreased rates ofcatheter-related bloodstream infection (41, 46, 49), but 1prospective controlled cohort study reported no differencein rates (48) attributable to simulation-based training.Studies showed mixed results for other major complica-tions and critical patient safety events.

Other Procedures and ProcessesThree RCTs reported data on other procedures and

processes. In 1 RCT, a simulation-based training curricu-lum for pediatric residents using high-fidelity models wasassociated with non–statistically significant increases inperformance of basic clinical procedural skills, such as bag–mask ventilation, venipuncture, peripheral venous catheterplacement, and lumbar puncture (52). Bachelor’s-levelnursing students made fewer medication administration er-rors in external training rotations when simulation trainingwas added to coursework (53). Among paramedic students,simulation-based training did not lead to improved perfor-mance during their first 15 intubations in terms of overallsuccess rate, success rate on first attempt, or complications(54).

Team and Systems PerformanceResearchers retrospectively investigated the effect of an

annual mandatory 1-day workshop and training programfor all midwifery (including community-based practitio-ners) and obstetric emergency staff in a tertiary care center(55). The workshop used simulation exercises for 7 com-mon obstetric emergencies: shoulder dystocia, postpartumhemorrhage, eclampsia, delivery of twins, breech presenta-tion, adult resuscitation, and neonatal resuscitation. Com-pared with the 2-year period before the training programwas implemented, there was a statistically significant de-crease in the rate of births with 5-minute Apgar scores of 6or less and hypoxic–ischemic encephalopathy in the 2-yearperiod after implementation. The decrease in rate of mod-erate to severe hypoxic–ischemic encephalopathy only ap-proached statistical significance.

In an RCT (56), primary care physicians in a largemultidisciplinary medical group were randomly assigned to1 of 3 groups: no simulation control, simulation alone, orsimulation combined with a physician leader program. Thesimulation training provided a series of interactive virtualencounters with patients who had newly diagnosed diabe-tes or who had indicated or contraindicated adjustments totheir insulin regimen. When combined for comparisonwith the control group, physicians in the simulation groupsprescribed renal-contraindicated metformin significantlyless often to patients with diabetes. In another RCT (57),residents who participated in full-scenario simulation train-ing for elective coronary artery bypass graft surgery hadincreased Anesthesiologists’ Nontechnical Skills Assess-

ment scores, and this difference was observed at 5-weekfollow-up.

Three studies reported patient outcomes aftersimulation-based training for resuscitation teams (14, 58,59). An RCT (58) reported no differences attributable tosimulation training for actual team performance on rates ofventilation, return of spontaneous circulation, or survivalto discharge. However, a prospective before–after study ex-amined resuscitation outcomes after implementation of theTeamSTEPPS team-building program coupled with simu-lation (59). This study reported several improvements incommunication, as well as reductions in time to computedtomography, intubation, and the operating room. Finally,in a retrospective case–control study (14), simulationtraining was associated with a higher correct response ratebased on the American Heart Association standards forresuscitation.

HarmsStudies generally provided additive or supplemental

interventions to training as usual, and no study reporteddata indicating increased potential for or actual harm topatients that resulted from implementing simulation tech-niques. However, it is conceivable that simulation exerciseswould place demand on valuable resources that could beapplied elsewhere in patient safety efforts. We found noevaluations of such considerations.


The Context for SimulationA meta-analysis (8) of simulation in education pro-

grams for health professionals found that 564 of 609 stud-ies (92.6%) examined techniques provided through dedi-cated simulation centers. Thirty-four studies (5.6%)examined simulation in situ, and 11 studies (1.8%) re-ported from both contexts. Among studies cited in ourreview, academic medical systems and academically affili-ated hospitals predominated (21–23, 27–29, 35, 36, 38,41–46, 49–54, 58, 60, 61). However, studies also reportedoutcomes of use of simulation in tertiary care facilities (25,45, 50, 58), trauma centers (44, 59), and multispecialtymedical groups (55, 56). We found no reported data onthe effect of context on the effectiveness of simulation ex-ercises for improving patient safety.

Implementing SimulationGaba (18) conceptualized a framework for simulation

techniques that may aid implementation and ultimatelyenhance patient safety (Table 2). The framework includes11 dimensions that form a comprehensive set of consider-ations proposed to enhance the development and the effec-tiveness of simulation exercises. Application of each dimen-sion guides specification and decision making on criticalchoices about the simulation exercise. In practice, objec-tives of implementing simulation are aligned with theneeds of learners and the goals of trainers from level of




participation to training in the particular simulation tech-nique. In addition, sufficient time for creating meaningfulexercises with debriefing, equipment matched to the sim-ulation need that recreates sufficient realism, and adequatespace or storage for in situ simulations will increase thelikelihood of success (18). Resources used in simulationsmust be available when needed and kept safe from beingused inappropriately for patient care (for example, expiredmedications). Technical support and maintenance may berequired for complex or high-tech simulators.

Rosen and colleagues (19) highlighted the importanceof cognitive fidelity (vs. physical fidelity) in a simulatedexercise: Simulations that engage the participant in waysthat cognitively best reflect the actual task are likely to bemore effective. Debriefing is considered crucial when im-plementing simulation requires instructor training (11, 12)and is considered a best practice in simulation-based med-ical education (62).

CostsThe cost of implementing simulation exercises ranges

from low to high, depending on the type of exercise andpersonnel and equipment resources involved (18). Instruc-tor and learner time are likely to be the most expensive andcrucial aspects of simulation in the long run. Start-up costsfor a comprehensive simulation center may be accountedfor differently from ongoing costs for exercises, whichcomplicates the ability to categorize the expected cost forsimulation as a patient safety strategy. Unfortunately, re-search addressing cost savings attributable directly to sim-ulation remains sparse, although some studies have re-ported up to a 7-to-1 return on simulation costs throughreduction in hospital days for bloodstream infections (21,49).

DISCUSSION

Simulation has continued to gain momentum in pa-tient safety efforts in the past decade because it allows forexercising and improving aspects of health care delivery

without any known risks to patients. Simulation has beenused in patient safety for the purposes of education, assess-ment, research, and integration of system-level strategies.These efforts have been reported in the literature as re-search about simulation: that is, research evaluating thetranslation of simulation-based education to enhanced pa-tient safety. In contrast, other research has focused on us-ing simulation as a laboratory to discover potential leveragepoints for patient safety (16).

Our review found that studies reporting patient out-comes or systems of care have been done primarily in aca-demic settings, although researchers have used simulationin diverse clinical specialties, experience levels, and caresettings. These studies varied in terms of individual quality,but the majority were randomized or had methodologicallysound controlled prospective designs. Researchers havereplicated standardized simulation training for central ve-nous catherization, and although this approach is promis-ing for patient safety in that area, we did not find otherexamples of replication studies in our review. We also didnot find analysis of contextual effects on the validity ofsimulation to improve patient safety. The generalizabilityof any one technique is likely to vary according to manyfactors, such as those in Gaba’s 11-dimensional framework(18), and the adequacy of resources dedicated to simula-tion (for example, debriefing).

At this juncture, simulation seems to have a favorableeffect on quicker acquisition and improved performance oftechnical skills. Although not yet thoroughly studied, sim-ulation of complex or high-stakes procedures seems to be apromising technique to increase patient-safe behavior atthe clinician and team levels. Simulation has the potentialto enhance patient safety through structured assessmentand debriefing in quality improvement initiatives. It hasbeen used to assess practices that would be difficult orunsafe to study empirically in real time with actual pa-tients. Likewise, simulation has been endorsed for ongoingcompetency and continuing education, as well as advance-ment to mastery-level practices.

A previous systematic review (7) reported that simula-tion contributes to enhanced knowledge acquisition andimproved clinical performance. Simulation techniqueshave been used in translating results from the within-simulation laboratory to patient- and health care system–level outcomes (17). Another systematic review (4) sug-gested that protected time for debriefing in a learningexperience is a crucial component of simulation tech-niques. To our knowledge, our review is the first to exam-ine the effects that simulation exercises have on patientsafety outcomes, and in particular outcomes in patientsoutside of simulation laboratory settings (that is, duringclinical care).

Our review has limitations. First, it is possible that thebroad search strategies missed studies that may be capturedwith targeted and comprehensive strategies dedicated toeach simulation technique, clinical specialty, or applica-

Table 2. Eleven Dimensions to Consider When Designingand Setting Up Simulation Exercises*

Purpose and Participants Setup

Purpose and aims of simulationactivity

Age of the patient beingsimulated

Type of knowledge, skills, attitudes,or behaviors addressed insimulation

Technology applicable orrequired for simulations

Extent of direct participationSite of simulation: in situ

clinical setting vs.dedicated simulation center

Feedback methodaccompanying simulation

Unit of participation in the simulation:individuals or teams

Experience level of simulationparticipants

Health care domain in which thesimulation is applied

Health care disciplines of personnelparticipating in the simulation

* Adapted from reference 18.




tion. Second, given the relative infancy of the research onsimulation exercises, the field may be prone to selectivereporting of studies with positive findings, leading to po-tential publication bias. Finally, we limited our assessmentof quality of evidence to study design and did not performa structured assessment of the strength of evidence. There-fore, the overall strength of the evidence for simulationexercises to improve patient safety should be interpretedwith caution.

In conclusion, simulation is a versatile technique thatcontinues to gain momentum in a variety of clinical set-tings and applications, including patient safety strategies.Although evidence is largely heterogeneous at this time,our review suggests the potential for simulation exercises tocontribute to patient safety through increased technicaland procedural performance and improved team perfor-mance. Limited research using health system–level obser-vations suggests that simulation may enhance patientsafety, although more research is needed on the potentialfor simulation to contribute to system-level differences inpatient safety outcomes. Systematic reviews of simulationfor specific procedures have begun reporting patient safetyoutcomes (26, 30); more reviews of this nature would en-hance our understanding of the overall contribution ofsimulation techniques to patient safety. Future systematicreviews would benefit from investigators using a consistentframework, such as that developed by Gaba (18), to de-scribe the intervention and its context and implementation.

From Stanford Center for Health Policy/Center for Primary Care andOutcomes Research and Stanford University School of Medicine, Stan-ford University Hospital and Clinics, Stanford, California.

Note: The Agency for Healthcare Research and Quality reviewedcontract deliverables to ensure adherence to contract requirements andquality, and a copyright release was obtained from the Agency forHealthcare Research and Quality before the manuscript was submittedfor publication.

Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof Stanford University, the Agency for Healthcare and Quality, or theU.S. Department of Health and Human Services.

Financial Support: From the Agency for Healthcare and Quality, U.S.Department of Health and Human Services (contract HHSA-290-2007-10062I).

Potential Conflicts of Interest: Mr. Schmidt: Grant (money to institu-tion): Agency for Healthcare Research and Quality. Ms. McDonald:Grant (money to institution): Agency for Healthcare Research and Qual-ity. All other authors have no disclosures. Disclosures can also be viewed atwww.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2572.

Requests for Single Reprints: Kathryn M. McDonald, MM, StanfordUniversity, 117 Encina Commons, Stanford, CA 94305-6019; e-mail,[email protected].


References1. Meltzer D, Manning WG, Morrison J, Shah MN, Jin L, Guth T, et al.Effects of physician experience on costs and outcomes on an academic generalmedicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137:866-74. [PMID: 12458986]2. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: anethical imperative. Acad Med. 2003;78:783-8. [PMID: 19088599]3. Ericsson KA. Deliberate practice and acquisition of expert performance: ageneral overview. Acad Emerg Med. 2008;15:988-94. [PMID: 18778378]4. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. Effect of practice onstandardised learning outcomes in simulation-based medical education. MedEduc. 2006;40:792-7. [PMID: 16869926]5. Semeraro F, Signore L, Cerchiari EL. Retention of CPR performance inanaesthetists. Resuscitation. 2006;68:101-8. [PMID: 16325986]6. Park CS. Simulation and quality improvement in anesthesiology. AnesthesiolClin. 2011;29:13-28. [PMID: 21295750]7. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Doessimulation-based medical education with deliberate practice yield better resultsthan traditional clinical education? A meta-analytic comparative review of theevidence. Acad Med. 2011;86:706-11. [PMID: 21512370]8. Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in healthprofessions education: a systematic review and meta-analysis. Acad Med. 2010;85:1589-602. [PMID: 20703150]9. Blum RH, Raemer DB, Carroll JS, Dufresne RL, Cooper JB. A method formeasuring the effectiveness of simulation-based team training for improving com-munication skills. Anesth Analg. 2005;100:1375-80. [PMID: 15845689]10. Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb Y. Simulation-basedtraining in anesthesia crisis resource management (ACRM): a decade of experi-ence. Simul Gaming. 2001;32:175-93.11. Fanning RM, Gaba DM. The role of debriefing in simulation-based learn-ing. Simul Healthc. 2007;2:115-25. [PMID: 19088616]12. Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefingwith good judgment: combining rigorous feedback with genuine inquiry. Anes-thesiol Clin. 2007;25:361-76. [PMID: 17574196]13. Lipman S, Daniels K, Cohen SE, Carvalho B. Labor room setting comparedwith the operating room for simulated perimortem cesarean delivery: a random-ized controlled trial. Obstet Gynecol. 2011;118:1090-4. [PMID: 22015877]14. Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghieWC. Simulation-based education improves quality of care during cardiac arrestteam responses at an academic teaching hospital: a case-control study. Chest.2008;133:56-61. [PMID: 17573509]15. Lighthall GK, Poon T, Harrison TK. Using in situ simulation to improvein-hospital cardiopulmonary resuscitation. Jt Comm J Qual Patient Saf. 2010;36:209-16. [PMID: 20480753]16. Gaba DM. The future’s here. We are it. Simul Healthc. 2006;1:1-2. [PMID:19088617]17. McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D. Evalu-ating the impact of simulation on translational patient outcomes. Simul Healthc.2011;6 Suppl:S42-7.18. Gaba DM. The future vision of simulation in healthcare. Simul Healthc.2007;2:126-35.19. Rosen MA, Salas E, Tannenbaum SI, Provonost P, King HB. Simulation-based training for teams in health care: designing scenarios, measuring perfor-mance, and providing feedback. In: Carayon P, ed. Handbook of HFE in HealthCare and Patient Safety. 2nd ed. Boca Raton, FL: CRC Press; 2011:571-92.20. Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D. Virtualreality colonoscopy simulation: a compulsory practice for the future colonosco-pist? Endoscopy. 2005;37:1198-204. [PMID: 16329017]21. Cohen J, Cohen SA, Vora KC, Xue X, Burdick JS, Bank S, et al. Multi-center, randomized, controlled trial of virtual-reality simulator training in acqui-sition of competency in colonoscopy. Gastrointest Endosc. 2006;64:361-8.[PMID: 16923483]22. Ferlitsch A, Schoefl R, Puespoek A, Miehsler W, Schoeniger-Hekele M,Hofer H, et al. Effect of virtual endoscopy simulator training on performance of




upper gastrointestinal endoscopy in patients: a randomized controlled trial.Endoscopy. 2010;42:1049-56. [PMID: 20972956]23. Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S, et al.Randomized controlled trial of virtual reality simulator training: transfer to livepatients. Am J Surg. 2007;194:205-11. [PMID: 17618805]24. Sedlack RE, Kolars JC. Computer simulator training enhances the compe-tency of gastroenterology fellows at colonoscopy: results of a pilot study.Am J Gastroenterol. 2004;99:33-7. [PMID: 14687137]25. Thomson M, Heuschkel R, Donaldson N, Murch S, Hinds R. Acquisitionof competence in paediatric ileocolonoscopy with virtual endoscopy training.J Pediatr Gastroenterol Nutr. 2006;43:699-701. [PMID: 17130753]26. Walsh CM, Sherlock ME, Ling SC, Carnahan H. Virtual reality simulationtraining for health professions trainees in gastrointestinal endoscopy. CochraneDatabase Syst Rev. 2012;6:CD008237. [PMID: 22696375]27. Blum MG, Powers TW, Sundaresan S. Bronchoscopy simulator effectivelyprepares junior residents to competently perform basic clinical bronchoscopy.Ann Thorac Surg. 2004;78:287-91. [PMID: 15223446]28. Duncan DR, Morgenthaler TI, Ryu JH, Daniels CE. Reducing iatrogenicrisk in thoracentesis: establishing best practice via experiential training in a zero-risk environment. Chest. 2009;135:1315-20. [PMID: 19017865]29. Tongprasert F, Wanapirak C, Sirichotiyakul S, Piyamongkol W, TongsongT. Training in cordocentesis: the first 50 case experience with and without acordocentesis training model. Prenat Diagn. 2010;30:467-70. [PMID:20440735]30. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual realitytraining for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev.2009:CD006575. [PMID: 19160288]31. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C,McClusky DA 3rd, et al. Proficiency-based virtual reality training significantlyreduces the error rate for residents during their first 10 laparoscopic cholecystec-tomies. Am J Surg. 2007;193:797-804. [PMID: 17512301]32. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J,Funch-Jensen P. Randomized clinical trial of virtual reality simulation for lapa-roscopic skills training. Br J Surg. 2004;91:146-50. [PMID: 14760660]33. Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, et al.Laparoscopic training on bench models: better and more cost effective than op-erating room experience? J Am Coll Surg. 2000;191:272-83. [PMID: 10989902]34. Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM.Fundamentals of laparoscopic surgery simulator training to proficiency improveslaparoscopic performance in the operating room—a randomized controlled trial.Am J Surg. 2010;199:115-20. [PMID: 20103076]35. Calatayud D, Arora S, Aggarwal R, Kruglikova I, Schulze S, Funch-JensenP, et al. Warm-up in a virtual reality environment improves performance in theoperating room. Ann Surg. 2010;251:1181-5. [PMID: 20485133]36. Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV,et al. Improving operative performance using a laparoscopic hernia simulator.Am J Surg. 2001;182:725-8. [PMID: 11839347]37. Rogers GM, Oetting TA, Lee AG, Grignon C, Greenlee E, Johnson AT,et al. Impact of a structured surgical curriculum on ophthalmic resident cataractsurgery complication rates. J Cataract Refract Surg. 2009;35:1956-60. [PMID:19878829]38. Kallstrom R, Hjertberg H, Svanvik J. Impact of virtual reality-simulatedtraining on urology residents’ performance of transurethral resection of the pros-tate. J Endourol. 2010;24:1521-8. [PMID: 20677993]39. Ma IW, Brindle ME, Ronksley PE, Lorenzetti DL, Sauve RS, Ghali WA.Use of simulation-based education to improve outcomes of central venous cath-eterization: a systematic review and meta-analysis. Acad Med. 2011;86:1137-47.[PMID: 21785310]40. Andreatta P, Chen Y, Marsh M, Cho K. Simulation-based training improvesapplied clinical placement of ultrasound-guided PICCs. Support Care Cancer.2011;19:539-43. [PMID: 20306091]41. Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use ofsimulation-based education to reduce catheter-related bloodstream infections.Arch Intern Med. 2009;169:1420-3. [PMID: 19667306]42. Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Useof simulation-based mastery learning to improve the quality of central venouscatheter placement in a medical intensive care unit. J Hosp Med. 2009;4:397-403. [PMID: 19753568]

43. Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB.Simulation-based mastery learning reduces complications during central venouscatheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697-701. [PMID: 19885989]44. Britt RC, Novosel TJ, Britt LD, Sullivan M. The impact of central linesimulation before the ICU experience. Am J Surg. 2009;197:533-6. [PMID:19249739]45. Evans LV, Dodge KL, Shah TD, Kaplan LJ, Siegel MD, Moore CL, et al.Simulation training in central venous catheter insertion: improved performancein clinical practice. Acad Med. 2010;85:1462-9. [PMID: 20736674]46. Khouli H, Jahnes K, Shapiro J, Rose K, Mathew J, Gohil A, et al. Perfor-mance of medical residents in sterile techniques during central vein catheteriza-tion: randomized trial of efficacy of simulation-based training. Chest. 2011;139:80-7. [PMID: 20705795]47. Martin M, Scalabrini B, Rioux A, Xhignesse MA. Training fourth-yearmedical students in critical invasive skills improves subsequent patient safety. AmSurg. 2003;69:437-40. [PMID: 12769219]48. Miranda JA, Trick WE, Evans AT, Charles-Damte M, Reilly BM, ClarkeP. Firm-based trial to improve central venous catheter insertion practices. J HospMed. 2007;2:135-42. [PMID: 17549773]49. Sherertz RJ, Ely EW, Westbrook DM, Gledhill KS, Streed SA, Kiger B,et al. Education of physicians-in-training can decrease the risk for vascular cath-eter infection. Ann Intern Med. 2000;132:641-8. [PMID: 10766683]50. Smith CC, Huang GC, Newman LR, Clardy PF, Feller-Kopman D, ChoM, et al. Simulation training and its effect on long-term resident performance incentral venous catheterization. Simul Healthc. 2010;5:146-51. [PMID:20651476]51. Velmahos GC, Toutouzas KG, Sillin LF, Chan L, Clark RE, Theodorou D,et al. Cognitive task analysis for teaching technical skills in an inanimate surgicalskills laboratory. Am J Surg. 2004;187:114-9. [PMID: 14706600]52. Gaies MG, Morris SA, Hafler JP, Graham DA, Capraro AJ, Zhou J, et al.Reforming procedural skills training for pediatric residents: a randomized, inter-ventional trial. Pediatrics. 2009;124:610-9. [PMID: 19651582]53. Sears K, Goldsworthy S, Goodman WM. The relationship between simu-lation in nursing education and medication safety. J Nurs Educ. 2010;49:52-5.[PMID: 19810664]54. Hall RE, Plant JR, Bands CJ, Wall AR, Kang J, Hall CA. Human patientsimulation is effective for teaching paramedic students endotracheal intubation.Acad Emerg Med. 2005;12:850-5. [PMID: 16141019]55. Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S, et al.Does training in obstetric emergencies improve neonatal outcome? BJOG. 2006;113:177-82. [PMID: 16907952]56. O’Connor PJ, Sperl-Hillen JM, Johnson PE, Rush WA, Asche SE, DuttaP, et al. Simulated physician learning intervention to improve safety and qualityof diabetes care: a randomized trial. Diabetes Care. 2009;32:585-90. [PMID:19171723]57. Bruppacher HR, Alam SK, LeBlanc VR, Latter D, Naik VN, Savoldelli GL,et al. Simulation-based training improves physicians’ performance in patient carein high-stakes clinical setting of cardiac surgery. Anesthesiology. 2010;112:985-92. [PMID: 20234305]58. Weidman EK, Bell G, Walsh D, Small S, Edelson DP. Assessing the impactof immersive simulation on clinical performance during actual in-hospital cardiacarrest with CPR-sensing technology: a randomized feasibility study. Resuscita-tion. 2010;81:1556-61. [PMID: 20724057]59. Capella J, Smith S, Philp A, Putnam T, Gilbert C, Fry W, et al. Teamworktraining improves the clinical care of trauma patients. J Surg Educ. 2010;67:439-43. [PMID: 21156305]60. Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Masterylearning of temporary hemodialysis catheter insertion by nephrology fellows usingsimulation technology and deliberate practice. Am J Kidney Dis. 2009;54:70-6.[PMID: 19376620]61. Griswold-Theodorson S, Hannan H, Handly N, Pugh B, Fojtik J, Saks M,et al. Improving patient safety with ultrasonography guidance during internaljugular central venous catheter placement by novice practitioners. Simul Healthc.2009;4:212-6. [PMID: 21330794]62. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review ofsimulation-based medical education research: 2003-2009. Med Educ. 2010;44:50-63. [PMID: 20078756]




Current Author Addresses: Mr. Schmidt and Ms. McDonald: StanfordCenter for Health Policy/Center for Primary Care and Outcomes Re-search, Stanford University, 117 Encina Commons, Stanford, CA94305-6019.Drs. Goldhaber-Fiebert and Ho: Stanford University School of Medi-cine, Stanford University Hospital and Clinics, 291 Campus Drive,Stanford, CA 94305.

Author Contributions: Conception and design: E. Schmidt, S.N.Goldhaber-Fiebert, K.M. McDonald.

Analysis and interpretation of the data: E. Schmidt, S.N. Goldhaber-Fiebert, L.A. Ho, K.M. McDonald.Drafting of the article: E. Schmidt, S.N. Goldhaber-Fiebert, L.A. Ho,K.M. McDonald.Critical revision of the article for important intellectual content: E.Schmidt, S.N. Goldhaber-Fiebert, L.A. Ho, K.M. McDonald.Final approval of the article: S.N. Goldhaber-Fiebert, K.M. McDonald.Obtaining of funding: K.M. McDonald.Administrative, technical, or logistic support: E. Schmidt, K.M. McDonald.Collection and assembly of data: E. Schmidt, S.N. Goldhaber-Fiebert.




Hospital-Initiated Transitional Care Interventions as a PatientSafety StrategyA Systematic ReviewStephanie Rennke, MD; Oanh K. Nguyen, MD; Marwa H. Shoeb, MD; Yimdriuska Magan, BS; Robert M. Wachter, MD;and Sumant R. Ranji, MD

Hospitals now have the responsibility to implement strategies toprevent adverse outcomes after discharge. This systematic reviewaddressed the effectiveness of hospital-initiated care transition strat-egies aimed at preventing clinical adverse events (AEs), emergencydepartment (ED) visits, and readmissions after discharge in generalmedical patients. MEDLINE, CINAHL, EMBASE, and Cochrane Da-tabase of Clinical Trials (January 1990 to September 2012) weresearched, and 47 controlled studies of fair methodological qualitywere identified. Forty-six studies reported readmission rates, 26reported ED visit rates, and 9 reported AE rates. A “bridging”

strategy (incorporating both predischarge and postdischarge inter-ventions) with a dedicated transition provider reduced readmissionor ED visit rates in 10 studies, but the overall strength of evidencefor this strategy was low. Because of scant evidence, no conclusionscould be reached on methods to prevent postdischarge AEs. Moststudies did not report intervention context, implementation, or cost.The strategies hospitals should implement to improve patient safetyat hospital discharge remain unclear.


THE PROBLEM

Nearly 1 in 5 Medicare patients is readmitted within30 days of discharge from the hospital (1). This proportionhas not changed substantially over the past several years (2)despite intense efforts to improve the discharge process.Patients are vulnerable to a wide range of adverse events(AEs) after discharge, with more than 20% of medical pa-tients sustaining a preventable AE within 3 weeks of dis-charge (3). Multiple issues contribute to ineffective caretransitions, including poor communication between inpa-tient and outpatient clinicians (4); medication changesduring hospitalizations (5); inadequate patient understand-ing of diagnoses, medications, and follow-up needs (6);discharging patients with incomplete diagnostic work-ups(7); and other, more general patient-related and health caresystem–related factors (8–10).

Several policy initiatives have recently been imple-mented to encourage improvements in transitional care.The Centers for Medicare & Medicaid Services publiclyreports hospitals’ risk-adjusted 30-day readmission rates forpatients hospitalized with pneumonia, acute myocardial in-farction, or congestive heart failure (11). The Centers re-cently announced that more than 2000 hospitals will sufferfinancial penalties of up to 1% of Medicare reimburse-ments because of high readmission rates (12). The Partner-ship for Patients initiative aims to decrease preventable re-admissions by 20% by the end of 2013 and has identifiedimproving transitional care as an opportunity to reducehealth care expenditures (13). Together, these policies con-stitute a mandate to hospitals to improve transitional careat hospital discharge.

Little information is available on effective transitionalcare strategies for general medical inpatients. Prominentnational organizations have recommended a range of inter-ventions (14), which are being implemented widely. How-

ever, little evidence supports their effect on readmissions orother important markers of postdischarge patient safety,such as emergency department (ED) visits and AEs occur-ring shortly after discharge. Moreover, a recent review (15)identified no interventions proven to reduce 30-day read-mission rates in general patient populations, although itdid not focus on hospital-initiated interventions. Becausefinancial penalties place the onus on hospitals to be primar-ily responsible for implementation of strategies to preventadverse outcomes after discharge, we conducted a system-atic review of the effectiveness of hospital-initiated caretransition interventions on reducing AEs, ED visits, andreadmissions after discharge in general medical patients.


We defined a “transitional care strategy” as 1 or agroup of interventions initiated before hospital dischargewith the aim of ensuring the safe and effective transition ofpatients from the acute inpatient setting to home. To syn-thesize a variety of published interventions, we classifiedspecific interventions on the basis of an existing taxonomyof transitional care interventions (16–21). We groupedtransitional care strategies into 3 categories according tothe timing and setting of intervention components: predis-charge, postdischarge, and “bridging” (including both pre-and postdischarge components) (Table 1) (15).

We defined postdischarge AEs as any of the followingpatient experiences—all representing clinically meaningful

See also:





injuries from medical care—occurring after hospital dis-charge: new or worsening symptoms, laboratory abnormal-ities (such as elevated international normalized ratio) ne-cessitating a change in clinical management, and injuries(such as adverse drug events, falls, or hospital-acquired in-fections) attributable at least in part to hospital care. Thisdefinition was based on classifications (3, 22) used in pre-vious studies that analyzed the epidemiology of postdis-charge AEs.

REVIEW PROCESSES

As part of this supplement on patient safety, our pur-pose was to evaluate the effect of transitional care strategiesinitiated in the hospital on adverse outcomes after dis-charge compared with usual discharge care. We searchedMEDLINE, CINAHL, EMBASE, and the Cochrane Da-tabase of Controlled Trials from January 1990 throughSeptember 2012 using a search strategy developed with theassistance of a medical librarian. We included English-language, randomized, controlled trials (RCTs) and non-randomized, controlled clinical trials that evaluated the ef-fect of a transitional care strategy initiated before hospitaldischarge on postdischarge AE rates, ED use, or readmis-sion rates after discharge home. To be included, studies

must have enrolled an undifferentiated population of adultgeneral medical patients. We excluded studies conductedin disease-specific populations, studies of other formal careprograms (such as disease management programs) thatwere not initiated in the hospital or did not explicitly targetcare transitions, and studies focusing on transition fromhospitalization to another acute or subacute care setting.We included studies that reported intervention costs onlyif one of the main outcomes was also reported.

Study investigators screened 20 248 titles identified bythe search strategy for relevance and rereviewed a sample ofexcluded titles for accuracy. Two investigators indepen-dently reviewed the full text of potentially relevant studies(n � 762) to determine study eligibility. Two investigatorsindependently reviewed the 47 studies that met inclusioncriteria. They extracted data on the following domains:study design, methodological quality, study setting, partic-ipants (type of health system, target population), details ofthe intervention components, and outcomes. Disagree-ments on specific fields were resolved by consensus anddiscussion with a third investigator if necessary. Reviewersrated the quality of individual studies using the CochraneCollaboration Effective Practice and Organisation of Carechecklist; they also rated the overall strength of evidencesupporting specific strategies according to the method usedfor the Agency for Healthcare Research and Quality evi-dence report for which this project was performed (23).The main outcomes extracted were AE rates and ED andreadmission rates within 30 days after hospital discharge.Additional outcomes included readmissions, ED visits, andAE rates up to 1 year after discharge. Given the heteroge-neity of interventions, study settings, and patient popula-tions, we chose not to perform a meta-analysis. See theSupplement (available at www.annals.org) for a completedescription of the search strategies; the detailed article flowdiagram; and evidence tables, including quality ratings.


BENEFITS AND HARMS

Of 47 eligible studies, 28 were RCTs (24–51) and 19were controlled clinical trials (52–70). Most were rated ashaving fair methodological quality (see Table 3 of theSupplement).

BenefitsPatient Populations, Risk Factors, and Settings

About half of the studies (n � 24) were conductedwithin the United States. The majority (n � 27) targetedolder adult populations, although definitions of “elderly”varied widely (enrolling patients older than age 55 years in1 case [25]). Twelve studies targeted individuals at “highrisk” for readmissions or AEs, although definitions of “highrisk” were inconsistent across studies. Seven studies tar-

Key Summary Points

Hospitals are charged with implementing transitional carestrategies—interventions initiated before hospital dischargeto facilitate the safe transition of patients across healthcare settings—to prevent adverse events, emergencydepartment visits, and readmissions after discharge.

Hospital-based or bridging (including in-hospital and post-discharge components) strategies to prevent adverse clini-cal outcomes after discharge can involve patient engage-ment, use of a dedicated transition provider, medicationreconciliation, and facilitation of communication with out-patient providers.

Low-strength evidence shows that use of a bridging inter-vention incorporating a dedicated transition provider, whocontacted patients before and after discharge, reducedemergency department visits and readmission rates in10 fair-quality studies.

Evidence on the effectiveness of strategies to preventpostdischarge adverse events is scant and inconclusive.

Few studies provide information on contextual factors,cost, or implementation of transitional care strategies.

Although hospitals may be penalized for excessive re-admission rates, strategies to improve the quality of caretransitions at hospital discharge for general medicalpatients remain undefined.

Supplement Hospital-Initiated Transitional Care Interventions



geted individuals according to medication-related indica-tions, including polypharmacy or receipt of a “high-risk”medication; again, these definitions varied across studies.The most common exclusion criteria used in individualstudies were the presence of cognitive impairment or de-mentia (n � 15) and lack of fluency in the dominant lan-guage of the country in which the intervention took place(n � 17). The exclusion of these individuals may limit thegeneralizability of study findings to specific groups gener-ally considered to be at lower risk for readmission and AEsand may have biased the study toward null results in somecases.

Characteristics of Transitional Care Strategies

Studies used a median of 4 separate interventions(range, 1 to 8) (Table 2 of the Supplement). Thirty studies(21 RCTs) used a bridging strategy with both pre- and

postdischarge intervention components, and 17 studies (7RCTs) included only hospital-based, predischarge inter-ventions. The strategies included a variety of separate in-terventions. The most commonly used interventions in-cluded patient engagement (n � 37), ranging from generalpatient education to more specific instruction on symptommanagement and medication counseling. Twenty-eightstudies included postdischarge outreach to patients by tele-phone (n � 10), home visit (n � 8), or both telephonecontact and at least 1 home visit (n � 10). Of the 30studies that included a bridging intervention, 20 includeda designated transition provider who had contact with thepatient in the hospital and in the outpatient setting afterdischarge (Table 2).

Effect of Transitional Care Strategies on Postdischarge AEs

Nine studies reported AE rates after discharge (29–32,38, 40, 44, 59, 70) (Table 4 of the Supplement). Of these,3 reported statistically significant reductions in postdis-charge AE rates (31, 44, 70). Gillespie and colleagues(31) found that a pharmacist-led intervention reducedmedication-related readmissions within 12 months of hos-pital discharge. The intervention targeted elderly patientsand involved inpatient monitoring, counseling, dischargeteaching and medication reconciliation, and postdischargetelephone follow-up. Schnipper and colleagues (44) re-ported that a similarly comprehensive pharmacist-led in-tervention reduced preventable drug AEs and reduced acomposite outcome of medication-related ED visits andhospital readmissions within 30 days of hospital discharge.Another pharmacist-led study (70) that included dischargemedication counseling without postdischarge follow-up re-duced adverse drug events in a Saudi Arabian population.Two additional studies (30, 59) reported reductions inpostdischarge AEs with pharmacist-led medication safetyinterventions; findings were not statistically significant, butboth studies were underpowered to detect important dif-ferences between intervention and control groups.

Table 1. Taxonomy of Interventions to Improve TransitionalCare at Hospital Discharge

Predischarge interventionsAssessment of risk for adverse events or readmissionsPatient engagement (e.g., patient or caregiver education)Creation of an individualized patient record (customized document in lay

language containing clinical and educational information for patients’use after discharge)

Facilitation of communication with outpatient providersMultidisciplinary discharge planning teamDedicated transition provider (who has in-person or telephone contact

with patient before and after discharge)Medication reconciliation

Postdischarge interventionsOutreach to patients (including follow-up telephone calls,

patient-activated hotlines, and home visits)Facilitation of clinical follow-up (including facilitated ambulatory provider

follow-up)Medication reconciliation after discharge

Bridging interventionsInclusion of at least 1 predischarge component and at least 1

postdischarge component

Table 2. Summary Strength of Evidence and Findings

Intervention and Strategies TotalStudies, n

Mean EPOCScore

Studies Reporting EDVisit or ReadmissionRate (at Any TimePoint), n

Statistically SignificantReduction inReadmissions or EDVisits

Findings

Hospital-only 17 3.53 16 6 Wide variation in types of interventions andproviders involved

Bridging strategy 30 4.83 30 12Dedicated transition provider 20 4.95 20 10 Most transition providers were nurses;

postdischarge patient contact was viatelephone call or home visit; probablyresource-intensive, but little informa-tion provided on cost or ease ofimplementation

No dedicated transition provider 10 4.6 10 2 Wide variation in types of interventionsand providers involved

ED � emergency department; EPOC � Effective Practice and Organisation of Care.

SupplementHospital-Initiated Transitional Care Interventions



Effect of Transitional Care Strategies on 30-Day Readmissionand ED Visit Rates

Forty-six studies reported readmission rates at intervalsranging from 15 days to 1 year after the index hospitaldischarge; 22 of these studies (12 RCTs) reported readmis-sion rates or ED visit rates 30 days or less after discharge(Table 5 of the Supplement). Eight studies (4 RCTs) re-ported statistically significant reductions in 30-day read-mission rates, ED visits, or a composite of the 2 outcomes.Six of the 8 studies used a bridging intervention that in-cluded a dedicated provider who had primary responsibil-ity for ensuring safe transitions (26, 27, 33, 34, 55, 67).Transition providers met with patients before discharge toprovide patient education and conducted posthospital out-reach to patients via telephone or home visits. Transitionproviders also created individualized, patient-centeredhealth records and communicated information about thehospitalization to the patient’s primary care provider.Three studies that evaluated the Care Transitions Interven-tion (CTI)—an intervention with a “transition coach” whoperformed postdischarge home visits that emphasized pa-tient education and self-management—reported reductionsin 30-day readmissions (26, 55, 67) when conducted inmanaged care systems, capitated delivery systems, andMedicare fee-for-service populations. Another similar in-tervention, Project RED, reduced 30-day ED visits at anurban safety net hospital (33). A nurse discharge advocatewas responsible for patient education and communicationof clinical information to the patient’s primary care pro-vider, and a clinical pharmacist reviewed the discharge planand medication management by telephone with the patientafter discharge.

Fourteen studies (8 RCTs) reported no statistically sig-nificant reductions in 30-day readmission or ED visit rates.These studies were broadly similar to the successful studiesin terms of sample size and methodologic quality. Fourused a bridging intervention with a dedicated transitionprovider. One, which evaluated the CTI in a Medicarefee-for-service population, reported a reduction in readmis-sions at 90 days after discharge (43).

ED Visits and Readmission Rates Beyond 30 DaysAfter Discharge

Twenty-six studies reported ED visit rates, readmis-sion rates, or a composite of the 2 outcomes at intervalsranging from 45 days to 1 year after the index discharge.Seven studies reported statistically significant reductions inreadmission rates, including 4 studies (39, 40, 43, 47) thatused a bridging intervention with a dedicated transitionprovider.

HarmsNone of the studies reported any harms associated

with transitional care interventions.


Although a majority of studies (n � 26) reported adetailed timeline of the implementation of each compo-nent of the transitional care strategy, fewer than one thirdexplicitly described the resources needed to implement thestrategy or the training protocols used in the intervention.No studies reported a plan for sustainability or long-termincorporation of the intervention into current clinical prac-tice. Studies also generally failed to include informationabout the health care system context in which the inter-vention was conducted. No studies reported on the localquality improvement infrastructure, safety culture, or otherimportant contextual elements that could have influencedthe success of the intervention.

The CTI was the only transitional care strategy thatwas “successfully” implemented and evaluated in multiplesettings, including many types of hospitals and integratedand nonintegrated health care systems (26, 43, 55, 67). Allother investigations of interventions that reduced 30-dayreadmissions or ED visits were single-center studies thatwere not replicated in multiple settings or diversepopulations.

Sixteen studies reported comparisons of health careutilization and associated costs for patients in the interven-tion group and patients receiving usual care. These costswere measured over varying intervals after discharge andused cost estimates from different sources. No studies re-ported the costs of the intervention itself. We thereforecould not draw any firm conclusions on the effect of tran-sitional care interventions on overall health care costs.

Contextual factors probably play a significant role indetermining the effectiveness of a transitional care strategy.These contextual factors may operate at the patient level(for example, an individual patient’s readmission risk), theorganizational level (such as a hospital’s quality improve-ment infrastructure and ability to support transitional careinterventions), and the health care system level (such asaccess to primary care). Unfortunately, the studies we iden-tified did not describe these factors. Because CTI was theonly strategy evaluated in different patient populations andhealth care systems, we could not draw conclusions on theeffect of context on effectiveness.

DISCUSSION

In this systematic review, we examined 47 studies in-volving 44 distinct hospital-initiated strategies aimed at re-ducing postdischarge AEs, ED visits, and readmissions. Weidentified 15 studies showing that interventions success-fully reduced readmission or ED visit rates after discharge,including 8 studies showing that interventions reduced 30-day readmission rates. Nearly all studies used a bridgingintervention, and 10 of the 15 used a dedicated transitionprovider who contacted patients before and after discharge.One of these strategies, the CTI, has been successfully im-plemented and evaluated in multiple patient populations




and health care systems; a similar intervention, ProjectRED, has been implemented in a safety net system. Al-though these strategies are relatively intensive and probablyrequire considerable resources, information on costs oftransitional care strategies was lacking. Because few studiesspecifically addressed the problem of postdischarge AEs, wecould not reach firm conclusions regarding effective strat-egies in this area.

Two recent systematic reviews (71, 72) also attemptedto identify interventions to improve the quality of caretransitions at hospital discharge. One of these focusedon the clinical handover from hospital to primary care, andthe other evaluated transitional care interventions for pa-tients with stroke and acute myocardial infarction. Thesereviews identified many flaws in the care transitions evi-dence base that we found as well. These flaws includedpossible selective reporting; heterogeneity in interventiontypes, patient populations enrolled, and outcomes mea-sured; limited description of implementation processes;and failure to report on important contextual aspects thatmay have influenced the success or failure of the transi-tional care strategy being studied.

Within our classification of interventions, the mannerin which the studies carried out specific interventions var-ied widely. For example, studies that deployed a dedicatedtransition provider used different types of providers (pri-marily nurses, but also pharmacists) who had varying levelsof contact with patients after discharge (ranging from sin-gle telephone calls to multiple home visits). Althoughmany studies enrolled elderly patients or patients consid-ered to be at high risk for readmission, these definitionswere also inconsistent. Strategies that involve adding ded-icated transition providers probably require considerableresources to implement and sustain effectiveness. However,fewer than one third of studies described the training pro-tocols or resources needed to implement a transitional carestrategy, and no studies reported a plan for interventionsustainability.

Although readmission risk is known to be linked toaccess to primary care and the overall level of health careresources within a community (73), most studies did notinclude information on the health system context in whichthe intervention was implemented. In addition, evenamong the most comprehensive intervention strategies re-viewed, there was little evidence of active engagement ofprimary care providers in the transitional care planningprocess. Primary care providers and the medical home maybe best positioned to detect and prevent AEs before an EDvisit or readmission, and thus active engagement of outpa-tient providers in discharge safety efforts may provefruitful.

Despite the rapid proliferation of transitional carestrategies in the race to reduce hospital readmissions, therehas been a notable lack of attention to the potential addi-tional benefit of strategies to reduce specific postdischargeAEs. Postdischarge AEs should also be targeted in quality

improvement efforts because they still represent significantfailures to ensure patient safety, even if they do not ulti-mately lead to ED visits or readmissions. Medication safetyinterventions led by clinical pharmacists seem to be apromising approach, indicating a need for larger trials withan explicit plan to measure clinically significant AEs. Fur-ther research in this field should also follow recently pub-lished recommendations (74) to standardize interventionnomenclature and reproducibility, identify target popula-tions most likely to benefit from specific interventions,measure patient-centered outcomes, and rigorously reportand evaluate cost and implementation factors.

Our study has several limitations. We focused on tran-sitional care strategies initiated during hospitalization forgeneral medical patient populations, and we excluded stud-ies conducted in disease-specific populations. Because cur-rent policy initiatives emphasize the role of hospitals inpreventing readmissions in all patients, we therefore aimedto identify strategies that hospitals could apply to broadpatient populations. Prior systematic reviews (18, 21, 72,75) have identified interventions that can reduce readmis-sion risk in patients with congestive heart failure, acutemyocardial infarction, or stroke, but these conditions col-lectively account for only about 10% of Medicare hospitaladmissions per year (2). Thus, a successful disease-specificapproach may not translate to reductions in overall read-mission rates. Proven disease-specific strategies, such as dis-ease management programs, often rely on customizedpatient self-management or medication adherence inter-ventions that may be less relevant for other diseaseprocesses.

We also included only studies that measured clinicallysignificant AEs, in an effort to emphasize patient-centeredoutcomes. This led to exclusion of some studies that mea-sured surrogate outcomes, such as studies of dischargemedication reconciliation that measured medication dis-crepancies but did not report data on clinical AEs (76, 77).Some of these strategies may yet prove to be effective atpreventing clinical AEs. Finally, publication bias may haveaffected the results of our review because the national focuson readmissions has catalyzed many efforts to improvetransitional care that have yet to be published in the peer-reviewed literature.

Hospitals are now faced with the challenge of reeval-uating their current transitional care practices in order toreduce 30-day readmission rates. Although emphasizing re-admissions may have good face validity, we believe thatpolicymakers’ focus on 30-day readmissions is problematic.Only a small proportion (approximately 20% from pub-lished studies) (78) of readmissions at 30 days are probablypreventable, and much of what drives hospital readmissionrates are patient- and community-level factors, such asmental illness, poor social support, and poverty, that arewell outside the hospital’s control (79, 80). Furthermore,high readmission rates can be the result of low mortalityrates, improved access to hospital care, and high admission




rates (81) and therefore may not always represent care tran-sitions failures. Because there are currently no reliablemethods to predict an individual patient’s readmission risk(82), hospitals face significant difficulties in determiningwhich patients should be targeted for transitional care in-terventions. Finally, because hospitals are expending re-sources on reducing readmissions, they may not be able toaddress other, more pressing patient safety issues. In thiscontext, our finding that only a few resource-intensive in-terventions seem to reduce readmission rates is especiallyproblematic.

In summary, we found that only a limited number ofbridging interventions involving a dedicated transition pro-vider seems to reduce readmissions and ED visits after hos-pital discharge to home. Among these, only the CTI hasbeen implemented in multiple settings and patient popu-lations. Few studies specifically targeted AEs after dis-charge, and the studies we identified provided little infor-mation about implementation factors, contextual factors,or cost. Although hospitals are now being penalized forexcessive readmission rates, the strategies that an individualhospital can implement to improve transitional care remainlargely undefined.

From the University of California, San Francisco, San Francisco,California.


Disclaimer: All statements expressed in this work are those of the authorsand should not in any way be construed as official opinions or positionsof the University of California, San Francisco; the Agency for HealthcareResearch and Quality; or the U.S. Department of Health and HumanServices.


Potential Conflicts of Interest: Dr. Rennke: Grant (money to self and toinstitution): AHRQ; Support for travel to meetings for the study or otherpurposes: AHRQ; Payment for writing or reviewing the manuscript (moneyto self and to institution): AHRQ; Provision of writing assistance, medicines,equipment, or administrative support (money to institution): AHRQ; Con-sultancy: Society Hospital of Medicine. Dr. Ranji: Grant (money to insti-tution): AHRQ. Dr. Magan: Grant (money to institution): AHRQ. Dr.Wachter: Grant (money to institution): AHRQ; Support for travel to meet-ings for the study or other purposes (money to institution): AHRQ; Boardmembership: Chair of the American Board of Internal Medicine; Grants/grants pending (money to institution): AHRQ; Payment for lectures includ-ing service on speakers’ bureaus: honorarium for lectures from more than100 health care organizations, mostly on patient safety, health care qual-ity, and hospitalists; Royalties: Lippincott Williams & Wilkins, McGraw-Hill; Payment for development of educational presentations: QuantiaMD;Payment for development of educational presentations (money to institution):IPC-The Hospitalist Company; Stock/stock options: PatientSafe Solutions,CRISI, EarlySense; Other: Compensation from John Wiley & Sons for

writing “Wachter’s World” blog, Benioff endowed chair in hospital med-icine, funded by the US-UK Fulbright Commission for a sabbatical atImperial College London from July to December 2011, unpaid memberof the Board of Directors, Quality Committee of Salem Hospital. Allother authors have no dislosures. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum�M12-2573.

Requests for Single Reprints: Stephanie Rennke, MD, University ofCalifornia, San Francisco, UCSF Mount Zion Medical Center, 1600Divisadero Street, San Francisco, CA 94115-1945; e-mail, [email protected].


References1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients inthe Medicare fee-for-service program. N Engl J Med. 2009;360:1418-28.[PMID: 19339721]2. Jha AK, Joynt KE, Orav EJ, Epstein AM. The long-term effect of premier payfor performance on patient outcomes. N Engl J Med. 2012;366:1606-15.[PMID: 22455751]3. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidenceand severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med. 2003;138:161-7. [PMID: 12558354]4. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW.Deficits in communication and information transfer between hospital-based andprimary care physicians: implications for patient safety and continuity of care.JAMA. 2007;297:831-41. [PMID: 17327525]5. Gleason KM, McDaniel MR, Feinglass J, Baker DW, Lindquist L, Liss D,et al. Results of the Medications at Transitions and Clinical Handoffs (MATCH)study: an analysis of medication reconciliation errors and risk factors at hospitaladmission. J Gen Intern Med. 2010;25:441-7. [PMID: 20180158]6. Makaryus AN, Friedman EA. Patients’ understanding of their treatment plansand diagnosis at discharge. Mayo Clin Proc. 2005;80:991-4. [PMID: 16092576]7. Moore C, McGinn T, Halm E. Tying up loose ends: discharging patients withunresolved medical issues. Arch Intern Med. 2007;167:1305-11. [PMID:17592105]8. Hasan O, Meltzer DO, Shaykevich SA, Bell CM, Kaboli PJ, Auerbach AD,et al. Hospital readmission in general medicine patients: a prediction model.J Gen Intern Med. 2010;25:211-9. [PMID: 20013068]9. Kirby SE, Dennis SM, Jayasinghe UW, Harris MF. Patient related factors infrequent readmissions: the influence of condition, access to services and patientchoice. BMC Health Serv Res. 2010;10:216. [PMID: 20663141]10. Robinson S, Howie-Esquivel J, Vlahov D. Readmission risk factors afterhospital discharge among the elderly. Popul Health Manag. 2012;15:338-51.[PMID: 22823255]11. U.S. Department of Health & Human Services. Hospital Compare. 11October 2012. Accessed at www.hospitalcompare.hhs.govon 29 November 2012.12. Rau J. Medicare to penalize 2,211 hospitals for excess readmissions. KaiserHealth News. 13 August 2012. Accessed at www.kaiserhealthnews.org/Stories/2012/August/13/medicare-hospitals-readmissions-penalties.aspx on 28 Decem-ber 2012.13. HealthCare.gov. Partnership for patients: better care, lower costs. Accessed atwww.healthcare.gov/center/programs/partnership/index.html on 29 November2012.14. Maynard GA, Budnitz TL, Nickel WK, Greenwald JL, Kerr KM, MillerJA, et al. 2011 John M. Eisenberg Patient Safety and Quality Awards. Mentoredimplementation: building leaders and achieving results through a collaborativeimprovement model. Innovation in patient safety and quality at the nationallevel. Jt Comm J Qual Patient Saf. 2012;38:301-10. [PMID: 22852190]15. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventionsto reduce 30-day rehospitalization: a systematic review. Ann Intern Med. 2011;155:520-8. [PMID: 22007045]




16. Chiu WK, Newcomer R. A systematic review of nurse-assisted case manage-ment to improve hospital discharge transition outcomes for the elderly. Prof CaseManag. 2007;12:330-6; quiz 337-8. [PMID: 18030153]17. Mistiaen P, Francke AL, Poot E. Interventions aimed at reducing problemsin adult patients discharged from hospital to home: a systematic meta-review.BMC Health Serv Res. 2007;7:47. [PMID: 17408472]18. Naylor MD, Aiken LH, Kurtzman ET, Olds DM, Hirschman KB. Thecare span: The importance of transitional care in achieving health reform. HealthAff (Millwood). 2011;30:746-54. [PMID: 21471497]19. Parker SG, Peet SM, McPherson A, Cannaby AM, Abrams K, Baker R,et al. A systematic review of discharge arrangements for older people. HealthTechnol Assess. 2002;6:1-183. [PMID: 12065067]20. Richards S, Coast J. Interventions to improve access to health and social careafter discharge from hospital: a systematic review. J Health Serv Res Policy. 2003;8:171-9. [PMID: 12869344]21. Shepperd S, McClaran J, Phillips CO, Lannin NA, Clemson LM, McClus-key A, et al. Discharge planning from hospital to home. Cochrane Database SystRev. 2010:CD000313. [PMID: 20091507]22. Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N,et al. Adverse events among medical patients after discharge from hospital.CMAJ. 2004;170:345-9. [PMID: 14757670]23. Shekelle PG, Wachter RM, Pronovost P. Chapter 2. Methods. In: MakingHealth Care Safer II: An Updated Critical Analysis of the Evidence for PatientSafety Practices. Comparative Effectiveness Review. Prepared by the SouthernCalifornia-RAND Evidence-based Practice Center under contract no.HHSA290200710062I. Rockville, MD: Agency for Healthcare Research andQuality. [In Press].24. Balaban RB, Weissman JS, Samuel PA, Woolhandler S. Redefining andredesigning hospital discharge to enhance patient care: a randomized controlledstudy. J Gen Intern Med. 2008;23:1228-33. [PMID: 18452048]25. Bolas H, Brookes K, Scott M, McElnay J. Evaluation of a hospital-basedcommunity liaison pharmacy service in Northern Ireland. Pharm World Sci.2004;26:114-20. [PMID: 15085948]26. Coleman EA, Parry C, Chalmers S, Min SJ. The care transitions interven-tion: results of a randomized controlled trial. Arch Intern Med. 2006;166:1822-8. [PMID: 17000937]27. Courtney M, Edwards H, Chang A, Parker A, Finlayson K, Hamilton K.Fewer emergency readmissions and better quality of life for older adults at risk ofhospital readmission: a randomized controlled trial to determine the effectivenessof a 24-week exercise and telephone follow-up program. J Am Geriatr Soc. 2009;57:395-402. [PMID: 19245413]28. Dellasega CA, Zerbe TM. A multimethod study of advanced practice nursepostdischarge care. Clin Excell Nurse Pract. 2000;4:286-93. [PMID: 11858450]29. Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N,et al. Effect of a nurse team coordinator on outcomes for hospitalized medicinepatients. Am J Med. 2005;118:1148-53. [PMID: 16194647]30. Gallagher PF, O’Connor MN, O’Mahony D. Prevention of potentiallyinappropriate prescribing for elderly patients: a randomized controlled trial usingSTOPP/START criteria. Clin Pharmacol Ther. 2011;89:845-54. [PMID:21508941]31. Gillespie U, Alassaad A, Henrohn D, Garmo H, Hammarlund-Udenaes M,Toss H, et al. A comprehensive pharmacist intervention to reduce morbidity inpatients 80 years or older: a randomized controlled trial. Arch Intern Med. 2009;169:894-900. [PMID: 19433702]32. Graumlich JF, Novotny NL, Stephen Nace G, Kaushal H, Ibrahim-Ali W,Theivanayagam S, et al. Patient readmissions, emergency visits, and adverseevents after software-assisted discharge from hospital: cluster randomized trial.J Hosp Med. 2009;4:E11-9. [PMID: 19479782]33. Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, JohnsonAE, et al. A reengineered hospital discharge program to decrease rehospitalization:a randomized trial. Ann Intern Med. 2009;150:178-87. [PMID: 19189907]34. Koehler BE, Richter KM, Youngblood L, Cohen BA, Prengler ID, ChengD, et al. Reduction of 30-day postdischarge hospital readmission or emergencydepartment (ED) visit rates in high-risk elderly medical patients through deliveryof a targeted care bundle. J Hosp Med. 2009;4:211-8. [PMID: 19388074]35. Lim WK, Lambert SF, Gray LC. Effectiveness of case management andpost-acute services in older people after hospital discharge. Med J Aust. 2003;178:262-6. [PMID: 12633482]

36. Lipton HL, Bird JA. The impact of clinical pharmacists’ consultations ongeriatric patients’ compliance and medical care use: a randomized controlled trial.Gerontologist. 1994;34:307-15. [PMID: 8076871]37. Martin F, Oyewole A, Moloney A. A randomized controlled trial of a highsupport hospital discharge team for elderly people. Age Ageing. 1994;23:228-34.[PMID: 8085509]38. Marusic S, Gojo-Tomic N, Erdeljic V, Bacic-Vrca V, Franic M, Kirin M,et al. The effect of pharmacotherapeutic counseling on readmissions and emer-gency department visits. Int J Clin Pharm. 2012. [PMID: 23007693]39. Naylor MD, Brooten D, Campbell R, Jacobsen BS, Mezey MD, PaulyMV, et al. Comprehensive discharge planning and home follow-up of hospital-ized elders: a randomized clinical trial. JAMA. 1999;281:613-20. [PMID:10029122]40. Naylor MD. Comprehensive discharge planning for hospitalized elderly: apilot study. Nurs Res. 1990;39:156-61. [PMID: 2188217]41. Nazareth I, Burton A, Shulman S, Smith P, Haines A, Timberal H. Apharmacy discharge plan for hospitalized elderly patients—a randomized con-trolled trial. Age Ageing. 2001;30:33-40. [PMID: 11322670]42. Nikolaus T, Specht-Leible N, Bach M, Oster P, Schlierf G. A randomizedtrial of comprehensive geriatric assessment and home intervention in the care ofhospitalized patients. Age Ageing. 1999;28:543-50. [PMID: 10604506]43. Parry C, Min SJ, Chugh A, Chalmers S, Coleman EA. Further applicationof the care transitions intervention: results of a randomized controlled trial con-ducted in a fee-for-service setting. Home Health Care Serv Q. 2009;28:84-99.[PMID: 20182958]44. Schnipper JL, Kirwin JL, Cotugno MC, Wahlstrom SA, Brown BA, TarvinE, et al. Role of pharmacist counseling in preventing adverse drug events afterhospitalization. Arch Intern Med. 2006;166:565-71. [PMID: 16534045]45. Scullin C, Scott MG, Hogg A, McElnay JC. An innovative approach tointegrated medicines management. J Eval Clin Pract. 2007;13:781-8. [PMID:17824872]46. Siu AL, Kravitz RL, Keeler E, Hemmerling K, Kington R, Davis JW, et al.Postdischarge geriatric assessment of hospitalized frail elderly patients. Arch In-tern Med. 1996;156:76-81. [PMID: 8526700]47. Stewart S, Pearson S, Luke CG, Horowitz JD. Effects of home-based inter-vention on unplanned readmissions and out-of-hospital deaths. J Am Geriatr Soc.1998;46:174-80. [PMID: 9475445]48. Thomas DR, Brahan R, Haywood BP. Inpatient community-based geriatricassessment reduces subsequent mortality. J Am Geriatr Soc. 1993;41:101-4.[PMID: 8426028]49. Weinberger M, Oddone EZ, Henderson WG. Does increased access toprimary care reduce hospital readmissions? Veterans Affairs Cooperative StudyGroup on Primary Care and Hospital Readmission. N Engl J Med. 1996;334:1441-7. [PMID: 8618584]50. Finn KM, Heffner R, Chang Y, Bazari H, Hunt D, Pickell K, et al.Improving the discharge process by embedding a discharge facilitator in a residentteam. J Hosp Med. 2011;6:494-500. [PMID: 22042739]51. Legrain S, Tubach F, Bonnet-Zamponi D, Lemaire A, Aquino JP, PaillaudE, et al. A new multimodal geriatric discharge-planning intervention to preventemergency visits and rehospitalizations of older adults: the optimization of med-ication in AGEd multicenter randomized controlled trial. J Am Geriatr Soc.2011;59:2017-28. [PMID: 22091692]52. Adler A, Lipkin C, Cooper L, Agolino M, Jones V. Effect of social workintervention on hospital discharge transition planning in a special needs popula-tion. Manag Care. 2009;18:50-3. [PMID: 19999254]53. Al-Rashed SA, Wright DJ, Roebuck N, Sunter W, Chrystyn H. The valueof inpatient pharmaceutical counselling to elderly patients prior to discharge.Br J Clin Pharmacol. 2002;54:657-64. [PMID: 12492615]54. Brand CA, Jones CT, Lowe AJ, Nielsen DA, Roberts CA, King BL, et al.A transitional care service for elderly chronic disease patients at risk of readmis-sion. Aust Health Rev. 2004;28:275-84. [PMID: 15595909]55. Coleman EA, Smith JD, Frank JC, Min SJ, Parry C, Kramer AM. Prepar-ing patients and caregivers to participate in care delivered across settings: the CareTransitions Intervention. J Am Geriatr Soc. 2004;52:1817-25. [PMID:15507057]56. Cowan MJ, Shapiro M, Hays RD, Afifi A, Vazirani S, Ward CR, et al. Theeffect of a multidisciplinary hospitalist/physician and advanced practice nursecollaboration on hospital costs. J Nurs Adm. 2006;36:79-85. [PMID: 16528149]57. Einstadter D, Cebul RD, Franta PR. Effect of a nurse case manager onpostdischarge follow-up. J Gen Intern Med. 1996;11:684-8. [PMID: 9120655]




58. Gow P, Berg S, Smith D, Ross D. Care co-ordination improves quality-of-care at South Auckland Health. J Qual Clin Pract. 1999;19:107-10. [PMID:10408752]59. Hellstrom LM, Bondesson A, Hoglund P, Midlov P, Holmdahl L, RickhagE, et al. Impact of the Lund Integrated Medicines Management (LIMM) modelon medication appropriateness and drug-related hospital revisits. Eur J Clin Phar-macol. 2011;67:741-52. [PMID: 21318595]60. Hogan DB, Fox RA. A prospective controlled trial of a geriatric consultationteam in an acute-care hospital. Age Ageing. 1990;19:107-13. [PMID: 2337005]61. Makowsky MJ, Koshman SL, Midodzi WK, Tsuyuki RT. Capturing out-comes of clinical activities performed by a rounding pharmacist practicing in ateam environment: the COLLABORATE study [NCT00351676]. Med Care.2009;47:642-50. [PMID: 19433997]62. Mudge A, Laracy S, Richter K, Denaro C. Controlled trial of multidisci-plinary care teams for acutely ill medical inpatients: enhanced multidisciplinarycare. Intern Med J. 2006;36:558-63. [PMID: 16911547]63. Palmer HC Jr, Halperin A, Elnicki M, Powers R, Kolar M, Evans K, et al.Effect of a patient care partnership project on cost and quality of care at anacademic teaching hospital. South Med J. 2002;95:1318-25. [PMID: 12540000]64. Scullin C, Hogg A, Luo R, Scott MG, McElnay JC. Integrated medicinesmanagement—can routine implementation improve quality? J Eval Clin Pract.2012;18:807-15. [PMID: 21504517]65. Steeman E, Moons P, Milisen K, De Bal N, De Geest S, De Froidmont C,et al. Implementation of discharge management for geriatric patients at risk ofreadmission or institutionalization. Int J Qual Health Care. 2006;18:352-8.[PMID: 16861721]66. Styrborn K. Early discharge planning for elderly patients in acutehospitals—an intervention study. Scand J Soc Med. 1995;23:273-85.[PMID: 8919370]67. Voss R, Gardner R, Baier R, Butterfield K, Lehrman S, Gravenstein S. Thecare transitions intervention: translating from efficacy to effectiveness. Arch InternMed. 2011;171:1232-7. [PMID: 21788540]68. Walker PC, Bernstein SJ, Jones JN, Piersma J, Kim HW, Regal RE, et al.Impact of a pharmacist-facilitated hospital discharge program: a quasi-experimental study. Arch Intern Med. 2009;169:2003-10. [PMID: 19933963]69. Wilkinson ST, Pal A, Couldry RJ. Impacting readmission rates and patientsatisfaction: results of a discharge pharmacist pilot program. Hospital Pharm.2011;46:876-83.70. Al-Ghamdi SA, Mahmoud MA, Alammari MA, Al Bekairy AM, AlwhaibiM, Mayet AY, et al. The outcome of pharmacist counseling at the time of

hospital discharge: an observational nonrandomized study. Ann Saudi Med.2012;32:492-7. [PMID: 22871618]71. Hesselink G, Schoonhoven L, Barach P, Spijker A, Gademan P, KalkmanC, et al. Improving patient handovers from hospital to primary care: a systematicreview. Ann Intern Med. 2012;157:417-28. [PMID: 22986379]72. Prvu Bettger J, Alexander KP, Dolor RJ, Olson DM, Kendrick AS, WingL, et al. Transitional care after hospitalization for acute stroke or myocardialinfarction: a systematic review. Ann Intern Med. 2012;157:407-16. [PMID:22986378]73. Epstein AM, Jha AK, Orav EJ. The relationship between hospital admissionrates and rehospitalizations. N Engl J Med. 2011;365:2287-95. [PMID:22168643]74. Bray-Hall ST. Transitional care: focusing on patient-centered outcomes andsimplicity [Editorial]. Ann Intern Med. 2012;157:448-9. [PMID: 22986380]75. Takeda A, Taylor SJ, Taylor RS, Khan F, Krum H, Underwood M. Clin-ical service organisation for heart failure. Cochrane Database Syst Rev. 2012;9:CD002752. [PMID: 22972058]76. Karapinar-Carkit F, Borgsteede SD, Zoer J, Smit HJ, Egberts AC, van denBemt PM. Effect of medication reconciliation with and without patient counsel-ing on the number of pharmaceutical interventions among patients dischargedfrom the hospital. Ann Pharmacother. 2009;43:1001-10. [PMID: 19491320]77. Schnipper JL, Hamann C, Ndumele CD, Liang CL, Carty MG, KarsonAS, et al. Effect of an electronic medication reconciliation application and processredesign on potential adverse drug events: a cluster-randomized trial. Arch InternMed. 2009;169:771-80. [PMID: 19398689]78. van Walraven C, Jennings A, Taljaard M, Dhalla I, English S,Mulpuru S, et al. Incidence of potentially avoidable urgent readmissionsand their relation to all-cause urgent readmissions. CMAJ. 2011;183:E1067-72. [PMID: 21859870]79. Joynt KE, Orav EJ, Jha AK. Thirty-day readmission rates for Medicarebeneficiaries by race and site of care. JAMA. 2011;305:675-81. [PMID:21325183]80. Philbin EF, Dec GW, Jenkins PL, DiSalvo TG. Socioeconomic status as anindependent risk factor for hospital readmission for heart failure. Am J Cardiol.2001;87:1367-71. [PMID: 11397355]81. Fisher ES, Wennberg JE, Stukel TA, Sharp SM. Hospital readmission ratesfor cohorts of Medicare beneficiaries in Boston and New Haven. N Engl J Med.1994;331:989-95. [PMID: 8084356]82. Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, FreemanM, et al. Risk prediction models for hospital readmission: a systematic review.JAMA. 2011;306:1688-98. [PMID: 22009101]




Current Author Addresses: Dr. Rennke: University of California, SanFrancisco, UCSF Mount Zion Medical Center, 1600 Divisadero Street,San Francisco, CA 94115-1945.Dr. Nguyen: University of California, San Francisco, UCSF LaurelHeights, Campus Box 1211, 3333 California Street, San Francisco, CA94143.Drs. Shoeb and Ranji: Department of Medicine, University of Califor-nia, San Francisco, 533 Parnassus Avenue, Box 0131, San Francisco, CA94143.Dr. Magan: Division of Hospital Medicine, University of California, SanFrancisco, 533 Parnassus Avenue, Box 0131, U-129, San Francisco, CA94143.Dr. Wachter: Department of Medicine, University of California, SanFrancisco, 533 Parnassus Avenue, Box 0120, San Francisco, CA 94143.

Author Contributions: Conception and design: S. Rennke, O.K.Nguyen, M.H. Shoeb, S.R. Ranji.Analysis and interpretation of the data: S. Rennke, O.K. Nguyen, M.H.Shoeb, Y. Magan, S.R. Ranji.Drafting of the article: S. Rennke, O.K. Nguyen, M.H. Shoeb, Y.Magan, S.R. Ranji.Critical revision of the article for important intellectual content: S.Rennke, O.K. Nguyen, M.H. Shoeb, Y. Magan, R.M. Wachter, S.R.Ranji.Final approval of the article: S. Rennke, O.K. Nguyen, M.H. Shoeb,R.M. Wachter, S.R. Ranji.Obtaining of funding: R.M. Wachter.Administrative, technical, or logistic support: Y. Magan.Collection and assembly of data: S. Rennke, O.K. Nguyen, M.H. Shoeb,Y. Magan, S.R. Ranji.




Strategies to Improve Patient Safety: The Evidence Base Matters

Documents

Transcript of Strategies to Improve Patient Safety: The Evidence Base Matters