Assessing and Improving aCulture of Safety in

Ambulatory and InpatientPediatric Settings

Tuesday, November 11, 200812:00 – 1:00 p.m. EST

© American Academy of Pediatrics 2008

Moderator: Marlene R. Miller, MD, MSc, FAAPVice Chair, Quality and SafetyJohns Hopkins Children’s Center, Baltimore, MarylandVice President, Quality TransformationNational Association of Children’s Hospitals and Related

Institutions (NACHRI), Alexandria, Virginia

DISCLOSURESFinancial Relationships

J. Bryan Sexton, PhD, Speakerhas disclosed a financial relationship with Lotus Forum,

Inc., which has licensed the rights to the Safety Attitudes Questionnaire (SAQ) described in this presentation. Dr. Sexton

is the inventor of the SAQ and a paid consultant to Lotus Forum, Inc. The terms of this arrangement are being managed by the

Johns Hopkins University in accordance with its conflict of interest policies. Dr. Sexton has also disclosed that he has

received research grants from Robert Wood Johnson and from the Agency for Healthcare Research and Quality (AHRQ).

The AAP resolved this potential conflict of interest by performing an advance peer review of the webinar content/slides to ensure a

balanced presentation.

DISCLOSURESFinancial Relationships

Melissa A. Singleton, MEd, Project Manager-Consultant

has disclosed a financial relationship with an entity producing, marketing, re-selling, or distributing health

care goods or services consumed by, or used on, patients. Her husband is employed by Walgreen Co. as

a Workforce Administration Manager (technology position) for the company’s call centers. The AAP

determined that this financial relationship does not relate to the educational assignment.

None of the other involved individuals (Moderator, Project Advisory Committee members, or Staff) has disclosed a

relevant financial relationship.

Refer to full AAP Disclosure Policy & Grid available below for download.

DISCLOSURESOff-Label/Investigational Uses

None of the individuals (Speakers, Moderators, Project Advisory Committee members, or Staff) has disclosed that

they intend to discuss or demonstrate pharmaceuticals and/or medical devices that are not approved.

Refer to full AAP Disclosure Policy & Grid available below for download.

This activity was funded through an educational grant from the

Physicians’ Foundation for Health Systems Excellence.

Visit our website:http://www.aap.org/saferhealthcare

Resources: Useful strategies, valuable information links, and expert advice on reducing or eliminating medical errors affecting children.

Webinars: Register for an upcoming, live Webinar, and earn a maximum of 1.0 AMA PRA Category 1 Credit™. Or, access a full archive, including audio, from one of the past Webinar offerings. Or, download just the

Podcast or slide set from an archive.

Latest News: Links to recent articles relating to pediatric patient safety.

Email List: An e-community dedicated to pediatric patient safety issues and information exchange with other clinicians.

Parents’ Corner: Resources to help parents understand what they can do to help ensure their optimal safety in the health care that their child

receives.

CME CREDITLive Webinar Only

The American Academy of Pediatrics (AAP) is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

 The AAP designates this educational activity for a

maximum of 1.0 AMA PRA Category 1 Credit™. Physicians should only claim credit commensurate with the extent of their participation in the activity.

 This activity is acceptable for up to 1.0 AAP credits. These

credits can be applied toward the AAP CME/CPD Award available to Fellows and Candidate Members of the American Academy of Pediatrics.

OTHER CREDITLive Webinar Only

This program is approved for 1.0 NAPNAP contact hours of which 0.0 contain pharmacology (Rx) content per the National Association of Pediatric Nurse Practitioners Continuing Education Guidelines.

 The American Academy of Physician Assistants accepts

AMA PRA Category 1 Credit(s)TM from organizations accredited by the ACCME.

Important Note:You must have been pre-registered for this webinar in

order to claim CME or other credit for your participation.

LEARNING OBJECTIVESUpon completion of the webinar, participants will be

able to:

• Describe the basic fundamentals of safety culture assessment and improvement.

• Review salient issues related to teamwork and teams at work.

• Apply the concepts presented to make safety culture accessible to frontline staff.

Speaker: J. Bryan Sexton, PhDAssistant ProfessorJohns Hopkins UniversityBaltimore, Maryland

Creating aCulture of Safety in Ambulatory

& Inpatient Pediatric Settings

Sexton Disclosure

“Lotus Forum, Inc., has licensed the rights to the SAQ described in this presentation. Dr. Sexton is the inventor of the SAQ and a paid consultant to Lotus Forum, Inc. The terms of this arrangement are being managed by the Johns Hopkins University in accordance with its conflict of interest policies.”

Additional Learning Objectives

• To recognize the value in respecting frontline caregiver wisdom is quality and safety efforts

• To understand the importance of methodological rigor in collecting, interpreting, feeding back and using safety culture data (i.e., good assessment is not just about the tool you use)

• To learn how to match CUSP, tools and next steps to specific culture strengths & weaknesses

Culture Assessment=Respecting the Wisdom of Frontline Caregivers

Surowiecki : inputs must be diverse and independent for collective wisdom to be potent

Representative-ness -vs- Unrepresentative Mess-Less than 60% response rate unreliable

Example from James Surowiecki, The Wisdom of Crowds, 2005

Safety Culture Primer

• “The way we do things around here”

• Measure of frontline caregiver consensus

• Predicts clinical and operational outcomes

• New field, evolving rapidly

• Assessing culture improvement is two-pronged:– Did the unit improve climate by 10 points or more?– Did the unit maintain a good culture of 60 points or more?

16

 

33.2%

54.8% 54.8%

73.3%

87.7%93.4%

0

10

20

30

40

50

60

70

80

90

100

C C M (T X ) MI MI

RN rates Physician Physician rates RN

% o

f res

pond

ents

repo

rtin

g ab

ove

adeq

uate

team

wor

k ICU Physician and RN Collaboration8 TX ICUs 230 RNs90 Staff PhysiciansThomas, Sexton, Helmreich. Crit Care Med. 2003

99 MI ICUs 20042221 RNs658 Staff Physicians(Crit Care/Med/Surg)

99 MI ICUs 20042221 RNs151 Crit Care Physicians

18

% Reporting Good Collaboration% Reporting Good Collaboration

76%

87%

68%

70%

84%

48% 88%75% 92%

58%

89%

63%

Makary MA, Sexton JB, Freischlag JA, Holzmueller CG, Millman EA, Rowen L, Pronovost PJ:Operating room teamwork among physicians and nurses: teamwork in the eye of the beholder. Journal of the American College of Surgeons. 2006 May;202(5):746-52.

Familiarity with others is a critical component of effective teamwork:

•74% of all commercial aviation accidents happen on the first day of a crew flying together•Familiarity trumps fatigue (simulator studies)•Highlights the importance of predictable patterns of behavior•Many teamwork tools, e.g., briefings are a proxy for familiarity

“I know the names of all the people I worked with during my last shift?”

(NACHRI 2008)• PICU RNs

• PICU Charge Nurse

• Intensivists/ICU Staff physicians

• Staff Physicians (Surgeons)

• Staff Physicians (Medicine)

• Respiratory Therapists

• 52%

• 83%

• 18%

• 17%

• 31%

• 31%

Teamwork Disconnect

•RN: Good teamwork means I am asked for my input

•MD: Good teamwork means the nurse does what I say

Teamwork Climate is the consensus of frontline caregiver assessments related to

collaboration

Example Teamwork Climate Scale Items:

• In this clinical area, it is difficult to speak up if I perceive a problem with patient care

• Disagreements in this clinical area are resolved appropriately (i.e. not who is right, but what is best for the patient)

• The physicians and nurses here work together as a well-coordinated team

22

Communication Breakdowns are frequently the root cause of…undesirable outcomes

23

 

0102030405060708090

100

Hospital (each bar = 1

% re

spon

dent

s re

port

ing

good

team

wor

kOR Teamwork Climate

by Hospital

Sexton JB, Makary MA, et al. Anesthesiology.

2006 Nov;105(5):877-84.

 

0

10

20

30

40

50

60

70

80

90

100

% o

f res

pond

ents

repo

rtin

g go

od te

amw

ork

clim

ate

Teamwork Climate Across PICUs 2008

Goa

l 80%

Nee

ds

imp

rove

men

t <

60%

NOTE: Teamwork climate is negatively correlated with annual nurse turnover rates, absenteeism, BSI, PE/DVT, delays, and burnout

PICU (each bar = 1 PICU)

 

0102030405060708090

100

PICU (each bar = 1 PICU)

% re

spon

dent

s in

PIC

U th

at a

gree

PICUs 2008: “Disagreements In This PICU Are Resolved Appropriately (i.e. not who is right, but what is best for the patient).”

0

10

20

30

40

50

60

70

80

90

100

% re

port

ing

good

team

wor

k cl

imat

e2004 Teamwork Climate Across Michigan ICUs

27

•Pronovost PJ, Berenholtz SM, et al. Improving patient safety in intensive care units in Michigan. Journal of Critical Care. 2008 June:23(2): 207-221.

0

10

20

30

40

50

60

70

80

90

100

% re

port

ing

good

team

wor

k cl

imat

e2004 Teamwork Climate Across Michigan ICUs

 

No BSI 21%No BSI 21% No BSI 44%No BSI 44% No BSI 31% No BSI 31%

No BSI = 5 months or more w/ zeroNo BSI = 5 months or more w/ zero

The strongest predictor of clinical excellence: caregivers feel comfortable speaking up if they perceive a problem with patient care

28

 

0

10

20

30

40

50

60

70

80

90

100

% o

f res

pond

ents

repo

rtin

g go

od te

amw

ork

clim

ate

2007 Teamwork Climate Across 45 ICUs

Goa

l 80%

Nee

ds

imp

rove

men

t <

60%

 

BSI:BSI: baseline to 08 baseline to 08

3.93 to .953.93 to .95t=2.96; p<.01

BSI:BSI: baseline to 08 baseline to 08

1.08 to .691.08 to .69t=.943; p<.36

BSI:BSI: baseline to 08 baseline to 08

5.84 to .645.84 to .64t=2.43; p<.03

* Preliminary estimates: RWJ Foundation grant 58292

Teamwork:

30

Usually Talk about:•Strategies & Tools

Frequently Overlook the role of:•Familiarity & Fatigue

Current Biology, Volume 17, Issue 20, 23 October 2007, Pages R877-R878Seung-Schik Yoo, Ninad Gujar, Peter Hu, Ferenc A. Jolesz and Matthew P. Walker

"The emotional centers of the brain were over 60 percent more reactive under conditions of sleep deprivation than in subjects who had obtained a normal night of sleep.”

Title: The human emotional brain without sleep - a prefrontal amygdala disconnect.

31

The Reality of Fatigue

• "Sleep appears to restore our emotional brain circuits, and in doing so prepares us for the next day's challenges and social interactions," says Walker. – 35 Hours without sleep = JERK

• In other words, we are emotional, at the expense of being logical, when we are tired…Seung-Schik Yoo, Ninad Gujar, Peter Hu, Ferenc A. Jolesz and Matthew P. Walker. Current Biology, Volume 17, Issue 20, 23 October 2007, Pages R877-R878

© Quality and Safety Research Group

• Maintain awareness of other crewmembers, their Maintain awareness of other crewmembers, their problems, and their workloadproblems, and their workload

• Value input from other crewmembersValue input from other crewmembers• Adjust interaction styles to different crewmembersAdjust interaction styles to different crewmembers• Acknowledge vulnerability to stress, both physical and Acknowledge vulnerability to stress, both physical and

psychological psychological

• The performance of 95.7% of the pilots was correctly The performance of 95.7% of the pilots was correctly classified by the analysis of attitudesclassified by the analysis of attitudes

Attitudes that predictcockpit performance:

Helmreich et al. 1986

"Do what you can, with what you have, where you are."

  --  Theodore Roosevelt

Improving Safety CultureImproving Safety Culture

34

Comprehensive Unit-based Safety Program (CUSP): An Intervention to

Learn from Mistakes and Improve Safety Culture

1. Begin Annual Assessment of Safety Culture2. Educate staff on science of safety

http://www.jhsph.edu/ctlt/training/patient_safety.html3. Identify defects4. Assign executive to adopt unit5. Begin Learning from Defects and Implementing

Tools

Pronovost J, Pronovost J, Patient Safety,Patient Safety, 20052005

“Safe Design”Tools for Learning from Mistakes and Improving Safety Culture

CUSP Teamwork Tools

Morning/Shift Briefings Daily Goals Shadowing Exercise SBAR Use Critical Language (“I need some

clarity”) Culture Debrief/Checkup Tool

CUSP Safety Tools Executive Partnership Identify Defects Learning from Defects Tool Science of Safety Training

• 45 Minute online course; free registration is required• http://distance.jhsph.edu/trams/index.cfm?

event=training.launch&trainingID=72 Culture Debrief/Checkup Tool

36

Best Practices: Curiosity• Help your staff to feel heard – if they feel unheard, they will find an

ear elsewhere, at your expense• Remember your role as a leader isn’t always to solve problems, it is,

at times to listen to staff and learn from them while you empathize• Show curiosity in staff feedback –

– Don’t be defensive about a less than perfect score,

if you are defensive: “Why was that so low…,” they will be defensive and not engage

instead engage “What can be done to remove barriers so that your concerns are addressed?”

37

Copyright © 2005 The Johns Hopkins Quality and Safety Research

Group

 

0

10

20

30

40

50

60

70

80

90

100%

of r

espo

nden

ts re

port

ing

good

saf

ety

clim

ate

Safety Climate Across Michigan ICUs

Copyright © 2005 The Johns Hopkins Quality and Safety Research

Group

 

WIC

U P

RE

CU

SP

WIC

U P

OS

T C

US

P

--S

ICU

PO

ST

CU

SP

--S

ICU

Tim

e 3 --W

ICU

Tim

e 3

SIC

U P

RE

CU

SP

0

10

20

30

40

50

60

70

80

90

100

--S IC U

P OS T

C U S P

--WIC U

T ime 3

% o

f res

pond

ents

repo

rtin

g go

od s

afet

y cl

imat

eSafety Climate Across Michigan ICUs

And two JHH CUSP ICUs

Pronovost, P., et al. 2005. Implementing and validating a comprehensive unit-based safety program. Journal of Patient Safety; 1(1):33-40.

* * * * * *

* Statistically Significant

60 Safety Climate 200665 Safety Climate 200770 Safety Climate 2008

59 Safety Climate 2005

The Take Home Points• Improving Quality: You should know your culture to

be effective stewards of limited quality resources• Culture is local – work unit culture trumps hospital

culture, and is related to clinical and operational outcomes

• To assess & improve culture– Tool, administration method, interpretation, framing and

debriefing• Culture Critters that introduce new Chaos:

– New Manager, New Location, New Technology

• Patient safety and quality with methodological rigor is a pioneering effort – the science of safety is racing to keep pace

• Be ready to answer the question: – “Are We Safer?” 42

Reference ListSexton, J.B., Thomas, E.J., & Helmreich, R.L. Error, stress, and teamwork in medicine and aviation: Cross sectional surveys. BMJ. 2000; Mar

18;320(7237):745-9

Sexton J.B., Thomas E, Pronovost P: Context of care and the patient care team: The Safety Attitudes Questionnaire. National Academies of Science Report on Engineering in Healthcare. Edited by Reid P, Compton W, Grossman J, Fanjiang G. Washington,DC, The National Academies Press, 2005, pp 119-23.

Thompson D, Holzmueller C, Hunt D, Cafeo C, Sexton J.B., Pronovost P. A morning briefing: setting the stage for a clinically and operationally good day. Jt Comm J Qual Patient Saf. 2005; Aug;31(8):476-479.

Pronovost PJ, Sexton JB. Assessing safety culture: guidelines and recommendations. Qual Saf Health Care 2005; 14:231-233.

Thomas E.J., Sexton J.B., Neilands, T.B., Frankel, A. and Helmreich, R.L. The effect of executive walk rounds on nurse safety climate attitudes: A randomized trial of clinical units. BMC Health Serv Res. 2005; Jun 8;5(1):4.

Sexton J.B., Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ. The Safety Attitudes Questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Services Research. 2006; Apr 3;6(1):44.

Makary MA, Pronovost PJ, Sexton J.B., Millman EA, Freischlag JA. Patient safety in surgery. Ann Surg. 2006; May;243(5):628-635.

Makary MA & Sexton J.B., Operating Room Teamwork Among Physicians and Nurses: Teamwork in the Eye of the Beholder. Journal of the American College of Surgeons. 2006 May;202(5):746-52.

Sexton J.B., Makary MA, Tersigni AR, Pryor D, Hendrich A, Thomas EJ, Holzmueller CG, Knight AP, Wu Y, and Pronovost PJ. Teamwork in the Operating Room: Frontline Perspectives among Hospitals and Operating Room Personnel. Anesthesiology. 2006 Nov; 105(5):877-84

Rose JS, Thomas C, Tersigni A, Sexton J.B., Pryor D. A Leadership Framework for Culture Change in Healthcare. Jt Comm J Qual Patient Saf. August 2006; 32(9):433-42.

Pronovost PJ, Berenholtz SM, Goeschel CA, Needham DM, Sexton JB, Thompson DA, Lubomski LA, Marsteller JA, Makary MA, Hunt E. Creating High Reliability in Health Care Organizations. HSR. August 2006, 41; 1599-617.

Sexton J.B., Holzmueller CG, Pronovost PJ, Thomas EJ, McFerran S, Nunes J, Thompson DA, Knight AP, Penning DH, Fox HE. Variation in Caregiver Perceptions of Teamwork Climate in Labor and Delivery Units. J Perinat. 2006; 26:463-470.

Huang DT, Clermont G, Sexton JB, Karlo CA, Miller RG, Weissfeld LA, Rowan KM, Angus DC. Perceptions of safety culture vary between the intensive care units of a single institution. Crit Care Med. 2007 Jan;35(1):165-76.

Makary MA, Mukherjee A, Sexton JB, Syin D, Goodrich E, Hartmann E, Rowan L, Behrens DC, Marohn M, Pronovost PJ. Operating Room Briefings and Wrong-Site Surgery. JACS. 2007 Feb;204(2):236-43.

Sexton JB, Paine LA, Manfuso J, Holzmueller CG, Martinez EA, Moore D, Hunt DG, Pronovost PJ. A Culture Check-up for Safety in “My Patient Care Area”. Joint Commission Journal on Quality and Patient Safety. 2007 Nov; 33(11): 699-703(5).

Pronovost PJ, Berenholtz SM, Goeschel C, Thom I, Watson SR, Holzmueller CG, Lyon JS, Lubomski LH, Thompson DA, Needham D, Hyzy R, Welsh R, Roth R, Bander J, Morlock L, Sexton JB. Improving patient safety in intensive care units in Michigan. Journal of Critical Care. 2008 June:23(2): 207-221.

QUESTION & ANSWER SESSION

Click on the Q&A button to submit your questions.

Sentinel Event AlertIssue 40, July 9, 2008Behaviors that undermine a culture of safety

Intimidating and disruptive behaviors can foster medical errors,(1,2,3) contribute to poor patient satisfaction and to preventable adverse outcomes,(1,4,5) increase the cost of care,(4,5) and cause qualified clinicians, administrators and managers to seek new positions in more professional environments. (1,6) Safety and quality of patient care is dependent on teamwork, communication, and a collaborative work environment. To assure quality and to promote a culture of safety, health care organizations must address the problem of behaviors that threaten the performance of the health care team.…

45

46

SAQ Background• Administered in over 2000 hospitals (USA, United Kingdom,

Switzerland, Germany, Norway, Sweden, Spain, Portugal, Italy, Turkey, Taiwan and New Zealand)– SAQ is a reliable tool and formally validated:

• Sexton J.B., Thomas E, Pronovost P: Context of care and the patient care team: The Safety Attitudes Questionnaire. National Academies of Science Report on Engineering in Healthcare. Washington, DC: The National Academies Press, 2005.

• Sexton J.B., Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ. The Safety Attitudes Questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Services Research. 2006; Apr 3;6(1):44.

• Sexton J.B., Makary MA, Tersigni AR, Pryor D, Hendrich A, Thomas EJ, Holzmueller CG, Knight AP, Wu Y, and Pronovost PJ. Teamwork in the Operating Room: Frontline Perspectives among Hospitals and Operating Room Personnel. Anesthesiology. 2006; in press.

• Sexton J.B., Holzmueller CG, Pronovost PJ, Thomas EJ, McFerran S, Nunes J, Thompson DA, Knight AP, Penning DH, Fox HE. Variation in Caregiver Perceptions of Teamwork Climate in Labor and Delivery Units. J Perinat.2006; in press.

• Pronovost PJ and Sexton J.B., Assessing safety culture: guidelines and recommendations. Qual Saf Health Care. 2005; 14:231-233

SAQ Teamwork Climate Validity

# sites

Resp Rate CFI TLI RMSEA SRMRw SRMRb

Cronbach ICC(1) ICC(2) Rwg

1) OR 68 77%0.99

0.98 0.05 0.02 0.02 0.79 0.14, p<.001 0.86 0.78

2) L&D 44 72% 0.95 0.92 0.12 0.04 0.09 0.78 0.06, p<.001 0.83 0.83

3-4) ICU 72 72% 0.96 0.94 0.07 0.03 0.05 0.78 0.09, p<.002 0.86 0.86

1. Sexton JB, Makary MA, et al. (2006) Teamwork in the Operating Room: Frontline Perspectives among Hospitals and Operating Room Personnel. Anesthesiology

2. Sexton JB, Holzmueller CG, et al. (2006) Variation in Caregiver Perceptions of Teamwork Climate in Labor and Delivery Units. Perinatology

3. Pronovost PJ, et al. 2008 Improving patient safety in intensive care units in Michigan. Journal of Critical Care. June:23(2): 207-221.

4. Sexton JB, Helmreich RL, et al. (2006) The Safety Attitudes Questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Services Research

Labor and Delivery Collaboration Map

48

Job PositionResponse Rate

(Returned/Admin)

Age Mean Years (± SD)

Obstetrician 67% (494/739) 45 (9.91)Anesthesiologist

54% (213/401) 44 (7.83)

Registered Nurse

77% (1877/2442)

42 (10.71)

LVN/OBa 81% (227/280) 42 (10.95)Nurse Manager

& Charge Nurse

79% (136/172) 46 (7.56)

Table 1: Labor & Delivery Respondent Demographics

OBSTETRICIAN ANESTHESIOLOGISTS

NURSE MANAGERS/CHARGE NURSES

REGISTEREDNURSES

LVN/OBTECHNICIANS

Register TodayFor the Next Safer Health Care for Kids Webinar

Simple Solutions to the Complicated Problem of Home MedicationThursday, December 18, 2008

12:00 – 1:00 p.m. EST

Register at: http://www.aap.org/saferhealthcare

under “Educational Offerings”