By Gail Gesin, PharmD, Brittany B. Russell, RN, BSN, CCRN, Andrew P. Lin, PharmD,H. James Norton, PhD, Susan L. Evans, MD, and John W. Devlin, PharmD

Background The impact of using a validated delirium screeningtool and different levels of education on surgical-trauma intensivecare unit (STICU) nurses’ knowledge about delirium is unclear. Objectives To measure the impact of using the Intensive CareDelirium Screening Checklist (ICDSC), with or without a multi-faceted education program, on STICU nurses’ knowledge andperceptions of delirium and their ability to evaluate it correctly.Methods The knowledge and perceptions of subject nursesabout delirium, and agreement between the independent assess-ments of delirium by the subject nurse and by a validated judge(who always used the ICDSC), were compared across 3 phases.Phase 1: No delirium screening tool and no education. Phase 2:ICDSC and minimal education (ie, ICDSC validation study only).Phase 3: ICDSC and multifaceted education (ie, pharmacist-leddidactic lecture, Web-based module, and nurse-led bedsidetraining). Results Nurses’ knowledge (mean [SD] score out of 10 points)was similar (P= .08) in phase 1 (6.1 [1.4]) and phase 2 (6.5 [1.4])but was greater (P= .001) in phase 3 (8.2 [1.4]). Agreementbetween nurses and the validated judge in the assessment ofdelirium increased from phase 1 (k=0.40) to phase 2 (k=0.62)to phase 3 (k=0.74). Nurses perceived use of the ICDSC asimproving their ability to recognize delirium.Conclusions Use of a multifaceted education program improvesboth nurses’ knowledge about delirium and their perceptionsabout its recognition. Implementation of the ICDSC improvesthe ability of STICU nurses to evaluate delirium correctly.(Am J Crit Care. 2012;21:e1-e11)

IMPACT OF A DELIRIUM

SCREENING TOOL AND

MULTIFACETED EDUCATION

ON NURSES’ KNOWLEDGE

OF DELIRIUM AND ABILITY

TO EVALUATE IT CORRECTLY

e1 AJCC�AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 www.ajcconline.org

©2012 American Association of Critical-Care Nursesdoi: http://dx.doi.org/10.4037/ajcc2012605

Delirium Assessment

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

Although education is increasingly being recog-nized as a critical component of efforts to implementdelirium screening, the types of education that areoptimal, the intensity of the pedagogical strategiesthat should be used, and the members of the ICUteam best suited to deliver this education remainunclear.24-27 Pharmacists play a key role in optimiz-ing the care of patients in the ICU and may be wellsuited to spearhead ICU delirium detection andeducation efforts, given pharmacists’ daily presencein the ICU, their experience in optimizing sedationtherapy, and their involvement with delirium recog-nition and prevention efforts.8,17,23,28

Although a number of reports describe deliriumscreening efforts in medical or mixed medical-surgicalICUs, few studies have described the implementationof delirium screening efforts in STICUs.17,18,24-27,29,30

Furthermore, although results of studies on theimpact of using a validated delirium screening tool,accompanied by education surrounding its use, onthe ability of ICU nurses to recognize delirium havebeen published, the relative impact of each of these

interventions on the knowledge of nurses about delir-ium and their ability to evaluate delirium remainsunknown. We therefore sought to measure the impactof using the ICDSC, with or without a multifacetededucation program led by a pharmacist and a nurse,on the knowledge and perceptions of STICU nursesabout delirium and their ability to evaluate delir-ium correctly.

MethodsA multiprofessional task force was convened to

improve delirium detection and management in theICUs at Carolinas Medical Center, an 813-bed com-munity teaching hospital with 140adult ICU beds located in Charlotte,North Carolina. As part of thisprocess, the ICDSC was chosen asthe delirium screening tool becauseof its ease of use, its ability to beused in patients with an impairedability to communicate, and itsability to identify subsyndromaldelirium.31-33 Since the RichmondAgitation Sedation Scale (RASS) wasalready in use at this institution, thefirst ICDSC domain was adapted toinclude a RASS score for each descriptor of level ofconsciousness (Appendix 1).34 The ICDSC worksheetwas also modified so that each ICDSC domain con-tained both expanded descriptions and key questionsthat could be used by clinicians to determine whetheran abnormality in a particular domain was present.27

The ICDSC provides a score from 0 to 8, with a scoreof 4 or greater being diagnostic for the presence of

Delirium, characterized by inattention, disorganized thinking, and a fluctuatingmental status, occurs in up to 70% of patients undergoing mechanical ventila-tion in surgical-trauma intensive care units (STICUs).1-4 Given that delirium isassociated with increased mortality, a longer duration of mechanical ventilation,and the potential for serious sequelae after leaving the ICU (eg, dementia and

prolonged neuropsychological impairment), current practice guidelines recommend that patientsbe routinely screened by using a validated delirium screening tool such as the Intensive CareDelirium Screening Checklist (ICDSC).5-16 However, despite an increasing awareness amongICU clinicians regarding the sequelae of delirium and the ever-increasing use of protocols fordelirium screening efforts in ICUs, most patients admitted to ICUs are not routinely screenedfor delirium.17-23 Common barriers to delirium screening with a validated tool reported by cli-nicians include the perceptions that the tool takes too long to complete, that use of a tool doesnot enhance clinicians’ ability to recognize delirium, and that assessment tools are too com-plex to use.22-24

About the AuthorsGail Gesin and Andrew P. Lin are pharmacists, BrittanyB. Russell is a nurse in the surgical-trauma intensive careunit, H. James Norton is a biostatistician, and Susan L.Evans is a surgeon in the Division of Trauma and SurgicalCritical Care at Carolinas Medical Center in Charlotte,North Carolina. John W. Devlin is an associate professorat The Northeastern University School of Pharmacy inBoston, Massachusetts.

Corresponding author: Gail Gesin, PharmD, Carolinas Med-ical Center, PO Box 32861, Charlotte, NC 28232 (e-mail:gail.gesin@carolinashealthcare.org).

www.ajcconline.org AJCC AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 e2

Delirium occurs in up to 70% ofmechanically ventilated surgical-traumaintensive careunits patients.

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

Evaluate patient on the basis of your observations during this nursing shift.

If symptom is present at any point during your shift, count as present.

1. Altered level of consciousness (Choose ONE from A-F): Note: May need to reassess patient if recent administration of sedation therapy

A. Exaggerated response to normal stimulation RASS = +1 or greater (Score 1 point)

B. Normal wakefulness RASS = 0 (Score 0 points)

C. Response to mild or moderate stimulation RASS = -1 or -2 (Score 1 point)(Note: Only score “1” if patient has NOT received recent sedatives or analgesics)

D. Patient recently received sedation/analgesia and RASS = -1 or -2? (Score 0 points)

E. Response only to intense and repeated stimulation (e.g., loud voice and pain) RASS = -3 or -4 **Stop assessment

_

F. No response RASS = -5 **Stop assessment _

2. Inattention (Score 1 point for any of the following abnormalities):A. Patient does not follow commands (eg, wiggle toes)B. Patient is easily distracted by external stimuli C. Difficulty in shifting focus

Does the patient follow you with their eyes when you move to the opposite side of the bed? D. No symptoms present (score 0 points)E. Uncertain (score “?”)

3. Disorientation (Score 1 point for any obvious abnormality):A. Significant mistake in place and or personB. Oriented x 3 (score 0 points)C. Uncertain (score “?”)

Does the patient know that he/she is in the hospital and not elsewhere (eg, shopping mall). Does the patient recog-nize ICU caregivers who have cared for him/her and not recognize those who have not?

4. Hallucinations or Delusions (Score 1 point for either): A. Equivocal evidence of hallucinations or a behavior due to hallucinations

(Hallucination = perception of something that is not there with NO stimulus) B. Delusions or gross impairment of reality testing

(Delusion = false belief that is fixed/unchanging)C. No symptoms present (score 0 points)D. Uncertain (score “?”)

Any hallucinations now or in past 24 hrs? Are you afraid of the people or things around you? Evaluate for fear that isinappropriate for the clinical situation.

5. Psychomotor Agitation or Retardation Score (1 point for either): A. Hyperactivity requiring the use of additional sedative drugs or restraints in order to control potential danger (eg,

pulling intravenous catheters out or hitting staff) B. Hypoactive or clinically noticeable psychomotor slowing or retardation C. No symptoms present (score 0 points)D. Uncertain (score “?”)

Based on documentation and observation over shift by primary caregiver. Family members can be a good resourcewith knowledge of patient’s baseline.

6. Inappropriate Speech or Mood (Score 1 point for either):A. Inappropriate, disorganized, or incoherent speech B. Inappropriate mood related to events or situationC. No symptoms present (score 0 points)D. Uncertain (score “?”)

Is the patient apathetic to current clinical situation (eg, lack of emotion)?Any gross abnormalities in speech or mood? Is patient inappropriately demanding?

7. Sleep/Wake Cycle Disturbance (Score 1 point for): A. Sleeping less than 4 hours at night B. Waking frequently at night (do not include wakefulness initiated by medical staff or loud environment) C. Sleeping ≥ 4 hours during day D. No symptoms present (score 0 points)E. Uncertain (score “?”)

Based on primary caregiver’s assessment

8. Symptom Fluctuation (Score 1 point for): Fluctuation of any of the preceding items (ie, 1-7) in 24 hours (eg, from one shift to another) A. Yes (score 1 point)B. No (score 0 points)

Based on primary caregiver’s assessment. Compare responses from previous shifts with your responses.

TOTAL ICSDC SCORE (Add 1-8)

Nurse Initials

Appendix 1 Adapted Intensive Care Delirium Screening Checklist (ICDSC). by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

delirium. Patients with a RASS of -3, -4, or -5 cannotbe evaluated for the presence of delirium giventheir unarousable state.

The 29-bed STICU at Carolinas Medical Center,a level I trauma center with trauma patients makingup approximately 25% of the STICU population,was chosen as the first ICU where the ICDSC wouldbe implemented. To this end, a multidisciplinaryteam consisting of a critical care pharmacist (doctorof pharmacy degree and postgraduate year 2 criticalcare pharmacy residency training), a postgraduateyear 1 pharmacy resident, a clinical nurse (baccalau-reate degree and CCRN), and an attending inten-sivist (board certified in surgical critical care) werecharged with developing a process for ICDSCimplementation and education. As the initial com-ponent of this process, a 3-phase implementation andevaluation study was designed to measure theimpact of using the ICDSC in practice and evaluatethe impact of using different types and intensitiesof education during the implementation process

(Figure 1). The study was conducted during a 5-month period starting February 1, 2010, and wasapproved by the institutional review board at Car-olinas Medical Center. Twenty nurses with at least 1year of experience in STICU were recruited and pro-vided consent for participation in the study. Informedconsent was not required from patients because theirparticipation was part of an institutional qualityimprovement effort.

The primary outcome evaluated across the 3study phases was the effect of each intervention onthe knowledge of each bedside nurse about delir-ium and its detection by using the ICDSC. To thisend, a bank of 30 multiple-choice questions wascompiled by 3 of the investigators (G.G., A.P.L.,J.W.D.). Three different assessments (A, B, and C),each with 10 questions, were randomly pulled fromthis 30-question bank (Appendix 2). During eachphase of the study, subject nurses were randomlyassigned to complete 1 of the 3 assessments. Nosubject nurse received the same assessment twice.

www.ajcconline.org AJCC AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 e4

Phase 1

Phase 2

Phase 3

Figure 1 Study methods used in the 3 study phases.Abbreviation: ICDSC, Intensive Care Delirium Screening Checklist.

None

Education

Subject nurse Validated judge

Method of delirium assessment

Screening methodof choice

ICDSC ICDSC• 30-slide live presentation by pharmacist• Webcast of the live presentation• Bedside demonstration of ICDSC

assessment by validated judge

• Provision of modified ICDSC screening form

• Provision of 2001 validating article

• No further verbal or written education

ICDSC

ICDSCICDSC

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

6 �AJCC�AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 www.ajcconline.org

Appendix 2 10-question multiple choice assessment to evaluate bedside nurses’ knowledge of delirium and the ICDSC

Abbreviations: ICDSC, Intensive Care Delirium Screening Checklist; ICU, intensive care unit.

1. Hyperactive delirium is characterized by:a. Apathy b. Attempts to remove cathetersc. Emotional labilityd. Lethargye. b and c

2. Hypoactive delirium is characterized by:a. Withdrawalb. Restlessnessc. Flat affectd. Apathye. a, c, and d

3. Order the subtypes of delirium from least commonly to most common observed:a. Hyperactive, hypoactive, mixedb. Hypoactive, hyperactive, mixedc. Hypoactive, mixed, hyperactived. All are equally common

4. During the ICU/hospital stay, delirium is associated with all of the following except:a. Increased mortalityb. Development of multiorgan dysfunctionc. Increased length of stayd. 3x greater reintubation ratee. Higher costs of care

5. After hospital discharge, delirium is associated with:a. Requirement for care in chronic care facilityb. Decreased functional status at 6 monthsc. No long-term sequelaed. a and b

6. All of the following have been shown in clinical trials to be risk factors for the development of delirium except:a. Increased ageb. Disruptive family/visitorsc. Use of physical restraintsd. Use of tubes and catheters

7. When unsure about how to score a specific item on the ICDSC, it is best to:a. Count the symptom as present and score 1 for that itemb. Count the symptom as absent and score 0 for that itemc. Mark uncertain for that symptom d. Skip that item

8. All of the following would result in the patient being deemed to have inattention by the ICDSC criteria except:a. The inability to follow a command (eg, wiggle toes)b. Being easily distracted by external stimulic. Being easily distracted by internal stimulid. Having difficulty in shifting focus (can’t follow you with their eyes when you move to the opposite side of the bed)

9. The difference between hallucinations and delusions is that:a. Hallucinations are the perception of something that is not there with no stimulus, and delusions are false beliefs that are

fixed/unchangingb. Delusions are the perception of something that is not there with no stimulus, and hallucinations are false beliefs that are

fixed/unchangingc. The terms are interchangeabled. Hallucinations are the perception of something that is not there with a known stimulus, and delusions are false beliefs that are

fixed/unchanging

10. A patient would be deemed to have inappropriate speech or mood by the ICDSC in all of the following situations except:a. The patient is apathetic when hearing that he or she most likely has only days to weeks left to liveb. The patient is sexually inappropriate toward staff and visitorsc. The patient is inconsolable when hearing that he or she most likely has only days to weeks left to lived. The patient demands a new meal tray 4 times, asks for the room to be cleaned and recleaned repeatedly, and asks to be discharged

immediately from the ICUe. b and c

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

The impact of 2 different educational interven-tions was evaluated as part of the study (Figure 1).Given that different clinicians retain information bydifferent learning styles, the educational interventionwas specifically developed to accommodate kines-thetic, auditory, and visual learners.35 In the phase2 educational intervention, geared toward visuallearners, nurses were provided with a copy of theoriginal 2001 ICDSC validation study to read atleast 24 hours before the start of the phase 2 assess-ment(s) of patients.15 Although no additional edu-cation was provided to the subject nurse at this time,the nurse was allowed to ask questions pertainingto use of the ICDSC, although these questions werenot encouraged. In the phase 3 educational interven-tion that incorporated all 3 learning strategies (ie,visual, auditory, and kinesthetic), each nurse subjectunderwent a multifaceted educational program ledby both a pharmacist and the validated judge thatconsisted of (1) a 30-slide live presentation by acritical care pharmacist, (2) a Webcast educationmodule of the same presentation (Appendix 3), and(3) a bedside demonstration of delirium screeningby a validated judge using the ICDSC in at least 1patient who was not a study subject. The presenta-tion highlighted the consequences of delirium, risk

factors for its development, and challenges associatedwith its detection. Additionally, it included a com-prehensive review of how to use the IDCSC properlythat was adapted from a previous ICDSC implemen-tation and education effort at Tufts Medical Center.26

Subject nurses were encouraged toask questions during all componentsof the intensive education in phase 3.

In addition to measuring theimpact of using the ICDSC and the2 different educational interventionssupporting its use on the knowledgeof each bedside nurse, we also soughtto measure the effect of each of theseinterventions on the ability of thesubject nurses to use the IDCSC cor-rectly. This effect was measured across the 3 studyphases by assessing the agreement between the vali-dated judge and the subject nurse for the correctevaluation of delirium (ie, delirium present, deliriumabsent, or unable to assess patient) for the patient(s)the subject nurse was caring for on the shift where adelirium evaluation was scheduled to take place.

Subject nurses are assigned to care for up to 2patients per shift. Patients were excluded if they wereless than 18 years of age. Given the importance of

www.ajcconline.org AJCC AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 e6

Appendix 3 Web-based education

The educationalinterventionaccommodatedkinesthetic, auditory, andvisual learners.

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

the standard deviation of that difference. In thiscase, an effect size of 0.7 means that the underlyingdifference in means is equal to 0.7 standard devia-tions. The mean percentages correct from phase 1versus 2, phase 1 versus 3, and phase 2 versus 3 werecompared by using paired Student t tests. A 2-tailedP value less than .05 was considered statisticallysignificant. The k statistic was used to evaluateagreement between the subject nurses and the vali-dated judge for the correct evaluation of deliriumand accounts for agreement beyond that expectedby chance. This was evaluated for a quantitative levelof significance per the guidelines proposed by Landisand Koch.36 Responses for each survey questionabout nurses’ perceptions were based on a 5-pointLikert scale (1= strongly disagree, 2 =disagree, 3 =neither agree nor disagree, 4 = agree, 5 = stronglyagree). The mean difference between phases 1 and3 was calculated for each question and evaluatedfor significance by using a Wilcoxon rank test. SAS,version 9.2, was used for all analyses (SAS Institute,Cary, North Carolina).

ResultsA total of 20 nurses were recruited to participate.

One subject was lost to follow-up before participa-tion in the first phase of the study. Subject nurseswere a mean of 33.8 (SD, 8.7) years old, workedprimarily on a day shift (63%), and had worked incritical care, and specifically the STICU at CarolinasMedical Center for a mean of 6.7 (SD, 4.4) and 4.4(SD, 3) years, respectively. Sixty-three percent had abaccalaureate degree and 53% had CCRN certifica-tion. A total of 90 paired assessments were completedduring the course of the study in 73 patients, with32 completed during phase 1, 32 during phase 2,and 26 during phase 3. The mean (SD) and median(interquartile range) number of assessments per-formed by each subject nurse were 4.74 (0.81) and5.0 (4.0-5.0), respectively. The patients were predom-inately male (63%), were a mean age of 55 (SD, 18)years, and had a mean score of 16.5 (SD, 7.7) on theAcute Physiology and Chronic Health Evaluation IIat admission. Slightly more than half (55%) wereundergoing mechanical ventilation at the time of thedelirium assessment and were primarily admitted toeither the trauma (39%), general surgical (25%), ortransplant (14%) services. Of the trauma patients eval-uated, 30% had a diagnosis of traumatic brain injury.

Nursing knowledge was similar between phase 1(mean, 6.1; SD, 1.4) and phase 2 (mean, 6.5; SD, 1.4;P= .08), but greater in phase 3 (mean, 8.2; SD, 1.4;P= .001). Agreement between subject nurses and thevalidated judge for the correct evaluation of delirium

allowing the results of our study to be generalizableto all patients admitted to a STICU setting, patientswith factors that might have precluded the evalua-tion of delirium (eg, a patient in a coma related toa traumatic brain injury) were not excluded from the

study. The validated judge (B.B.R.),an experienced STICU clinical nurse,served as the gold standard fordelirium assessments. She was for-mally trained regarding use of theICDSC by an expert in the field ofdelirium (J.W.D.) at Tufts MedicalCenter in Boston, Massachusetts,using a process that was used in aprior study26 of implementation of

the ICDSC to screen for delirium. Neither routinepatient care nor the administration of analgesia,sedation, or psychoactive medications were modifiedfor the purposes of the study; however, changes mayhave occurred as a component of usual care.

During each phase, the subject nurse was askedif his or her patient had delirium. Subject nursesdid not use a delirium screening tool in phase 1;however, they were required to use the ICDSC inphases 2 and 3 (Figure 1). The validated judge usedthe ICDSC for all delirium assessments. Deliriumassessments of the identified patient were conductedfirst by the subject nurse and then independently,but consecutively, in an immediate fashion, by thevalidated judge. During each delirium assessment,the validated judge was allowed to question thesubject nurse about events that had occurred in theprior 12-hour shift that could be pertinent to theICDSC evaluation (eg, the number of hours thepatient had slept). The results of the validated judge’sassessments were not shared with the subject nurses.Finally, in an effort to characterize nurses’ percep-tions about ICU delirium identification and use ofthe ICDSC in each phase of the study, each subject

completed a survey that was adaptedfrom a previously validated surveydeveloped by experts in the field ofICU delirium.22

Data were presented by usingthe most appropriate descriptivestatistic (eg, means and standarddeviations, counts, and percentages).For the primary outcome of nursingknowledge assessed via the multiple-

choice test scores (percentage correct), 20 subjectswere needed to detect an effect size of 0.7 with analpha of .05 and a power of 80%. An effect size fora paired Student t test in this situation is the ratio ofthe mean difference between time periods relative to

e7 �AJCC�AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 www.ajcconline.org

Evaluationsincluded the

nurses’ ability touse a delirium tool correctly.

The proportion ofnurses who per-

ceived delirium tobe challenging to

assess decreased.

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

across the 3 study phases is summarized in theTable. Using the Landis and Koch criteria, agreementimproved from fair in phase 1 (k=0.40; 95% CI, 0.11-0.69) to substantial agreement in phases 2 (k=0.62;95% CI, 0.39-0.69) and remained at this level inphase 3 (k=0.74; 95% CI, 0.69-0.95).

Across the 3 study phases, the proportion ofnurses who perceived delirium to be challenging toassess in ICU patients decreased (89.5% in phase 1,to 78.9% in phase 2, to 63.2% in phase 3), and theproportion of nurses who agreed with the statementthat the ICDSC makes delirium easier to identify intheir patient(s) increased (57.9% in phase 1, to78.9% in phase 2, to 89.5% in phase 3; Figure 2).The mean difference on the 5-point Likert scalebetween phases 1 and 3 for each of these questionswas -0.53 (P= .07) and 0.58 (P= .06), respectively.

DiscussionOur study is the first to demonstrate in an STICU

that the addition of a multifaceted educationalintervention that incorporates both didactic andbedside pedagogical strategies and that is deliveredby both critical care pharmacists and nurses improvesnurses’ knowledge about delirium beyond the knowl-edge seen with minimal education alone. Further-more, the study demonstrates that use of the ICDSCimproves the ability of nurses to evaluate patientscorrectly for delirium. The interventions used in ourstudy also improve nurses’ perceptions about theimportance of using a validated instrument to screenfor delirium. Finally, our study demonstrates thatcritical care pharmacists can play a key role in col-laborating with critical care nurses and physicianswhen delirium screening and educational effortsare being developed and implemented in the ICU.

Previously described delirium screening effortsin the STICU have been focused on characterizingthe reliability between the bedside nurse and atrained expert for assessing the presence of deliriumafter implementation of the Confusion AssessmentMethod for the Intensive Care Unit (CAM-ICU)accompanied by a comprehensive education pro-gram.17,25,30,37,38 However, by combining these compo-nents as 1 intervention, the relative impact ofimplementing a delirium screening tool or variousintensities of education about its use on the abilityof nurses to evaluate delirium correctly remainsunknown. Because the symptoms of delirium thatthe CAM-ICU and the ICDSC evaluate differ, onecannot extrapolate results from studies using theCAM-ICU to results of studies using the ICDSC.39,40

A variety of educational strategies have beendescribed to boost efforts to implement delirium

screening.17,18,24-27,29,30 These strategies can be catego-rized as being live or Web-based, formal or informal,scenario-based or not scenario-based, classroom(eg, didactic) or bedside-based, and intermittent orcontinuous. Presentation content is commonlydescribed to include in-depth descriptions of seda-tion and delirium scoring tools.17,26,27,29 A simpledidactic and bedside educational intervention thatincorporated script concordance theory, and thattook an average of 1.5 hours to conduct, increasedthe ability of ICU nurses to evaluate delirium prop-erly by using a scale (12% vs 82%, P< .001) and usethis scale correctly (8% vs 62%, P< .001).26 An abbre-viated educational intervention that took an averageof 15 minutes to complete that accompanied imple-mentation of the ICDSC to physicians in the medical

www.ajcconline.org AJCC AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 e8

Table Overall agreement between subjectnurse and validated judge for deliriumassessment

Study phase ĸ value (95% CI)No. in agreement/total (%)

1

2

3

0.40 (0.11-0.69)

0.62 (0.39-0.69)

0.74 (0.69-0.95)

22/32 (69)

26/32 (81)

23/26 (88)

Figure 2 Perceptions of study nurses regarding delirium screening.Abbreviation: ICDSC, Intensive Care Delirium Screening Checklist; ICU, intensivecare unit.

Delirium is challenging toassess in ICU patients

The ICDSC makes delirium easier to identify

in my patient(s)

Phase 1 Phase 2 Phase 3

0 50 100

% of nurses who agree

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

assess. Additionally, although improvements innurses’ perceptions were noted between phases 1and 3, the difference between the phases was notsignificant. These findings highlight the importanceof developing a continuous quality improvementplan for delirium screening that measures bothscreening compliance and quality. Strategies toobtain end-user feedback and provide supplementaldidactic and bedside nursing education when neededhave been described and should be considered tohelp sustain the benefits gained during the initialimplementation phase.17,25,29

Our study has several strengths, including thefact that we separated implementation of the ICDSCand use of the educational interventions into differ-ent phases and that the comprehensive educationalintervention evaluated was validated in a priorstudy.26 Furthermore, the expert evaluator used theICDSC during all assessments and was formallytrained regarding its use at Tufts Medical Center.Last, the external validity of our study is high giventhat no patients were excluded. The low rate of delir-ium detected in our study (13%, 12 of 90 assess-ments) deserves mention because it is substantiallylower than the rate detected in other STICU stud-ies.2,3,4,8,18 This can be explained, in part, by our useof a screening tool (ie, the ICDSC) that has a lowerreported sensitivity than the tool used in otherpublished studies (ie, CAM-ICU) and our inclusionof patients with conditions where delirium cannotbe evaluated (eg, severe traumatic brain injury).39

Additionally, we did not have a requirement formechanical ventilation and thus, unlike other stud-ies in this population, our patients may have had alower baseline risk for delirium. Finally, unlikeother studies where delirium assessment was per-formed from ICU admission to the time of dis-charge, we evaluated patients only at a single pointin time. Given the fluctuation of delirium symp-toms, increasing the frequency and/or duration ofdelirium assessments would most likely increasethe likelihood for detecting delirium. As most ofour patients either did not have delirium or hadfactors that precluded its assessment, we cannotdraw conclusions specific to the subgroup of patientswith delirium. Therefore, further investigation isneeded in a study that is larger and less inclusivethan ours.

Although all paired assessments were conductedas soon as possible after each educational phasewas completed, we did not standardize the timebetween education delivery and the time of theassessments and thus subject nurse recall of theinformation provided could have varied. It remains

ICU improved their ability to identify delirium cor-rectly.27 Finally, use of printed materials such ashandouts, pocket cards, bulletin boards, and posters

has been incorporated as a compo-nent of the education process.17,18,24,25,29

Although a plethora of differentstrategies have been reported toeducate ICU clinicians about delir-ium, few studies17,18,24-27,29,30 havecharacterized the optimal type andintensity of education that shouldbe used. These data are importantbecause many of the reported barri-ers to delirium assessment in theICU are tied to the need for greatereducation.22-24 However, with an

ever-increasing scarcity of health care resources,evidence to justify expanded educational efforts tosupport ICU delirium screening pro grams is greatlyneeded. We found that an intensive strategy consist-ing of live pharmacist-led lectures, Web-based train-ing, and a nurse-led bedside demonstration improveddelirium-related knowledge among nurses far morethan simply handing nurses the original ICDSC val-idation study. In our study, the time required todeliver the multifaceted education to each subjectnurse was approximately 45 to 60 minutes. Althoughour results suggest that institutions will need todevote substantial resources (particularly the timespent by clinical nurse educators and/or pharmacists)to optimize the delivery of a multifaceted educationintervention such as the one used in our study,cheaper, more efficient educational strategies maywork just as well (eg, a “train-the-trainer” approach,

where a trained staff nurse trainstheir colleagues at the bedside afterthey have completed the Web-based education).

Perceptions among nurses andpharmacists about delirium and itsidentification pose a significantbarrier to the success of efforts toimplement delirium screening.22-24

Educational efforts designed withthese barriers in mind may impart

a positive influence on such perceptions and con-tribute to the success of screening. Our educationalprogram included information on not just the epi-demiology and consequences of delirium (eg, theimportance of screening) but also on how the ICDSCshould be properly used. Although the number ofnurses who reported that delirium was challengingto assess decreased after education, 63% continuedto report that delirium remained challenging to

e9 �AJCC�AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 www.ajcconline.org

Use of the Intensive Care

Delirium ScreeningChecklist improved

nurses’ ability toevaluate delirium

correctly.

Reported barriersto delirium assess-ment in the inten-sive care unit are

tied to the need forgreater education.

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

to be determined if individual learning styles influ-ence the knowledge gained from multifaceted edu-cational interventions focused on improving use ofdelirium screening tools, given that we did not eval-uate the preferred learning styles of the nurses atbaseline. In an effort to enroll nurses who weremotivated to complete all phases of the study, wedid not randomly select nurses to participate fromthe study ICU and thus our population of nursesmay not have been representative of the entire studySTICU. We failed to evaluate the number of nurseswho actually read the ICDSC-validating article, andin some situations, nurses may have been forced tocomplete the multiple choice assessments and sur-veys in the sometimes loud and distracting environ-ment of the ICU. Finally, we did not collect specificdata regarding how long it took each nurse to com-plete each ICDSC assessment.

ConclusionsDelirium screening with a validated tool is a

key component of patient care in the ICU. It isencouraging that subject nurses viewed the ICDSCas an easy-to-use delirium screening method. Delir-ium screening efforts should be accompanied by aneducational intervention that incorporates bothdidactic and bedside pedagogical methods, usesboth live and Web-based strategies, and includescritical care pharmacists in key roles. Implementa-tion of the ICDSC in a STICU improved the abilityof nurses to evaluate patients for delirium correctly.Further research is required to investigate the sus-tainability of the gains observed in our study, theeffect of delirium assessments conducted by STICUnurses on patients’ outcomes, and the impact ofimplementing the ICDSC and multifaceted educa-tion about its use in other types of ICUs.

ACKNOWLEDGMENTSWe thank the nurses of the STICU for making this studypossible, Meredith Moore, RN, for her contributions tothe data analysis, and Yoanna Skrobik, MD, for herreview of the manuscript.

FINANCIAL DISCLOSURESFinancial support for this project was provided by theDepartments of Nursing and Pharmacy at the CarolinasMedical Center in Charlotte, North Carolina.

REFERENCES 1. American Psychiatric Association. Diagnostic and Statistical

Manual of Mental Disorders. 4th ed, text revision. Arlington,VA: American Psychiatric Association; 2000.

2. Pandharipande P, Cotton BA, Shintani A, et al. Prevalenceand risk factors for development of delirium in surgical andtrauma intensive care unit patients. J Trauma. 2008;65:34-41.

3. Pandharipande P, Cotton BA, Shintani A, et al. Motoricsubtypes of delirium in mechanically ventilated surgicaland trauma intensive care unit patients. Intensive CareMed. 2007;33:1726-1731.

4. Angles EM, Robinson TN, Biffl WL, et al. Risk factors fordelirium after major trauma. Am J Surg. 2008;196:864-870.

5. Ely EW, Shintani A, Truman B, et al. Delirium as a predictorof mortality in mechanically ventilated patients in the inten-sive care unit. JAMA. 2004;291:1753-1762.

6. Shehabi Y, Riker RR, Bokesch PM, et al. Delirium durationand mortality in lightly sedated, mechanically ventilatedintensive care patients. Crit Care Med. 2010;38:2311-2318.

7. Lin SM, Liu CY, Wang CH, et al. The impact of delirium onthe survival of mechanically ventilated patients. Crit CareMed. 2004;32:2254-2259.

8. Lat I, McMillian W, Taylor S, et al. The impact of deliriumon clinical outcomes in mechanically ventilated surgicaland trauma patients. Crit Care Med. 2009;37:1898-1905.

9. Skrobik YS. Delirium prevention and treatment. Crit CareClin. 2009;25:585-591.

10. Rockwood K, Cosway S, Carver D, et al. The risk of dementiaand death after delirium. Age Ageing. 1999;28:551-556.

11. Jackson JC, Hart RP, Gordon SM, et al. Six-month neuropsy-chological outcome of medical intensive care unit patients.Crit Care Med. 2003;31:1226-1234.

12. Jacobi J, Fraser GL, Coursin DB, et al. Clinical practiceguidelines for the sustained use of sedatives and analgesicsin the critically ill adult. Crit Care Med. 2002;30:119-141.

13. Young J, Murthy L, Westby M, et al. Diagnosis, prevention,and management of delirium: summary of NICE guidance.BMJ. 2010;341:c3704.

14. Martin J, Heyman A, Basell K, et al. Evidence and consensus-based German guidelines for the management of analgesia,sedation and delirium in intensive care: short version. GerMed Sci. 2010;8:Doc02.

15. Bergeron N, Dubois M, Dumont M, et al. Intensive caredelirium screening checklist: evaluation of a new screeningtool. Intensive Care Med. 2001;27:859-864.

16. Girard TD, Jackson JC, Pandharipande PP, et al. Deliriumas a predictor of long-term cognitive impairment in survivorsof critical illness. Crit Care Med. 2010;38:1513-1520.

17. Soja SL, Pandharipande P, Fleming SB, et al. Implementation,reliability testing, and compliance monitoring of the confu-sion assessment method for the intensive care unit intrauma patients. Intensive Care Med. 2008;34:1263-1268.

18. Page VJ, Navarange S, Gama S, et al. Routine deliriummonitoring in a UK critical care unit. Crit Care. 2009;13:R16.

19. Patel RP, Gambrell M, Speroff T, et al. Delirium and sedationin the intensive care unit: survey of behaviors and attitudesof 1384 healthcare professionals. Crit Care Med. 2009;37:825-832.

20. Cadogan FL, Riekerk B, Vreeswijk R, et al. Current awarenessof delirium in the intensive care unit: a postal survey in theNetherlands. Neth J Med. 2009;67:296-300.

21. Forsgren LM, Eriksson M. Delirium—awareness, observationand interventions in intensive care units: a national surveyof Swedish ICU head nurses. Intensive Crit Care Nurs. 2010;26:296-303.

22. Devlin JW, Fong JJ, Howard EP, et al. Assessment of delir-ium in the intensive care unit: nursing practices and per-ceptions. Am J Crit Care. 2008;17:555-566.

23. Devlin JW, Bhat S, Roberts RJ, et al. Current perceptions andpractices surrounding the recognition and treatment of delir-ium in the intensive care unit: a survey of 250 critical carepharmacists from eight states. Ann Pharmacother. 2011;45:1217-1229.

24. Riekerk B, Pen EJ, Hofhuis JG, et al. Limitations and practi-calities of CAM-ICU implementation, a delirium scoringsystem, in a Dutch intensive care unit. Intensive Crit CareNurs. 2009;25:242-249.

25. van den Boogaard M, Pickkers P, van der Hoeven H, et al.Implementation of a delirium assessment tool in the ICUcan influence haloperidol use. Crit Care. 2009;13:R131.

26. Devlin JW, Marquis F, Riker RR, et al. Combined didacticand scenario-based education improves the ability of inten-sive care unit staff to recognize delirium at the bedside.Crit Care. 2008;12:R19.

27. Devlin JW, Fong JJ, Schumaker G, et al. Use of a validated

www.ajcconline.org AJCC AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 e10

eLettersNow that you’ve read the article, create or contribute to anonline discussion on this topic. Visit www.ajcconline.organd click “Submit a response” in either the full-text orPDF view of the article.

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

delirium assessment tool improves the ability of physiciansto identify delirium in medical intensive care unit patients.Crit Care Med. 2007;35:2721-2725.

28. Marshall J, Finn CA, Theodore AC. Impact of a clinicalpharmacist-enforced intensive care unit sedation protocolon duration of mechanical ventilation and hospital stay.Crit Care Med. 2008;36:427-433.

29. Pun BT, Gordon SM, Peterson JF, et al. Large-scale imple-mentation of sedation and delirium monitoring in the inten-sive care unit: a report from two medical centers. Crit CareMed. 2005;33:1199-1205.

30. Robinson BR, Mueller EW, Henson K, et al. An analgesia-delirium-sedation protocol for critically ill trauma patientsreduces ventilator days and hospital length of stay. J Trauma.2008;65:517-526.

31. Ouimet S, Riker R, Bergeron N, et al. Subsyndromal deliriumin the ICU: evidence for a disease spectrum. Intensive CareMed. 2007;33:1007-1013.

32. Roberts B, Rickard CM, Rajbhandari D, et al. Multicentrestudy of delirium in ICU patients using a simple screeningtool. Aust Crit Care. 2005;18:6-16.

33. Skrobik Y, Devlin JW. The intensive care delirium screeningchecklist has many potential benefits over the nursingdelirium screening scale. Crit Care Med. 2010;38:1610-1611.

34. Sessler CN, Gosnell M, Grap MJ, et al. The RichmondAgitation-Sedation Scale: validity and reliability in adult

intensive care patients. Am J Respir Crit Care Med. 2002;166:1338-1344.

35. Smith A. Learning styles of registered nurses enrolled inan online nursing program. J Prof Nurs. 2010;26:49-53.

36. Landis RJ, Koch GG. The measurement of observer agree-ment for categorical data. Biometrics. 1977;33:159-174.

37. Ely EW, Margolin R, Francis J, et al. Evaluation of deliriumin critically ill patients: validation of the confusion assess-ment method for the intensive care unit (CAM-ICU). CritCare Med. 2001;29:1370-1379.

38. Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechani-cally ventilated patients: validity and reliability of the con-fusion assessment method for the intensive care unit(CAM-ICU). JAMA. 2001;286:2703-2710.

39. van Eijk MMJ, van Marum RJ, Klijn IAM, et al. Comparisonof delirium assessment tools in a mixed intensive careunit. Crit Care Med. 2009;37:1881-1885.

40. Neufeld KJ, Hayat MJ, Coughlin JM, et al. Evaluation of twointensive care delirium screening tools for non-critically illhospitalized patients. Psychosomatics. 2011;52:133-140.

www.ajcconline.org AJCC AMERICAN JOURNAL OF CRITICAL CARE, January 2012, Volume 21, No. 1 e11

To purchase electronic or print reprints, contact TheInnoVision Group, 101 Columbia, Aliso Viejo, CA 92656.Phone, (800) 899-1712 or (949) 362-2050 (ext 532); fax,(949) 362-2049; e-mail, reprints@aacn.org.

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from

Gail Gesin, Brittany B. Russell, Andrew P. Lin, H. James Norton, Susan L. Evans and John W. DevlinDelirium and Ability to Evaluate It CorrectlyImpact of a Delirium Screening Tool and Multifaceted Education on Nurses' Knowledge of

http://ajcc.aacnjournals.org/Published online ©2012 American Association of Critical-Care Nurses

10.4037/ajcc2012605 e1-e11 21 2012;Am J Crit Care

http://ajcc.aacnjournals.org/cgi/external_ref?link_type=PERMISSIONDIRECTPersonal use only. For copyright permission information:

http://ajcc.aacnjournals.org/subscriptions/Subscription Information

http://ajcc.aacnjournals.org/misc/ifora.xhtmlInformation for authors

http://www.editorialmanager.com/ajcc Submit a manuscript

http://ajcc.aacnjournals.org/subscriptions/etoc.xhtmlEmail alerts

. 532. Fax: (949) 362-2049. Copyright ©2016 by AACN. All rights reserved(AACN) published bimonthly by AACN, 101 Columbia, Aliso Viejo, CA 92656. Telephone: (800) 899-1712, (949) 362-2050, ext. The American Journal of Critical Care is an official peer-reviewed journal of the American Association of Critical-Care Nurses

by AACN on June 13, 2018http://ajcc.aacnjournals.org/Downloaded from