C-CHEWS_CHD

8/12/2019 C-CHEWS_CHD

1/9

ORIGINAL ARTICLE

Validation of the Cardiac Childrens Hospital Early Warning

Score: An Early Warning Scoring Tool to PreventCardiopulmonary Arrests in Children with Heart Disease

Mary C. McLellan, BSN, RN, CPN,* Kimberlee Gauvreau, ScD, andJean A. Connor, DNSc, RN, CPNP

*Cardiovascular Program Inpatient Unit, Department of Cardiology and Cardiovascular and Critical Care Services,

Boston Childrens Hospital, Boston, Mass, USA

A B S T R A C T

Objective. Most inpatient pediatric arrests are preventable by early recognition/treatment of deterioration. Childrenwith cardiac disease have the highest arrest rates; however, early warning scoring systems have not been validated inthis population. The objective of this study was to validate the Cardiac Childrens Hospital Early Warning Score(C-CHEWS) tool in inpatient pediatric cardiac patients. The associated escalation of care algorithm directs: routinecare (score 02), increased assessment/intervention (34), or cardiac intensive care unit (CICU) consult/transfer(5).Design. Sensitivity and specificity were estimated based on retrospective review of patients that experiencedunplanned CICU transfer/arrest (n = 64) and a comparison sample (n = 248) of admissions. The previously validatedPediatric Early Warning Score (PEWS) tool was used for comparison. Patients highest C-CHEWS scores werecompared with calculated PEWS scores. Area under the receiver operating characteristic (AUROC) curve wascalculated for PEWS and C-CHEWS to measure discrimination.

Results. The AUROC curve for C-CHEWS was 0.917 compared with PEWS 0.785 (P < .001). The algorithm

AUROC curve was 0.902 vs. PEWS of 0.782. C-CHEWS algorithm sensitivity was 96.9 (score

2), 79.7 (

4), and67.2 (5) vs. PEWS of 81.1(2), 37.5 (4), and 23.4 (5). C-CHEWS specificity was 58.1 (2), 85.5 (4), and 93.6(5) vs. PEWS of 81.1 (2), 94.8 (4) and 97.6 (5). Lead time of elevated C-CHEWS scores (2) was a median of9.25 hours prior to event vs. PEWS, which was 2.25 hours and lead time for critical C-CHEWS scores ( 5) was 2hours vs. 0 hours for PEWS (P< .001).Conclusions. C-CHEWS has excellent discrimination to identify deterioration in children with cardiac disease andperformed significantly better than PEWS both as an ordinal variable and when choosing cut points to maximize

AUROC. C-CHEWS has a higher sensitivity than PEWS at all cut points.

Key Words. Cardiopulmonary Arrest Prevention; Congenital Heart Disease; Early Warning Score

Introduction

P ediatric cardiac patients have higher inci-dences of inhospital cardiac arrests than non-cardiac patients.16 Cardiac patients accounted for36% of the 3323 pediatric inhospital arrestsreported to Get with the Guidelines Resuscita-tion (previously known as the National Registryfor Cardiopulmonary Resuscitation) over 8 yearsby 265 institutions.3 Unlike other pediatric popu-lations for whom respiratory failure is the typicalarrest etiology,3,615 arrhythmias account for 41%

This article did not receive any extramural or commer-cial support.

This article was presented at the Cardiology 2012 con-ference in Orlando, FL. February 2012 and at the Pedi-atric Cardiac Intensive Care Society conference in

Miami, FL December 2012 as an oral abstract.This research was funded by Boston Childrens Hospi-tal Program for Patient Safety and Quality and theBoston Childrens Hospital Cardiovascular Program.

1

2013 Wiley Periodicals, Inc. Congenit Heart Dis. 2013;:


2/9

of cardiac patients arrest events.1,36 Of the chil-dren that do experience an arrest, cardiac patientsare younger than noncardiac patients, with morethan three-quarters


3/9

Figure

1.

C-CHEWSTool.

Congenit Heart Dis. 2013;:

C-CHEWS Tool Validation 3


4/9

A score3 is the first cut point on the C-CHEWSalgorithm that triggers an escalation response ofresources.

Data Collection

The Pediatric Early Warning Score18,19was used asthe gold standard for comparison as it was the only

validated pediatric tool available at the time of

this study.18,19

The lead investigator trained allnurse data collectors in abstraction and comple-tion of study forms. The highest documentedC-CHEWS scores from the patient cohort wereextracted from each patients electronic healthrecord. Based upon the charted documentation,trained nurse data collectors calculated the PEWSscore for the same time point as the patientshighest C-CHEWS score. To ensure reliability ofdata, an independent inter-rater reliable studynurse intermittently reviewed the collected datarecords to identify, address and reeducate data col-

lection team member regarding any discrepanciesin data abstraction to maintain inter-rater reliabil-ity of>90% within the data collection team.

In addition to the highest C-CHEWS score,PEWS and C-CHEWS scores were abstracted onthe case patients for up to 18 hours prior to thearrest or unplanned transfer to determine lead-time scores to these events.

Data were not included during time periodswhen patients were hospitalized on different unitlocations during their hospital stay (i.e., postop-erative period in the CICU). Each time the patientcame to the inpatient unit, whether as an admis-sion or transfer from another unit, it was consid-ered a separate admission event as some patientshad multiple admissions over the study period.

Data Analysis

The 919 documented observations used to calcu-late the C-CHEWS and PEWS scores included

Figure 2. C-CHEWS escalation of care algorithm.

Table 1. Demographics

Variables # of Cases (n =64) # of Controls (n = 248) Pvalue

Gender .002

Male 44 (69%) 116 (47%)Female 20 (31%) 132 (53%)

Admitting service .91Cardiology 34 (53%) 126 (51%)Cardiac surgery 22 (34%) 89 (36%)

Medicine 7 (11%) 24 (10%)Surgery 1 (2%) 9 (4%)

Age, median years 0.5 (range 0.061.3) 2.8 (range 0.054.9) .001

McLellan et al.4



5/9

documented C-CHEWS scores, vital signs, clini-cians observations, and clinical notes. The pairedt-test was used to compare the highest C-CHEWSscores and associated PEWS score to assess statis-tical significance of any observed differences

between the scores of the two tools. Area underthe receiver operating characteristic curve(AUROC) was generated for the PEWS and theC-CHEWS. Sensitivity, specificity, as well asnegative and positive predictive values were calcu-lated based upon selected cut points for the twoscores; 95% confidence intervals were generatedfor each measure. For the purpose of this study,sensitivity for the PEWS and C-CHEWS toolsis defined as the probability that a patient scores ator above a certain cut point score given that thepatient experienced an arrest and/or unplannedtransfer to an ICU. Specificity, for the purpose ofthis study, is defined as the probability that apatient scores at or below the cut point score giventhe patient had not experienced an arrest orunplanned transfer. Positive predictive value(PPV) is the probability that a patient will experi-ence an arrest or unplanned transfer to an ICUgiven that the patient scored at or above a certaincut point. The negative predictive value (NPV) isthe probability that a patient will not experiencean arrest or unplanned transfer given that theirscore is below the cut point. All data were analyzedusing STATA version 12 (StataCorp LP, College

Station, TX, USA).

Results

Demographics

The case patients were significantly younger thanthe control patients (P=.001), which is consistent

with previously published studies citing higherrates of cardiopulmonary arrests in cardiacpatients < 1 year of age (Table 2).3,4,20 Case patientsthat were male had a significantly higher occur-rence (P = .002) of unplanned transfer or arrestcompared to controls although it is unclear theetiology of this difference.

Sensitivity and Specificity

For a score 3, the sensitivity of the PEWS was54.7% (95% confidence interval [CI] 41.7, 67.2)compared with the C-CHEWS, which was 95.3%(95% CI 86.9, 99.0). For a score 5, which couldtrigger resources from the CICU, the sensitivity ofthe PEWS was 23.4% (95% CI 13.8, 35.7) com-pared with the C-CHEWS, which was 67.2%

(95% CI 54.3, 78.4) (Table 2). For a score 3, the

specificity of the PEWS was 86.3% (95% CI 81.4,90.3) compared with the C-CHEWS, which was76.2% (95% CI 70.4, 81.4) (Table 2). For a score5, the specificity of the PEWS was 97.6% (95%CI 94.8, 99.1) compared with the C-CHEWS,

which was 93.6% (95% CI 89.7, 96.3) (Table 2).For a score 3, the PPV for the PEWS was

50.7% (95% CI 38.4, 63.0) compared with theC-CHEWS 50.8% (95% CI 41.6, 60.1) (Table 3).

The PPV for a score 5 for the PEWS was 71.4%(95% CI 47.8, 88.7) compared with theC-CHEWS 72.9% (95% CI 59.6, 83.6) (Table 3).

For a score

3, the NPV for the PEWS was88.1% (95% CI 83.3, 91.9) compared with theC-CHEWS 98.4% (95% CI 95.5, 99.7) (Table 3).

The NPV for a score 5 for the PEWS was 83.2%(95% CI 78.4, 87.3) compared with theC-CHEWS 91.7% (95% CI 87.6, 94.8) (Table 3).

Using the scores as an ordinal variable (09 forPEWS and 011 for C-CHEWS), the C-CHEWShad a higher AUROC (0.917) compared with thePEWS (0.785) (P


6/9

cut points of varying combinations (Table 4). Inselecting a single cut point as to the best discrimi-nator between patients who do and do not experi-ence an unplanned transfer or arrest, specifically asingle score at or above which an intervention

would be required, a score 3 for C-CHEWS(AUROC 0.858) and score of 2 for PEWS(AUROC 0.741) would have the highest AUROC.In examining two cut points to define green,yellow, or red ranges the best choice for theC-CHEWS is the existing algorithm (Figure 2)(AUROC 0.907) where the best choice for thePEWS would be 01, 23, and 4 (AUROC0.782).

Lead Time

The times of elevated C-CHEWS and PEWSscores prior to the event times were compared toassess potential lead time for clinical intervention.For the cut point 3, the median for theC-CHEWS was 9.25 hours (range 021 hours)compared with 2.25 hours (range 020 hours) for

the PEWS. For the cut point5, the C-CHEWSwas a median of approximately 2 hours (range020 hours) compared with the median of thePEWS that was zero hours (range 016 hours).

The time of elevated scores prior to the event time

was significantly longer for the C-CHEWS thanthe PEWS for both cut points (P< .001, Wilcoxonsigned-rank test).

Discussion

The results of our study demonstrate theC-CHEWS achieved statistically significanthigher discrimination than the PEWS in identify-ing cardiovascular patients who may experience anarrest or ICU transfer than those who may not.Discrimination of the C-CHEWS algorithm wassuperior both when using the scores as ordinal

variables and when choosing cut points intendedto maximize discrimination. At all cut points, thesensitivity of the C-CHEWS was higher than thePEWS, especially as the condition of the patientsdeteriorated. Although the specificity of the twotools were comparable, the NPV of the C-CHEWS was higher than the PEWS indicatingpatients with low C-CHEWS scores have a highprobability of not experiencing an arrest orunplanned ICU transfer.

In this study, the lead time for elevatedC-CHEWS scores (3) was a median of 7 hours

longer than elevated PEWS scores (3), and thelead time for critical C-CHEWS scores (5) was 2hours longer than critical PEWS scores (5). Thisincreased lead time should allow for earlier activa-tion of resources to at-risk patients bedsides toprovide earlier treatment of deterioration and pre-

vention of cardiopulmonary arrests or unplannedtransfers.

An objective scoring tool adjusts for familiaritywith the patient and can heighten awareness ofslow deterioration. The C-CHEWS score is cal-culated using the current vital signs and clinical

assessment of the patient thus providing a real-time score to the clinician as to whether thepatient may be deteriorating. The score does notrely on further information (i.e., calculated urineoutput, patient history, and lab values) obtainedaway from the bedside to provide essential or nec-essary data. The inter-rater reliability of the tooldemonstrated that the level of nursing experiencedid not change the objectivity of the scores.

The use of a three-tiered algorithm helps toprevent taxing resources by triggering a localizedresponse first, allowing the team to intervene and

Figure 3. Area under the receiving operating characteristiccurve of the PEWS and C-CHEWS.

Table 4. AUROC of the Algorithm Cut Points

C-CHEWS PEWS

Overall scores 0.917 0.785Using a single cut point

Score 2 0.775 0.741Score 3 0.858 0.705

Using two cut pointsScores 02, 34, 5 (exist ing algorithm) 0.907 0.714

Scores 02, 35, 6 0.896 0.713Scores 02, 36, 7 0.887 *Scores 01, 23, 4 0.872 0.782Scores 01, 24, 5 0.884 0.770Scores 01, 25, 6 0.855 0.763

*Only five patients PEWS 7 and all 5 were transferred to CICU.

McLellan et al.6



7/9

potentially reversing the patients deterioration.Once a score has been elevated to the red tier,resources outside of the patients unit are mobi-lized to evaluate the patient. Should other centersconsider adapting the C-CHEWS system for their

use, merging the three-tiered algorithm into theirexisting response systems may be beneficial to effi-ciently direct resources to patients. It should benoted, this tool is intended as an early warningguide to measure deterioration; it is not intendedas an acuity measurement.

An early warning score can be an effective toolfor nurses to use when communicating concernabout subtle changes in the patient as the scoreprovides a common language between nurses andphysician colleagues.21 The tool supports clinical

judgment of the bedside nurses by quantifyingtheir assessments of their patients and using anagreed upon algorithm for evaluating and treatingthe patients. This framework may empower nursesand interns to contact attending physicians morereadily.21,22 The C-CHEWS tool and companionEscalation of Care Algorithm provides a standard-ized assessment and approach to deterioratingpatients, ensuring that there is the appropriate dis-persal of resources allocated to address the acuitylevel of the patient population.

Limitations

This study was a retrospective review limited byreview of data recorded in the electronic healthrecords. Although the analysis was conducted witha retrospective cohort, all data points wereabstracted from an established practice of nursingdocumentation standards that require patientsC-CHEWS scores to be documented a minimumof every 4 hours in the patients electronic healthrecords. The case group was verified from anongoing, quality initiative in the hospital to trackall inpatient transfers and arrests in partnership

with the Get With the Guidelines-Resuscitation.

Identified cases were further reviewed for inclu-sion criteria at the time of cohort abstraction.The C-CHEWS tool has more variables within

each domain compared with similar early warningscoring tools.2,2328 The increase in variables wasdeemed necessary to accommodate the wide agerange and baseline conditions of this patient popu-lation. Some of the assessment parameters may beredundant as they may occur simultaneously (i.e.,cool skin, mottled torso, and tachycardia, but thepresence of any one of these three variables wouldgenerate the same severity of score). This study

did not examine which variable was most powerfulor which one could be removed without loss ofpredictive power. That process may be useful butit is another study for future consideration.

This tool is currently being validated with non-

cardiac patients. The institution also is trackingwhether there is a sustainable decrease inunplanned CICU transfers on the inpatientcardiac unit as the implementation of theC-CHEWS tool and companion Escalation ofCare Algorithm.

This study was a single-center experience withan acute pediatric cardiovascular patient popula-tion and may not be generalized to other pediatricunits, even cardiovascular ones. In addition, the

validation of the PEWS tool occurred in a cohortof pediatric medical patients admitted to a generalmedicine unit19 whereas the C-CHEWS was

validated with a specific high-risk population andmay not be generalizable to other populations. Amulticenter prospective trial would be helpful inassessing if this tool may be generalized to abroader population.

Conclusion

The C-CHEWS tool has provided real-timetrigger responses that have activated necessaryresources for pediatric cardiovascular patients

who are deteriorating on the inpatient cardiac

unit. Arrest prevention will improve patient out-comes and survival for these hospitalized pediatriccardiovascular patients. In conclusion, theC-CHEWS is an early warning scoring tool spe-cific for this high-risk, vulnerable population andmay assist clinicians in recognizing and treatingthese patients early thus preventing arrests orunplanned transfers to the CICU.

Acknowledgements

Multidisciplinary panel: Roger E Breitbart MD, Jane C

Romano MS, RN, Monica Kleinman, MD, and SuzanneReidy MS, RN, NE-BC.

Staffing coordination: Adrienne P. Sullivan RN, BSN.

Study nurses: Lauren Asay and Maeve Giangregorio.

Nurse data collectors: Lauren Asay, Katherine Byrne,Laura Connelly, Elizabeth Cove, Kristen Galofaro,Jennifer Kenny, and Christine Thornton.

Database design: Vlad Gankin.

Manuscript support and review: Sandra Mott PhD, CPN,RN-BC.


C-CHEWS Tool Validation 7


8/9

Data entry: Taylor Boggs and Shawna Collins.

Author Contributions

Mary C McLellanconcept/design, protocol develop-ment, overseeing data collection, and manuscript drafting,revision and approval.

Kimberlee Gauvreaudata analysis/interpretation,statistics.

Jean A Connorconcept/design, overseeing protocoldevelopment, data collection, data analysis and manuscriptrevision and approval.

Corresponding Author: Mary C. McLellan, BSN,RN, CPN, Cardiovascular Program Inpatient Unit,Boston Childrens Hospital, 300 Longwood Ave,Boston, MA 02115, USA. Tel: (+1) 617-355-8083; Fax:(+1) 617-734-1034; E-mail: mary.mclellan@childrens

.harvard.edu

Conflict of interest:The authors do not have any disclo-sures or conflicts of interest to declare.

Accepted in final form: July 13, 2013.

References

1 Berg MD, Nadkarni VM, Zuercher M, Berg RA.In-hospital pediatric cardiac arrest. Pediatr Clin

North Am. 2008;55:589604.2 Hunt EA, Zimmer KP, Rinke ML, et al. Transition

from a traditional code team to a medical emer-gency team and categorization of cardiopulmonaryarrests in a childrens center. Arch Pediatr Adolesc

Med. 2008;162:117122.3 Ortmann L, Prodhan P, Gossett J, et al. Outcomes

after in-hospital cardiac arrest in children withcardiac disease: a report from Get With theGuidelinesResuscitation. Circulation. 2011;124:23292337.

4 Parra DA, Totapally BR, Zahn E, et al. Outcome ofcardiopulmonary resuscitation in a pediatric cardiacintensive care unit. Crit Care Med. 2000;28:32963300.

5 Rhodes JF, Blaufoux AD, Sieden HS, et al. Cardiac

arrest in infants after congenital heart surgery. Cir-culation. 1999;100 (suppl II):II194II199.

6 Samson RA, Nadkarni VM, Meaney PA, Carey SM,Berg MD, Berg RA for the American Heart

Association National Registry of CPR Investigators.Outcomes of in-hospital ventricular fibrillation inchildren.N Engl J Med. 2006;354:23282339.

7 Lopez-Herce J, Garcia C, Dominguez P, et al. forthe Spanish Group of Cardiopulmonary Arrest inChildren. Characteristics and outcome of cardio-pulmonary arrest in children. Resuscitation. 2004;63:311320.

8 Meert KL, Donaldson MS, Nadkarni V, et al.,The Pediatric Emergency Care Applied ResearchNetwork. Multicenter cohort on in-hospital pediat-ric cardiac arrest. Pediatr Crit Care Med. 2009;10:544553.

9 Reis AG, Nadkarni VM, Perondi MB, Grisi S, BergRA. A prospective investigation into the epidemiol-ogy of in-hospital pediatric cardiopulmonary resus-citation using the International Utstein reportingstyle.Pediatrics. 2002;109:200209.

10 Samson RA, Berg MD, Berg RA. Cardiopulmonaryresuscitation algorithms, defibrillation and opti-mized ventilation during resuscitation. Curr Opin

Anaesthesiol. 2006;19:146156.11 Tibballs J, Kinney S. Reduction of hospital mortal-

ity and of preventable cardiac arrest and death onintroduction of a pediatric medical emergency team.

Pediatr Crit Care Med. 2009;10:306312.12 Tibballs J, Kinney S, Duke T, Oakley E, Hennessy

M. Reduction of paediatric in-patient cardiac arrestand death with a medical emergency team: prelimi-nary results.Arch Dis Child. 2005;90:11481152.

13 Tume L. The deterioration of children in ward areasin a specialist childrens hospital. Nurs Crit Care.2007;12:1219.

14 Young KD, Seidel JS. Pediatric cardiopulmonaryresuscitation: a collective review. Ann Emerg Med.1999;33:195205.

15 Nadkarni VM, Larkin GL, Peberdy MA, et al. Firstdocumented rhythm and clinical outcomes fromin-hospital cardiac arrest among children andadults.JAMA. 2006;295:5057.

16 Kleinman M, Romano J. Childrens HospitalBoston Early Warning Score: early detection + earlytreatment = better outcomes. FIRST Do No Harm.

August 2010. Available at: http://www.mass.gov/eohhs/docs/borim/newsletters/qps-august-2010.pdf

Accessed January 18, 2013.17 McLellan MC, Connor JA. The Cardiac Childrens

Hospital Early Warning Score (C-CHEWS). JPediatr Nurs. 2012;28:171178. Available at:http://dx.doi.org/10.1016/j.pedn.2012.07.009 Accessed

January 16, 2013.18 Monaghan A. Detecting and managing deteriora-

tion in children. Paediatr Nurs. 2005;17:3235.19 Tucker KM, Brewer TL, Baker RB, Demeritt B,

Vossmeyer MT. Prospective evaluation of a pediat-ric inpatient early warning scoring system. J SocPediatr Nurs. 2008;14:7985.

20 Flick RP, Sprung J, Harrison TE, et al. Periopera-tive cardiac arrests in children between 1988 and2005 at a tertiary referral center. Anesthesiology.2007;106:226237.

21 Andrews T, Waterman H. Packaging: a groundedtheory of how to report physiological deteriorationeffectively.J Adv Nurs. 2005;52:473481.

22 Rattay JE, Lauder W, Ludwick R, et al. Indicatorsof acute deterioration in adult patients nurses in

McLellan et al.8

mailto:[email protected]:[email protected]://www.mass.gov/eohhs/docs/borim/newsletters/qps-august-2010.pdfhttp://www.mass.gov/eohhs/docs/borim/newsletters/qps-august-2010.pdfhttp://dx.doi.org/10.1016/j.pedn.2012.07.009http://dx.doi.org/10.1016/j.pedn.2012.07.009http://dx.doi.org/10.1016/j.pedn.2012.07.009http://dx.doi.org/10.1016/j.pedn.2012.07.009http://www.mass.gov/eohhs/docs/borim/newsletters/qps-august-2010.pdfhttp://www.mass.gov/eohhs/docs/borim/newsletters/qps-august-2010.pdfmailto:[email protected]:[email protected]


9/9

C-CHEWS_CHD

Documents

Transcript of C-CHEWS_CHD