Original Pediatric

ORIGINAL RESEARCH

Indicators of pain in neonates at risk for neurological impairment

Bonnie Stevens, Patrick McGrath, Annie Dupuis, Sharyn Gibbins, Joseph Beyene, Lynn Breau,

Carol Camfield, Gordon Allen Finley, Linda S. Franck, Alexandra Howlett, Celeste Johnston,

Patricia McKeever, Karel OBrien, Arne Ohlsson & Janet Yamada

Accepted for publication 22 August 2008

Correspondence to B. Stevens:

e-mail: [email protected]

Bonnie Stevens PhD RN

Signy Hildur Eaton Chair in Paediatric

Nursing Research

Associate Chief Nursing Research

The Hospital for Sick Children,

Toronto, Ontario, Canada

Patrick McGrath PhD OC FRSC

Vice President, Research

IWK Health Centre, Dalhousie University,

Halifax, Nova Scotia, Canada

Annie Dupuis PhD

Biostatistician

Child Health Evaluative Sciences,

Research Institute, The Hospital for

Sick Children, Toronto, Ontario, Canada

Sharyn Gibbins PhD RN

Director of Interdisciplinary Research

Sunnybrook Health Sciences Centre,


Joseph Beyene PhD

Scientist

Child Health Evaluative Sciences, Research

Institute, The Hospital for Sick Children,


Lynn Breau PhD

Registered Psychologist

School of Nursing, Dalhousie University,


Carol Camfield MD FRCPC

Professor of Pediatrics

Department of Pediatric Neurology,

IWK Health Centre


STEVENS B. , MCGRATH P. , DUPUIS A. , GIBBINS S. , BEYENE J. , BREAU L. ,STEVENS B. , MCGRATH P. , DUPUIS A. , GIBBINS S. , BEYENE J. , BREAU L. ,

CAMFIELD C. , ALLEN FINLEY G., FRANCK L.S . , HOWLETT A., JOHNSTON C.,CAMFIELD C., ALLEN FINLEY G., FRANCK L.S. , HOWLETT A., JOHNSTON C. ,

MCKEEVER P. , OBRIEN K. , OHLSSON A. & YAMADA J. (2009)MCKEEVER P. , OBRIEN K., OHLSSON A. & YAMADA J. (2009) Indicators of

pain in neonates at risk for neurological impairment. Journal of Advanced Nursing

65(2), 285296

doi: 10.1111/j.1365-2648.2008.04854.x

AbstractTitle. Indicators of pain in neonates at risk for neurological impairment.

Aim. This paper is a report of a study to compare the importance and usefulness

ratings of physiological and behavioural indicators of pain in neonates at risk for

neurological impairment by nurse clinicians and pain researchers.

Background. Neonates at risk for neurological impairment have not been system-

atically included in neonatal pain measure development and how clinicians and

researchers view pain indicators in these infants is unknown.

Methods. Data triangulation was undertaken in three Canadian Neonatal Intensive

Care Units using data from: (a) 149 neonates at high, moderate and low risk for

neurological impairment, (b) 95 nurse clinicians from the three units where infant

data were collected and (c) 14 international pain researchers. Thirteen indicators

were assessed following heel lance in neonates and 39 indicators generated from

nurse clinicians and pain researchers were assessed for importance and accuracy.

Data were collected between 2004 and 2005.

Results. Across risk groups, indicators with the highest accuracy for discriminating

pain among neonates were: brow bulge (7783%), eye squeeze (7584%), naso-

labial furrow (7981%), and total facial expression (7883%). Correlations

between nurse ratings and neonatal accuracy scores ranged from moderate to none

(mild risk r = 052, P = 007; moderate r = 043, P = 015; high r = 012,P = 069). Researchers demonstrated a better understanding of the importance of

pain indicators (mild risk, r = 091, P < 0001; moderate 085, P < 0001; 00002;

high r = 064, P = 0019) than nurse clinicians.

Conclusion/Discussion. Facial actions were rated as the most important indicators

of neonatal pain. However, as neurological impairment risk increased, physiological

indicators were rated more important by nurse clinicians and pain researchers,

opposite to pain indicators demonstrated by neonates.

Keywords: neonates, neurological impairment, nurses, pain indicators, researcherscontinued on page 286

2008 The Authors. Journal compilation 2008 Blackwell Publishing Ltd 285

JAN JOURNAL OF ADVANCED NURSING

Gordon Allen Finley MD FRCPC FAAP

Medical Director

Pediatric Pain Management, Centre for

Pediatric Pain Research, IWK Health Centre


Linda S. Franck PhD RN RSCN

Professor of Childrens Nursing Research

Institute of Child Health, Centre for Nursing

and Allied Health Professions Research,

Great Ormond Street Hospital for Children,

London, UK

Alexandra Howlett MD FRCPC

Neonatologist

Department of Pediatrics,

IWK Health Centre,


Celeste Johnston DEd RN

James McGill Professor

School of Nursing, McGill University,

Montreal, Quebec, Canada

Patricia McKeever PhD RN

Senior Scientist

Bloorview Kids Foundation Chair in

Childhood Disability Studies,

and Professor

Lawrence S. Bloomberg Faculty of Nursing,

University of Toronto,

Ontario, Canada

Karel OBrien MD FRCPC

Interim Pediatrican-in Chief

Department of Pediatrics, Mount Sinai

Hospital Toronto, Ontario, Canada

Arne Ohlsson MD FRCPC

Professor

Departments of Paediatrics, Obstetrics and

Gynaecology, and Health Policy,

Management and Evaluation, University of

Toronto,

and Director

Evidence Based Neonatal Care and

Outcomes Research,

Department of Pediatrics,

Mount Sinai Hospital,

Ontario, Canada

Janet Yamada MSc, RN

Research Associate

Child Health Evaluative Sciences,

Research Institute, Hospital for Sick

Children, Toronto, Ontario, Canada

Introduction

Responses of healthy term and preterm neonates to painful

procedures have been well documented (Johnston et al.

1997, Stevens et al. 1999, Thase et al. 2000, Craig et al.

2002). These behavioural and physiological responses have

been established as the basis for the development and

validation of multiple neonatal pain assessment measures by

researchers worldwide (See summary of neonatal pain

measures in Duhn & Medves 2004, Stevens et al. 2007a).

However, risk for neurological impairment (NI) has not

been adequately considered in pain assessment in neonates

by either clinicians or researchers. Risk for NI is determined

by a multiplicity of factors, including congenital syndrome/

chromosomal abnormalities (e.g. congenital trisomies), birth

trauma (e.g. fractures, nerve injuries), extreme preterm birth

[e.g. intraventricular haemorrhage (IVH); necrotising entero-

colitis (NEC)], and acquired illnesses with central nervous

system involvement (Robertson et al. 1998). Risk for NI is

an important consideration for pain assessment by clinicians

and researchers as infants at the highest risk for NI have

been reported to undergo statistically significantly more

painful procedures (e.g. heel lances) and to receive less

analgesia than infants at minimal or no risk for NI (Stevens

et al. 2003). Although physiological and behavioural pain

indicators in infants at risk for NI have been described

(Stevens et al. 2003, 2007b), little is known about how

these indicators are interpreted by the nurses who care for

these neonates in the clinical setting or by neonatal pain

researchers who generate knowledge to promote evidence-

based pain assessment and management. Pain may be

inaccurately assessed and managed in neonates at risk for

NI when reciprocal interaction between infant and care

provider is not adequately considered and when measures

that were not validated for neonatal and infant populations

at risk for NI are used. Therefore, triangulation of data

from multiple sources that is guided through a conceptual

framework will enhance our understanding and enhance-

ment of pain in this population.

Background

Two studies have been undertaken to explore healthcare

professionals pain perceptions. In the first study, 99 health-

care professionals completed the Pain Opinion Questionnaire

aimed at understanding beliefs and attitudes about pain

assessment and management in neonates at risk for NI. They

believed that these neonates have less pain than those who are

not at risk for NI, and that neonates at higher levels of NI are

likely to have less pain (Breau et al. 2006). In the second

B. Stevens et al.

286 2008 The Authors. Journal compilation 2008 Blackwell Publishing Ltd

study, 95 healthcare professionals reviewed nine video clips

depicting preterm and term neonates having a heel lance

(Breau et al. 2004). Prior to the video, the level of NI was

provided. Ratings of pain, distress, and time to calm did not

vary statistically significantly with level of NI risk, but ratings

of the effectiveness of a non-pharmacological intervention

were statistically significantly lower as risk increased. Neither

study included an assessment of how healthcare professionals

rated the importance and/or usefulness of individual pain

indicators. No research to our knowledge has examined these

ratings in neonatal/infant pain researchers.

Conceptual model

The Socio Communication Model of Infant Pain (Craig &

Pillai Riddell 2003) provided the conceptual framework for

this study. This model asserts that infant pain should be

viewed as a dynamic, interactive process between infant and

caregiver. Two stages of inquiry (Pain Expression; Pain

Experience) are central to this model. Pain expression is

operationalized as the response of infants to a painful (tissue-

damaging) procedure. As pain is subjective (International

Association for the Study of Pain, 2003) and, given the

infants incapacity of verbal report, behavioural and physi-

ological indicators are accepted as proxies for self-report

(Anand & Craig 1996). Furthermore, due to the infants

vulnerable nature and their dependency on caregivers, their

pain is intrinsically linked to the caregiver. Pain Experience is

determined by how: (a) clinicians who care for these infants

and (b) researchers who develop infant pain measures

interpret the pain expression of infants at risk for NI, in

either clinical practice or the development of neonatal pain

measures. To link the stages of inquiry proposed in the

conceptual framework with our data collection methods, we

elicited the perspectives of clinicians and researchers on the

importance and usefulness of potential pain indicators,

observed the behavioural and physiological indicators of

pain in infants undergoing a painful event, and through the

process of data triangulation, determined how pain expres-

sion and pain experience were influenced by NI risk status in

infants.

The study

Aim

The aim of this study was to compare the importance and

usefulness ratings of physiological and behavioural indicators

of pain in neonates at risk for NI by nurse clinicians and pain

researchers.

Design

Three sets of data were obtained from three sources and

integrated through a process of triangulation. Triangulation

involves using multiple sources, methods and investigators to

corroborate evidence to shed light on a theme or perspective

(Creswell 1998). Nurses in clinical practice and pain

researchers opinions on the importance and usefulness of

these indicators of pain were compared to the neonates

expressions of pain during a painful procedure. This study

was one of five investigations in a programme of research on

Pain in Infants at risk for Neurological Impairment (PINI)

supported by a grant from the Canadian Institutes of Health

Research (MCT 63143).

Participants

Neonates

The first set of data was collected from a prospective obser-

vational cohort study of 149 neonates at three tertiary level

neonatal intensive care units (NICUs) in central and eastern

Canada (Stevens et al. 2007b). Eligible neonates were hos-

pitalized in the NICU,>25 weeks gestational age,

Pain researchers

The third set of data was collected from 14 international,

multidisciplinary pain researchers as part of a Delphi consen-

sus exercise (Stevens et al. 2006). These experts were identified

by the 13 PINI study investigators and from the International

Association for the Study of Pain Directory ofMembers, 2004.

Eligible participants had at least 2 years of research experience

related to the assessment andmanagement of pain in neonates,

infants and young children, had published in peer-reviewed

pain journals, had presented at major paediatric pain meetings

or were known to be involved in current research in neonatal/

infant pain. Co-investigators in the PINI programme of

research were excluded.

Data collection

Neonatal physiological data were collected using a comput-

erized multi-channel data collection system (Compumedics

USA Ltd., El Paso, TX, USA) coordinated with behavioural

data collected by audiotaping and videotaping infants during

baseline, procedure, and return to baseline phases of a routine

heel lance procedure. Data collection procedures/protocols

that have been validated in multiple studies by Stevens and

Johnston are described elsewhere (Johnston et al. 1996,

Stevens et al. 1999, 2003, 2007b). All data were collected for

the three studies between 2004 and 2005. Demographic data

were abstracted from neonates medical records.

Physiological pain indicators

Changes from baseline in heart rate, heart rate variability and

oxygen saturation were assessed. In previous research on

healthy term and preterm neonates, heart rate and oxygen

saturation have been the most prevalent physiological indi-

cators of pain and have been included in several composite

neonatal pain measures (van Dijk et al. 2000, Debillon et al.

2001), including the Premature Infant Pain Profile (Stevens

et al. 1996).

All physiological and behavioural data were collected using

the Compumedics E-Series ProFusion PSG documentation

software (Compumedics USA Ltd.). Disposable ECG elec-

trodes and pulse oximetry probes were placed on the

neonates and ECG, respiratory rate and oxygen saturation

were continuously recorded using a cardiorespiratory mon-

itor and personal computer (1000 HZ sampling rate). An

electronic event marker was used to mark each phase of the

heel lance procedure.

Behavioural pain indicators

Facial actions were collected by videotaping the neonates

pain response on an 8 mm camcorder (Sharp, Panasonic or

Sony) and audiotaping through all phases of the heel lance

procedure. Facial actions were independently coded second-

to-second according to a reliable and valid measure of facial

expression, the Neonatal Facial Coding System (NFCS)

(Grunau & Craig 1990, Stevens et al. 2007a) by two trained

coders with established inter- and intra-rater reliability

(>95%). A final score based on percentage of time the action

was present was calculated for each individual indicator. A

total facial action score was calculated by summing the

individual scores.

Procedure for nurse and researcher data collection

A 39- item self-administered questionnaire was developed by

the investigators to include indicators representing neonatal

physiological, behavioural and contextual pain responses as

well as pain indicators identified by parents and healthcare

professionals (Stevens et al. 2006). Face and content validity

were established with the 13 study investigators and with 10

advanced practice nurses with expertise in pain assessment

and management from three local neonatal and paediatric

intensive care units. As a result of this validation, only minor

changes were made to the formatting and wording of the

questionnaires.

The questionnaire was organized to assess four domains: (a)

physical indicators (11 items); (b) facial actions (ten items); (c)

vocal behaviour/cry indicators (nine items); (d) neonatal

activity (nine items). Each participant was asked to give

opinions on the importance and usefulness of the pain

indicators for neonates at high, moderate, and low risk for

NI. Importance and usefulness were each rated on a Ten-point

analogue scale, where a score of 1 = not important/useful at

all and 10 = extremely important/useful. Importance referred

to how crucial the pain indicator was in accurately and

consistently identifying pain following a painful tissue dam-

aging procedure. Usefulness referred to the feasibility (i.e.

how easily the pain indicator was to observe and score) and

clinical utility (i.e. how useful the pain indicator was for

making decisions about individualized pain management

(Stevens & Gibbins 2002). An indicator could be important

for accurately measuring pain but not useful because of the

difficulties in using it to assess pain in clinical practice.

An explanation of the purpose and description of the

research was given to participants, with opportunities for

clarification of questions prior to completion of the ques-

tionnaire. Definitions of pain indicators were provided and

respondents were asked to add any additional indicators they

felt were important or useful. Demographic information and

years of experience with neonates at risk for NI data were

also obtained from each participant. The questionnaire took

approximately 2030 minutes to complete.

B. Stevens et al.


Questionnaires were distributed to eligible researchers

electronically (e-mail) by a research administrative assistant

who was not familiar with the study or participants and who

could not link the participants identity with any response.

Reminder emails were sent to individuals 2 weeks after the

initial email, and after the questionnaires were returned the

indicators were ordered by importance and usefulness ratings

and returned to the participants for a second round of

consensus rating. The ratings from the second round were

considered final and were used for the analysis.

Questionnaires were distributed to eligible nurse clinicians

in paper and pencil form in each clinical setting by a research

nurse. They were completed individually during break

periods or at educational sessions and were returned in

unmarked envelopes in a central collection designation to

ensure confidentiality and anonymity.

Ethical considerations

All included studies in this program of research were

approved by the Research Ethics Boards at the universities

and university-affiliated participating paediatric hospitals.

Written consent was obtained from parents for the cohort

study of neonates. Participation in the surveys was voluntary

and return of completed questionnaires by nurse clinicians

and pain research implied consent to participate.

Data analysis

The neonatal behavioural and physiological pain indicators

constituted the gold standard for acute procedural pain.

These indicators were compared to the nurse and researcher

ratings in terms of importance and usefulness. Cutoff values

for each indicator were established to classify infants as

being in pain or not in pain. These estimates of pain

accuracy gave a good indication of which variables were

important in identifying pain vs. which variables provided

little or no information. This information served as the basis

for comparing nurse and researcher assessments to the

information gained from the neonates at different levels of

NI risk.

Baseline data served as the no pain state which was

compared to the pain state (defined as the most invasive

stick phase of the heel lance procedure) so that each neonate

served as their own control. For each variable, all possible

cutoffs were evaluated for their ability to classify infants as

being in no pain or pain. All values below the given cutoff

were classified as no pain and all values above the cutoff

were classified as pain. Each variables accuracy was

calculated as 1 (the total misclassification rate) and was

expressed as a percent value. The cutoff that maximized

accuracy was chosen as the best cutoff for that variable.

Preliminary analyses showed similar optimum cutoffs and no

differences in patterns across the three NI risk groups so that,

in subsequent analyses, a single cutoff was selected for all

neonates.

Changes from baseline values were also examined, with

change from baseline to the warming phase of the heel lance

representing no pain and change from baseline to stick phase

of the heel lance representing pain. Data from the warming

phase suggested that warming itself was an intervention that

decreased heart rate. Consequently, the direction of change

from baseline to warming or warming to baseline was

randomly assigned for each neonate to obtain an estimate of

the variability of change between two no pain states.

Estimates of accuracy based on the same sample used to

derive the best cutoff values were considered optimistic.

Unbiased estimates were obtained using bootstrapping to

create replicate datasets of the pain indicator data. To create

each replicate, a neonate was randomly selected from the

pool of subjects. After each draw, the selected neonate was

replaced in the same pool and the draw was repeated until the

sample size was equal to that of the original sample. The best

cutoff value was estimated for each of 1000 bootstrap

samples created to establish a reference distribution for the

sample of 149 neonates. This cutoff was applied to the data

on neonates that were not selected for the given sample.

Analysis of data from neonates not included in the bootstrap

sample provided an estimate of the optimism that could be

used to correct estimates of accuracy, sensitivity and speci-

ficity derived from the original data.

Estimates of accuracy based on the neonatal data were

compared to the importance of the pain indicators, rather

than their usefulness, as rated in the researchers opinion

surveys. Evaluating the usefulness of pain indicators would

require information on how difficult it would be to measure

these pain variables and the neonatal data could not provide

this information. Accuracy is expressed was a percentage

whereas nurse and researcher ratings were expressed on a

scale from 0 to 10. To compare the gold standard pain

indicators (i.e. neonatal data) to nurse and researcher ratings,

a linear relationship between accuracy and importance rating

was assumed. An accuracy of 50% for discriminating the

pain and no pain states is no better than chance, and thus

corresponds to a rating of 0. Conversely, 100% accuracy

corresponds to a rating of 10. Figures were created to

compare the accuracy of each pain indicator to the nurse and

researcher ratings, and the correlation coefficient for each

comparison was calculated. Results were reported using 95%

confidence intervals. SAS statistical software, version 9.1

JAN: ORIGINAL RESEARCH Indicators of pain in neonates


(SAS Institute Inc., Carey, NC, USA) was used to analyze the

data.

Results

The number of pain indicators that were common to all three

data sets (i.e. neonates, nurses and pain researchers) reduced

the number of variables from 39 to 13; these indicators

formed the basis for the analysis and included 10 facial

expression variables: brow bulge, eye squeeze, nasolabial

furrow, open lips, chin quiver, lip purse, taut tongue,

horizontal stretch mouth, vertical stretch mouth, total facial

expression and three physiologic variables: increase or

decrease in heart rate and decrease in oxygen saturation.

Table 1 Neonatal characteristics of study sample by group*

High risk for NI Moderate risk for NI Low risk for NI F, P-value

GA at birth (weeks) 3400(527)n = 54

3111(534)n = 45

3142(393)n = 50

F(2,146) = 541, P = 0005

Birth weight (grams) 222774(113444)n = 54

78473(109709)n = 45

165688(73569)n = 50

F(2,146) = 462, P = 0011

Apgar score (5 minutes) 723(214)n = 53

769(134)n = 45

826(127)n = 50

F(2,145) = 503, P = 0008

SNAPP: PE score 2085 (2165)n = 53

2424(2120)n = 45

1314(1666)n = 50

F(2,145) = 392 P = 0022

NTISS score 1558(882)n = 53

1667(770)n = 45

1444(521)n = 50

F(2,145) = 107, P = 0346

GA, gestational age; SNAPP: PE, score for neonatal acute physiology-perinatal extension; NTISS, neonatal therapeutic intervention scoring

system; NI, neurological impairment.

All values expressed as means (standard deviation).

*Adapted from Stevens et al. 2007b.

Table 2 Nurse and pain researcher charac-

teristics*Nurses

N = 95

Pain researchers

N = 14

Respondents professional

role

General duty nurse = 75

CNS-NNP = 8

Nurse educator = 6

Other = 2

N = 91

Nurse scientists = 9

Psychologist = 4

Physical Therapist = 1

N = 14

Mean number of years in

current position

984 (790) (minimum 79 years;maximum 30 years)

N = 88

1339 (656) (minimum5 years; maximum 24 years)

N = 13

Professional contact with

infants who have or

at risk for neurological

impairment

None = 2

Very little = 9

Moderate amount = 27

Great deal = 52

N = 90

None = 4

Very little = 2

Moderate amount = 3

Great deal = 5

N = 14

Non-professional contact

with infants who have or

are at risk for neurological

impairment

None = 20

Very little = 45


Great deal = 9

N = 91

None = 2

Very little = 8

Moderate amount = 3

Great deal = 1

N = 14

Academic or school-based

learning about infants at

risk for neurological

impairment

None = 10

Very little = 29


Great deal = 14

N = 91

None = 2

Very Little = 3

Moderate amount = 3

Great deal = 6

N = 14

*Adapted from Stevens et al. 2006.

CNS-NNP, clinical nurse specialist-neonatal nurse practitioner.

B. Stevens et al.


Additional variables that were suggested by the nurses and

pain researchers (e.g., body movements, consolability, finger

splay) were not observed in neonates in the existing data and

therefore were not included in these analyses.

The characteristics of the neonates, nurses and pain

researchers are summarized in Tables 1 and 2. As reported

previously (Stevens et al. 2007b), neonates at the highest risk

for NI had higher birth weights, increased maturity, higher

NBRS (Neonatal Neurobiologic Risk Score) and lower Apgar

scores than the other two groups. Neonates at moderate risk

for NI had statistically significantly higher severity of illness

scores, spent more days on the ventilator in low flow oxygen/

air. The number of days until no further apnea, bradycardia

and/or desaturation spells occurred was the highest in the

moderate NI risk group.

Estimates of accuracy for each of the infant pain

indicators across NI risk groups revealed that the four

variables with the highest accuracy for discriminating pain

and no pain in infants were: brow bulge (7783% within

the three risk groups), eye squeeze (7584%), nasolabial

Table 3 Optimal cutpoints* for the variables with highest accuracy

for discriminating between pain and no pain states

Variables Cutpoint (95% CI)

Total facial 1085 (99; 120)Brow bulge 125 (6; 24)Eye squeeze 105 (3; 12)Naso-labial furrow 95 (7; 18)

*Cutpoints were selected to maximize accuracy; all observations

below the cutpoint are classified as no pain and all observations

above the cutpoint are classified as pain states.

Table 4 Mild risk group

Infants Nurses Researchers

Sensitivity Specificity Accuracy Mean ratings Mean ratings

Total facial 71 (51; 87) 93 (77; 99) 83 (70; 91) 86 93Brow bulge 69 (49; 85) 91 (73; 98) 80 (67; 90) 81 91Eye squeeze 64 (43; 81) 94 (78; 99) 79 (66; 89) 82 86Naso-labial furrow 70 (49; 86) 89 (71; 97) 79 (66; 86) 75 79Horizontal stretch mouth 43 (24;63) 97 (81; 100) 70 (56; 82) 61 59Increase in mean heart rate 78 (59; 91) 56 (35; 75) 67 (53; 80) 86 73Open lips 71 (51; 86) 57 (37; 75) 64 (50; 77) 62 62Vertical stretch mouth 29 (13; 50) 97 (82; 100) 63 (49; 76) 63 52Taut tongue 23 (10; 44) 98 (83; 100) 61 (47; 75) 58 59Decrease in oxygen saturation 47 (28; 67) 72 (50; 87) 59 (44; 72) 82 71Chin quiver 3 (0; 19) 100 (86; 100) 52 (38; 66) 67 41Decrease in mean heart rate 96 (81; 99) 5 (1; 22) 52 (38; 66) 76 38Lip purse 1 (0; 15) 99 (85; 100) 51 (37; 65) 59 24

Table 5 Moderate risk group



Total facial 69 (45; 86) 89 (67; 97) 78 (63; 89) 84 91Brow bulge 62 (40; 81) 92 (72; 99) 77 (62; 88) 79 90Eye squeeze 58 (36; 78) 92 (73; 99) 75 (59; 57) 81 84Naso-labial furrow 65 (43; 84) 93 (74; 99) 79 (64; 90) 73 78Horizontal stretch mouth 44 (23; 66) 100 (83; 100) 72 (56; 84) 59 56Increase in mean heart rate 75 (53; 90) 43 (23; 65) 59 (43; 73) 88 72Open lips 67 (44; 84) 58 (36; 78) 62 (46; 77) 60 61Vertical stretch mouth 21 (7; 44) 89 (68; 98) 55 (38; 70) 60 51Taut tongue 17 (5; 41) 97 (78; 100) 57 (40; 73) 57 61Decrease in oxygen saturation 62 (39; 81) 78 (55; 92) 70 (54; 83) 86 74Chin quiver 5 (0; 24) 100 (83; 100) 52 (36; 67) 65 39Decrease in mean heart rate 98 (81; 100) 4 (1; 22) 51 (35; 66) 81 51Lip purse 1 (0; 18) 100 (83; 100) 50 (34; 66) 58 21



furrow (7981%), and total facial expression (7883%).

The cut-off values for these variables are presented in

Table 3. No statistically significant differences were

observed in the neonatal data between accuracy estimates

among the three risk groups. However, there was a non-

statistically significant but consistent trend towards higher

indicator accuracy in the most severe NI risk group

compared to the mild group. The physiological variables

used in the analysis (i.e. increase and decrease in mean heart

rate and decrease in oxygen saturation) demonstrated poor

accuracy (Tables 46).

The nurse-neonate correlations were lower than

researcher-neonate correlations. The level of statistical sig-

nificance for the high NI group was P = 0014, for the

moderate NI group was P = 015 and for the mild NI group

was P = 0025. Nurse-infant correlations ranged from 052,

P = 007 for the mild NI risk group, 043, P = 015 for

moderate NI risk and zero correlation, 012, P = 069 forthe high NI risk group. Nurses said that physiological

variables were more important for assessing pain in the high

NI risk group compared to the mild NI risk group (paired t-

test, P = 009). However, when the mild and high risk groups

were compared, there was an increase in accuracy for the

behavioural variables and a decrease in accuracy for the

physiological variables. With increases in risk for NI,

variables were ranked as less important by the nurses,

whereas a trend towards increased accuracy with increasing

severity existed. There was little variability in nurses ratings,

with average mean ratings ranging from 55 to 88/10 across

all NI risk groups (Figure 1).

Pain researchers showed very good understanding of the

relative importance of the different pain variables, as dem-

onstrated by the high correlation between their ratings and

accuracy scores of the neonates. The correlation between

neonate and researcher data was very high for mild 091,

P < 00001 and moderate NI risk groups (085, P = 00002),

but lower for the high NI risk group (064, P = 0019). These

researchers ranked variables as less important for the high NI

risk infants; however, they assigned a greater range of mean

ratings to the variables (2093) (Figure 1).

When infants in each NI risk group were compared against

each other, they were highly correlated in how they expressed

pain: mild compared to moderate risk group correlation =

091; mild compared to high risk group correlation = 092;

moderate compared to high risk group correlation = 079.

Discussion

Study limitations

Although this study illustrates the importance of taking the

perspectives of nurse clinicians and pain researchers into

consideration, several limitations merit discussion. First, only

a limited number of pain indicators could be included in the

study specifically those that were available across the

neonate, nurse and pain researcher data sets. Although it is

unreasonable to consider evaluating a much broader scope of

neonatal pain variables, there is the possibility of including

frequently-emerging indicators such as gross and fine body

movements and indicators of cortical involvement in future

studies of this nature.

Second, only attributes of importance and feasibility were

investigated for nurses and pain researchers. As it is essential

to establish validity of pain assessment indicators further

across a wide spectrum of neonates with varying levels of risk

for NI, our aim was to focus on and maximize the construct

Table 6 Severe risk group



Total facial 82 (59; 94) 83 (62; 94) 82 (68; 92) 79 89Brow bulge 82 (61; 94) 84 (64; 95) 83 (69; 92) 75 87Eye squeeze 81 (60; 94) 86 (66; 96) 84 (70; 93) 76 81Naso-labial furrow 85 (64; 96) 78 (57; 91) 81 (68; 91) 69 76Horizontal stretch mouth 61 (38; 81) 93 (74; 99) 78 (63; 89) 57 54Increase in mean heart rate 76 (55; 90) 50 (30; 70) 63 (48; 76) 88 74Open lips 76 (54; 90) 57 (37; 75) 66 (50; 79) 57 57Vertical stretch mouth 59 (37; 80) 86 (66; 96) 73 (58; 85) 57 48Taut tongue 47 (25; 71) 92 (73; 99) 72 (57; 85) 55 59Decrease in oxygen saturation 51 (30; 71) 59 (38; 78) 55 (40; 69) 88 74Chin quiver 4 (0; 22) 98 (82; 100) 54 (39; 69) 61 36Decrease in mean heart rate 93 (76; 98) 2 (0; 18) 48 (33; 63) 85 54Lip purse 4 (0; 23) 96 (79; 100) 53 (38; 68) 57 20

B. Stevens et al.


Mean nurse ratings vs. accuracy Mild

50 60 70 80 90 100

108

64

20

BBESNF

OLCQ

LP TT HMVM

TF

DH

IHDO

R = 052

Mean expert ratings vs. accuracy

50 60 70 80 90 100

108

64

20

BBESNF

OL

CQ

LP

TTHMVM

TF

DH

IHDO

R = 091

Moderate

50 60 70 80 90 100

018

64

20

BBES

NF

OLCQ

LP TT HMVM

TFDHIH DO

R = 043

50 60 70 80 90 100

108

64

20

BBES

NF

OL

CQ

LP

TTHM

VM

TF

DH

IH DO

R = 085

Severe

50 60 70 80 90 100Accuracy

108

64

20

BBESNF

OLCQ

LP TT HMVM

TFDH

IHDO

R = 012

50 60 70 80 90 100Accuracy

108

64

20

BBES

NF

OL

CQ

LP

TTHM

VM

TF

DH

IHDO

R = 064

Figure 1 Accuracy Ratings of Nurses and Researchers for Pain Indicators by NI risk group. Mean nurse and researcher ratings are plotted

against estimated accuracy for each physiological and behaviour variable. TF, total facial; BB, brow bulge; ES, eye squeeze; NF, naso-labial

furrow; HM, horizontal stretch mouth; IH, increase in mean HR; OL, open lips; VM, vertical stretch mouth; TT, taut tongue; DO, decrease in

O2sat; CQ, chin quiver ; DH, decrease in mean HR and LP, lip purse. The dashed line represents perfect agreement, assuming that 50% accuracy

(random chance) is equivalent to a rating of 0 and 100% accuracy represents a rating of 10. The solid line is a least squares fit of the data.



of accuracy. Therefore, sensitivity and specificity, although

reported, were not of primary concern in this study. Future

research with a control situation will be required to establish

the optimum cut-offs for pain to maximize sensitivity and

specificity.

Comparison with previous research

In this study, the accuracy of pain assessment in neonates at

varying levels of risk for NI was determined from the

perspectives of practising neonatal nurses and neonatal/infant

pain research experts compared to data generated by the

neonates during a painful procedure. Ultimately, it is impor-

tant to determine whether existing pain measures are valid

and reliable for assessing pain in newborn infants across the

broad spectrum of risk for NI and consistent with the

conceptualization for this study, understanding the percep-

tions of healthcare providers about pain is crucial.

As corroborated by other research (Stevens et al. 2007c),

behavioural indicators were more consistently accurate and

representative of the neonates response to pain. Nurses and

researchers considered these indicators important. This find-

ing continues to support the external validity of multidimen-

sional behavioural measures such as the NFCS (Grunau &

Craig 1990), yet fuels the discussion on whether composite

pain measures such as the PIPP (Premature Infant Pain

Profile; Stevens et al. 1996) and DAN (Douleur Aigue du

Nouveau-ne; Carbajal et al.1997) have construct validity and

clinical utility. Recent findings in an in-depth series of factor

analyses indicate that physiological indicators (e.g. heart rate,

heart rate variability, oxygen saturation), although contrib-

uting less to the total pain response, still give important

information on the level of pain intensity for infants across

NI groups undergoing painful procedures (Stevens et al.

2007c).

Of particular interest in the present study was the high pain

researcher-neonate pain indicator correlations compared to

moderate nurse clinician-neonate correlations. This finding

could be attributed to our enhanced knowledge about

neonatal pain indicators, much of which may have been

directly contributed by researchers through the development

of pain assessment measures or evaluation of pain-relieving

strategies in neonates and infants. They may have also

benefited from the two rounds of the Delphi study, where

they received feedback in the initial round which may have

led to more precise identification of indicator importance in

the second round. However, it is reasonable to expect that

NICU nurses would have more practical expertise with

neonates undergoing painful procedures and familiarity with

neonates of all levels of NI risk; thus, would be able to

discriminate more accurately between pain indicators

amongst NI risk groups, should they exist. Yet, increased

risk for NI was rated as less important by nurses even though

the results showed a trend towards increased accuracy of pain

indicators with increasing risk of NI. Further exploration of

the meaning of this finding in terms of nursing assessment of

neonatal pain is required to better understand this result.

There was little variability in nurses ratings of pain

indicators; they considered that most indicators were of

moderate to high importance for discriminating pain in

neonates. Although there is no obvious reason for this

finding, nurses may have difficulty discriminating important

from unimportant indicators, or they may not believe that

these indicators represent pain or that pain is important in

these neonates. Conversely, one might wonder why the pain

researcher-neonate correlations were so high, yet were lower

(similar to nurses) for the most at-risk NI neonates. Does this

suggest that both research and clinical experts are consistent

What is already known about this topic

Neonates at risk for neurological impairment have notbeen systematically included in the development of

neonatal pain measures.

The validity of existing pain measures is unknown forthis population of vulnerable infants.

What this paper adds

Behavioural indicators were rated as the most impor-tant indicators in the neonates response to pain by

both nurse clinicians and pain researchers.

Pain researchers ratings of pain indicators showedhigher correlations with neonatal physiological and

behavioural pain indicators compared to nurses clini-

cians ratings.

Ratings of both pain researchers and nurse clinicianson the importance of behavioural and physiological

indicators were opposite to the actual pain responses

demonstrated by infants.

Implications for practice and/or policy

The influence of context on process and outcomesneeds to be unravelled if we hope to improve clinical

outcomes, particularly decreased pain intensity, for

these vulnerable neonates.

Development of infant pain assessment policies shouldinclude input from both healthcare professionals and

pain researchers.

B. Stevens et al.


in their thinking, namely that as NI risk status increases,

physiological variables become more important than

behavioural indicators, resulting in their assignment of lower

importance to behavioural indicators but higher importance

to physiological indicators? Pain researchers ratings sug-

gested a higher level of association with neonatal variables;

yet, both nurses and pain researchers opinions were oppo-

site to the pain indicators demonstrated by neonates. That is,

nurses and pain researchers thought behavioural indicators

were more important in the mild NI risk group than in the

severe NI risk group, and physiological indicators were more

important in the high NI risk group than in the low risk

group. Conversely, neonates demonstrated (by the accuracy

scores) that behavioural indicators were more important in

the high risk group compared to the low risk group, and

physiological indicators were more important in the low risk

group compared to the high risk group. These results are

puzzling and raise the question of whether the attributes of

importance and usefulness capture the essence of what is

crucial in pain assessment with neonates.

Conclusion

The clinical implications of these findings relate directly to

how nurses evaluate pain in hospitalized neonates. There are

clearly differences in the importance and validity of various

indicators from the perspectives of nurses and pain research-

ers that could influence the nature and frequency of pain

assessment and management in these vulnerable neonates.

These differences are not dissimilar to the quandaries of

knowledge translation, where we are unsure of the influence

of context (e.g. opinions and values of clinicians) on the

translation of evidence into practice. Unraveling the influence

of context on process and outcomes is important if we hope

to improve clinical outcomes, particularly decreased pain

intensity, for these vulnerable neonates.

Development of infant pain assessment policies should

include input from both healthcare professionals and pain

researchers. Further research is warranted on how nurse

clinicians conceptualize pain in infants and on the develop-

ment and evaluation of knowledge translation strategies that

will effectively translate infant pain research into practice.

Acknowledgements

We acknowledge the research nurses who participated in the

collection and management of study data including Marilyn

Ballantyne, Anne Jack, Marie Bagg, Mary Anne Fagan, Janet

Narciso, Kim Caddell, Janet Chee Salena Mohammed-

Breault, Sandy Lin and Karolina Kupczyk. Finally, we are

grateful for the willingness of the infants families who agreed

to participate in this study.

Funding

Funding is acknowledged from the Canadian Institutes of

Health Research (MOP-37884) and the Bloorview Childrens

Hospital Foundation. We would also like to acknowledge

financial support from the Ontario Ministry of Health for

the Career Scientist Award and the Signy Hildur Eaton Chair

in Paediatric Nursing Research at the Hospital for Sick

Children to B. Stevens, a Canada Research Chair Award to

P. McGrath, and a Canadian Institutes of Health Research

Fellowship to J. Yamada.

Author contributions

BJS, PM, SG, LB, CSC, GAF, LF, AH, CJ, PMcK & AO were

responsible for the study conception and design. CSC, AH &

JY performed the data collection. BJS, PM, AD, JB & CJ

performed the data analysis. BJS, PM, AD, SG, JB, LF & JY

were responsible for the drafting of the manuscript. BJS, PM,

SG, JB, CSC, GAF, LF, KOB, AO & JY made critical

revisions to the paper for important intellectual content. BJS,

AD & JB provided statistical expertise. BJS, PM, SG, JB, LB,

GAF, AH, CJ, PMcK, KOB & AO obtained funding. AH &

JY provided administrative, technical or material support.

BJS, CSC & AH supervised the study.

References

Anand K.J. & Craig K.D. (1996) New perspectives on the definition

of pain. Pain 67(1), 36.

Breau L.M., McGrath P.J., Stevens B., Beyene J., Camfield C.S.,

Finley G.A., Franck L., Howlett A., OBrien K. & Ohlsson A.

(2004) Healthcare professionals perceptions of pain in infants at

risk for neurological impairment. BMC Pediatrics 4(1), 2331.

Breau L.M., McGrath P.J., Stevens B., Beyene J., Camfield C., Finley

G.A., Franck L., Gibbins S., Howlett A., McKeever P., OBrien K.

& Ohlsson A. (2006) Judgments of pain in the neonatal intensive

care setting: a survey of direct care staffs perceptions of pain in

infants at risk for neurological impairment. The Clinical Journal of

Pain 22(2), 122129.

Carbajal R., Paupe A., Hoenn E., Lenclen R. & Olivier-Martin M.

(1997) [APN: evaluation behavioral scale of acute pain in newborn

infants]. Archives de pediatrie 4(7), 623628.

Craig K.D. & Pillai Riddell R. (2003) Social influences, culture and

ethnicity. In Pediatric Pain: Biological and social context. Progress

in Pain Research and Management, Vol. 26 (McGrath P.J. &

Finley G.A., eds), IASP Press Seattle, Seattle, WA, pp. 159182.

Craig K.D., Korol C.T. & Pillai Riddell R. (2002) Challenges of

judging pain in vulnerable infants. Clinics in Perinatology, 29(3),

445457.



Creswell J.W. (1998) Qualitative Inquiry and Research Design:

Choosing Among Five Traditions. Sage Publications Inc, Thousand

Oaks, CA.

Debillon T., Zupan V., Ravault N., Magny J.F. & Dehan M.

(2001) Development and initial validation of the EDIN scale, a

new tool for assessing prolonged pain in preterm infants.

Archives of disease in childhood. Fetal and Neonatal Edition

85(1), F36F41.

van Dijk M., de Boer J.B., Koot H.M., Tibboel D., Passchier J. &

Duivenvoorden H.J. (2000) The reliability and validity of the

COMFORT scale as a postoperative pain instrument in 0 to

3-year-old infants. Pain 84(23), 367377.

Duhn L.J. & Medves J.M. (2004) A systematic integrative review of

infant pain assessment tools. Advances in Neonatal Care 4(3),

126140.

Grunau R. & Craig K. (1990) Facial activity as a measure of neonatal

pain expression. In Advances in Pain Research and Therapy,

Pediatric Pain (Tyler D. & Krane E., eds), Raven Press Ltd,

New York, pp. 147153.

International Association for the Study of Pain (2003) IASP

Task Force on Taxonomy: IASP Pain Terminology. Retri-

eved from http://www.iasp-pain.org/AM/Template.cfm?Section=

Home&Template=/CM/HTMLDisplay.cfm&ContentID=6632 on

26 February 2008.

Johnston C.C., Stevens B., Yang F. & Horton L. (1996) Develop-

mental changes in response to heelstick in preterm infants:

a prospective cohort study. Developmental Medicine and Child

Neurology 38(5), 438445.

Johnston C.C., Stremler R.L., Stevens B.J. & Horton L.J. (1997)

Effectiveness of oral sucrose and simulated rocking on pain

response in preterm neonates. Pain 72(12), 193199.

Robertson C.M., Svenson L.W. & Joffres M.R. (1998) Prevalence of

cerebral palsy in Alberta. The Canadian Journal of Neurological

Sciences 25(2), 117122.

Stevens B. & Gibbins S. (2002) Clinical utility and clinical signifi-

cance in the assessment and management of pain in vulnerable

infants. Clinics in Perinatology 29(3), 459468.

Stevens B., Johnston C., Petryshen P. & Taddio A. (1996) Premature

infant pain profile: development and initial validation. The Clinical

Journal of Pain 12(1), 1322.

Stevens B., Johnston C., Franck L., Petryshen P., Jack A. & Foster G.

(1999) The efficacy of developmentally sensitive interventions and

sucrose for relieving procedural pain in very low birth weight

neonates. Nursing Research 48(1), 3543.

Stevens B., McGrath P., Gibbins S., Beyene J., Breau L., Camfield C.,

Finley A., Franck L., Howlett A., McKeever P., OBrien K.,

Ohlsson A. & Yamada J. (2003) Procedural pain in newborns at

risk for neurologic impairment. Pain 105(12), 2735.

Stevens B., McGrath P., Yamada J., Gibbins S., Beyene J., Breau L.,

Camfield C., Finley A., Franck L., Howlett A., Johnston C.,

McKeever P., OBrien K. & Ohlsson A. (2006) Identification of

pain indicators for infants at risk for neurological impairment: a

Delphi consensus study. BMC Pediatrics, 6, 1.

Stevens B., Pillai R.B., Oberlander T. & Gibbins S. (2007a) Assess-

ment of pain in neonates and infants. In Pain in Neonates and

Infants (Stevens B. & McGrath P., eds), Elsevier, Edinburgh, UK,

pp. 6790.

Stevens B., McGrath P., Gibbins S., Beyene J., Breau L., Camfield C.,

Finley A., Franck L., Howlett A., Johnston C., McKeever P.,

OBrien K., Ohlsson A. & Yamada J. (2007b) Determining

behavioural and physiological responses to pain in infants

at risk for neurological impairment. Pain 127(12), 94

102.

Stevens B., Franck L., Gibbins S., McGrath P.J., Dupuis A., Yamada

J., Beyene J., Camfield C., Finley G.A., Johnston C., OBrien K. &

Ohlsson A. (2007c) Determining the structure of acute pain

responses in vulnerable neonates. The Canadian Journal of Nurs-

ing Research 39(2), 3247.

Thase M.E., Friedman E.S., Berman S.R., Fasiczka A.L., Lis J.A.,

Howland R.H. & Simons A.D. (2000) Is cognitive behavior ther-

apy just a nonspecific intervention for depression? A retrospective

comparison of consecutive cohorts treated with cognitive behavior

therapy or supportive counseling and pill placebo. Journal of

Affective Disorders 57(13), 6371.

The Journal of Advanced Nursing (JAN) is an international, peer-reviewed, scientific journal. JAN contributes to the

advancement of evidence-based nursing, midwifery and health care by disseminating high quality research and

scholarship of contemporary relevance and with potential to advance knowledge for practice, education, management

or policy. JAN publishes research reviews, original research reports and methodological and theoretical papers.

For further information, please visit the journal web-site: http://www.journalofadvancednursing.com

B. Stevens et al.


Original Pediatric

Documents

Transcript of Original Pediatric