BMJ Paediatrics Open is committed to open peer review. As...
Transcript of BMJ Paediatrics Open is committed to open peer review. As...
BMJ Paediatrics Open is committed to open peer review. As part of this commitment we make the peer review history of every article we publish publicly available. When an article is published we post the peer reviewers’ comments and the authors’ responses online. We also post the versions of the paper that were used during peer review. These are the versions that the peer review comments apply to. The versions of the paper that follow are the versions that were submitted during the peer review process. They are not the versions of record or the final published versions. They should not be cited or distributed as the published version of this manuscript. BMJ Paediatrics Open is an open access journal and the full, final, typeset and author-corrected version of record of the manuscript is available on our site with no access controls, subscription charges or pay-per-view fees (http://bmjpaedsopen.bmj.com). If you have any questions on BMJ Paediatrics Open’s open peer review process please email
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
Validation of a classification system for treatment-related
mortality in children with cancer
Journal: BMJ Paediatrics Open
Manuscript ID bmjpo-2017-000082
Article Type: Original article
Date Submitted by the Author: 25-May-2017
Complete List of Authors: Hassan, Hadeel; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Rompola, Melpomeni; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology
Kinsey, Sally; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Glaser, Adam; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Phillips, Bob; University of York, Centre for Reviews and Dissemination; Bob Phillips
Keywords: Oncology, Haematology, Palliative Care
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open on 11 M
ay 2018 by guest. Protected by copyright.
http://bmjpaedsopen.bm
j.com/
bmjpo: first published as 10.1136/bm
jpo-2017-000082 on 30 October 2017. D
ownloaded from
Confidential: For Review O
nlyTitle:
Validation of a classification system for treatment-related mortality in children with cancer
Authors:
H. Hassan 1, 2, M. Rompola
1, 2, A.W. Glaser
1, 2, S. E. Kinsey
1, 2, R. S. Phillips
1, 3
Affiliations:
1 Department of Paediatric Haematology and Oncology, Leeds Teaching Hospital NHS Trust,
Leeds, UK
2 Leeds Institute of Cancer and Pathology,
University of Leeds, Leeds, UK
3 Centre for Reviews and Dissemination, University of York, York, UK
Full address for correspondence:
Dr Hadeel Hassan
Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, LS2 9JT, UK
Telephone: +44 (0)113 343 2596
Email address: [email protected]
Page 1 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyAbstract
Background: Death not directly due to cancer has been termed “treatment-related mortality”
(TRM). Appreciating the differences between TRM and disease-related death is critical in
directing strategies to improve supportive care, interventions delivered or disease progression.
Recently, a global collaboration developed and validated a consensus-based classification tool
and attribution system.
Objectives: To evaluate the newly developed consensus-based definition of TRM and cause-
of-death attribution system outside the centre it was initially validated (Toronto, Canada).
Setting: The paediatric haematology and oncology department at Leeds Teaching Hospital in
Leeds, UK.
Participants: Two consultants and two clinical research associates (CRA).
Methods: Thirty medical records of the most recent deaths in children with cancer, two and
four weeks prior to death were anonymised and presented to the participants. Reviewers
independently classified deaths as “treatment related mortality” or “not treatment related”
according to the algorithm developed. When TRM occurred, reviewers applied the cause-of-
death attribution system. Inter-relater reliability was assessed using the Kappa statistic (k).
Main outcome: Inter-relater reliability between CRA and consultants.
Results: Reliability of the classification was deemed almost perfect between CRA and
consultants (k=0.86, 95% confidence interval 0.72-0.97). Ten deaths were classified as TRM;
of which infection was the most frequent cause identified. Reviewers disagreed on the primary
cause of death (e.g. respiratory vs infection) when applying the cause-of-death attribution
system in 6 cases. The study identified how the algorithm may not detect TRM in patients
receiving non-curative therapy.
Page 2 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConclusions: The classification and cause of death attribution system could be implemented
in different health care settings. Adaptation of the classification tool in patients receiving non-
curative interventions and the cause of death attribution system should be considered.
What is known about the subject?
• Treatment-related mortality (TRM) is poorly defined in paediatric oncology and
haematology.
• A global collaboration developed and validated a consensus-based classification tool
and attribution system (in Toronto, Canada).
What this study adds?
• This study has further demonstrated criterion validity of the newly developed
classification tool in Leeds, UK, outside the centre it was initially tested.
• This study identified that the classification tool may not detect TRM in patients
receiving non-curative therapy, and proposes an alternative algorithm for these
patients.
• This study identifies the need for further research exploring the use of the attribution
system.
Page 3 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyBackground
Survival of children diagnosed with malignancies has improved with recent estimates
suggesting more than 80% of those diagnosed in high income countries will survive [1]. This
still means 20% of children will die - around 80,000 deaths worldwide in 2012 [1]. The cause
of death may be due directly to the disease, or complications or toxicities of interventions
delivered. The increasing success of cancer-directed interventions has contributed to a higher
relative proportion of deaths due to associated toxicities [2]. Death not directly due to the
cancer has been broadly termed “treatment-related mortality” (TRM). This includes death
from infection, bleeding, and organ dysfunction [3]. Incidence varies according to the
underlying disease, stage of treatment and socio-economic status [4]. Identifying the nature of
the mortality is critical in helping researchers and clinicians improve survival in particular
diagnostic groups by focusing on strategies to improve supportive care when death is
predominantly through TRM.
Despite the importance of TRM, varying definitions have been used. Systematic reviews [5]
[6] identified significant heterogeneity in TRM definitions used in randomised trials. This
inspired a global collaboration to develop a consensus-based classification tool for ascribing
death as TRM, and further specific attribution of cause of death [7], primarily to be used by
clinical research assistants (CRA). Reliability for the TRM classification was “almost
perfect” between medical and CRA reviewers that had been involved in developing the
system (kappa=0.92, 95% confidence interval 0.78-1.00) [7].
An ideal classification system for treatment-related mortality should be applicable across
different countries, treatment protocols, and health care settings. To assess this, it is important
to attempt to further validate the classification and attribution system in a different setting,
with professionals uninvolved in the development of the system. This paper reports such a
Page 4 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyvalidation at a large regional paediatric oncology centre in a different continent to the one in
which the system was originally developed.
Objectives
This study aimed to evaluate the newly developed consensus-based definition of TRM and
cause-of-death attribution system at a regional paediatric oncology centre in Leeds, England.
Methods
This study was approved by the University of Leeds School of Medicine Ethics committee
(Ref: MREC15-118) and did not require NHS ethics approval. Eligible patient records were
those of patients treated for a malignancy or who underwent a haematopoietic stem cell
transplant (HSCT) for a non-malignant diagnosis at Leeds Children’s Hospital (Leeds, UK)
while aged 18 years or younger at diagnosis. Five cases were excluded as patients had died
following relapse after the age of 18, or the medical records were not located. All included
patients died between 2014 and 2016. Data from the clinical records were anonymised, and
presented in a random order. Thirty patient records were included. Each set of records was
presented twice, once with information from 2 weeks prior to death, once with the
information extending back to 4 weeks prior to death, leading to a total of 60 assessments
being made. 4 participants were identified to review the case notes; the two CRAs were a data
analyst (AF) and research nurse (JT), and the two senior clinicians were a consultant
paediatric oncologist and consultant paediatric haematologist (AG and SK).
The study was undertaken on a single afternoon. After agreeing and signing a consent form,
participants received a 10 minute educational presentation explaining how to use the system,
and how the study would be undertaken. The reviewers then independently classified each
death according to the algorithm (fig. 1). For cases assessed as TRM, the reviewers were
asked to apply the cause-of-death attribution system (table 1). Following the individual
Page 5 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlycompletion of the assessments, a moderated group discussion was undertaken with notes
recorded by two facilitators and used to supplement the themes of the discussion.
Inter-relater reliability was assessed using the Kappa statistic (k). Criterion validity was
assessed by assuming classification by the Consultants as the gold standard. Group consensus
classification between and within the CRAs and Consultant group was evaluated using the
Cohen’s kappa statistic, and across all individuals using the Fleiss’ kappa statistic. The
strength of agreement was defined as slight (0.00-020), fair (0.21-0.40), moderate (0.41-0.60)
substantial (0.61-0.80) and almost perfect (0.81-1.00) [8]. Based on the previously published
study [7], we decided to include 30 cases for analysis. If the null hypothesis was 0.1, power
would be 0.75. A further 30 cases would be reviewed if validity was inadequate (defined a
priori as k<0.6). Calculation of the k statistic was completed using the R studio irr package
and a bootstrap with 200 iterations was used to calculate 95% confidence intervals [9].
Comparison of cause of death was qualitative, to provide further insight reviewers
participated in a discussion of the use of the algorithm and cause of death attribution system.
Page 6 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyFigure 1: Classification of TRM in children with cancer, taken from (7).
Page 7 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyTable 1 Summary of cause of death attribution system. Full system is supplied in a
supplementary S1 file.
Probable cause of death Possible cause of death
Infection Infection with associated
microbiologically
documented organism.
Infection without associated
microbiologically documented organism.
Haemorrhage Symptomatic
haemorrhage.
Symptomatic haemorrhage not shown by
imaging or pathology.
Thrombosis Symptomatic thrombosis
or embolism shown by
imaging or pathology.
Symptomatic thrombosis or embolism not
shown by imaging or pathology.
Cardiac Symptomatic cardiac-
dysfunction defined by
ECG, cardiac imaging or
pathology.
Symptomatic arrhythmia excluding sinus
tachycardia or bradycardia not shown by
electrocardiography (ECG).
Immunomediated Acute anaphylaxis,
deteriorating or acute-
severe graft versus host
disease,
haemophagocytic
lymphohistocytosis, or
cytokine-release
syndrome.
Stable graft-versus-host disease.
Metabolic Symptomatic tumour
lysis syndrome.
CNS Symptomatic CNS
dysfunction shown by
imaging or pathology.
Symptomatic CNS dysfunction not shown
by imaging or pathology.
Respiratory Symptomatic respiratory
distress with ventilator
support.
Symptomatic respiratory distress without
ventilator support.
Gastrointestinal
system
Symptomatic abdominal
disease resulting in or
perforation shown by
imaging or pathology.
Symptomatic abdominal disease resulting
in or perforation not shown by imaging or
pathology.
Renal system Acute kidney injury
requiring intervention.
External causes E.g. suicide.
Page 8 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyResults
Demographics
Age of identified patients ranged from less than 1 to 17 years and 57% were male. 67% were
diagnosed with solid tumours, and 33% were diagnosed with malignant haematological
conditions. Collectively, sarcomas were the most frequent solid tumour diagnosed (40% of
total), followed by CNS tumours (30%). 25% of these patients had presented with metastatic
disease at diagnosis. All patients with malignant haematological conditions were diagnosed
with leukaemias. 40% of these patients had either received a transplant or presented with
relapsed disease. A summary is enclosed in a supplementary file.
Classification of treatment-related mortality
Ten deaths (33%) were classified as TRM. 10% of patients diagnosed with solid tumours,
and 80% of patients diagnosed with malignant haematological conditions were classified as
TRM. Reliability of classification was almost perfect between CRAs and consultants, with a
k statistic of 0.86 (95% confidence interval 0.72-0.97, with disagreement on 3 deaths). There
was also almost perfect agreement between CRAs (k=0.96, 95% confidence interval 0.87-
1.00, disagreement on one record) and between consultants (k=0.85, 95% confidence interval
0.67-0.97, with disagreement on two deaths) (See Table 3).
When the 2- and 4-week data were examined, there was a single difference between the
assessments of each of the four assessors (see Table 3).
Cause of death attribution system
Table 2 summarises the diagnoses of the ten patients whose deaths were classified as TRM,
alongside the cause of death as attribution by each assessor.
In brief, there was unanimous agreement on cause of death of 3 cases (J, V, AA), the other 7
cases has inconsistencies in either the nature of cause of death (e.g. infection vs. immune
Page 9 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlymediated), or strength of evidence (i.e. probably vs. possible). In one complex scenario (AD),
the CRAs did not feel able to record a cause of death.
Post-review discussion
The reviewers all agreed that the algorithm was straight forward to use, and that it would be
beneficial to have a standardised tool and attribution system to use in trials nationally and
globally.
Three difficulties with the system were identified: firstly, how to address the patients who
may die from TRM whilst receiving palliative care and particularly how the algorithm could
be used as part of palliative care trial. One case was a patient taking palliative etoposide
following a diagnosis of relapsed ALL, developed a febrile illness, and died. Secondly, how
should deaths in children after the completion of treatment be classified? A child (case V)
who had completed treatment for standard risk AML died of overwhelming pneumococcal
septicaemia 6 months after end of treatment while in complete remission. Another further
case the assessors had difficulty with (case M) followed presentation acutely with signs of
raised intra-cranial pressure and large mass noted on CT. This patient died on the operating
table whilst receiving surgical intervention.
Attribution of cause of death was felt to be more difficult than ascribing a death as TRM or
not. The case the reviewers felt was most difficult to assign was patient who died of multi-
organ failure following a HSCT (AD); both CRAs independently decided to not attribute a
cause of death.
Page 10 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review Only
Table 2 Summary of cause of death attribution by reviewers for deaths classified as TRM. Bold font=probable causes of death.
Case Diagnosis CRA 1 CRA 2 Consultant 1 Consultant 2
J AML prior to HSCT
Immunomediated Immunomediated Immunomediated Immunomediated
M ATRT Acute symptomatic
intracranial haemorrhage
Acute symptomatic
intracranial
haemorrhage
NR Acute symptomatic intracranial
haemorrhage
V AML Infection Infection Infection Infection
X B-cell ALL Acute symptomatic
intracranial haemorrhage
Acute symptomatic
intracranial haemorrhage Infection Infection
Y T-cell ALL Respiratory Infection Acute symptomatic
intracranial haemorrhage
Infection
Z B-cell ALL Respiratory Infection Respiratory Infection
AA B-cell ALL Infection Infection Infection Infection
AB Ependymoma NR Infection NR Infection
AC T-cell ALL Worsening symptomatic
graft versus host disease.
Worsening symptomatic
graft versus host disease.
Worsening symptomatic
graft versus host disease.
Acute symptomatic pulmonary
haemorrhage
AD B-cell ALL post
HSCT
Unclear Difficult case Worsening symptomatic
graft versus host disease.
Acute kidney Injury
Page 11 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. Protected by copyright. http://bmjpaedsopen.bmj.com/ bmjpo: first published as 10.1136/bmjpo-2017-000082 on 30 October 2017. Downloaded from
Confidential: For Review O
nlyTable 3 Summary of kappa statistic and 95% confidence intervals of independent
reviewers, consultants, CRAs and between CRA and consultants for all total case
reviews, 4 weeks and 2 weeks prior to death using the cause of death attribution system.
Inter-rater
comparison
Total k (95%
confidence interval)
4 weeks k (95%
confidence interval)
2 weeks k (95%
confidence interval)
Independent
reviewers
0.92 (0.83-0.98) 0.91(0.76-1.00) 0.92 (0.79-1.00)
Consultants 0.85 (0.67-0.97) 0.85 (0.59-1.00) 0.84 (0.59-1.00)
CRA 0.96 (0.87-1.00) 0.85 (0.59-1.00) 0.85 (0.60-1.00)
CRA vs consultants 0.86 (0.72-0.97) 0.87 (0.66-1.00) 0.86 (0.67-1.00)
Page 12 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyDiscussion
This study is, to the best of our knowledge, the first revalidation of the standardised definition
of treatment-related mortality and cause of death attribution system for paediatric cancer
patients [7]. It demonstrates the system is reliable and established criterion validity in an
alternative centre and health care system with different treatment protocols. It can be used
after very limited training, with “almost perfect” agreement between assessors irrespective of
discipline (Fleiss kappa 0.92, 95% CI 0.83-0.98). The study confirms the observations of the
development group and shows that information from two weeks prior to the death of a patient
is sufficient to consistently attribute death to TRM or disease.
Challenges and areas for future development
Our study highlights specific challenges with the system as it currently exists, both with the
classification of TRM and the attribution of a specific cause of death. Fundamentally, this
approach defines deaths as either ‘treatment related’ or ‘cancer related’. This gives rise to a
semantic challenge; “treatment-related mortality” implies that deaths that come under this
term occur directly because of the therapies delivered. However, the classification system
classifies deaths which occur prior to commencement of anti-cancer therapy, which are not
directly attributable to the cancer (for example, tumour lysis syndrome in high-count
leukaemia) as cases of TRM. This clash of language and “common sense” may confuse users
of the classification tool, for example, case M in which a patient presented acutely with signs
of raised intra-cranial pressure and died on the operating table.
A deeper challenge to this system addressed the philosophical distinction of assigning all
deaths into one of two categories; cancer or treatment related. There is a convincing argument
that a third category of death should be attributable, “other non-cancer death”, for those who
Page 13 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlydie of an event or illness external to their malignancy. This problem is particularly evident if
the current system is to be used after the completion of treatment. For example, a patient dies
as a passenger in an air traffic accident, 4 years after treatment for a localised Wilms tumour.
In the current system, this death is classified as TRM, even though the death is unrelated to
the child’s cancer diagnosis.
Conversely, it is also, important to note how a diagnosis of cancer may be included in ‘non-
cancer deaths’. For example a patient may commit suicide some years after the completion of
treatment because of the psychological effects of their diagnosis or treatment. Any addition to
the system would need to be sensitive to these potential issues.
We have identified the need to further refine the approach to categorising cause of death in
cancer patients receiving care without intent to cure. This is particularly important if the
system is applied in ‘routine’ settings, assessing deaths in the palliative setting rather than in
the original setting of use within a curative trial. Increasingly individuals destined not to be
cured are living for lengthier periods due to participation in clinical trials/studies. This group
of patients currently have all deaths classified as “not treatment-related mortality” as
clinicians would have specified progressive disease, or that cancer therapy has no curative
intent. This algorithm may fail to identify a significant group of patients who may die of
causes amenable to better supportive care whilst receiving palliative care. For example, a
patient can die of overwhelming sepsis whilst receiving palliative etoposide for refractory
neuroblastoma. This could be addressed by modifying the algorithm, for this type of use.
Another proposal includes using a separate classification tool for patients on palliative care
trials (fig. 2). The counter-argument to this suggested change is the risk of adding complexity
to a simple, effective, tool which can be used by people of different skills and health care
settings globally. It will also have similar issues with interpretation as with initial algorithm.
Page 14 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyReviewers failed to agree on cause of death in 6 episodes. Differences in cause of death
allocated could be attributed to the reviewers’ previous experience, clinical expertise and
interpretation of the clinical records, particularly in light of potential previous direct clinical
involvement with the cases under review. Currently, the tool is intended for use by any CRA,
however, it may require users to have a certain level of experience or clinical expertise, and
agreement may be reduced with CRA who are new to the role.
Understanding and interpretation of the system as proposed for attribution of a specific
mechanistic cause of death could potentially be improved by dedicating more time during the
presentations and using clinical scenarios. Alternatively, the cause of death attribution system
could be further refined. Currently reviewers, attribute death directly to one, most likely,
cause. As multiple organ systems may be involved, exploring the option of listing more than
one cause could be considered. Another possibility includes refining the current grouping, for
example rather than having ‘respiratory’ and ‘infection’ as 2 separate entities, ‘respiratory
infection’ could be listed, making it easier for the reviewer to identify the primary cause.
Page 15 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyFigure 2: Proposed classification of TRM in children receiving non-curative therapy
(NCT) only.
Page 16 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConclusions
We have been able to confirm the reproducibility and criterion validity of the TRM
classification system. We believe this supports the hypothesis that the classification system
can be implemented easily and effectively in different health care settings, thereby improving
consistency and accuracy of outcome reporting in clinical trials. The TRM classification
system will be of immense value in the evaluation of deaths in the palliative setting. We
propose the addition of a separate classification tool in patients on palliative trials.
Page 17 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConflict of interest statement
The authors declare no conflict of interest.
Acknowledgments
The authors would like to thank the reviewers for participating in this study. This work was
supported by Candlelighters charity who funded Dr Hadeel Hassan’s PhD, for which this
research was conducted. H.H wrote the manuscript and R.S.P edited the manuscript. M.R,
S.E.K and A.G contributed to the review process.
Page 18 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyReferences
1. CRUK. Children's cancers mortality in Europe and worldwide. 2014 [cited 2016
September]; Available from: http://www.cancerresearchuk.org/health-
professional/cancer-statistics/childrens-cancers/mortality#heading-Three.
2. Kremer, L.C., et al., A worldwide collaboration to harmonize guidelines for the long-
term follow-up of childhood and young adult cancer survivors: a report from the
International Late Effects of Childhood Cancer Guideline Harmonization Group.
Pediatr Blood Cancer, 2013. 60(4): p. 543-9.
3. Blanco, E., et al., Non-relapse mortality in pediatric acute lymphoblastic leukemia: a
systematic review and meta-analysis. Leuk Lymphoma, 2012. 53(5): p. 878-85.
4. White, Y., V.P. Castle, and A. Haig, Pediatric oncology in developing countries:
challenges and solutions. J Pediatr, 2013. 162(6): p. 1090-1, 1091.e1.
5. Ethier, M.C., et al., Lack of clarity in the definition of treatment-related mortality:
pediatric acute leukemia and adult acute promyelocytic leukemia as examples. Blood,
2011. 118(19): p. 5080-3.
6. Tran, T.H., et al., Lack of treatment-related mortality definitions in clinical trials of
children, adolescents and young adults with lymphomas, solid tumors and brain
tumors: a systematic review. BMC Cancer, 2014. 14: p. 612.
7. Alexander, S., et al., Classification of treatment-related mortality in children with
cancer: a systematic assessment. Lancet Oncol, 2015. 16(16): p. e604-10.
8. McHugh, M.L., Interrater reliability: the kappa statistic. Biochemia Medica, 2012.
22(3): p. 276-282.
9. Team, R., RStudio: Integrated Development for R. RStudio, Inc. 2015.
Figure legends
Figure 1: Classification of TRM in children with cancer, taken from (7).
Figure 2: Proposed classification of TRM in children receiving non-curative therapy (NCT)
only.
Supplementary files legend list
Supplemental table S1: Cause-of-death attribution system.
Supplemental table S2: Diagnosis of included patients results of classification by reviewers.
Page 19 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
190x338mm (96 x 96 DPI)
Page 20 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For R
eview O
nly
Page 21 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySupplemental file S1
Probable cause of death Possible cause of death
Infection A.1 Clinically or
radiographically
documented infection with
associated microbiologically
documented organism
1. Clinically or
radiographically
documented infection
without associated
microbiologically
documented organism.
Haemorrhage B.1 Acute symptomatic
intracranial haemorrhage
shown by imaging or
pathology.
B.2 Acute symptomatic
pulmonary haemorrhage
shown by imaging or
pathology.
B.3 Acute symptomatic
bleeding resulting in
hypotension, urgent
transfusion or fluid bolus.
2. Acute symptomatic
pulmonary haemorrhage not
shown by imaging or
pathology
Thrombosis C.1 Acute symptomatic
intracranial thrombosis or
embolism shown by
imaging or pathology.
C.2 Acute symptomatic
pulmonary thrombosis or
embolism shown by
imaging or pathology.
C.3 Acute symptomatic
hepatic thrombosis or
embolism shown by
imaging or pathology.
3. Acute symptomatic
pulmonary thrombosis
or embolism not shown
by imaging or
pathology.
Cardiac D.1 Acute symptomatic
arrhythmia excluding
sinus tachycardia or
bradycardia shown by
ECG.
D.2 Acute symptomatic
cardiac- dysfunction defined
by ECG, cardiac imaging or
pathology.
4. Acute symptomatic
arrhythmia excluding sinus
tachycardia or bradycardia
not shown by ECG.
Immunomediated
E.1 Acute allergic
reaction, anaphylaxis with
symptomatic
bronchospasm, oedema,
angioedema, or
hypotension.
E.2 Worsening
symptomatic graft
5. Stable graft-versus-host
disease.
Page 22 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyversus host disease.
E.3 Acute symptomatic
haemophagocytic
lymphohistocytosis,
macrophage activation
syndrome or cytokine-
release syndrome.
Metabolic F. Clinically diagnosed
tumour lysis syndrome
with cardiac arrhythmia,
seizure, or creatinine
concentrations greater
than 3 times the ULN.
CNS G.1 Acute symptomatic
CNS necrosis shown by
imaging or pathology
G.2 Acute symptomatic
encephalopathy shown by
imaging or
electroencephalography
G.2 Acute symptomatic
hydrocephalus or raised
intracranial pressure shown
by imaging, pathology, or
measurement of intracranial
pressure
G.4 Seizure lasting at least
30 minutes within 48 hours
of death
6.1 Acute symptomatic CNS
necrosis not shown by
imaging or pathology
6.2 Acute symptomatic
encephalopathy not shown
by imaging or
electroencephalography
6.2 Acute symptomatic
hydrocephalus or raised
intracranial pressure not
shown by imaging,
pathology, or measurement
of intracranial pressure
6.4 Seizure between 5 and
30 minutes within 48 hours
of death
Respiratory H. Acute symptomatic
respiratory distress with
ventilator support
7. Acute symptomatic
respiratory distress without
ventilator support
Gastrointestinal system I.1 Acute symptomatic
bowel disease resulting in
necrosis, obstruction or
perforation shown by
imaging or pathology
I.2 Acute, clinically
diagnosed hepatic
dysfunction associated with
conjugated bilirubin
concentrations greater than
10 x ULN, ammonium
concentrations greater than
2.5 x ULN, or international
normalised greater than 2.5
times the ULN
I.3 Acute, clinically
diagnosed pancreatitis with
haemorrhage, peritonitis,
8.1 Acute symptomatic
bowel disease resulting in
necrosis, obstruction or
perforation not shown by
imaging or pathology
8.2 Acute, clinically
diagnosed hepatic
dysfunction associated with
conjugated bilirubin
concentrations greater than
1.5 and less than 10 x ULN,
ammonium concentrations
greater than 1.5 and less
than 2.5 x ULN, or
international normalised
greater than 1.5 but less than
2.5 times the ULN
Page 23 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlynecrosis or haemodynamic
instability (evidenced by
hypotension, urgent
transfusion, fluid bolus, or
vasopressers
Renal system J. Acute kidney injury with
dialysis or renal replacement
therapy (planned or
received)
External causes K.1 Unintentional injury
(e.g accident)
K.2 Suicide
K.3 Homicide
ULN: Upper Limit of Normal ECG: electrocardiogram CNS: central nervous system
Page 24 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySupplemental file S2
Case Diagnosis CRA 1 CRA 2 Consultant
1
Consultant
2
A Germ cell CNS tumour Not TRM Not TRM Not TRM Not TRM
B Diffuse intrinsic
pontine glioma (DIPG)
Not TRM Not TRM Not TRM Not TRM
C Thalamic astrocytoma Not TRM Not TRM Not TRM Not TRM
D Relapsed B-cell ALL Not TRM Not TRM Not TRM Not TRM
E Spinal cord
glioblastoma
multiforme
Not TRM Not TRM Not TRM Not TRM
F Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
G Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
H Metastatic
rhabdomyosarcoma
Not TRM Not TRM Not TRM Not TRM
I Philadelphia positive
ALL
Not TRM Not TRM Not TRM Not TRM
J AML prior to HSCT TRM TRM TRM TRM
K Metastatic
rhabdomyosarcoma
Not TRM Not TRM Not TRM Not TRM
L Soft tissue
myoepithelial
carcinoma
Not TRM Not TRM Not TRM Not TRM
M ATRT TRM TRM Not TRM TRM
N Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
O High risk
neuroblastoma
Not TRM Not TRM Not TRM Not TRM
P Osteosarcoma Not TRM Not TRM Not TRM Not TRM
Q DIPG Not TRM Not TRM Not TRM Not TRM
R Osteosarcoma Not TRM Not TRM Not TRM Not TRM
Page 25 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyCase Diagnosis CRA 1 CRA 2 Consultant
1
Consultant
2
S High risk
neuroblastoma
Not TRM Not TRM Not TRM Not TRM
T Malignant melanoma of
the CNS
Not TRM Not TRM Not TRM Not TRM
U Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
V AML TRM TRM TRM TRM
W Epithelial sarcoma Not TRM Not TRM Not TRM Not TRM
X ALL TRM TRM TRM TRM
Y T-cell ALL TRM TRM TRM TRM
Z B-cell ALL TRM TRM TRM TRM
AA B-cell ALL TRM TRM TRM TRM
AB Ependymoma TRM TRM Not TRM TRM
AC T-cell ALL TRM TRM TRM TRM
AD B-cell ALL post HSCT TRM TRM TRM TRM
Page 26 of 26
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
Validation of a classification system for treatment-related
mortality in children with cancer
Journal: BMJ Paediatrics Open
Manuscript ID bmjpo-2017-000082.R1
Article Type: Original article
Date Submitted by the Author: 20-Sep-2017
Complete List of Authors: Hassan, Hadeel; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Rompola, Melpomeni; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology
Kinsey, Sally; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Glaser, Adam; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Phillips, Bob; University of York, Centre for Reviews and Dissemination; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology
Keywords: Oncology, Haematology, Palliative Care
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open on 11 M
ay 2018 by guest. Protected by copyright.
http://bmjpaedsopen.bm
j.com/
bmjpo: first published as 10.1136/bm
jpo-2017-000082 on 30 October 2017. D
ownloaded from
Confidential: For Review O
nlyTitle:
Validation of a classification system for treatment-related mortality in children with cancer
Authors:
H. Hassan 1, 2
, M. Rompola 1, 2
, A.W. Glaser 1, 2
, S. E. Kinsey 1, 2
, R. S. Phillips 1, 3
Affiliations:
1 Department of Paediatric Haematology and Oncology, Leeds Teaching Hospital NHS Trust,
Leeds, UK
2 Leeds Institute of Cancer and Pathology,
University of Leeds, Leeds, UK
3 Centre for Reviews and Dissemination, University of York, York, UK
Full address for correspondence:
Dr Hadeel Hassan
Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, LS2 9JT, UK
Telephone: +44 (0)113 343 2596
Email address: [email protected]
Page 1 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyAbstract
Background: Death not directly due to cancer has been termed “treatment-related mortality”
(TRM). Appreciating the differences between TRM and disease-related death is critical in
directing strategies to improve supportive care, interventions delivered or disease progression.
Recently, a global collaboration developed and validated a consensus-based classification tool
and attribution system.
Objectives: To evaluate the reliability of the newly developed consensus-based definition of
TRM and explore the use of the cause-of-death attribution system outside the centre it was
initially validated (Toronto, Canada).
Setting: The paediatric haematology and oncology department at Leeds Teaching Hospital in
Leeds, UK.
Participants: Two consultants and two clinical research associates (CRA).
Methods: Thirty medical records of the most recent deaths in children with cancer, two and
four weeks prior to death were anonymised and presented to the participants. Reviewers
independently classified deaths as “treatment related mortality” or “not treatment related”
according to the algorithm developed. When TRM occurred, reviewers applied the cause-of-
death attribution system to identify the primary cause of death. Inter-relater reliability was
assessed using the Kappa statistic (k).
Main outcome: Inter-relater reliability between CRA and consultants.
Results: Reliability of the classification was deemed “very good” between CRA and
consultants (k=0.86, 95% confidence interval 0.72-0.97). Ten deaths were classified as TRM;
of which infection was the most frequent cause identified. Reviewers disagreed on the primary
cause of death (e.g. respiratory versus infection) when applying the cause-of-death attribution
Page 2 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlysystem in 6 cases and probable and possible causes in 4 cases. The study identified how the
algorithm may not detect TRM in patients receiving non-curative therapy.
Conclusions: The classification and cause of death attribution system could be implemented
in different health care settings. Adaptation of the classification tool in patients receiving non-
curative interventions and the cause of death attribution system should be considered.
What is known about the subject?
• Treatment-related mortality (TRM) is poorly defined in paediatric oncology and
haematology.
• A global collaboration developed and validated a consensus-based classification tool
and attribution system (in Toronto, Canada).
What this study adds?
• The classification tool is not designed to detect TRM in patients receiving non-curative
therapy.
• The classification and cause of death attribution system can be used in different health
care settings.
Page 3 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyBackground
Survival of children diagnosed with malignancies has improved with recent estimates
suggesting more than 80% of those diagnosed in high income countries will survive [1]. This
still means 20% of children will die - around 80,000 deaths worldwide in 2012 [1]. The cause
of death may be due directly to the disease, or complications or toxicities of interventions
delivered. The increasing success of cancer-directed interventions has contributed to a higher
relative proportion of deaths due to associated toxicities [2]. Death not directly due to the
cancer has been broadly termed “treatment-related mortality” (TRM). This includes death
from infection, bleeding, and organ dysfunction [3]. Incidence varies according to the
underlying disease, stage of treatment and socio-economic status [4]. Identifying the nature of
the mortality is critical in helping researchers and clinicians improve survival in particular
diagnostic groups by focusing on strategies to improve supportive care when death is
predominantly through TRM.
Despite the importance of TRM, varying definitions have been used. Systematic reviews [5]
[6] identified significant heterogeneity in TRM definitions used in randomised trials. This
inspired a global collaboration to develop a consensus-based classification tool for ascribing
death as TRM, and further specific attribution of cause of death [7], primarily to be used by
clinical research assistants (CRA). Reliability for the TRM classification was “very good”
between medical and CRA reviewers that had been involved in developing the system
(kappa=0.92, 95% confidence interval 0.78-1.00) [7].
An ideal classification system for treatment-related mortality should be applicable across
different countries, treatment protocols, and health care settings. To assess this, it is important
to attempt to further validate the classification and attribution system in a different setting,
with professionals uninvolved in the development of the system. This paper reports such
Page 4 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyvalidation at a large regional paediatric oncology centre in a different continent to the one in
which the system was originally developed.
Objectives
This study aimed to evaluate the reliability of the newly developed consensus-based definition
of TRM and explore the use of the cause-of-death attribution system at a regional paediatric
oncology centre in Leeds, England.
Methods
This study was approved by the University of Leeds School of Medicine Ethics committee
(Ref: MREC15-118) and did not require NHS ethics approval. Eligible patient records were
those of patients treated for a malignancy or who underwent a haematopoietic stem cell
transplant (HSCT) for a non-malignant diagnosis at Leeds Children’s Hospital (Leeds, UK)
while aged 18 years or younger at diagnosis. Five cases were excluded as patients had died
following relapse after the age of 18, or the medical records were not located. All included
patients died between 2014 and 2016. Thirty patient records were included. Copies of the
clinical records, with information from both 2 weeks prior to death, and with the information
extending back to 4 weeks prior to death were anonymised. This resulted in 60 sets of
anonymised case-notes (30 patients, each with 2 time periods) which were presented in a
different random order for each assessor. 4 participants were identified to review the case
notes; the two CRAs were a data analyst (AF) and research nurse (JT), and the two senior
clinicians were a consultant paediatric oncologist and consultant paediatric haematologist
(AG and SK).
The study was undertaken on a single afternoon. After agreeing and signing a consent form,
participants received a 10 minute educational presentation explaining how to use the system,
and how the study would be undertaken. The reviewers then independently classified each
Page 5 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlydeath according to the algorithm (fig. 1). For cases assessed as TRM, the reviewers were
asked to apply the cause-of-death attribution system (supplemental file 1) to identify a
primary cause of death. Following the individual completion of the assessments, a moderated
group discussion was undertaken with notes recorded by two facilitators and used to
supplement the themes of the discussion.
Inter-relater reliability was assessed using the Kappa statistic (k). Criterion validity was
assessed by assuming classification by the Consultants as the gold standard. Group consensus
classification between and within the CRAs and Consultant group was evaluated using the
Cohen’s kappa statistic, and across all individuals using the Fleiss’ kappa statistic. The
strength of agreement was defined as slight (0.00-020), fair (0.21-0.40), moderate (0.41-
0.60), good (0.61-0.80) and very good (0.81-1.00) [8]. A numerical code was used to
combine agreement/disagreement between the individual consultants (TRM was recorded as
“0” and non TRM outcomes were recorded as “1” in Excel). When calculating inter-relater
reliability between the CRAs and consultants if individual disagreement was recorded then
the outcome was recorded as “2”. Based on the previously published study [7], we decided
to include 30 cases for analysis. A sample size of 27 deaths determined whether k was good
(i.e., ≥0·61), with a power of 0·80, and two-sided α of 0·05 and assuming that treatment-
related mortality accounted for 20% of deaths (6). A further 30 cases would be reviewed if
validity was inadequate (defined a priori as k<0.6). Calculation of the k statistic was
completed using the R studio irr package, and a bootstrap with 200 iterations was used to
calculate 95% confidence intervals [9]. Comparison of cause of death was qualitative, and to
provide further insight reviewers participated in a discussion of the use of the algorithm and
cause of death attribution system.
Page 6 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyFigure 1: Classification of TRM in children with cancer, taken from (7)
Page 7 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly Results
Demographics
Age of identified patients ranged from less than 1 to 17 years and 57% (17) were male. 67%
(20) were diagnosed with solid tumours, and 33% (10) were diagnosed with malignant
haematological conditions. Collectively, sarcomas were the most frequent solid tumour
diagnosed (12 cases, 40% of total), followed by CNS tumours (9 cases, 30% of total). 27% (8
cases) of these patients had presented with metastatic disease at diagnosis. All patients with
malignant haematological conditions were diagnosed with leukaemia. 40% (12 cases) of
patients had either received a transplant or presented with relapsed disease. A summary is
enclosed in a supplementary file 2.
Classification of treatment-related mortality
Ten deaths (33%) were identified as TRM by at least one reviewer. Fifteen percent (3/20) of
patients diagnosed with solid tumours, and 80% (8/10) of patients diagnosed with malignant
haematological conditions who died were classified as TRM. Reliability of classification was
very good between CRAs and consultants, with a k statistic of 0.86 (95% confidence interval
0.72-0.97, with disagreement on 3 deaths). There was also very good agreement between
CRAs (k=0.96, 95% confidence interval 0.87-1.00, disagreement on one record) and between
consultants (k=0.85, 95% confidence interval 0.67-0.97, with disagreement on two deaths)
(See Table 1).
When the 2 and 4 week data were examined, there no difference between the assessments of
each of the four assessors (see Table 1).
Cause of death attribution system
Table 2 summarises the diagnoses of the ten patients whose deaths were classified as TRM,
alongside the cause of death as attribution by each assessor.
Page 8 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyIn brief, there was unanimous agreement on cause of death of 3 cases (J, V, AA), the other 7
cases has inconsistencies in either the nature of cause of death (e.g. infection vs. immune
mediated), or strength of evidence (i.e. probable vs. possible). In one complex scenario (AD),
the CRAs did not feel able to record a cause of death.
Post-review discussion
The reviewers all agreed that the algorithm was straight forward to use, and that it would be
beneficial to have a standardised tool and attribution system to use in trials nationally and
globally.
Three difficulties with the system were identified: firstly, how to address the patients who
may die from TRM whilst receiving non-curative therapy and particularly how the algorithm
could be used as part of palliative care trial. In one particular case a patient taking palliative
etoposide following a diagnosis of relapsed ALL developed a febrile illness, and died.
Secondly, how should deaths in children after the completion of treatment be classified? A
child (case V) who had completed treatment for standard risk AML died of overwhelming
pneumococcal septicaemia 6 months after end of treatment whist in complete remission.
Another further case the assessors had difficulty with (case M) followed presentation acutely
with signs of raised intra-cranial pressure and large mass noted on CT. This patient died on
the operating table whilst receiving surgical intervention.
Attribution of cause of death was felt to be more difficult than ascribing a death as TRM or
not. The case the reviewers felt was most difficult to assign was patient who died of multi-
organ failure following a HSCT (AD); both CRAs independently decided to not attribute a
cause of death.
Page 9 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyTable 1 Summary of kappa statistic and 95% confidence intervals of independent
reviewers, consultants, CRAs and between CRA and consultants for all total case
reviews, 4 weeks and 2 weeks prior to death using the cause of death attribution system.
*calculated using the Fleiss kappa statistic (between 4 reviewers).
**calculated using the Cohen’s kappa statistic (between 2 reviewers or 2 groups).
Inter-rater
comparison
Total k (95%
confidence interval)
4 weeks k (95%
confidence interval)
2 weeks k (95%
confidence interval)
Independent
reviewers *
0.92 (0.83-0.98) 0.91(0.76-1.00) 0.92 (0.79-1.00)
Consultants ** 0.85 (0.67-0.97) 0.85 (0.59-1.00) 0.84 (0.59-1.00)
CRA ** 0.96 (0.87-1.00) 0.85 (0.59-1.00) 0.85 (0.60-1.00)
CRA vs
consultants**
0.86 (0.72-0.97) 0.87 (0.66-1.00) 0.86 (0.67-1.00)
Page 10 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review Only
Table 2 Summary of cause of death attribution by reviewers for deaths classified as TRM. Bold font=probable causes of death.
Case Diagnosis CRA 1 CRA 2 Consultant 1 Consultant 2
J AML prior to HSCT
Immunomediated Immunomediated Immunomediated Immunomediated
M ATRT Acute symptomatic
intracranial haemorrhage
Acute symptomatic
intracranial
haemorrhage
NR Acute symptomatic intracranial
haemorrhage
V AML Infection Infection Infection Infection
X B-cell ALL Acute symptomatic
intracranial haemorrhage
Acute symptomatic
intracranial haemorrhage Infection Infection
Y T-cell ALL Respiratory Infection Acute symptomatic
intracranial haemorrhage
Infection
Z B-cell ALL Respiratory Infection Respiratory Infection
AA B-cell ALL Infection Infection Infection Infection
AB Ependymoma NR Infection NR Infection
AC T-cell ALL Worsening symptomatic
graft versus host disease.
Worsening symptomatic
graft versus host disease.
Worsening symptomatic
graft versus host disease.
Acute symptomatic pulmonary
haemorrhage
AD B-cell ALL post
HSCT
Unclear Difficult case Worsening symptomatic
graft versus host disease.
Acute kidney Injury
Page 11 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. Protected by copyright. http://bmjpaedsopen.bmj.com/ bmjpo: first published as 10.1136/bmjpo-2017-000082 on 30 October 2017. Downloaded from
Confidential: For Review Only
Page 12 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. Protected by copyright. http://bmjpaedsopen.bmj.com/ bmjpo: first published as 10.1136/bmjpo-2017-000082 on 30 October 2017. Downloaded from
Confidential: For Review O
nlyDiscussion
Strengths of study
This study is, to the best of our knowledge, the first revalidation of the standardised definition
of treatment-related mortality and cause of death attribution system for paediatric cancer
patients [7]. It demonstrates that the system is reliable and established its validity in an
alternative centre and health care system with different treatment protocols. It can be used
after very limited training, with “very good” agreement between assessors irrespective of
discipline (Fleiss kappa 0.92, 95% CI 0.83-0.98). The study confirms the observations of the
development group and shows that information from two weeks prior to the death of a patient
is sufficient to consistently attribute death to TRM or disease.
Limitations of study
Although consultants’ opinions are considered gold-standard, in this study we identified how
even experienced clinicians may disagree on use of the algorithm. Consultants disagreed on
the classification of death in two cases- this may have occurred due to the individual
consultant’s clinical experience, or previous contact with the patients. Even though the cases
were anonymised and randomised the physicians may have recognised the patient due to their
potential clinical involvement in direct patient care. The differences identified highlight how
the TRM classification tool is unlikely to ever have perfect agreement between reviewers
irrespective of experience, and clinical, and scientific knowledge.
In this study reviewers attributed death to one primary probable, or possible, cause. Whilst
developing the study protocol, we decided to limit the number of causes of death for
simplicity. However, reviewers found it challenging to identify only one cause of death, and
distinguish between probable and possible causes.
Page 13 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySince the development of this study, a standard operating procedure TRM web-based tool has
been published (https://www.sungresearch.com/trm-training-manual/) and includes working
examples. Use of this tool when delivering the training package should help clarify how to
use the cause-of-death attribution system and minimise misunderstanding. Currently, the web-
based tool is available in English, having the tool available in other languages could
potentially reduce confusion and improve harmonisation across clinical trials.
Challenges and areas for future development
Our study highlights specific challenges with the system as it currently exists, both with the
classification of TRM and the attribution of a specific cause of death. Fundamentally, this
approach defines deaths as either ‘treatment related’ or ‘cancer related’. This gives rise to a
semantic challenge; “treatment-related mortality” implies that deaths that come under this
term occur directly because of the therapies delivered. However, the classification system
classifies deaths which occur prior to commencement of anti-cancer therapy, which are not
directly attributable to the cancer (for example, tumour lysis syndrome in high-count
leukaemia) as cases of TRM. This clash of language and “common sense” may confuse users
of the classification tool, for example, case M in which a patient presented acutely with signs
of raised intra-cranial pressure and died on the operating table.
A deeper challenge to this system addressed the philosophical distinction of assigning all
deaths into one of two categories; cancer or treatment related. There is a convincing argument
that a third category of death should be attributable, “other non-cancer death”, for those who
die of an event or illness external to their malignancy. This problem is particularly evident if
the current system is to be used after the completion of treatment. For example, a patient dies
as a passenger in an air traffic accident, 4 years after treatment for a localised Wilms tumour.
Page 14 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyIn the current system, this death is classified as TRM, even though the death is unrelated to
the child’s cancer diagnosis.
Conversely, it is also, important to note how a diagnosis of cancer may be included in ‘non-
cancer deaths’. For example a patient may commit suicide some years after the completion of
treatment because of the psychological effects of their diagnosis or treatment. Any addition to
the system would need to be sensitive to these potential issues.
We have identified the need to further refine the approach to categorising cause of death in
cancer patients receiving care without intent to cure. This is particularly important if the
system is applied in ‘routine’ settings, assessing deaths in the palliative setting rather than in
the original setting of use within a curative trial. Increasingly individuals destined not to be
cured are living for lengthier periods due to participation in clinical trials/studies. This group
of patients currently have all deaths classified as “not treatment-related mortality” as
clinicians would have specified progressive disease, or that cancer therapy has no curative
intent. This algorithm may fail to identify a significant group of patients who may die of
causes amenable to better supportive care whilst receiving palliative care. For example, a
patient can die of overwhelming sepsis whilst receiving palliative etoposide for refractory
neuroblastoma. This could be addressed by modifying the algorithm, for this type of use.
Another proposal includes using a separate classification tool for patients on non-curative
therapy trials (fig. 2) although this should be further developed in conjunction with palliative
care physicians and researchers. The counter-argument to this suggested change is the risk of
adding complexity to a simple, effective, tool which can be used by people of different skills
and health care settings globally. It will also have similar issues with interpretation as with
initial algorithm.
Page 15 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyReviewers failed to agree on a primary cause of death in 6 episodes and probable and possible
causes in 4 cases. Differences in cause of death allocated could be attributed to the reviewers’
previous experience, clinical expertise and interpretation of the clinical records, particularly in
light of potential previous direct clinical involvement with the cases under review. Currently,
the tool is intended for use by any CRA, however, it may require users to have a certain level
of experience or clinical expertise, and agreement may be reduced with CRA who are new to
the role.
Understanding and interpretation of the system as proposed for attribution of a specific
mechanistic cause of death could potentially be improved by dedicating more time during the
presentations and using the newly developed web-based training tool. Alternatively, the cause
of death attribution system could be refined, and multiple causalities permitted.
Page 16 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyFigure 2: Proposed classification of TRM in children receiving non-curative therapy
(NCT) only.
Page 17 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConclusions
We have been able to confirm the reproducibility and criterion validity of the TRM
classification system. We believe this supports the hypothesis that the classification system
can be implemented easily and effectively in different health care settings, thereby improving
consistency and accuracy of outcome reporting in clinical trials. The TRM classification
system will be of immense value in the evaluation of deaths in the palliative setting. We
propose the addition of a separate classification tool in patients on palliative trials.
Page 18 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConflict of interest statement
The authors declare no conflict of interest.
Acknowledgments
The authors would like to thank the reviewers for participating in this study. This work was
supported by Candlelighters charity who funded Dr Hadeel Hassan’s PhD, for which this
research was conducted. H.H wrote the manuscript and R.S.P edited the manuscript. M.R,
S.E.K and A.G contributed to the review process.
Page 19 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyReferences
1. CRUK. Children's cancers mortality in Europe and worldwide. 2014 [cited 2016
September]; Available from: http://www.cancerresearchuk.org/health-
professional/cancer-statistics/childrens-cancers/mortality#heading-Three.
2. Kremer, L.C., et al., A worldwide collaboration to harmonize guidelines for the long-
term follow-up of childhood and young adult cancer survivors: a report from the
International Late Effects of Childhood Cancer Guideline Harmonization Group.
Pediatr Blood Cancer, 2013. 60(4): p. 543-9.
3. Blanco, E., et al., Non-relapse mortality in pediatric acute lymphoblastic leukemia: a
systematic review and meta-analysis. Leuk Lymphoma, 2012. 53(5): p. 878-85.
4. White, Y., V.P. Castle, and A. Haig, Pediatric oncology in developing countries:
challenges and solutions. J Pediatr, 2013. 162(6): p. 1090-1, 1091.e1.
5. Ethier, M.C., et al., Lack of clarity in the definition of treatment-related mortality:
pediatric acute leukemia and adult acute promyelocytic leukemia as examples. Blood,
2011. 118(19): p. 5080-3.
6. Tran, T.H., et al., Lack of treatment-related mortality definitions in clinical trials of
children, adolescents and young adults with lymphomas, solid tumors and brain
tumors: a systematic review. BMC Cancer, 2014. 14: p. 612.
7. Alexander, S., et al., Classification of treatment-related mortality in children with
cancer: a systematic assessment. Lancet Oncol, 2015. 16(16): p. e604-10.
8. McHugh, M.L., Interrater reliability: the kappa statistic. Biochemia Medica, 2012.
22(3): p. 276-282.
9. Team, R., RStudio: Integrated Development for R. RStudio, Inc. 2015.
Figure legends
Figure 1: Classification of TRM in children with cancer, taken from (7).
Figure 2: Proposed classification of TRM in children receiving non-curative therapy (NCT)
only.
Supplementary files legend list
Supplemental table S1: Cause-of-death attribution system.
Supplemental table S2: Diagnosis of included patients results of classification by reviewers.
Page 20 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
81x45mm (300 x 300 DPI)
Page 21 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
45x25mm (300 x 300 DPI)
Page 22 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySupplemental file S1
Probable cause of death Possible cause of death
Infection A.1 Clinically or
radiographically
documented infection with
associated microbiologically
documented organism
1. Clinically or
radiographically
documented infection
without associated
microbiologically
documented organism.
Haemorrhage B.1 Acute symptomatic
intracranial haemorrhage
shown by imaging or
pathology.
B.2 Acute symptomatic
pulmonary haemorrhage
shown by imaging or
pathology.
B.3 Acute symptomatic
bleeding resulting in
hypotension, urgent
transfusion or fluid bolus.
2. Acute symptomatic
pulmonary haemorrhage not
shown by imaging or
pathology
Thrombosis C.1 Acute symptomatic
intracranial thrombosis or
embolism shown by
imaging or pathology.
C.2 Acute symptomatic
pulmonary thrombosis or
embolism shown by
imaging or pathology.
C.3 Acute symptomatic
hepatic thrombosis or
embolism shown by
imaging or pathology.
3. Acute symptomatic
pulmonary thrombosis
or embolism not shown
by imaging or
pathology.
Cardiac D.1 Acute symptomatic
arrhythmia excluding
sinus tachycardia or
bradycardia shown by
ECG.
D.2 Acute symptomatic
cardiac- dysfunction defined
by ECG, cardiac imaging or
pathology.
4. Acute symptomatic
arrhythmia excluding sinus
tachycardia or bradycardia
not shown by ECG.
Immunomediated
E.1 Acute allergic
reaction, anaphylaxis with
symptomatic
bronchospasm, oedema,
angioedema, or
hypotension.
5. Stable graft-versus-host
disease.
Page 23 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyE.2 Worsening
symptomatic graft
versus host disease.
E.3 Acute symptomatic
haemophagocytic
lymphohistocytosis,
macrophage activation
syndrome or cytokine-
release syndrome.
Metabolic F. Clinically diagnosed
tumour lysis syndrome
with cardiac arrhythmia,
seizure, or creatinine
concentrations greater
than 3 times the ULN.
CNS G.1 Acute symptomatic
CNS necrosis shown by
imaging or pathology
G.2 Acute symptomatic
encephalopathy shown by
imaging or
electroencephalography
G.2 Acute symptomatic
hydrocephalus or raised
intracranial pressure shown
by imaging, pathology, or
measurement of intracranial
pressure
G.4 Seizure lasting at least
30 minutes within 48 hours
of death
6.1 Acute symptomatic CNS
necrosis not shown by
imaging or pathology
6.2 Acute symptomatic
encephalopathy not shown
by imaging or
electroencephalography
6.2 Acute symptomatic
hydrocephalus or raised
intracranial pressure not
shown by imaging,
pathology, or measurement
of intracranial pressure
6.4 Seizure between 5 and
30 minutes within 48 hours
of death
Respiratory H. Acute symptomatic
respiratory distress with
ventilator support
7. Acute symptomatic
respiratory distress without
ventilator support
Gastrointestinal system I.1 Acute symptomatic
bowel disease resulting in
necrosis, obstruction or
perforation shown by
imaging or pathology
I.2 Acute, clinically
diagnosed hepatic
dysfunction associated with
conjugated bilirubin
concentrations greater than
10 x ULN, ammonium
concentrations greater than
2.5 x ULN, or international
normalised greater than 2.5
times the ULN
8.1 Acute symptomatic
bowel disease resulting in
necrosis, obstruction or
perforation not shown by
imaging or pathology
8.2 Acute, clinically
diagnosed hepatic
dysfunction associated with
conjugated bilirubin
concentrations greater than
1.5 and less than 10 x ULN,
ammonium concentrations
greater than 1.5 and less
than 2.5 x ULN, or
international normalised
Page 24 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyI.3 Acute, clinically
diagnosed pancreatitis with
haemorrhage, peritonitis,
necrosis or haemodynamic
instability (evidenced by
hypotension, urgent
transfusion, fluid bolus, or
vasopressers
greater than 1.5 but less than
2.5 times the ULN
Renal system J. Acute kidney injury with
dialysis or renal replacement
therapy (planned or
received)
External causes K.1 Unintentional injury
(e.g accident)
K.2 Suicide
K.3 Homicide
ULN: Upper Limit of Normal ECG: electrocardiogram CNS: central nervous system
Page 25 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySupplemental file S2
Case Diagnosis CRA 1 CRA 2 Consultant
1
Consultant
2
A Germ cell CNS tumour Not TRM Not TRM Not TRM Not TRM
B Diffuse intrinsic
pontine glioma (DIPG)
Not TRM Not TRM Not TRM Not TRM
C Thalamic astrocytoma Not TRM Not TRM Not TRM Not TRM
D Relapsed B-cell ALL Not TRM Not TRM Not TRM Not TRM
E Spinal cord
glioblastoma
multiforme
Not TRM Not TRM Not TRM Not TRM
F Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
G Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
H Metastatic
rhabdomyosarcoma
Not TRM Not TRM Not TRM Not TRM
I Philadelphia positive
ALL
Not TRM Not TRM Not TRM Not TRM
J AML prior to HSCT TRM TRM TRM TRM
K Metastatic
rhabdomyosarcoma
Not TRM Not TRM Not TRM Not TRM
L Soft tissue
myoepithelial
carcinoma
Not TRM Not TRM Not TRM Not TRM
M ATRT TRM TRM Not TRM TRM
N Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
O High risk
neuroblastoma
Not TRM Not TRM Not TRM Not TRM
P Osteosarcoma Not TRM Not TRM Not TRM Not TRM
Q DIPG Not TRM Not TRM Not TRM Not TRM
R Osteosarcoma Not TRM Not TRM Not TRM Not TRM
Page 26 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyCase Diagnosis CRA 1 CRA 2 Consultant
1
Consultant
2
S High risk
neuroblastoma
Not TRM Not TRM Not TRM Not TRM
T Malignant melanoma
of the CNS
Not TRM Not TRM Not TRM Not TRM
U Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
V AML TRM TRM TRM TRM
W Epithelial sarcoma Not TRM Not TRM Not TRM Not TRM
X ALL TRM TRM TRM TRM
Y T-cell ALL TRM TRM TRM TRM
Z B-cell ALL TRM TRM TRM TRM
AA B-cell ALL TRM TRM TRM TRM
AB Ependymoma TRM TRM Not TRM TRM
AC T-cell ALL TRM TRM TRM TRM
AD B-cell ALL post HSCT TRM TRM TRM TRM
Page 27 of 27
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
Validation of a classification system for treatment-related
mortality in children with cancer
Journal: BMJ Paediatrics Open
Manuscript ID bmjpo-2017-000082.R2
Article Type: Original article
Date Submitted by the Author: 10-Oct-2017
Complete List of Authors: Hassan, Hadeel; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Rompola, Melpomeni; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology
Kinsey, Sally; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Glaser, Adam; University of Leeds, Leeds Institute of Cancer and Pathology; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology Phillips, Bob; University of York, Centre for Reviews and Dissemination; Leeds Teaching Hospitals NHS Trust, Paediatric Haematology and Oncology
Keywords: Oncology, Haematology, Palliative Care
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open on 11 M
ay 2018 by guest. Protected by copyright.
http://bmjpaedsopen.bm
j.com/
bmjpo: first published as 10.1136/bm
jpo-2017-000082 on 30 October 2017. D
ownloaded from
Confidential: For Review O
nlyTitle:
Validation of a classification system for treatment-related mortality in children with cancer
Authors:
H. Hassan 1, 2
, M. Rompola 1, 2
, A.W. Glaser 1, 2
, S. E. Kinsey 1, 2
, R. S. Phillips 1, 3
Affiliations:
1 Department of Paediatric Haematology and Oncology, Leeds Teaching Hospital NHS Trust,
Leeds, UK
2 Leeds Institute of Cancer and Pathology,
University of Leeds, Leeds, UK
3 Centre for Reviews and Dissemination, University of York, York, UK
Full address for correspondence:
Dr Hadeel Hassan
Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, LS2 9JT, UK
Telephone: +44 (0)113 343 2596
Email address: [email protected]
Page 1 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyAbstract
Background: Death not directly due to cancer has been termed “treatment-related mortality”
(TRM). Appreciating the differences between TRM and disease-related death is critical in
directing strategies to improve supportive care, interventions delivered or disease progression.
Recently, a global collaboration developed and validated a consensus-based classification tool
and attribution system.
Objectives: To evaluate the reliability of the newly developed consensus-based definition of
TRM and explore the use of the cause-of-death attribution system outside the centre it was
initially validated (Toronto, Canada). In the initial study reviewers listed multiple causes of
death. In this study, reviewers identified a primary cause for simplicity.
Setting: The paediatric haematology and oncology department at Leeds Teaching Hospital in
Leeds, UK.
Participants: Two consultants and two clinical research associates (CRA).
Methods: Thirty medical records of the most recent deaths in children with cancer, two and
four weeks prior to death were anonymised and presented to the participants. Reviewers
independently classified deaths as “treatment related mortality” or “not treatment related”
according to the algorithm developed. When TRM occurred, reviewers applied the cause-of-
death attribution system to identify the primary cause of death. Inter-relater reliability was
assessed using the Kappa statistic (k).
Main outcome: Inter-relater reliability between CRA and consultants.
Results: Reliability of the classification was deemed “very good” between CRA and
consultants (k=0.86, 95% confidence interval 0.72-0.97). Ten deaths were classified as TRM;
of which infection was the most frequent cause identified. Reviewers disagreed on the primary
cause of death (e.g. respiratory versus infection) when applying the cause-of-death attribution
Page 2 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlysystem in 6 cases and probable and possible causes in 4 cases. The study identified how the
algorithm may not detect TRM in patients receiving non-curative therapy.
Conclusions: The classification and cause of death attribution system could be implemented
in different health care settings. Adaptation of the classification tool in patients receiving non-
curative interventions and the cause of death attribution system should be considered.
What is known about the subject?
• Treatment-related mortality (TRM) is poorly defined in paediatric oncology and
haematology.
• A global collaboration developed and validated a consensus-based classification tool
and attribution system (in Toronto, Canada).
What this study adds?
• The classification tool is not designed to detect TRM in patients receiving non-curative
therapy.
• The classification and cause of death attribution system can be used in different health
care settings.
Page 3 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyBackground
Survival of children diagnosed with malignancies has improved with recent estimates
suggesting more than 80% of those diagnosed in high income countries will survive [1]. This
still means 20% of children will die - around 80,000 deaths worldwide in 2012 [1]. The cause
of death may be due directly to the disease, or complications or toxicities of interventions
delivered. The increasing success of cancer-directed interventions has contributed to a higher
relative proportion of deaths due to associated toxicities [2]. Death not directly due to the
cancer has been broadly termed “treatment-related mortality” (TRM). This includes death
from infection, bleeding, and organ dysfunction [3]. Incidence varies according to the
underlying disease, stage of treatment and socio-economic status [4]. Identifying the nature of
the mortality is critical in helping researchers and clinicians improve survival in particular
diagnostic groups by focusing on strategies to improve supportive care when death is
predominantly through TRM.
Despite the importance of TRM, varying definitions have been used. Systematic reviews [5]
[6] identified significant heterogeneity in TRM definitions used in randomised trials. This
inspired a global collaboration to develop a consensus-based classification tool for ascribing
death as TRM, and further specific attribution of cause of death [7], primarily to be used by
clinical research assistants (CRA). Reliability for the TRM classification was “very good”
between medical and CRA reviewers that had been involved in developing the system
(kappa=0.92, 95% confidence interval 0.78-1.00) [7].
An ideal classification system for treatment-related mortality should be applicable across
different countries, treatment protocols, and health care settings. To assess this, it is important
to attempt to further validate the classification and attribution system in a different setting,
with professionals uninvolved in the development of the system. This paper reports such
Page 4 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyvalidation at a large regional paediatric oncology centre in a different continent to the one in
which the system was originally developed. In the original study reviewers attributed death to
multiple probable or possible causes. Whilst developing the study protocol, we decided to
limit the number of causes of death to one primary probable or possible cause for simplicity.
Objectives
This study aimed to evaluate the reliability of the newly developed consensus-based definition
of TRM and explore the use of the cause-of-death attribution system at a regional paediatric
oncology centre in Leeds, England. In the initial study reviewers listed multiple causes of
death. However, in this study reviewers identified a primary cause for simplicity.
Methods
This study was approved by the University of Leeds School of Medicine Ethics committee
(Ref: MREC15-118) and did not require NHS ethics approval. Eligible patient records were
those of patients treated for a malignancy or who underwent a haematopoietic stem cell
transplant (HSCT) for a non-malignant diagnosis at Leeds Children’s Hospital (Leeds, UK)
while aged 18 years or younger at diagnosis. Five cases were excluded as patients had died
following relapse after the age of 18, or the medical records were not located. All included
patients died between 2014 and 2016. Thirty patient records were included. Copies of the
clinical records, with information from both 2 weeks prior to death, and with the information
extending back to 4 weeks prior to death were anonymised. This resulted in 60 sets of
anonymised case-notes (30 patients, each with 2 time periods) which were presented in a
different random order for each assessor. 4 participants were identified to review the case
notes; the two CRAs were a data analyst (AF) and research nurse (JT), and the two senior
clinicians were a consultant paediatric oncologist and consultant paediatric haematologist
(AG and SK).
Page 5 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyThe study was undertaken on a single afternoon. After agreeing and signing a consent form,
participants received a 10 minute educational presentation explaining how to use the system,
and how the study would be undertaken. The reviewers then independently classified each
death according to the algorithm (fig. 1). For cases assessed as TRM, the reviewers were
asked to apply the cause-of-death attribution system (supplemental file 1) to identify a
primary cause of death. Following the individual completion of the assessments, a moderated
group discussion was undertaken with notes recorded by two facilitators and used to
supplement the themes of the discussion.
Inter-relater reliability was assessed using the Kappa statistic (k). Criterion validity was
assessed by assuming classification by the Consultants as the gold standard. Group consensus
classification between and within the CRAs and Consultant group was evaluated using the
Cohen’s kappa statistic, and across all individuals using the Fleiss’ kappa statistic. The
strength of agreement was defined as slight (0.00-020), fair (0.21-0.40), moderate (0.41-
0.60), good (0.61-0.80) and very good (0.81-1.00) [8]. A numerical code was used to
combine agreement/disagreement between the individual consultants (TRM was recorded as
“0” and non TRM outcomes were recorded as “1” in Excel). When calculating inter-relater
reliability between the CRAs and consultants if individual disagreement was recorded then
the outcome was recorded as “2”. Based on the previously published study [7], we decided
to include 30 cases for analysis. A sample size of 27 deaths determined whether k was good
(i.e., ≥0·61), with a power of 0·80, and two-sided α of 0·05 and assuming that treatment-
related mortality accounted for 20% of deaths (6). A further 30 cases would be reviewed if
validity was inadequate (defined a priori as k<0.6). Calculation of the k statistic was
completed using the R studio irr package, and a bootstrap with 200 iterations was used to
calculate 95% confidence intervals [9]. Comparison of cause of death was qualitative, and to
Page 6 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyprovide further insight reviewers participated in a discussion of the use of the algorithm and
cause of death attribution system.
Page 7 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyFigure 1: Classification of TRM in children with cancer, taken from (7)
Page 8 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly Results
Demographics
Age of identified patients ranged from less than 1 to 17 years and 57% (17) were male. 67%
(20) were diagnosed with solid tumours, and 33% (10) were diagnosed with malignant
haematological conditions. Collectively, sarcomas were the most frequent solid tumour
diagnosed (12 cases, 40% of total), followed by CNS tumours (9 cases, 30% of total). 27% (8
cases) of these patients had presented with metastatic disease at diagnosis. All patients with
malignant haematological conditions were diagnosed with leukaemia. 40% (12 cases) of
patients had either received a transplant or presented with relapsed disease. A summary is
enclosed in a supplementary file 2.
Classification of treatment-related mortality
Ten deaths (33%) were identified as TRM by at least one reviewer. Fifteen percent (3/20) of
patients diagnosed with solid tumours, and 80% (8/10) of patients diagnosed with malignant
haematological conditions who died were classified as TRM. Reliability of classification was
very good between CRAs and consultants, with a k statistic of 0.86 (95% confidence interval
0.72-0.97, with disagreement on 3 deaths). There was also very good agreement between
CRAs (k=0.96, 95% confidence interval 0.87-1.00, disagreement on one record) and between
consultants (k=0.85, 95% confidence interval 0.67-0.97, with disagreement on two deaths)
(See Table 1).
When the 2 and 4 week data were examined, there no difference between the assessments of
each of the four assessors (see Table 1).
Cause of death attribution system
Table 2 summarises the diagnoses of the ten patients whose deaths were classified as TRM,
alongside the cause of death as attribution by each assessor.
Page 9 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyIn brief, there was unanimous agreement on cause of death of 3 cases (J, V, AA), the other 7
cases has inconsistencies in either the nature of cause of death (e.g. infection vs. immune
mediated), or strength of evidence (i.e. probable vs. possible). In one complex scenario (AD),
the CRAs did not feel able to record a cause of death.
Post-review discussion
The reviewers all agreed that the algorithm was straight forward to use, and that it would be
beneficial to have a standardised tool and attribution system to use in trials nationally and
globally.
Three difficulties with the system were identified: firstly, how to address the patients who
may die from TRM whilst receiving non-curative therapy and particularly how the algorithm
could be used as part of palliative care trial. In one particular case a patient taking palliative
etoposide following a diagnosis of relapsed ALL developed a febrile illness, and died.
Secondly, how should deaths in children after the completion of treatment be classified? A
child (case V) who had completed treatment for standard risk AML died of overwhelming
pneumococcal septicaemia 6 months after end of treatment whist in complete remission.
Another further case the assessors had difficulty with (case M) followed presentation acutely
with signs of raised intra-cranial pressure and large mass noted on CT. This patient died on
the operating table whilst receiving surgical intervention.
Attribution of cause of death was felt to be more difficult than ascribing a death as TRM or
not. The case the reviewers felt was most difficult to assign was patient who died of multi-
organ failure following a HSCT (AD); both CRAs independently decided to not attribute a
cause of death.
Page 10 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyTable 1 Summary of kappa statistic and 95% confidence intervals of independent
reviewers, consultants, CRAs and between CRA and consultants for all total case
reviews, 4 weeks and 2 weeks prior to death using the cause of death attribution system.
*calculated using the Fleiss kappa statistic (between 4 reviewers).
**calculated using the Cohen’s kappa statistic (between 2 reviewers or 2 groups).
Inter-rater
comparison
Total k (95%
confidence interval)
4 weeks k (95%
confidence interval)
2 weeks k (95%
confidence interval)
Independent
reviewers *
0.92 (0.83-0.98) 0.91(0.76-1.00) 0.92 (0.79-1.00)
Consultants ** 0.85 (0.67-0.97) 0.85 (0.59-1.00) 0.84 (0.59-1.00)
CRA ** 0.96 (0.87-1.00) 0.85 (0.59-1.00) 0.85 (0.60-1.00)
CRA vs
consultants**
0.86 (0.72-0.97) 0.87 (0.66-1.00) 0.86 (0.67-1.00)
Page 11 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review Only
Table 2 Summary of cause of death attribution by reviewers for deaths classified as TRM. Bold font=probable causes of death.
Case Diagnosis CRA 1 CRA 2 Consultant 1 Consultant 2
J AML prior to HSCT
Immunomediated Immunomediated Immunomediated Immunomediated
M ATRT Acute symptomatic
intracranial haemorrhage
Acute symptomatic
intracranial
haemorrhage
NR Acute symptomatic intracranial
haemorrhage
V AML Infection Infection Infection Infection
X B-cell ALL Acute symptomatic
intracranial haemorrhage
Acute symptomatic
intracranial haemorrhage Infection Infection
Y T-cell ALL Respiratory Infection Acute symptomatic
intracranial haemorrhage
Infection
Z B-cell ALL Respiratory Infection Respiratory Infection
AA B-cell ALL Infection Infection Infection Infection
AB Ependymoma NR Infection NR Infection
AC T-cell ALL Worsening symptomatic
graft versus host disease.
Worsening symptomatic
graft versus host disease.
Worsening symptomatic
graft versus host disease.
Acute symptomatic pulmonary
haemorrhage
AD B-cell ALL post
HSCT
Unclear Difficult case Worsening symptomatic
graft versus host disease.
Acute kidney Injury
Page 12 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. Protected by copyright. http://bmjpaedsopen.bmj.com/ bmjpo: first published as 10.1136/bmjpo-2017-000082 on 30 October 2017. Downloaded from
Confidential: For Review Only
Page 13 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. Protected by copyright. http://bmjpaedsopen.bmj.com/ bmjpo: first published as 10.1136/bmjpo-2017-000082 on 30 October 2017. Downloaded from
Confidential: For Review O
nlyDiscussion
This study is, to the best of our knowledge, the first revalidation of the standardised definition
of treatment-related mortality and cause of death attribution system for paediatric cancer
patients [7]. It demonstrates that the system is reliable and established its validity in an
alternative centre and health care system with different treatment protocols. It can be used
after very limited training, with “very good” agreement between assessors irrespective of
discipline (Fleiss kappa 0.92, 95% CI 0.83-0.98). The study confirms the observations of the
development group and shows that information from two weeks prior to the death of a patient
is sufficient to consistently attribute death to TRM or disease.
Although consultants’ opinions are considered gold-standard, in this study we identified how
even experienced clinicians may disagree on use of the algorithm. Consultants disagreed on
the classification of death in two cases- this may have occurred due to the individual
consultant’s clinical experience, or previous contact with the patients. Even though the cases
were anonymised and randomised the physicians may have recognised the patient due to their
potential clinical involvement in direct patient care. The differences identified highlight how
the TRM classification tool is unlikely to ever have perfect agreement between reviewers
irrespective of experience, and clinical, and scientific knowledge.
In this study reviewers attributed death to one primary probable, or possible, cause. Whilst
developing the study protocol, we decided to limit the number of causes of death for
simplicity. However, reviewers found it challenging to identify only one cause of death, and
distinguish between probable and possible causes. In the original study reviewers attributed
death to multiple probable or possible causes. Despite this difference, reviewers only agreed
on the causes of death on 3 of 9 cases classified as TRM. This is similar to the results in this
study of which reviewers agreed on 3 of 10 cases.
Page 14 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySince the development of this study, a standard operating procedure TRM web-based tool has
been published (https://www.sungresearch.com/trm-training-manual/) and includes working
examples. Use of this tool when delivering the training package should help clarify how to
use the cause-of-death attribution system and minimise misunderstanding. Currently, the web-
based tool is available in English, having the tool available in other languages could
potentially reduce confusion and improve harmonisation across clinical trials.
Our study highlights specific challenges with the system as it currently exists, both with the
classification of TRM and the attribution of a specific cause of death. Fundamentally, this
approach defines deaths as either ‘treatment related’ or ‘cancer related’. This gives rise to a
semantic challenge; “treatment-related mortality” implies that deaths that come under this
term occur directly because of the therapies delivered. However, the classification system
classifies deaths which occur prior to commencement of anti-cancer therapy, which are not
directly attributable to the cancer (for example, tumour lysis syndrome in high-count
leukaemia) as cases of TRM. This clash of language and “common sense” may confuse users
of the classification tool, for example, case M in which a patient presented acutely with signs
of raised intra-cranial pressure and died on the operating table.
A deeper challenge to this system addressed the philosophical distinction of assigning all
deaths into one of two categories; cancer or treatment related. There is a convincing argument
that a third category of death should be attributable, “other non-cancer death”, for those who
die of an event or illness external to their malignancy. This problem is particularly evident if
the current system is to be used after the completion of treatment. For example, a patient dies
as a passenger in an air traffic accident, 4 years after treatment for a localised Wilms tumour.
In the current system, this death is classified as TRM, even though the death is unrelated to
the child’s cancer diagnosis.
Page 15 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConversely, it is also, important to note how a diagnosis of cancer may be included in ‘non-
cancer deaths’. For example a patient may commit suicide some years after the completion of
treatment because of the psychological effects of their diagnosis or treatment. Any addition to
the system would need to be sensitive to these potential issues.
We have identified the need to further refine the approach to categorising cause of death in
cancer patients receiving care without intent to cure. This is particularly important if the
system is applied in ‘routine’ settings, assessing deaths in the palliative setting rather than in
the original setting of use within a curative trial. Increasingly individuals destined not to be
cured are living for lengthier periods due to participation in clinical trials/studies. This group
of patients currently have all deaths classified as “not treatment-related mortality” as
clinicians would have specified progressive disease, or that cancer therapy has no curative
intent. This algorithm may fail to identify a significant group of patients who may die of
causes amenable to better supportive care whilst receiving palliative care. For example, a
patient can die of overwhelming sepsis whilst receiving palliative etoposide for refractory
neuroblastoma. This could be addressed by modifying the algorithm, for this type of use.
Another proposal includes using a separate classification tool for patients on non-curative
therapy trials (fig. 2) although this should be further developed in conjunction with palliative
care physicians and researchers. The counter-argument to this suggested change is the risk of
adding complexity to a simple, effective, tool which can be used by people of different skills
and health care settings globally. It will also have similar issues with interpretation as with
initial algorithm.
Reviewers failed to agree on a primary cause of death in 6 episodes and probable and possible
causes in 4 cases. Differences in cause of death allocated could be attributed to the reviewers’
previous experience, clinical expertise and interpretation of the clinical records, particularly in
light of potential previous direct clinical involvement with the cases under review. Currently,
Page 16 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlythe tool is intended for use by any CRA, however, it may require users to have a certain level
of experience or clinical expertise, and agreement may be reduced with CRA who are new to
the role.
Understanding and interpretation of the system as proposed for attribution of a specific
mechanistic cause of death could potentially be improved by dedicating more time during the
presentations and using the newly developed web-based training tool. Alternatively, the cause
of death attribution system could be refined, and multiple causalities permitted.
Page 17 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyFigure 2: Proposed classification of TRM in children receiving non-curative therapy
(NCT) only.
Page 18 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConclusions
We have been able to confirm the reproducibility and criterion validity of the TRM
classification system. We believe this supports the hypothesis that the classification system
can be implemented easily and effectively in different health care settings, thereby improving
consistency and accuracy of outcome reporting in clinical trials. The TRM classification
system will be of immense value in the evaluation of deaths in the palliative setting. We
propose the addition of a separate classification tool in patients receiving palliative treatment.
Page 19 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyConflict of interest statement
The authors declare no conflict of interest.
Acknowledgments
The authors would like to thank the reviewers for participating in this study. This work was
supported by Candlelighters charity who funded Dr Hadeel Hassan’s PhD, for which this
research was conducted. H.H wrote the manuscript and R.S.P edited the manuscript. M.R,
S.E.K and A.G contributed to the review process.
Page 20 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyReferences
1. CRUK. Children's cancers mortality in Europe and worldwide. 2014 [cited 2016
September]; Available from: http://www.cancerresearchuk.org/health-
professional/cancer-statistics/childrens-cancers/mortality#heading-Three.
2. Kremer, L.C., et al., A worldwide collaboration to harmonize guidelines for the long-
term follow-up of childhood and young adult cancer survivors: a report from the
International Late Effects of Childhood Cancer Guideline Harmonization Group.
Pediatr Blood Cancer, 2013. 60(4): p. 543-9.
3. Blanco, E., et al., Non-relapse mortality in pediatric acute lymphoblastic leukemia: a
systematic review and meta-analysis. Leuk Lymphoma, 2012. 53(5): p. 878-85.
4. White, Y., V.P. Castle, and A. Haig, Pediatric oncology in developing countries:
challenges and solutions. J Pediatr, 2013. 162(6): p. 1090-1, 1091.e1.
5. Ethier, M.C., et al., Lack of clarity in the definition of treatment-related mortality:
pediatric acute leukemia and adult acute promyelocytic leukemia as examples. Blood,
2011. 118(19): p. 5080-3.
6. Tran, T.H., et al., Lack of treatment-related mortality definitions in clinical trials of
children, adolescents and young adults with lymphomas, solid tumors and brain
tumors: a systematic review. BMC Cancer, 2014. 14: p. 612.
7. Alexander, S., et al., Classification of treatment-related mortality in children with
cancer: a systematic assessment. Lancet Oncol, 2015. 16(16): p. e604-10.
8. McHugh, M.L., Interrater reliability: the kappa statistic. Biochemia Medica, 2012.
22(3): p. 276-282.
9. Team, R., RStudio: Integrated Development for R. RStudio, Inc. 2015.
Figure legends
Figure 1: Classification of TRM in children with cancer, taken from (7).
Figure 2: Proposed classification of TRM in children receiving non-curative therapy (NCT)
only.
Supplementary files legend list
Supplemental table S1: Cause-of-death attribution system.
Supplemental table S2: Diagnosis of included patients results of classification by reviewers.
Page 21 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
81x45mm (300 x 300 DPI)
Page 22 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
45x25mm (300 x 300 DPI)
Page 23 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySupplemental file S1
Table 1: Cause-of-death attribution system.
Probable cause of death Possible cause of death
Infection A.1 Clinically or
radiographically
documented infection with
associated microbiologically
documented organism
1. Clinically or
radiographically
documented infection
without associated
microbiologically
documented organism.
Haemorrhage B.1 Acute symptomatic
intracranial haemorrhage
shown by imaging or
pathology.
B.2 Acute symptomatic
pulmonary haemorrhage
shown by imaging or
pathology.
B.3 Acute symptomatic
bleeding resulting in
hypotension, urgent
transfusion or fluid bolus.
2. Acute symptomatic
pulmonary haemorrhage not
shown by imaging or
pathology
Thrombosis C.1 Acute symptomatic
intracranial thrombosis or
embolism shown by
imaging or pathology.
C.2 Acute symptomatic
pulmonary thrombosis or
embolism shown by
imaging or pathology.
C.3 Acute symptomatic
hepatic thrombosis or
embolism shown by
imaging or pathology.
3. Acute symptomatic
pulmonary thrombosis
or embolism not shown
by imaging or
pathology.
Cardiac D.1 Acute symptomatic
arrhythmia excluding
sinus tachycardia or
bradycardia shown by
ECG.
D.2 Acute symptomatic
cardiac- dysfunction defined
by ECG, cardiac imaging or
pathology.
4. Acute symptomatic
arrhythmia excluding sinus
tachycardia or bradycardia
not shown by ECG.
Immunomediated
E.1 Acute allergic
reaction, anaphylaxis with
symptomatic
bronchospasm, oedema,
angioedema, or
hypotension.
5. Stable graft-versus-host
disease.
Page 24 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyE.2 Worsening
symptomatic graft
versus host disease.
E.3 Acute symptomatic
haemophagocytic
lymphohistocytosis,
macrophage activation
syndrome or cytokine-
release syndrome.
Metabolic F. Clinically diagnosed
tumour lysis syndrome
with cardiac arrhythmia,
seizure, or creatinine
concentrations greater
than 3 times the ULN.
CNS G.1 Acute symptomatic
CNS necrosis shown by
imaging or pathology
G.2 Acute symptomatic
encephalopathy shown by
imaging or
electroencephalography
G.2 Acute symptomatic
hydrocephalus or raised
intracranial pressure shown
by imaging, pathology, or
measurement of intracranial
pressure
G.4 Seizure lasting at least
30 minutes within 48 hours
of death
6.1 Acute symptomatic CNS
necrosis not shown by
imaging or pathology
6.2 Acute symptomatic
encephalopathy not shown
by imaging or
electroencephalography
6.2 Acute symptomatic
hydrocephalus or raised
intracranial pressure not
shown by imaging,
pathology, or measurement
of intracranial pressure
6.4 Seizure between 5 and
30 minutes within 48 hours
of death
Respiratory H. Acute symptomatic
respiratory distress with
ventilator support
7. Acute symptomatic
respiratory distress without
ventilator support
Gastrointestinal system I.1 Acute symptomatic
bowel disease resulting in
necrosis, obstruction or
perforation shown by
imaging or pathology
I.2 Acute, clinically
diagnosed hepatic
dysfunction associated with
conjugated bilirubin
concentrations greater than
10 x ULN, ammonium
concentrations greater than
2.5 x ULN, or international
normalised greater than 2.5
times the ULN
8.1 Acute symptomatic
bowel disease resulting in
necrosis, obstruction or
perforation not shown by
imaging or pathology
8.2 Acute, clinically
diagnosed hepatic
dysfunction associated with
conjugated bilirubin
concentrations greater than
1.5 and less than 10 x ULN,
ammonium concentrations
greater than 1.5 and less
than 2.5 x ULN, or
international normalised
Page 25 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyI.3 Acute, clinically
diagnosed pancreatitis with
haemorrhage, peritonitis,
necrosis or haemodynamic
instability (evidenced by
hypotension, urgent
transfusion, fluid bolus, or
vasopressers
greater than 1.5 but less than
2.5 times the ULN
Renal system J. Acute kidney injury with
dialysis or renal replacement
therapy (planned or
received)
External causes K.1 Unintentional injury
(e.g accident)
K.2 Suicide
K.3 Homicide
ULN: Upper Limit of Normal ECG: electrocardiogram CNS: central nervous system
Page 26 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nly
Page 27 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlySupplemental file S2
Case Diagnosis CRA 1 CRA 2 Consultant
1
Consultant
2
A Germ cell CNS tumour Not TRM Not TRM Not TRM Not TRM
B Diffuse intrinsic
pontine glioma (DIPG)
Not TRM Not TRM Not TRM Not TRM
C Thalamic astrocytoma Not TRM Not TRM Not TRM Not TRM
D Relapsed B-cell ALL Not TRM Not TRM Not TRM Not TRM
E Spinal cord
glioblastoma
multiforme
Not TRM Not TRM Not TRM Not TRM
F Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
G Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
H Metastatic
rhabdomyosarcoma
Not TRM Not TRM Not TRM Not TRM
I Philadelphia positive
ALL
Not TRM Not TRM Not TRM Not TRM
J AML prior to HSCT TRM TRM TRM TRM
K Metastatic
rhabdomyosarcoma
Not TRM Not TRM Not TRM Not TRM
L Soft tissue
myoepithelial
carcinoma
Not TRM Not TRM Not TRM Not TRM
M ATRT TRM TRM Not TRM TRM
N Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
O High risk
neuroblastoma
Not TRM Not TRM Not TRM Not TRM
P Osteosarcoma Not TRM Not TRM Not TRM Not TRM
Q DIPG Not TRM Not TRM Not TRM Not TRM
R Osteosarcoma Not TRM Not TRM Not TRM Not TRM
Page 28 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from
Confidential: For Review O
nlyCase Diagnosis CRA 1 CRA 2 Consultant
1
Consultant
2
S High risk
neuroblastoma
Not TRM Not TRM Not TRM Not TRM
T Malignant melanoma
of the CNS
Not TRM Not TRM Not TRM Not TRM
U Metastatic Ewing’s
sarcoma
Not TRM Not TRM Not TRM Not TRM
V AML TRM TRM TRM TRM
W Epithelial sarcoma Not TRM Not TRM Not TRM Not TRM
X ALL TRM TRM TRM TRM
Y T-cell ALL TRM TRM TRM TRM
Z B-cell ALL TRM TRM TRM TRM
AA B-cell ALL TRM TRM TRM TRM
AB Ependymoma TRM TRM Not TRM TRM
AC T-cell ALL TRM TRM TRM TRM
AD B-cell ALL post HSCT TRM TRM TRM TRM
Page 29 of 29
https://mc.manuscriptcentral.com/bmjpo
BMJ Paediatrics Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on 11 May 2018 by guest. P
rotected by copyright.http://bm
jpaedsopen.bmj.com
/bm
jpo: first published as 10.1136/bmjpo-2017-000082 on 30 O
ctober 2017. Dow
nloaded from