· Confidential: For Review Only A systematic review and critical appraisal of prognostic models...

Confidential: For Review OnlyA systematic review and critical appraisal of prognostic models for outcome prediction in patients with chronic

obstructive pulmonary disease

Journal: BMJ

Manuscript ID BMJ.2018.046909.R2

Article Type: Research

BMJ Journal: BMJ

Date Submitted by the Author: 05-Jul-2019

Complete List of Authors: Bellou, Vanesa; University of Ioannina Medical School, Department of Hygiene and Epidemiology; University of Ioannina Medical School, Department of Respiratory MedicineBelbasis, Lazaros; University of Ioannina Medical School, Department of Hygiene and EpidemiologyKonstantinidis, Athanasios; University of Ioannina Medical School, Department of Respiratory MedicineTzoulaki, Ioanna; Imperial College London, Department of Epidemiology and BiostatisticsEvangelou, Evangelos; University of Ioannina Medical School, Department of Hygiene and Epidemiology; Imperial College London, Department of Epidemiology and Biostatistics

Keywords: Systematic review, Prediction model, Chronic obstructive pulmonary disease, Prognosis, Critical appraisal, Risk of bias

https://mc.manuscriptcentral.com/bmj

BMJ

Confidential: For Review Only

1

A systematic review and critical appraisal of prognostic models for

outcome prediction in patients with chronic obstructive pulmonary disease

Vanesa Bellou1,2, Lazaros Belbasis1, Athanasios K Konstantinidis2, Ioanna Tzoulaki1,3, Evangelos

Evangelou1,3

1Department of Hygiene and Epidemiology, University of Ioannina Medical School, Ioannina, Greece

2Department of Respiratory Medicine, University Hospital of Ioannina, University of Ioannina Medical School,

Ioannina, Greece

3Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK

Corresponding author:

Evangelos Evangelou,

Department of Hygiene and Epidemiology,

University of Ioannina Medical School,

Ioannina, Greece

e-mail: [email protected]

Acknowledgements:

Vanesa Bellou is supported by a PhD scholarship funded by Greek State Scholarship Foundation.

Lazaros Belbasis is supported by a PhD scholarship funded by Greek State Scholarship Foundation.

We would like to thank Prof Karel Moons for his constructive comments on the study design and the final

draft of the manuscript.

Competing risk:

None.

Authorship:

VB, LB, IT and EE designed the study. VB and LB performed the literature search, the data extraction,

and wrote the first draft of the manuscript. All the authors wrote the final version of the manuscript. EE

Page 1 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


2

accepts full responsibility for the work and conduct of the study, had access to the data, and controlled the

decision to publish.

Page 2 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


3

Abstract

Objective: To map and assess prognostic models for outcome prediction in patients with chronic

obstructive pulmonary disease (COPD).

Design: Systematic review.

Data sources: PubMed until November 2018 and hand-searched references from eligible articles.

Eligibility criteria for study selection: Studies developing, validating or updating a prediction model in

COPD patients focusing on any potential clinical outcome.

Results: Our systematic search yielded 228 eligible articles, describing the development of 408 prognostic

models, the external validation of 39 models, and the validation of 20 prediction models derived from

other diseases. The 408 prognostic models were developed in three clinical settings: outpatients (n = 239,

59%), hospitalized patients (n = 155, 38%), and patients attending the emergency department (n = 14,

3%). Among the 408 prognostic models, the most prevalent endpoints were mortality (n = 209, 51%), risk

for acute exacerbation of COPD (n = 42, 10%), and risk for readmission after the index hospitalisation (n

= 36, 9%). Overall, the most commonly used predictors were age (n = 166, 40%), forced expiratory volume

in one second (n = 85, 21%), sex (n = 74, 18%), body mass index (n = 66, 16%), and smoking (n = 65,

16%). Out of the 408 prognostic models, 100 (25%) were internally validated, and 93 (23%) examined

the calibration of the developed model. For 285 models (70%) a model presentation was not available,

and only 56 models (14%) were presented through the full equation. Model discrimination was available

for 311 models (76%) using the c-statistic. Thirty-eight (9%) were externally validated, but in only 12 of

these the external validation was performed by a fully independent team. We identified only seven

prediction models with an overall low risk of bias using PROBAST. These models were ADO, B-AE-D,

B-AE-D-C, extended ADO, updated ADO, updated BODE, and a model developed by Bertens et al. A

meta-analysis of c-statistic was performed for 12 prognostic models and the summary estimates ranged

from 0.611 to 0.769.

Conclusions: In the present study, a detailed mapping and assessment of the prognostic models for

outcome prediction in COPD patients was performed. Overall, we identified several methodological

pitfalls in their development, and a low rate of external validation. Future research should focus on the

improvement of existing models through update and external validation, as well as the assessment of

Page 3 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


4

safety, clinical effectiveness and cost-effectiveness of the application of these prognostic models in

clinical practice through impact studies.

Page 4 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


5

Summary box

What is already known on this topic?

Historically, spirometry and age have been identified as the most important prognostic indicators

in COPD.

GOLD guidelines recommended the use of multivariable prediction models to assess the prognosis

of patients, instead of single predictors such as spirometry or history of exacerbations alone.

No systematic overview has been published to summarize and critically appraise all multivariable

prognostic models for outcome prediction in COPD patients.

What this study adds?

There are more than 400 prognostic models for outcome prediction in COPD patients in an

ambulatory, hospital or emergency department setting, with the majority focusing on mortality.

Only a minority of the developed models have been externally validated and the majority were

characterized by major issues in the statistical analysis and, particularly, in internal validation,

model calibration and presentation, and handling of missing data.

After applying PROBAST, ADO, B-AE-D, B-AE-D-C, extended ADO, updated ADO, updated

BODE, and a model developed by Bertens et al were developed in studies assessed as low risk of

bias.

We performed meta-analysis for 12 prognostic models which have been validated in at least three

non-overlapping populations.

Page 5 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


6

Introduction

Chronic obstructive pulmonary disease (COPD) is a major public health issue. COPD accounts for at

least 2.9 million deaths annually [1] and is a leading cause of morbidity and mortality with its prevalence

projected to increase over the following years. Morbidity associated with the disease entails physician

visits, emergency department visits and hospitalizations [2], all of which lead to a substantial economic

burden. The greatest proportion of the costs is attributed to COPD exacerbations [2].

COPD is a quite heterogeneous disease and stratifying cases according to prognosis would raise the

possibility of a precision medicine approach. For many years now, forced expiratory volume in one second

(FEV1) and age have been considered to be the most important prognostic indicators in COPD [3]. More

recently, a wide variety of individual clinical factors have been also linked to COPD prognosis.

Prognostic models, in general, have two distinct uses; they classify patients in groups with different

prognosis, as well as estimate prognosis for individual patients. While these are two different ways of

looking at the same information, they differ fundamentally and the ultimate goal is to guide therapeutic

and further diagnostic choices [4]. Use of a composite index to assess prognosis in COPD patients may

provide a more comprehensive way of evaluation, incorporating a cluster of systemic manifestations of

the disease [5]. Furthermore, in patients with COPD, multivariable prognostic models for various clinical

outcomes could be used in clinical practice to assist decision making about hospitalization or admission

to intensive care units and treatment strategy [6].

Numerous prognostic models, combining multiple predictors for COPD-related outcomes, have been

developed. Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines recommend the

use of multivariable prediction models to assess the prognostic profile and facilitate follow-up of patients,

instead of single predictors such as spirometry or history of exacerbations alone. Also, in the latest GOLD

statement, BODE index is proposed as a tool to determine who needs referral for lung transplantation

consideration [7].

In this study, we aim to systematically summarize the reported multivariable prognostic models

developed for predicting subsequent outcomes in patients diagnosed with COPD, to map their

characteristics, and to examine whether they have undergone external validation. We will apply risk of

bias assessment of the methodological features of the available studies developing or validating prognostic

models using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). For prognostic models

Page 6 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


7

that have multiple validation studies, we will perform a meta-analysis for model performance and model

calibration in order to obtain more accurate estimates.

Methods

We designed this systematic review following the Checklist for critical Appraisal and data extraction

for systematic Reviews of prediction Modelling Studies (CHARMS) and the recent guidance by Debray

et al [8,9]. A protocol of this article was published on PROSPERO (registration number

CRD42017069247).

Literature search

We systematically searched PubMed from inception to November 11, 2018 to capture all studies

developing and/or validating a prognostic model for outcomes in COPD patients. Based on previous

research [10,11], we created the following search algorithm: (predict* OR progn* OR "risk prediction"

OR "risk score" OR "risk calculation" OR "risk assessment" OR "c statistic" OR discrimination OR

calibration OR AUC OR "area under the curve" OR "area under the receiver operator characteristic curve")

AND ("chronic obstructive pulmonary disease" OR emphysema OR "chronic bronchitis" OR COPD). The

literature search was independently conducted by two researchers (VB, LB) and discrepancies were

resolved by a third researcher (IT). We further hand-searched the references of each eligible article for

potential additional eligible studies.

Eligibility criteria

We included all studies that reported the development or validation of at least one multivariable model

for predicting the risk for any clinical outcome in COPD patients. A detailed description of the PICOTS

[8,9] for this review is presented in Table 1. To consider a study as eligible, we followed the definition of

prognostic model studies as proposed by the Transparent Reporting of a multivariable prediction model

for Individual Prognosis Or Diagnosis (TRIPOD) statement. [12] Accordingly, it should specifically

report the development, the update or the external validation of a prognostic model used in COPD patients

for making individualized predictions, either in its objectives or conclusions. A study was also eligible if

the development or update of a prognostic model could be deduced by the available information through

the full-text (e.g., model presentation, measures of predictive performance for a multivariable model).

Page 7 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


8

Eligible outcomes included any possible clinical endpoint of COPD patients, such as mortality,

exacerbations and hospitalizations.

The eligible studies could report the development of multivariable models, the external validation of

an existing model, and/or the update of an existing model. Model updating may range from simple

adjustment of the baseline risk/hazard, to additional adjustment of predictors’ weights using the same or

different adjustment factors, to re-estimating predictor weights and adding new predictors or removing

existing predictors from the original model [13]. External validation studies aim to assess the predictive

performance of an existing model in an independent population [13]. We included external validation

studies that explicitly estimated and presented a measure of model performance. We also considered

studies validating prediction models originally developed for other diseases, in COPD population. Also,

included articles should report original research, study humans and be written in English.

We excluded studies developing or validating diagnostic models to detect or exclude COPD presence

in suspected patients, studies examining only independent prognostic factors, methodological studies, and

COPD case finding or COPD screening studies. Also, we excluded studies that developed search

algorithms to identify existing COPD cases based on administrative data. Given that prognostic models

estimate a probability of a certain outcome for an individual over a specified time horizon, we excluded

cross-sectional studies, because in this study design predictors and the outcome are measured

concurrently. However, cohort studies performing an external validation of a model derived from a cross-

sectional study were deemed eligible.

Data extraction

To facilitate the data extraction process, a standardized form was constructed by three researchers

(VB, LB, IT) following recommendations by CHARMS checklist [8]. Data extraction was independently

performed by two researchers (VB, LB). From all eligible articles, we extracted information on first

author, year and journal of publication, and model name. From articles describing model development,

we extracted the following information: study design, study population, geographical location, predicted

outcome, definition of outcome, prediction horizon, definition of COPD, modelling method, method of

internal validation, number of participants and number of events, number and type of predictors in final

model, model presentation, and measures of predictive performance (discrimination, calibration,

classification, overall performance). Potential measures of discrimination were c-statistic, D-statistic;

potential measures of classification were sensitivity, specificity, positive and negative predictive value

Page 8 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


9

and predictive accuracy; potential measures for overall performance were R2 and Brier score, and potential

measures for assessment of calibration were calibration plot, calibration-in-the-large, calibration slope,

Hosmer-Lemeshow test, Harrel’s E statistic, and calibration test [14,15]. Furthermore, we evaluated

whether the authors only reported the apparent performance of a prediction model or if they examined

overfitting using internal validation. Additionally, we examined whether a shrinkage of regression

coefficients toward zero was performed in eligible studies and which methodology was applied. We

considered that the authors adjusted for optimism sufficiently if they re-evaluated the performance of a

model in internal validation and performed shrinkage of model coefficients as well. We extracted whether

the authors performed decision curve analysis and net benefit analysis to evaluate the clinical usefulness

of a model [15,16]. Moreover, for each eligible study, we examined whether the authors reported the

presence of missing data on examined outcomes and/or variables included in prediction models; and, if

so, we recorded how missing data were treated. Furthermore, we extracted information on how continuous

variables were handled, and whether non-linear trends for continuous predictors were assessed by

applying polynomials, fractional polynomials or cubic splines. If the handling of continuous predictors

was not described explicitly, we scrutinized the full-text and the tables of the respective papers to derive

this information from the reported effect sizes. In the case that this process was inconclusive, we described

the handling of continuous predictors as unclear.

In articles examining the performance of the same prediction model on various outcomes or multiple

timepoints, we retained the prediction model referring to the outcome or timepoint mentioned as primary

analysis of the study. In cases where a primary timepoint was not specified, we considered the prediction

with the longest horizon as the primary analysis of the study, because longer follow-up duration would

lead to a larger number of events. Whenever a study described model performance both in an overall

sample and in specific subgroups of population, we extracted the analysis on the total population.

From articles describing model external validation, we extracted study population, geographic

location, number of participants and events, the model’s performance and calibration. If an article

described multiple models, we performed separate data extraction for each model. For each model

externally validated in multiple articles, we included in our analysis only external validation studies with

non-overlapping populations. Furthermore, we examined whether the research team performing the

external validation was independent to the research team developing the prediction model.

Risk of bias assessment

Page 9 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


10

We appraised the presence of bias in the studies developing or externally validating prognostic

models using PROBAST, which is a risk of bias assessment tool designed for systematic reviews of

diagnostic or prognostic prediction models [17,18]. It contains a multitude of questions in 4 different

domains: participants, predictors, outcome and statistical analysis. Questions are answered with yes,

probably yes, probably no, no and no information depending on the characteristics of the study. If a domain

contains at least one question signalled as no or probably no it is considered high risk. To be considered

low risk a domain should contain all questions answered with yes or probably yes. Overall risk of bias is

graded as low risk when all domains were considered low risk, while overall risk of bias is considered

high risk when at least one of the domains is considered high risk. Risk of bias assessment was

independently performed by two researchers (VB, LB).

PROBAST describes the assessment of both development studies and external validation studies.

Often, articles describe the development of multiple prediction models using different populations or

different statistical approaches. Hence, differences in the risk of bias assessment is expected among

different prediction models developed in the same article. For this reason, we chose to report the risk of

bias assessment per developed prediction model and not per article. Furthermore, articles may describe

the external validation of multiple prediction models in the same population or in multiple different

populations. For this reason, we refer to external validation efforts and we report the risk of bias

assessment per external validation effort.

Statistical analysis

We calculated and reported descriptive statistics to summarize the characteristics of the models; for

continuous variables, we calculated the median and interquartile range and, for binary variables, we

calculated the respective percentages.

For the prediction models that were examined in more than 2 independent datasets (excluding the

model development dataset), we performed a random-effects meta-analysis to calculate a summary

estimate for model performance and model calibration. We also considered for the meta-analysis, those

prediction models that were internally validated through bootstrapping or cross-validation and were

externally validated in only two independent datasets. We followed a recently published framework for

the meta-analysis of prediction models [9,19]. In the case that a measure of uncertainty (i.e., standard error

or 95% confidence interval) was not available for mean c-statistic, we used a formula to approximate the

standard error of mean c-statistic based on number of events and number of participants [9,19,20]. The

Page 10 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


11

statistical analysis was performed on R version 3.5.0. For the meta-analysis of prediction models, we used

the R package ‘metamisc’ [19].

Results

Out of the 17,538 screened papers, 228 papers were eligible (Figure 1). These articles described the

development of 408 prognostic models in COPD patients, the external validation of 39 prognostic models,

and the application of 20 prognostic models originally developed for other health outcomes than prognosis

of COPD patients. One of the eligible papers was identified through the hand-search of references from

eligible articles [21]. The prognostic models were mainly developed in United States of America (n = 91,

22%), Spain (n = 57, 14%), and United Kingdom (n = 34, 8%), whereas 80 models were developed in

multicentre studies from multiple countries (20%). For the derivation cohorts, the median sample size was

409 (IQR, 163 to 1033) and the median number of events was 63 (IQR, 36 to 188). For the internal

validation cohorts, the median sample size was 831 (IQR, 225 to 4192) and the median number of events

was 77 (IQR, 40 to 370).

The eligible prognostic models were developed in a variety of clinical settings. 239 (59%) prognostic

models were developed in an outpatient setting and 155 (38%) were developed on a sample of hospitalized

patients. Also, 14 (3%) prognostic models were developed for COPD patients attending the emergency

department. The developed models focused on a wide range of clinical outcomes. The most commonly

used endpoints were mortality (n = 209, 51%), exacerbation (n = 42, 10%) and re-admission (n = 36, 9%).

Supplementary Table 1 presents a summary of the predicted outcomes per clinical setting. Also, 24

prognostic scores focused on a composite outcome. The most commonly used predictors were age (n =

166, 40%), forced expiratory volume in one second (n = 85, 21%), sex (n = 74, 18%), body mass index (n

= 66, 16%), smoking (n = 65, 16%), previous exacerbations (n = 53, 13%), previous hospitalisations (n =

50, 12%), BODE index (n = 43, 10%), mMRC dyspnea scale (n = 42, 10%), and Charlson comorbidity

index (n = 35, 9%). Figure 2 presents the predictors which were used in at least 20 models, and Figure 3

presents the 10 most common predictors stratified by clinical setting.

Below, we describe the methodological and clinical characteristics for a total of 408 prognostic

models, based on clinical setting.

Prognostic models for outpatients

Page 11 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


12

The majority of the prognostic models (n = 239, 59%) were developed on a sample of COPD patients

examined in an outpatient facility (Supplementary Table 3). For the derivation cohort, the median sample

size was 431 (IQR, 244 to 1000) and the median number of events was 63 (IQR, 33 to 155). For the

internal validation cohort, the median sample size was 249 (IQR, 204 to 3468) and the median number of

events was 150 (IQR, 64 to 1642). The most common clinical endpoints examined by these models were:

mortality (n = 124, 52%), exacerbations (n = 40, 17%), spirometric indices (n = 25, 10%), hospitalization

(n = 16, 7%), treatment failure during an acute exacerbation (n = 8, 4%), composite outcome (n = 9, 4%).

The most commonly used predictors in these models were: age (n = 105, 44%), FEV1 (n = 69, 29%),

smoking (n = 54, 22%), BMI (n = 51, 22%), sex (n = 43, 18%), previous exacerbations (n = 43, 18%),

BODE index (n = 43, 18%), previous hospitalisations (n = 28, 12%), and diabetes mellitus (n = 24, 10%).

198 (83%) of these models reported a c-statistic, while the remaining 41 (17%) did not report a

discrimination metric. 172 of the ambulatory patient models only assessed apparent performance in the

development study. One prognostic model performed temporal validation, while the remaining performed

cross-validation (n = 28, 12%), bootstrapping (n = 25, 10%), random split (n = 12, 5%) or a combination

of methods (n = 1). 192 (80%) prognostic models were not calibrated. 48 (20%) prognostic models

assessed calibration and the most frequent method was Hosmer-Lemeshow test (n = 35, 15%). Various

modelling methods were applied, with the most frequent being Cox regression (n = 90, 38%), logistic

regression (n = 79, 34%), negative binomial regression (n = 21, 9%), and linear regression (n = 16, 7%).

In 12 prognostic models, shrinkage of regression coefficients was performed to reduce overfitting.

Application of a uniform shrinkage factor to all the regression coefficients was used in 9 models,

application of a penalized maximum likelihood method to estimate the regression coefficients was

described for 1 prognostic model and lasso regression was applied in 2 prognostic models to perform

shrinkage for selection of predictors. For 17 prognostic models a non-linear association between

continuous predictors and predicted outcome was examined using the following methods: polynomials (n

= 7), restricted cubic splines (n = 6), fractional polynomials (n = 2), and Box-Tidwell transformation (n =

2). A significant amount (n = 177, 73%) did not have any type of model presentation, while only 24 (10%)

reported the full regression equation. The most common type of presentation was sum score (n = 31, 13%).

Only 1 study performed decision analysis [22]. In this study, net benefit and decision curves are available

for updated ADO index. Net benefit is a category of decision analysis, comparing benefits and harms

directly after transforming them on the same scale. A detailed description of all the methodological

characteristics is presented in Table 2.

Page 12 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


13

Prognostic models for hospitalized patients

155 models were developed in patients hospitalized in medical wards, intensive care units or

rehabilitation centers (Supplementary Table 4). The median sample size of the derivation cohort was 303

(IQR, 102 to 920) and the median number of events was 67 (IQR, 37 to 311). The median sample size of

the internal validation cohort was 4131 (IQR, 731 to 4840) and the median number of events was 333

(IQR, 35 to 370). The most prevalent outcomes assessed were mortality (n = 78, 50%), re-admission after

an index hospitalisation (n = 36, 23%), failure of non-invasive ventilation (n = 14, 9%) and composite

outcomes (n = 13, 8%). The following are the predictors encountered in the majority of the prognostic

models: age (n = 56, 36%), sex (n = 30, 19%), PaCO2 (n = 24, 15%), previous hospitalisations (n = 20,

13%), length of hospital stay (n = 20, 13%), Charlson comorbidity index (n = 19, 12%), pH (n = 18, 12%),

heart failure (n = 16, 10%), BMI (n = 15, 10%), and serum albumin (n = 15, 10%).

31 of 155 prognostic models (20%) were developed for patients hospitalised in intensive care units

to predict mortality (n = 22), weaning success (n = 2), need for mechanical ventilation (n = 6), and duration

of mechanical ventilation (n = 1). The most commonly used predictors were the following: age (n = 12),

Glasgow or Japan Coma Scale (n = 9), APACHE II (n = 8), sex (n = 6), pH (n = 6), hemoglobin (n = 6),

serum albumin (n = 6), heart failure (n = 6), and hypertension (n = 6).

Only 102 prognostic models (66%) reported a c-statistic, while 53 models (34%) did not assess

discrimination. 131 prognostic models (85%) did not perform internal validation and in the few models

that this was performed, bootstrapping (n = 9, 6%), random split (n = 7, 5%), cross-validation (n = 3, 2%)

or a combination of aforementioned methods (n = 2, 1%) were used. Also 3 prognostic models (2%)

performed temporal validation. 115 prognostic tools (74%) did not assess their calibration. Hosmer-

Lemeshow test (n = 34, 22%) was the most frequently used method of calibration. Most of the prognostic

models did not have a model presentation (n = 104, 67%). The most common presentation was sum score

(n = 16, 10%), while a regression formula was available for 27 prognostic models (17%). The most

frequently used regression methods were logistic regression (n = 112, 73%) and Cox regression (n = 21,

14%). For 4 prognostic models, shrinkage was applied to reduce overfitting. Application of a uniform

shrinkage factor to all the regression coefficients was performed for 2 models, the penalized maximum

likelihood approach was used in 1 model, whereas, for 1 model, lasso regression was applied. For 3

prognostic models, the non-linear association of predictors with the predicted outcome was considered

Page 13 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


14

using polynomials (n = 1), fractional polynomials (n = 1) and Box-Tidwell transformation (n = 1). One

study after developing a prognostic model performed a decision analysis [23].

Prognostic models for patients presenting at the emergency department

Only 14 prognostic models were developed for patients that attend the emergency department

(Supplementary Table 5) with a median sample size of 1195 (IQR, 871 to 1250) and a median number of

events of 77 (IQR, 40 to 137) in the derivation cohort. The median sample size of internal validation

cohort was 1235 (IQR, 266 to 1244) and the median number of events was 52 (IQR, 29 to 66). The

outcomes that were examined were mortality (n = 7, 50%), change in physical activity (n = 2), composite

outcome (n = 2), hospitalization (n = 1), ICU admission (n = 1) and treatment failure after a visit in

emergency department for an acute exacerbation (n = 1). 5 of these models examined a long term

prediction horizon (more than 1 month). The most prevalent variables included in these models were:

long-term oxygen therapy or non-invasive ventilation at home (n = 8, 57%), age (n = 5, 36%), mMRC

dyspnea scale (n = 5, 36%), Charlson comorbidity index (n = 4, 29%), PaCO2 (n = 3, 21%), use of

inspiratory accessory muscles and paradoxical breathing (n = 3, 21%), Glasgow or Japan coma scale (n =

3, 21%).

3 of these models did not report an assessment of their discrimination, while 11 reported a c-statistic.

5 models did not perform any internal validation, while 8 models used a random split of the dataset. A

single model used bootstrapping for internal validation. The modeling methods employed by these models

were logistic regression (n = 10), linear regression (n = 2) and machine learning (n = 2). A shrinkage

procedure was not applied for any model.

External validation studies

38 of 408 prognostic models (9%) were externally validated at least once. However, only twelve of

these models (3%) were externally validated by a fully independent research team. The prognostic models

that were externally validated more than 5 times were the following: ADO (17 cohorts), BODE (13

cohorts), BODEx (8 cohorts) and CODEX (7 cohorts).

Four prognostic models (i.e., DOSE index, SAFE index, mBODE% index and COPD Severity Score)

were developed in cross-sectional studies, and these models were not described in the aforementioned

section of models. We retained only their external validation in cohort studies, which were 12 for DOSE

Page 14 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


15

index, 1 for COPD Severity Score, SAFE and mBODE% index. Table 3 presents all the external validation

studies of the prognostic models for outcome prediction in COPD patients.

Risk of bias assessment

We assessed the risk of bias of all studies developing or externally validating a prognostic model

using PROBAST. In Figure 4, we present a summary of risk of bias assessment of developed models by

domain. Seven prognostic models were assessed as low risk of bias and all these models were developed

for ambulatory COPD patients (i.e, ADO index, B-AE-D index, B-AE-D-C index, extended ADO index,

updated BODE index, updated ADO index, and a model developed by Bertens et al). Table 4 presents the

clinical setting, the predicted outcome and the time horizon, the events per variable number, the shrinkage

method and the optism-corrected C-statistic for these 7 prognostic models with low risk of bias. Also, the

predictors included in these 7 prognostic models are presented in Table 5. For one of these models (i.e.,

extended ADO index), a model presentation was not available. For the remaining 6 models, the model

equations are described in Table 6. Overall, 338 models were of low risk of bias for participants, 394

models were of low risk of bias for predictors, 402 models were of low risk of bias for outcome, but only

10 models were of low risk of bias for statistical analysis.

We additionally assessed a total of 116 validation efforts (Figure 5). Of these efforts, only 5 were

graded as low risk of bias using PROBAST. These pertained to one validation of the model developed by

Bertens et al, one validation of DECAF, one validation of BAP-65 and 2 validations of PEARL. The

remaining validation efforts were high risk of bias.

Validation of prognostic models originally developed for other diseases

Furthermore, 28 papers examined the predictive ability of 20 prognostic models originally developed

for other disease outcome or other patients (Supplementary Table 6). Specifically, these models are

APACHE II and III, CHA2DS2-VASC score, Charlson comorbidity index, CURB-65, CRB-65, CREWS,

Elixhauser comorbidity index, Framingham risk score, GRACE, HOSPITAL score, LACE, MDA,

MODS, NEWS, NRS 2002, PSI, Salford-NEWS, SAPS and SOFA. Overall, the predictive ability of these

models was examined for the following outcomes: mortality, exacerbation, hospitalization, non-invasive

ventilation failure, or high-cost patients.

Meta-analysis of prognostic models

Page 15 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


16

Overall, we performed a total of 19 meta-analyses of c-statistic for 12 prognostic models (i.e., ADO

index, APACHE II, BOD index, BODE index, BODEx index, CODEX, COTE index, CURB-65, DOSE

index, LACE index, updated ADO index, updated BODE index). For ADO index, APACHE II, BODE

index, BODEx index, and CODEX, we performed 2 different meta-analyses for 2 distinct outcomes,

whereas for DOSE index we performed 3 different meta-analyses for 3 distinct outcomes. 11 meta-

analyses examined the risk of mortality, 2 meta-analyses examined the risk of acute exacerbation of

COPD, 5 meta-analyses examined the risk of re-admission or mortality, and 1 meta-analysis was focused

on failure of non-invasive ventilation. In 12 meta-analyses of c-statistic, we observed large between-study

heterogeneity estimates (i.e., I2 >50%). Summary c-statistic estimates ranged from 0.611 for DOSE index

in prediction of a composite outcome to 0.769 for APACHE II in prediction of mortality. Figure 6 presents

a forest plot of all conducted meta-analyses. Table 7 presents the results of the meta-analyses of c-

statistics. We could not perform meta-analysis of calibration measures, because they were not adequately

reported in the external validation studies.

Discussion

Our systematic search yielded a detailed map of more than 400 prognostic models for the prediction

of clinical outcomes in COPD patients. These scores were developed on a wide range of clinical settings,

including outpatient services, emergency departments, medical wards, intensive care units and primary

care structures. We identified 7 prognostic models that were developed in low risk of bias studies as

assessed with PROBAST, and all these models were externally validated at least once. Also, we

complemented our systematic review and bias assessment with a meta-analysis of c-statistic for 12

prognostic models.

The majority of the prognostic models were developed in Western countries; more than half were

developed in USA, Spain and UK. Although COPD is a quite prevalent chronic disease in low and middle-

income countries [24], only a trivial number of prognostic models were developed or validated in Asia,

Africa or South America. In the developing world, the main risk factors for COPD are history of

tuberculosis and exposure to indoor air pollution [25]. Previous literature has shown a more favorable

prognosis in COPD inflicted by biomass fuel than in smoking-induced COPD [26,27]. We found only one

paper performing an external validation of BODEx index and COTE index on individuals with COPD

Page 16 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


17

associated with biomass fuels [28]. However, this study was conducted in Spain. Our literature search

indicates that currently developed prognostic models could not be generalized in developing countries,

given that they have not been validated in these populations, except an external validation of BODE index

in Brazilian population [29].

Our systematic review unveiled a number of methodological pitfalls in the development of the models

which is also reflected in the risk of bias assessment. Only one-fourth of the models were internally

validated and a tenth of the models were externally validated. The performance of a prognostic model is

overestimated when simply determined on the sample of subjects that was used to construct the model.

Internal validation provides a more accurate estimate of model performance in new subjects, when it is

properly performed, i.e. using bootstrapping or cross-validation techniques [30]. To ensure the

generalizability of a prognostic model in populations with different characteristics, external validation

studies are required [13]. However, independent populations with large sample sizes of COPD patients

and available COPD-specific information (used as predictors in the prognostic models) can be hard to

obtain to measure external validity. This necessitates the use of suitable internal validation techniques to

provide an optimism-adjusted performance for the population in which the model was originally

developed. Nonetheless, an evaluation of model performance in a different sample is not sufficient to

address overfitting and studies developing prognostic models should also apply shrinkage, which is a

methodology to reduce overfitting by re-adjusting the regression coefficients. Indeed, our systematic

review depicted that only a very small number of prognostic models performed shrinkage.

An important finding of our systematic review was that only one-fourth of the models assessed

calibration, which is the accuracy of absolute risk estimates, i.e. it informs clinicians how similar the

predicted absolute risk is to the true (observed) risk in groups of patients classified in different risk strata

[31]. Also, the majority of the models either did not report any method of handling missing data or

performed a complete-case analysis. Missing data often lead to biased estimates if not imputed, because

it can distort the performance of a prediction model if the missingness of values is related to other known

characteristics [32]. Additionally, in about half of the prognostic models, continuous predictors were

dichotomized or categorized and only for a small percentage of prognostic models, the non-linearity of

continuous predictors was examined. However, it is already shown that categorizing continuous predictors

into 2 or more categories leads to weaker prediction performance than analysing predictors on a

Page 17 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


18

continuous scale, due to significant loss in information [33]. Additionally, non-linear associations can be

efficiently modelled using restricted cubic splines or fractional polynomials [34].

Another key issue we should highlight is that discrimination and classification statistics that are

usually reported in prognostic model studies do not inform us on the clinical value of a model. Decision

analysis is required to evaluate whether the implementation of a prognostic model in clinical practice

would be beneficial i.e. do more good than harm [16]. However, only 2 studies performed decision

analysis. Moreover, the applicability of a prediction model in clinical practice depends on the model

presentation. In clinical practice, decision trees, sum scores, nomograms and risk charts are commonly

used in decision-making. Sum scores and decision trees are more suitable for acute care settings, while

risk charts and nomograms allow for a more detailed risk assessment and are more fitted for outpatient

settings. However, more than two-thirds of the developed models did not have any type of model

presentation. Lack of presentation of a predictive tool does not allow its use in clinical practice.

Additionally, lack of reporting of the regression formula in many of the prognostic models hinders future

efforts for validation, update and recalibration [35].

The variables most commonly used in the development of prognostic models were age, FEV1, sex,

BMI, smoking, previous exacerbations, previous hospitalisations, mMRC dyspnea scale, BODE index

and Charlson comorbidity index. The aforementioned variables are either anthropometric features,

important factors in the natural progression of the disease, or markers of disease severity. They are easily

measured so they are available in settings where limited resources are accessible (e.g. primary care) and

in acute care facilities where prompt decisions are ought to be made (e.g. emergency department). Another

advantage of these predictors is low risk for measurement bias which leads to a smaller possibility for

exposure misclassification. Finally, these variables have been identified as individual prognostic factors

in COPD [3,36–40]. However, variability in the top predictors was observed when the predictors were

stratified by clinical setting. For example, in the prognostic models designed based on COPD patients

presenting at the emergency department, the most commonly used predictor was the use of long-term

oxygen therapy or non-invasive ventilation at home, which is uncommon in other settings. Also, smoking

was a frequently used predictor only in models derived from outpatient setting, and it was only rarely used

as a predictor in hospitalized patients. Furthermore, comorbid conditions, either in the form of

multidimensional indices such as Charlson comorbidity index, or as distinct conditions (e.g., diabetes

mellitus, cardiovascular disorders), were widely used and ranked among the most common predictors to

Page 18 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


19

predict clinical outcomes in all settings. Serum albumin and arterial blood gases were used almost

exclusively as predictors in the hospitalized patients and patients visiting the emergency department.

The most extensively validated prognostic models were BODE index [41] and ADO index [42].

BODE index is the most established prognostic model in COPD and was developed to predict mortality

[41]. In GOLD statement, BODE index is used in the prediction of mortality and in clinical decision-

making for lung transplantation and post-discharge follow-up of patients. The predictors included in

BODE index are BMI, FEV1, dyspnea and exercise capacity. Despite the lack of calibration in the original

study of BODE model, it has been validated and updated extensively in medical literature. The updated

BODE index, a recalibration of BODE index, is among models with low risk of bias.

ADO index was based on the predictors used to develop BODE. It entails FEV1, dyspnea and age

[42]. The elimination of 6 minute walking distance that was used in BODE, was based on the rationale of

a more easily applicable model, even by primary care physicians in settings with limited resources, rather

than respiratory professionals alone. Despite the good predictive performance that ADO depicted in its

development study, it showed poor calibration. This led to a re-calibration of ADO index in an independent

population resulting in an updated ADO, as well as an extended version of the re-calibrated model with

the addiation of two variables. ADO index, updated ADO index, and extension of updated ADO index

had low risk of bias and have been externally validated.

Three additional prognostic models presented low risk of bias and were developed for the outpatient

setting. B-AE-D index, and its update B-AE-D-C, were developed for stable COPD patients being at

GOLD stage II to IV to predict 2-year all-cause mortality. The prognostic model developed by Bertens et

al was the only low-risk-of-bias prognostic model that was developed to predict the risk for future

exacerbation in 2 years in stable COPD patients.

An essential step before the application of prediction models in clinical practice is the external

validation in indepdendent populations with different clinical characteristics, and the comparison of

performance among different prognostic models to identify the models with the best discrimination and

calibration. A large-scale effort was recently published to externally validate and compare multiple

prognostic models for COPD patients [43]. The researchers used the methodology of network meta-

analysis to compare the performance of eight multivariable prognostic models and two different GOLD

classifications in 24 cohort studies. In this analysis, updated ADO index had the best ability to predict 3-

year mortality in patients with COPD, followed by the updated BODE index and e-BODE index. However,

Page 19 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


20

the researchers pointed out that the approach of network meta-analysis has not integrated yet the synthesis

of calibration measures [43].

Recommendations and policy implications

Based on the aforementioned methodological pitfalls, the following recommendations could be stated

to improve the research on prognostic models for outcome prediction in COPD patients. First, model

development studies should adjust for overfitting by performing internal validation(mainly through non-

random split or resampling techniques such as bootstrapping) and shrinkage techniques, and provide an

optimism-adjusted performance. Second, model calibration should be examined. If a prognostic model

presents poor calibration, efforts should be made to improve its calibration by updating it either through

re-calibration or through addition of new variables. Third, researchers should apply imputation techniques

when missing data are observed, and they should report the full equation of the prognostic model to allow

its external validation and update by independent research teams. Fourth, continuous predictors should

not be dichotomized and potential non-linear association with the outcome should be examined using

fractional polynomials or restricted cubic splines [44].

We should point out that the vast majority of prognostic models predicted the risk for mortality. Other

clinically important outcomes, such as risk for exacerbation, a very common outcome in randomized

clinical trials for COPD treatment, attracted much less attention. Also, the predictive ability of existing

models focused on European and North American populations and could not be easily generalized. Thus,

there is a need for external validation studies of existing models in other populations.

External validation studies are not sufficient to guarantee the clinical utility of a prediction model. To

select a prediction model for implementation in clinical practice impact studies are needed [13]. These are

randomized clinical trials applying a prognostic model in a clinical setting and assessing its clinical utility

for decision making. However, we found only one impact study in the literature [45]. This study concluded

that the DECAF score, a prognostic model [46] that was initially developed for patients hospitalized with

an exacerbation to predict in-hospital mortality is safe, clinically effective and cost-effective in the

selection of COPD patients with an exacerbation that could be treated at home [45,47].

Comparison with other studies

Page 20 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


21

A previously published systematic review identified 15 prognostic models (either original models or

updates of existing models) for stable COPD patients that were published until September 2010 [6]. This

systematic review mainly focused on the description of clinical characteristics of prognostic models, i.e.

population characteristics and predictors. In contrast, our systematic review included a much broader

spectrum of COPD patients by additionally detecting prognostic models for hospitalized COPD patients

and for COPD patients visiting the emergency department. As a consequence, we captured a total of 408

prognostic models from various clinical settings. Furthermore, we reported a detailed presentation of

methodological characteristics in multivariable prognostic models for COPD. We additionally performed

a meta-analysis for prognostic models with multiple external validation studies, and we assessed the risk

of bias using PROBAST.

Strenghts and limitations

The major strength of our study is that it provides an overall mapping of the available research on

prognostic models for outcome prediction in COPD patients. We collected all published prognostic

models used to forecast any clinical outcome that may occur in the course of COPD. We present a detailed

description of the characteristics of the developed models as well as updates and validation studies of

existing models. Another important aspect of our paper is the critical appraisal of prognostic models in

COPD using the PROBAST tool. We also performed a meta-analysis of c-statistics for prognostic models

that were externally validated in multiple independent populations.

A limitation of our study is the inability to perform meta-analysis of calibration measures for

prognostic models, due to poor reporting of calibration in the validation studies. Also, in the meta-analyses

of c-statistics, we observed large between-study heterogeneity. Potential sources of heterogeneity could

the differences in clinical setting, patient characteristics and time horizons across the validation studies,

but we could not perform meta-regression analyses or sensitivity analyses due to the small number of

external validation studies per prognostic model [34].

Conclusions

Our manuscript constitutes a map of the research on multivariable prognostic models for outcome

prediction in COPD patients to summarize their methodological characteristics, their calibration and

performance. There is an abundance of prognostic models for patients with COPD, so it can be challenging

for healthcare professionals to decide on which one to use in their specific setting or population. Future

Page 21 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


22

prognostic research should steer towards recalibration or update of existing prognostic models with the

addition of new predictors to enhance their prognostic performance. Studies updating existing models

should sufficiently estimate optimism-adjusted performance and calibration measures by applying

appropriate internal validation and adjust for overfitting by applying shrinkage techniques. Future studies

should also perform multiple imputation to handle missing data as well as examine non-linearity of

continuous predictors.

Moreover, to ensure the generalizability of prognostic models, validation studies in populations with

different characteristics, in regards of setting and inclusion criteria, are needed. Prognostic tools with good

calibration and external validity should inform clinical practice as well as be recommended by guidelines

after they undergo impact studies, which examine the impact of using the model for a specific outcome in

clinical practice.

Page 22 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


23

References

1 Soriano JB, Rodríguez-Roisin R. Chronic obstructive pulmonary disease overview: epidemiology,

risk factors, and clinical presentation. Proc Am Thorac Soc 2011;8:363–7.

doi:10.1513/pats.201102-017RM

2 From the Global Strategy for the Diagnosis, Management and Prevention of COPD, Global

Initiative for Chronic Obstructive Lung Disease (GOLD) 2017. 2017.http://goldcopd.org

3 Celli BR, Casanova Macario C. The use of multidimensional indices. In: Controversies in COPD.

European Respiratory Society 2015. 143–60. doi:10.1183/2312508X.10019714

4 Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med

2000;19:453–73. doi:10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5

5 Celli BR. Predictors of mortality in COPD. Respir Med 2010;104:773–9.

doi:10.1016/j.rmed.2009.12.017

6 van Dijk WD, van den Bemt L, van den Haak-Rongen S, et al. Multidimensional prognostic

indices for use in COPD patient care. A systematic review. Respir Res 2011;12:151.

doi:10.1186/1465-9921-12-151

7 Vogelmeier CF, Criner GJ, Martinez FJ, et al. Global Strategy for the Diagnosis, Management,

and Prevention of Chronic Obstructive Lung Disease 2017 Report: GOLD Executive Summary.

Eur Respir J 2017;49. doi:10.1183/13993003.00214-2017

8 Moons KGM, de Groot JAH, Bouwmeester W, et al. Critical Appraisal and Data Extraction for

Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist. PLoS Med

2014;11:e1001744. doi:10.1371/journal.pmed.1001744

9 Debray TPA, Damen JAAG, Snell KIE, et al. A guide to systematic review and meta-analysis of

prediction model performance. BMJ 2017;356:i6460.

10 Geersing G-J, Bouwmeester W, Zuithoff P, et al. Search Filters for Finding Prognostic and

Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews. PLoS One

2012;7:e32844. doi:10.1371/journal.pone.0032844

11 Damen JAAG, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the

Page 23 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


24

general population: systematic review. BMJ 2016;:i2416. doi:10.1136/bmj.i2416

12 Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction

model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ

2015;350:g7594.

13 Moons KGM, Kengne AP, Grobbee DE, et al. Risk prediction models: II. External validation,

model updating, and impact assessment. Heart 2012;98:691–8. doi:10.1136/heartjnl-2011-301247

14 Steyerberg EW. Clinical Prediction Models. New York, NY: : Springer New York 2009.

doi:10.1007/978-0-387-77244-8

15 Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a

framework for traditional and novel measures. Epidemiology 2010;21:128–38.

doi:10.1097/EDE.0b013e3181c30fb2

16 Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction

models, molecular markers, and diagnostic tests. BMJ 2016;:i6. doi:10.1136/bmj.i6

17 Wolff RF, Moons KGM, Riley RD, et al. PROBAST: A Tool to Assess the Risk of Bias and

Applicability of Prediction Model Studies. Ann Intern Med 2019;170:51. doi:10.7326/M18-1376

18 Moons KGM, Wolff RF, Riley RD, et al. PROBAST: A Tool to Assess Risk of Bias and

Applicability of Prediction Model Studies: Explanation and Elaboration. Ann Intern Med

2019;170:W1. doi:10.7326/M18-1377

19 Debray TP, Damen JA, Riley RD, et al. A framework for meta-analysis of prediction model

studies with binary and time-to-event outcomes. Stat Methods Med Res 2018;:962280218785504.

doi:10.1177/0962280218785504

20 Snell KI, Ensor J, Debray TP, et al. Meta-analysis of prediction model performance across

multiple studies: Which scale helps ensure between-study normality for the C -statistic and

calibration measures? Stat Methods Med Res 2018;27:3505–22. doi:10.1177/0962280217705678

21 Madkour AM, Adly NN. Predictors of in-hospital mortality and need for invasive mechanical

ventilation in elderly COPD patients presenting with acute hypercapnic respiratory failure. Egypt

J Chest Dis Tuberc 2013;62:393–400. doi:10.1016/j.ejcdt.2013.07.003

Page 24 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


25

22 Puhan MA, Hansel NN, Sobradillo P, et al. Large-scale international validation of the ADO index

in subjects with COPD: an individual subject data analysis of 10 cohorts. BMJ Open 2012;2.

doi:10.1136/bmjopen-2012-002152

23 Zhong X, Lee S, Zhao C, et al. Reducing COPD readmissions through predictive modeling and

incentive-based interventions. Health Care Manag Sci 2019;22:121–39. doi:10.1007/s10729-017-

9426-2

24 Ntritsos G, Franek J, Belbasis L, et al. Gender-specific estimates of COPD prevalence: a

systematic review and meta-analysis. Int J Chron Obstruct Pulmon Dis 2018;13:1507–14.

doi:10.2147/COPD.S146390

25 Bellou V, Belbasis L, Konstantinidis AK, et al. Elucidating the risk factors for chronic obstructive

pulmonary disease: an umbrella review of meta-analyses. Int J Tuberc Lung Dis 2019;23:58–66.

doi:10.5588/ijtld.18.0228

26 Pérez-Padilla R, Ramirez-Venegas A, Sansores-Martinez R. Clinical Characteristics of Patients

With Biomass Smoke-Associated COPD and Chronic Bronchitis, 2004-2014. Chronic Obstr

Pulm Dis (Miami, Fla) 2014;1:23–32. doi:10.15326/jcopdf.1.1.2013.0004

27 Vestbo J, Celli B. Chronic obstructive pulmonary disease: different risk factors and different

natural histories? Am J Respir Crit Care Med 2014;190:968–70. doi:10.1164/rccm.201409-

1705ED

28 Golpe R, Mengual-Macenlle N, Sanjuán-López P, et al. Prognostic Indices and Mortality

Prediction in COPD Caused by Biomass Smoke Exposure. Lung 2015;193:497–503.

doi:10.1007/s00408-015-9731-9

29 Faganello MM, Tanni SE, Sanchez FF, et al. BODE index and GOLD staging as predictors of 1-

year exacerbation risk in chronic obstructive pulmonary disease. Am J Med Sci 2010;339:10–4.

doi:10.1097/MAJ.0b013e3181bb8111

30 Steyerberg EW, Harrell FE, Borsboom GJ, et al. Internal validation of predictive models:

efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81.

31 Alba AC, Agoritsas T, Walsh M, et al. Discrimination and Calibration of Clinical Prediction

Models. JAMA 2017;318:1377. doi:10.1001/jama.2017.12126

Page 25 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


26

32 Rubin DB. Inference and missing data. Biometrika 1976;63:581–92. doi:10.1093/biomet/63.3.581

33 Collins GS, Ogundimu EO, Cook JA, et al. Quantifying the impact of different approaches for

handling continuous predictors on the performance of a prognostic model. Stat Med

2016;35:4124–35. doi:10.1002/sim.6986

34 Riley RD, Ensor J, Snell KIE, et al. External validation of clinical prediction models using big

datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ

2016;:i3140. doi:10.1136/bmj.i3140

35 Bonnett LJ, Snell KIE, Collins GS, et al. Guide to presenting clinical prediction models for use in

clinical settings. BMJ 2019;:l737. doi:10.1136/bmj.l737

36 Nishimura K, Izumi T, Tsukino M, et al. Dyspnea is a better predictor of 5-year survival than

airway obstruction in patients with COPD. Chest 2002;121:1434–40.

37 Schols AM, Slangen J, Volovics L, et al. Weight loss is a reversible factor in the prognosis of

chronic obstructive pulmonary disease. Am J Respir Crit Care Med 1998;157:1791–7.

doi:10.1164/ajrccm.157.6.9705017

38 Landbo C, Prescott E, Lange P, et al. Prognostic value of nutritional status in chronic obstructive

pulmonary disease. Am J Respir Crit Care Med 1999;160:1856–61.

doi:10.1164/ajrccm.160.6.9902115

39 Yang H, Xiang P, Zhang E, et al. Is hypercapnia associated with poor prognosis in chronic

obstructive pulmonary disease? A long-term follow-up cohort study. BMJ Open 2015;5:e008909.

doi:10.1136/bmjopen-2015-008909

40 Lange P, Halpin DM, O’Donnell DE, et al. Diagnosis, assessment, and phenotyping of COPD:

beyond FEV₁. Int J Chron Obstruct Pulmon Dis 2016;11:3–12. doi:10.2147/COPD.S85976

41 Celli BR, Cote CG, Marin JM, et al. The body-mass index, airflow obstruction, dyspnea, and

exercise capacity index in chronic obstructive pulmonary disease. N Engl J Med 2004;350:1005–

12. doi:10.1056/NEJMoa021322

42 Puhan MA, Garcia-Aymerich J, Frey M, et al. Expansion of the prognostic assessment of patients

with chronic obstructive pulmonary disease: the updated BODE index and the ADO index. Lancet

Page 26 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


27

(London, England) 2009;374:704–11. doi:10.1016/S0140-6736(09)61301-5

43 Guerra B, Haile SR, Lamprecht B, et al. Large-scale external validation and comparison of

prognostic models: an application to chronic obstructive pulmonary disease. BMC Med

2018;16:33. doi:10.1186/s12916-018-1013-y

44 Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for

development and an ABCD for validation. Eur Heart J 2014;35:1925–31.

doi:10.1093/eurheartj/ehu207

45 Echevarria C, Gray J, Hartley T, et al. Home treatment of COPD exacerbation selected by

DECAF score: a non-inferiority, randomised controlled trial and economic evaluation. Thorax

2018;73:713–22. doi:10.1136/thoraxjnl-2017-211197

46 Steer J, Gibson J, Bourke SC. The DECAF Score: predicting hospital mortality in exacerbations

of chronic obstructive pulmonary disease. Thorax 2012;67:970–6. doi:10.1136/thoraxjnl-2012-

202103

47 Cook R, Thomas V, Martin R, et al. People with chronic obstructive pulmonary disease

exacerbations prefer early discharge, then treatment at home. BMJ 2019;364:k5339.

doi:10.1136/bmj.k5339

48 Abu Hussein N, Ter Riet G, Schoenenberger L, et al. The ADO index as a predictor of two-year

mortality in general practice-based chronic obstructive pulmonary disease cohorts. Respiration

2014;88:208–14. doi:10.1159/000363770

49 Ansari K, Keaney N, Kay A, et al. Body mass index, airflow obstruction and dyspnea and body

mass index, airflow obstruction, dyspnea scores, age and pack years-predictive properties of new

multidimensional prognostic indices of chronic obstructive pulmonary disease in primary care.

Ann Thorac Med 2016;11:261. doi:10.4103/1817-1737.191866

50 Boeck L, Soriano JB, Brusse-Keizer M, et al. Prognostic assessment in COPD without lung

function: the B-AE-D indices. Eur Respir J 2016;47:1635–44. doi:10.1183/13993003.01485-2015

51 Echevarria C, Steer J, Heslop-Marshall K, et al. The PEARL score predicts 90-day readmission or

death after hospitalisation for acute exacerbation of COPD. Thorax 2017;72:686–93.

doi:10.1136/thoraxjnl-2016-209298

Page 27 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


28

52 Esteban C, Arostegui I, Moraza J, et al. Development of a decision tree to assess the severity and

prognosis of stable COPD. Eur Respir J 2011;38:1294–300. doi:10.1183/09031936.00189010

53 Jones RC, Price D, Chavannes NH, et al. Multi-component assessment of chronic obstructive

pulmonary disease: an evaluation of the ADO and DOSE indices and the global obstructive lung

disease categories in international primary care data sets. NPJ Prim care Respir Med

2016;26:16010. doi:10.1038/npjpcrm.2016.10

54 Marin JM, Alfageme I, Almagro P, et al. Multicomponent indices to predict survival in COPD:

the COCOMICS study. Eur Respir J 2013;42:323–32. doi:10.1183/09031936.00121012

55 Morales DR, Flynn R, Zhang J, et al. External validation of ADO, DOSE, COTE and CODEX at

predicting death in primary care patients with COPD using standard and machine learning

approaches. Respir Med 2018;138:150–5. doi:10.1016/j.rmed.2018.04.003

56 Motegi T, Jones RC, Ishii T, et al. A comparison of three multidimensional indices of COPD

severity as predictors of future exacerbations. Int J Chron Obstruct Pulmon Dis 2013;8:259–71.

doi:10.2147/COPD.S42769

57 Ou C-Y, Chen C-Z, Yu C-H, et al. Discriminative and predictive properties of multidimensional

prognostic indices of chronic obstructive pulmonary disease: a validation study in Taiwanese

patients. Respirology 2014;19:694–9. doi:10.1111/resp.12313

58 Quintana JM, Esteban C, Unzurrunzaga A, et al. Predictive score for mortality in patients with

COPD exacerbations attending hospital emergency departments. BMC Med 2014;12:66.

doi:10.1186/1741-7015-12-66

59 Waschki B, Kirsten A, Holz O, et al. Physical activity is the strongest predictor of all-cause

mortality in patients with COPD: a prospective cohort study. Chest 2011;140:331–42.

doi:10.1378/chest.10-2521

60 Zhang J, Rutten FH, Cramer MJ, et al. The importance of cardiovascular disease for mortality in

patients with COPD: a prognostic cohort study. Fam Pract 2011;28:474–81.

doi:10.1093/fampra/cmr024

61 Germini F, Veronese G, Marcucci M, et al. Validation of the BAP-65 score for prediction of in-

hospital death or use of mechanical ventilation in patients presenting to the emergency

Page 28 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


29

department with an acute exacerbation of COPD: a retrospective multi-center study from the

Italian Society of Emerg. Eur J Intern Med 2019;61:62–8. doi:10.1016/j.ejim.2018.10.018

62 Sangwan V, Chaudhry D, Malik R. Dyspnea, Eosinopenia, Consolidation, Acidemia and Atrial

Fibrillation Score and BAP-65 Score, Tools for Prediction of Mortality in Acute Exacerbations of

Chronic Obstructive Pulmonary Disease: A Comparative Pilot Study. Indian J Crit Care Med

2017;21:671–7. doi:10.4103/ijccm.IJCCM_148_17

63 Chan HP, Mukhopadhyay A, Chong PLP, et al. Prognostic utility of the 2011 GOLD

classification and other multidimensional tools in Asian COPD patients: a prospective cohort

study. Int J Chron Obstruct Pulmon Dis 2016;11:823–9. doi:10.2147/COPD.S96790

64 Crook S, Frei A, Ter Riet G, et al. Prediction of long-term clinical outcomes using simple

functional exercise performance tests in patients with COPD: a 5-year prospective cohort study.

Respir Res 2017;18:112. doi:10.1186/s12931-017-0598-6

65 Stolz D, Kostikas K, Blasi F, et al. Adrenomedullin refines mortality prediction by the BODE

index in COPD: the ‘BODE-A’ index. Eur Respir J 2014;43:397–408.

doi:10.1183/09031936.00058713

66 Herer B, Chinet T. Acute exacerbation of COPD during pulmonary rehabilitation: outcomes and

risk prediction. Int J Chron Obstruct Pulmon Dis 2018;13:1767–74. doi:10.2147/COPD.S163472

67 Horita N, Koblizek V, Plutinsky M, et al. Chronic obstructive pulmonary disease prognostic

score: A new index. Biomed Pap Med Fac Univ Palacky Olomouc Czech Repub 2016;160:211–8.

doi:10.5507/bp.2016.030

68 Moy ML, Teylan M, Danilack VA, et al. An index of daily step count and systemic inflammation

predicts clinical outcomes in chronic obstructive pulmonary disease. Ann Am Thorac Soc

2014;11:149–57. doi:10.1513/AnnalsATS.201307-243OC

69 Neo H-Y, Xu H-Y, Wu H-Y, et al. Prediction of Poor Short-Term Prognosis and Unmet Needs in

Advanced Chronic Obstructive Pulmonary Disease: Use of the Two-Minute Walking Distance

Extracted from a Six-Minute Walk Test. J Palliat Med 2017;20:821–8.

doi:10.1089/jpm.2016.0449

70 Pedone C, Scarlata S, Forastiere F, et al. BODE index or geriatric multidimensional assessment

Page 29 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


30

for the prediction of very-long-term mortality in elderly patients with chronic obstructive

pulmonary disease? a prospective cohort study. Age Ageing 2014;43:553–8.

doi:10.1093/ageing/aft197

71 Strassmann A, Frei A, Haile SR, et al. Commonly Used Patient-Reported Outcomes Do Not

Improve Prediction of COPD Exacerbations: A Multicenter 4½ Year Prospective Cohort Study.

Chest 2017;152:1179–87. doi:10.1016/j.chest.2017.09.003

72 Golpe R, Suárez-Valor M, Veres-Racamonde A, et al. Octogenarian patients with chronic

obstructive pulmonary disease: Characteristics and usefulness of prognostic indexes. Med Clin

(Barc) 2018;151:53–8. doi:10.1016/j.medcli.2017.09.011

73 Chan HP, Mukhopadhyay A, Chong PLP, et al. Role of BMI, airflow obstruction, St George’s

Respiratory Questionnaire and age index in prognostication of Asian COPD. Respirology

2017;22:114–9. doi:10.1111/resp.12877

74 Wildman MJ, Harrison DA, Welch CA, et al. A new measure of acute physiological derangement

for patients with exacerbations of obstructive airways disease: the COPD and Asthma Physiology

Score. Respir Med 2007;101:1994–2002. doi:10.1016/j.rmed.2007.04.002

75 Almagro P, Yun S, Sangil A, et al. Palliative care and prognosis in COPD: a systematic review

with a validation cohort. Int J Chron Obstruct Pulmon Dis 2017;12:1721–9.

doi:10.2147/COPD.S135657

76 Liu D, Peng S-H, Zhang J, et al. Prediction of short term re-exacerbation in patients with acute

exacerbation of chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis

2015;10:1265–73. doi:10.2147/COPD.S83378

77 Miravitlles M, Izquierdo I, Herrejón A, et al. COPD severity score as a predictor of failure in

exacerbations of COPD. The ESFERA study. Respir Med 2011;105:740–7.

doi:10.1016/j.rmed.2010.12.020

78 Villalobos N, Davidson R, Ghori UK, et al. External Validation of the COmorbidity Test. COPD

2017;14:513–7. doi:10.1080/15412555.2017.1354981

79 Hoogendoorn M, Feenstra TL, Boland M, et al. Prediction models for exacerbations in different

COPD patient populations: comparing results of five large data sources. Int J Chron Obstruct

Page 30 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


31

Pulmon Dis 2017;12:3183–94. doi:10.2147/COPD.S142378

80 Echevarria C, Steer J, Heslop-Marshall K, et al. Validation of the DECAF score to predict

hospital mortality in acute exacerbations of COPD. Thorax 2016;71:133–40.

doi:10.1136/thoraxjnl-2015-207775

81 Jones RC, Donaldson GC, Chavannes NH, et al. Derivation and validation of a composite index

of severity in chronic obstructive pulmonary disease: the DOSE Index. Am J Respir Crit Care

Med 2009;180:1189–95. doi:10.1164/rccm.200902-0271OC

82 Rolink M, van Dijk W, van den Haak-Rongen S, et al. Using the DOSE index to predict changes

in health status of patients with COPD: a prospective cohort study. Prim Care Respir J

2013;22:169–74. doi:10.4104/pcrj.2013.00033

83 Esteban C, Quintana JM, Moraza J, et al. BODE-Index vs HADO-score in chronic obstructive

pulmonary disease: Which one to use in general practice? BMC Med 2010;8:28.

doi:10.1186/1741-7015-8-28

84 Esteban C, Quintana JM, Aburto M, et al. The health, activity, dyspnea, obstruction, age, and

hospitalization: prognostic score for stable COPD patients. Respir Med 2011;105:1662–70.

doi:10.1016/j.rmed.2011.05.005

85 Cote CG, Pinto-Plata VM, Marin JM, et al. The modified BODE index: validation with mortality

in COPD. Eur Respir J 2008;32:1269–74. doi:10.1183/09031936.00138507

86 Stolz D, Meyer A, Rakic J, et al. Mortality risk prediction in COPD by a prognostic biomarker

panel. Eur Respir J 2014;44:1557–70. doi:10.1183/09031936.00043814

87 Antón A, Güell R, Gómez J, et al. Predicting the result of noninvasive ventilation in severe acute

exacerbations of patients with chronic airflow limitation. Chest 2000;117:828–33.

88 Bertens LCM, Reitsma JB, Moons KGM, et al. Development and validation of a model to predict

the risk of exacerbations in chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon

Dis 2013;8:493–9. doi:10.2147/COPD.S49609

89 Bloch KE, Weder W, Bachmann LM, et al. Model-Based versus Clinical Prediction of the

Spirometric Response to Lung Volume Reduction Surgery. Respiration 2004;71:611–8.

Page 31 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


32

doi:10.1159/000081762

90 Confalonieri M, Garuti G, Cattaruzza MS, et al. A chart of failure risk for noninvasive ventilation

in patients with COPD exacerbation. Eur Respir J 2005;25:348–55.

doi:10.1183/09031936.05.00085304

91 Connors AF, Dawson N V, Thomas C, et al. Outcomes following acute exacerbation of severe

chronic obstructive lung disease. The SUPPORT investigators (Study to Understand Prognoses

and Preferences for Outcomes and Risks of Treatments). Am J Respir Crit Care Med

1996;154:959–67. doi:10.1164/ajrccm.154.4.8887592

92 Esteban C, Castro-Acosta A, Alvarez-Martínez CJ, et al. Predictors of one-year mortality after

hospitalization for an exacerbation of COPD. BMC Pulm Med 2018;18:18. doi:10.1186/s12890-

018-0574-z

93 Kerkhof M, Freeman D, Jones R, et al. Predicting frequent COPD exacerbations using primary

care data. Int J Chron Obstruct Pulmon Dis 2015;10:2439–50. doi:10.2147/COPD.S94259

94 Lindenauer PK, Grosso LM, Wang C, et al. Development, validation, and results of a risk-

standardized measure of hospital 30-day mortality for patients with exacerbation of chronic

obstructive pulmonary disease. J Hosp Med 2013;8:428–35. doi:10.1002/jhm.2066

95 Murata GH, Gorby MS, Kapsner CO, et al. A multivariate model for the prediction of relapse

after outpatient treatment of decompensated chronic obstructive pulmonary disease. Arch Intern

Med 1992;152:73–7.

96 Stanford RH, Nag A, Mapel DW, et al. Claims-based risk model for first severe COPD

exacerbation. Am J Manag Care 2018;24:e45–53.

97 Tabak YP, Sun X, Johannes RS, et al. Development and validation of a mortality risk-adjustment

model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med

Care 2013;51:597–605. doi:10.1097/MLR.0b013e3182901982

98 Zafari Z, Sin DD, Postma DS, et al. Individualized prediction of lung-function decline in chronic

obstructive pulmonary disease. CMAJ 2016;188:1004–11. doi:10.1503/cmaj.151483

99 Lau CS, Siracuse BL, Chamberlain RS. Readmission After COPD Exacerbation Scale:

Page 32 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


33

determining 30-day readmission risk for COPD patients. Int J Chron Obstruct Pulmon Dis

2017;12:1891–902. doi:10.2147/COPD.S136768

100 Roche N, Chavaillon J-M, Maurer C, et al. A clinical in-hospital prognostic score for acute

exacerbations of COPD. Respir Res 2014;15:99. doi:10.1186/s12931-014-0099-9

101 Almagro P, Soriano JB, Cabrera FJ, et al. Short- and medium-term prognosis in patients

hospitalized for COPD exacerbation: the CODEX index. Chest 2014;145:972–80.

doi:10.1378/chest.13-1328

Page 33 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


34

Figure 1. Flow chart of literature search for prognostic model in COPD patients.

17,538 articles reviewed by title screening

7758 articles reviewed by abstract screening

228 eligible articles published until November 11, 2018

2868 articles reviewed by full text screening

1 eligible article was identified by hand-search

of references

Page 34 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


35

Figure 2. Predictors included in at least 20 of 408 prognostic models for COPD patients by category of predictor.

Page 35 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


36

Figure 3. The ten more frequently used predictors in the 408 prognostic models for COPD patients presented by clinical setting.

Page 36 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


37

Figure 4. Risk of bias assessment based on 4 domains across the 408 prognostic models for outcome prediction in patients with COPD using PROBAST.

Page 37 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


38

Figure 5. Risk of bias assessment based on 4 domains across external validation studies of prognostic models for outcome prediction in patients with COPD using PROBAST.

Page 38 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


39

Figure 6. Summary c-statistic for 19 meta-analyses of prognostic models for outcome prediction in patients with COPD.

Page 39 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


40

Table 1. Key items for framing the systematic review aim, search strategy, and study inclusion and exclusion criteria, following the PICOTS guidance [8,9].

Item DefinitionPopulation Patients diagnosed with COPDIntervention Any prognostic model to predict any possible clinical outcome in COPD

patients, to distinguish COPD patients with poor prognosis (i.e., who will develop any unfavorable outcome), to aid decision-making in acute care and treatment planning long term

Comparator Not applicableOutcomes Any clinical outcome reported by prognostic modelsTiming Predictors measured at any timepoint in the clinical course of COPD

and preceeding the outcome; outcome measured short-term or long-term without applying any specific limitation in the prediction horizon

Setting Patients visiting ambulatory health care facilities, patients hospitalized or patients visiting the emergency department

Page 40 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


41

Table 2. Methodological characteristics of prediction models developed for outcome prediction in COPD patients.

Emergency department (14 models)

Inpatient(155 models)

Outpatient(239 models)

Overall(408 models)

Internal validationNon-random split 0 3 (1.9 %) 1 (0.4 %) 4 (1.0 %)Random split 8 (57.2 %) 7 (4.5 %) 12 (5.0 %) 27 (6.6 %)Bootstrapping 1 (7.1 %) 9 (5.8 %) 24 (10.0 %) 34 (8.3 %)Cross-validation 0 3 (1.9 %) 28 (11.7 %) 31 (7.6 %)Combination of methods 0 2 (1.3 %) 2 (0.8 %) 4 (1.0 %)None 5 (35.7 %) 131 (84.5 %) 172 (72.0 %) 308 (75.5 %)

Modelling methodCox hazard model 0 21 (13.5 %) 90 (37.7 %) 111 (27.2 %)Logistic regression model 10 (71.4 %) 111 (71.6 %) 79 (33.1 %) 200 (49.0 %)Machine learning 2 (14.3 %) 3 (1.9 %) 2 (0.8 %) 7 (1.7 %)Linear regression model 0 1 (0.6 %) 16 (6.7 %) 17 (4.2 %)Generalized linear model 0 2 (1.3 %) 6 (2.5 %) 8 (2.0 %)Negative binomial model 0 2 (1.3 %) 21 (8.8 %) 23 (5.6 %)Weibull regression model 0 0 15 (6.3 %) 15 (3.7 %)Other methods 2 (14.3%) 4 (2.6 %) 5 (2.1 %) 11 (2.7 %)More than 1 method 0 1 (0.7 %) 3 (1.3 %) 4 (1.0 %)Not reported 0 10 (6.5 %) 2 (0.8 %) 12 (2.9 %)

Handling of missing dataImputation 4 (28.6 %) 13 (8.4 %) 18 (7.5 %) 35 (8.6 %)No missing values 0 1 (0.6 %) 17 (7.1 %) 18 (4.4 %)Exclusion of patients 7 (50.0 %) 28 (18.1 %) 47 (19.7 %) 82 (20.1 %)Not reported 3 (21.4 %) 106 (68.4 %) 153 (64.0 %) 262 (64.2 %)Inappropriate handling 0 7 (4.5 %) 4 (1.7 %) 11 (2.7 %)

Model discriminationC-statistic 11 (78.6 %) 102 (65.8 %) 198 (82.8%) 311 (76.2 %)None 3 (21.4 %) 53 (34.2 %) 41 (17.2 %) 97 (23.8)

Model presentationFull equation 5 (35.7 %) 27 (17.4 %) 24 (10.0 %) 56 (13.7 %)Sum score 3 (21.4 %) 16 (10.3 %) 30 (12.6 %) 49 (12.0 %)

Page 41 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


42

Emergency department (14 models)

Inpatient(155 models)

Outpatient(239 models)

Overall(408 models)

Decision tree 2 (14.3 %) 1 (0.6 %) 3 (1.3 %) 6 (1.5 %)Nomogram 0 1 (0.6 %) 3 (1.3 %) 4 (1.0 %)Risk chart 0 3 (1.9 %) 0 3 (0.7 %)More than 1 method 0 3 (1.9 %) 1 (0.4 %) 4 (1.0%)None 4 (28.6 %) 104 (67.1 %) 178 (74.5 %) 286 (70.1 %)

Model calibrationHosmer-Lemeshow test 5 (35.7 %) 34 (21.9 %) 35 (14.6 %) 74 (18.1 %)Calibration plot 0 1 (0.6 %) 4 (1.7 %) 5 (1.2 %)More than 1 method 1 (7.1 %) 3 (1.9 %) 4 (1.7 %) 8 (2.0 %)Other 0 1 (0.6 %) 3 (1.2 %) 4 (1.0 %)None 8 (57.1 %) 116 (74.8 %) 193 (80.8 %) 317 (77.7 %)

Handling of continuous predictorsContinuous 4 (28.6 %) 47 (30.3 %) 87 (36.4 %) 138 (33.8 %)Categorical/dichotomous 10 (71.4 %) 87 (56.1 %) 64 (26.8 %) 161 (39.5 %)Mixed handling 0 5 (3.2 %) 31 (13.0 %) 36 (8.8 %)Not included 0 15 (9.7 %) 37 (15.5 %) 52 (12.7 %)Unclear 0 1 (0.7 %) 20 (8.4 %) 21 (5.1 %)

Non-linearityPolynomials 0 1 (0.7 %) 7 (3 %) 8 (2 %)Fractional polynomials 0 1 (0.7 %) 2 (0.8 %) 3 (0.7 %)Restricted cubic splines 0 0 6 (2.5 %) 6 (1.5 %)Box-Tidwell transformation 0 1 (0.7 %) 2 2 (0.8 %) 3 (0.7 %)None 14 (100 %) 152 (98.1 %) 222 (92.9 %) 388 (95.1%)

Page 42 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


43

Table 3. External validation studies for prognostic models for outcome prediction in COPD patients.

Reference Model name Cohort name Sample size (cases/total) Outcome c-statistic (95% CI) Calibration

Abu-Hussein, 2014 [48] ADO index ICE COLD ERIC study 35/409 Mortality (2 years) 0.79

Hosmer-Lemeshow test, calibration plot,

calibration-in-the-large

Abu-Hussein, 2014 [48] ADO index Swiss COPD Cohort 17/237 Mortality (2 years) 0.76


calibration-in-the-largeAnsari, 2016* [49] ADO index NR 154/458 Mortality (10 years) 0.70 (0.66 to 0.75) NRBoeck, 2016* [50] ADO index PROMISE study 82/229 Mortality (5 years) 0.68 (0.61 to 0.74) Hosmer-Lemeshow testEchevarria, 2017*

[51] ADO index Cohort 1 309/824 Re-admission or mortality (90 days) 0.67 (0.63 to 0.71) NR

Echevarria, 2017* [51] ADO index Cohort 2 297/802 Re-admission or mortality (90

days) 0.64 (0.60 to 0.67) NR

Echevarria, 2017* [51] ADO index Cohort 3 330/791 Re-admission or mortality (90

days) 0.58 (0.54 to 0.62) NR

Esteban, 2011* [52] ADO index NR 112/348 Mortality (5 years) 0.74 (0.69 to 0.80) Hosmer-Lemeshow test

Jones, 2016* [53] ADO index Optimum Patient Care NR/7105 Hospitalization (1 year) 0.57 (0.50 to 0.64) NRMarin, 2013* [54] ADO index COCOMICS study NR/3633 Mortality (10 years) 0.68 NR

Morales, 2018* [55] ADO index UK Clinical Practice Research Datalink 8083/52,684 Mortality (3 years) 0.732 (0.727 to 0.738) NR

Motegi, 2013* [56] ADO index NR 64/183 AECOPD (1 year) 0.64 (0.56 to 0.73) NROu, 2014* [57] ADO index NR 114/594 Mortality (33 months) 0.70 NR

Puhan, 2012 [22] ADO index

Basque study, Cardiovascular Health Study, Copenhagen City Heart Study,

Jackson Heart Study, Lung Health Study, National Emphysema Treatment Trial, Phenotype and Course of COPD PAC-

COPD Study, PLATINO, Quality of Life of COPD Study Group

1350/13,683 Mortality (3 years) 0.82 (0.81 to 0.83)Hosmer-Lemeshow test,

calibration plot, calibration-in-the-large

Quintana, 2014* [58] ADO index NR NR/2067 Mortality (in-hospital) 0.78 NR

Page 43 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


44


Waschki, 2011* [59] ADO index NR 26/170 Mortality (4 years) 0.77 NR

Zhang, 2011* [60] ADO index NR 60/405 Mortality (5 years) 0.66 (0.59 to 0.73) NRBoeck, 2016 [50] B-AE-D index COCOMICS study 307/2047 Mortality (2 years) 0.65 (0.62 to 0.69) Hosmer-Lemeshow testBoeck, 2016 [50] B-AE-D index COMIC study 82/675 Mortality (2 years) 0.67 (0.61 to 0.72) Hosmer-Lemeshow test

Boeck, 2016 [50] † B-AE-D index ProCOLD study 36/160 Mortality (2 years) 0.60 (0.52 to 0.69) Hosmer-Lemeshow test

Boeck, 2016 [50] † B-AE-D-C index ProCOLD study 36/160 Mortality (2 years) 0.69 (0.60 to 0.78) Hosmer-Lemeshow test

Germini, 2018* [61] BAP-65 NR 165/2908 Mortality (in-hospital) 0.64 (0.59 to 0.68) Calibration plot

Sangwan, 2017* [62] BAP-65 NR 9/50 Mortality (in-hospital) 0.92 (0.83 to 1.00) NR

Chan, 2016* [63]v BOD index NR NR/1110 Mortality 0.72 NRCrook, 2017* [64] BOD index NR 77/409 Mortality (5 years) 0.60 (0.52 to 0.68) NRStolz, 2014* [65] BOD index PROMISE 43/549 Mortality (2 years) 0.65 NRZhang, 2011* [60] BOD index NR 60/405 Mortality (5 years) 0.66 (0.59 to 0.73) NRFaganello, 2010*

[29] BODE index NR 60/120 AECOPD (1 year) 0.62 NR

Herer, 2018* [66] BODE index NR 32/125 AECOPD (1 month) 0.78 (0.69 to 0.85) NRHorita, 2016* [67] BODE index NETT 287/607 Mortality (5 years) 0.62 NRMarin, 2013 [54] BODE index COCOMICS study 230/3633 Mortality (1 year) 0.682 NR

Motegi, 2013* [56] BODE index NR 64/183 AECOPD (1 year) 0.65 (0.56 to 0.73) NR

Moy, 2014* [68] BODE index NR 167 Number of AECOPD (16 months) 0.62 NR

Neo, 2017* [69] BODE index NR 17/124 Mortality (18 months) 0.72 (0.58 to 0.86) NRPedone, 2014* [70] BODE index SARA study 141/468 Mortality (5 years) 0.64 NR

Puhan, 2009* [42] BODE index Swiss Barmelweid Cohort 79/232 Mortality (3 years) 0.67 Hosmer-Lemeshow test, calibration plot

Puhan, 2009* [42] BODE index Spanish Phenotype and Course of COPD 41/342 Mortality (3 years) 0.62 Hosmer-Lemeshow test, calibration plot

Stolz, 2014*[65] BODE index PROMISE study 26/549 Mortality (1 year) 0.745 NRStrassman, 2017*

[71] BODE index ICE COLD ERIC study 408 Number of AECOPD (54 months) 0.69 (0.64 to 0.74) NR

Page 44 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


45


Waschki, 2011* [59] BODE index NR 26/169 Mortality (4 years) 0.76 NR

Echevarria, 2017* [51] BODEx index Cohort 1 309/824 Re-admission or mortality (90

days) 0.65 (0.61 to 0.69) NR


days) 0.64 (0.60 to 0.68) NR


days) 0.62 (0.58 to 0.66) NR

Golpe, 2018* [72] BODEx index NR 154/594 Mortality (1 year) 0.77 (0.74 to 0.80) NRGolpe, 2015* [28] BODEx index NR 12/142 Mortality 0.81 (0.73 to 0.87) NRMarin, 2013 [54] BODEx index COCOMICS study 1225/3633 Mortality (10 years) 0.642 NROu, 2014* [57] BODEx index NR 114/594 Mortality (1 year) 0.702 NR

Strassman, 2017* [71] BODEx index ICE COLD ERIC 408 Number of AECOPD (54

months) 0.72 (0.67 to 0.77) NR

Chan, 2017 [73] BOSA NR NR/772 Mortality (5 years) 0.69 (0.64 to 0.74) NRWildman, 2007

[74] CAPS Case Mix Programme Database NR/NR Mortality (in-hospital) 0.70 (0.69 to 0.72) Hosmer-Lemeshow test

Almagro, 2017 [75] CODEX NR 122/697 Mortality (1 year) 0.68 (0.62 to 0.72) NREchevarria, 2017*

[51] CODEX Cohort 1 309/824 Re-admission or mortality (90 days) 0.69 (0.65 to 0.73) NR

Echevarria, 2017* [51] CODEX Cohort 2 297/802 Re-admission or mortality (90

days) 0.66 (0.63 to 0.70) NR

Echevarria, 2017* [51] CODEX Cohort 3 330/791 Re-admission or mortality (90

days) 0.62 (0.58 to 0.66) NR

Golpe, 2018* [72] CODEX NR 154/594 Mortality (1 year) 0.80 (0.77 to 0.83) NR

Liu, 2015* [76] CODEX NR 86/176 AECOPD or mortality (90 days) 0.57 (0.49 to 0.66) NR

Morales, 2018* [55] CODEX UK Clinical Practice Research Datalink 8083/52,684 Mortality (3 years) 0.652 (0.646 to 0.648) NR

Miravitlles, 2011* [77]

COPD Severity Score ESFERA study 97/346 Treatment failure 0.72 NR

Golpe 2018* [72] COTE index NR 154/594 Mortality (1 year) 0.63 (0.60 to 0.67) NRGolpe, 2015* [28] COTE index NR 12/142 Mortality (32 months) 0.66 (0.57 to 0.73) NR

Page 45 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


46


Morales, 2018* [55] COTE index UK Clinical Practice Research Datalink 8083/52,684 Mortality (3 years) 0.674 (0.667 to 0.680) NR

Villalobos, 2017* [78] COTE index NR 54/317 Mortality (4 months) 0.63 (0.55 to 0.71) Cox-Snell residuals

Hoogendoorn, 2017 [79] CPI COPDGene, OLIN, RECODE, ECLIPSE,

UPLIFT 13254 N of total AECOPD NR NR

Ou, 2014* [57] CPI NR 114/594 Mortality (33 months) 0.72 NREchevarria, 2016

[80] DECAF NR NR/880 Mortality (30 days) 0.79 (0.75 to 0.83) Calibration plot

Sangwan, 2017* [62] DECAF NR 9/50 Mortality (in-hospital) 0.91 (0.79 to 1.00) NR

Boeck, 2016* [50] DOSE PROMISE study 82/229 Mortality (5 years) 0.58 (0.52 to 0.64) Hosmer-Lemeshow testEchevarria, 2017*

[51] DOSE Cohort 1 309/824 Re-admission or mortality (90 days) 0.63 (0.59 to 0.67) NR

Echevarria, 2017* [51] DOSE Cohort 2 297/802 Re-admission or mortality (90

days) 0.59 (0.55 to 0.64) NR

Echevarria, 2017* [51] DOSE Cohort 3 330/791 Re-admission or mortality (90

days) 0.61 (0.57 to 0.65) NR

Herer, 2018* [66] DOSE NR 32/125 AECOPD (1 month) 0.50 (0.41 to 0.59) NRJones, 2009 [81] DOSE London COPD cohort 50/175 Hospitalization 0.755 NRJones, 2016 [53] DOSE Optimum Patient Care NR/7105 Hospitalization (1 year) 0.64 (0.57 to 0.71) NR

Marin, 2013* [54] DOSE COCOMICS study 1225/3633 Mortality (10 years) 0.618 NRMorales, 2018*

[55] DOSE UK Clinical Practice Research Datalink 8083/52684 Mortality (3 years) 0.645 (0.638 to 0.651) NR

Motegi, 2013 [56] DOSE NR 64/183 AECOPD (1 year) 0.75 (0.67 to 0.82) NR

Rolink, 2013* [82] DOSE NR NR/209 Change in health status (2 years) 0.62 (0.50 to 0.74) NR

Strassman, 2017* [71] DOSE ICE COLD ERIC 408 Number of AECOPD (54

months) 0.68 (0.62 to 0.73) NR

Marin, 2013 [54] e-BODE index COCOMICS study 1225/3633 Mortality (10 years) 0.66 NR

Puhan, 2012 [22] Extended ADO index

Barmelweid study, CHS, Basque Study, JHS and SEPOC 338/3693 Mortality (3 years) 0.74 (0.71 to 0.77)


calibration-in-the-largeEsteban, 2010 [83] HADO NR 71/543 Mortality (3 years) 0.81 (0.76 to 0.86) NR

Page 46 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


47


Quintana, 2014 [58] HADO NR 36/1887

Mortality (in-hospital or within 1 week of ED

presentation)0.68 NR

Esteban, 2011 [84] HADO-AH score NR 112/348 Mortality (5 years) 0.76 (0.71 to 0.81) Hosmer-Lemeshow test,

calibration plot

Cote, 2008 [85] mBODE% index BODE cohort 206/444 Mortality 0.72 (0.66 to 0.78) NR

Stolz, 2014 [86] Model 1 NR 49/638 Mortality (3 years) 0.72 NRStolz, 2014 [86] Model 2 NR 49/638 Mortality (3 years) 0.73 NRAnton, 2000 [87] NR NR 12/15 Success of NIV NR NR

Bertens, 2013 [88] NR NR 793/222 AECOPD (2 years) 0.66 (0.61 to 0.71) Calibration plotBloch, 2004 [89] NR NR 27/60 FEV1 change (6 months) 0.75 (0.62 to 0.87) NR

Confalonieri, 2005 [90] NR NR NR/145 NPPV failure (in-hospital) 0.71 (0.63 to 0.79) Hosmer-Lemeshow test

Connors, 1996 [91] NR NR 124/416 Mortality (6 months) 0.731 Calibration plotEsteban, 2011 [52] NR NR 112/348 Mortality (5 years) 0.74 (0.69 to 0.80) Hosmer-Lemeshow testEsteban, 2018 [92] NR NR 492/3235 Mortality (1 year) 0.76 (0.74 to 0.78) Hosmer-Lemeshow testKerkhof, 2015 [93] NR NR NR/2713 Frequent exacerbations 0.74 (0.71 to 0.76) Calibration plotLindenauer, 2013

[94] NR Cohort 1 NR/259,911 Mortality (30 days) 0.73 Overfitting index

Lindenauer, 2013 [94] NR Cohort 2 NR/279,377 Mortality (30 days) 0.72 Overfitting index

Lindenauer, 2013 [94] NR Cohort 3 NR/NR Mortality (30 days) 0.74 Overfitting index

Murata, 1992 [95] NR NR 47/476 Relapse (48 hours) NR NRStanford, 2018 [96] NR NR NR/NR Hospitalisation (1 year) 0.71 NR

Tabak, 2013 [97] NR NR 957/33,327 Mortality (in-hospital) 0.84 (0.82 to 0.85) Hosmer-Lemeshow test, calibration plot

Zafari, 2016 [98] NR EUROSCOP 542 FEV1 NA NR

Zafari, 2016 [98] NR Pan-Canadian Early Detection of Lung Cancer Study 940 FEV1 NA NR

Echevarria, 2017 [51] PEARL score Cohort 1 297/802 Re-admission or mortality (90

days) 0.68 (0.64 to 0.72) Calibration plot

Page 47 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


48


Echevarria, 2017 [51] PEARL score Cohort 2 330/791 Re-admission or mortality (90

days) 0.70 (0.66 to 0.73) Calibration plot

Lau, 2017 [99] RACE State Inpatient Database 17,294/258,113 Re-admission (30 days) NR NR

Roche, 2014 [100] Roche 2008 score NR 45/1824 Mortality (in-hospital) 0.77 (0.72 to 0.81) NR

Marin, 2013* [54] SAFE COCOMICS study 1225/3633 Mortality (10 years) 0.671 NRHerer, 2018* [66] SCOPEX NR 32/125 AECOPD (1 month) 0.74 (0.65 to 0.81) NRStrassman, 2017*

[71] SCOPEX ICE COLD ERIC 408 Number of AECOPD (54 months) 0.70 (0.65 to 0.75) NR

Almagro, 2014* [101]

Updated ADO index ESMI study NR/557 Mortality (1 years) 0.64 (0.61 to 0.67) NR

Morales, 2018* [55]

Updated ADO index UK Clinical Practice Research Datalink 8083/52,684 Mortality (3 years) 0.724 (0.719 to 0.730) NR

Neo, 2017* [69] Updated ADO index NR 17/124 Mortality (18 months) 0.685 (0.55 to 0.82) NR

Puhan, 2012 [22] Updated ADO index

Barmelweid study, CHS, Basque Study, JHS and SEPOC cohorts 338/3693 Mortality (3 years) 0.73 (0.70 to 0.76)


calibration-in-the-large

Boeck, 2016* [50] Updated BODE index PROMISE study 82/211 Mortality (5 years) 0.62 (0.55 to 0.68) Hosmer-Lemeshow test

Puhan, 2009 [42] Updated BODE index Spanish Phenotype and Course of COPD 41/342 Mortality (3 years) 0.61 Hosmer-Lemeshow test,

calibration plotWaschki, 2011*

[59]Updated

BODE index NR 26/170 Mortality (4 years) 0.75 NR

Abbreviations: AECOPD, acute exacerbation of chronic obstructive pulmonary disease; CI, confidence interval; COPDSS, COPD

severity score; NIV, non-invasive ventilation; NR, not reported.

* External validation studies conducted by fully independent research teams.

† In ProCOLD study a measure for dyspnea was not available. B-AE-D index and B-AE-D-C index were externally validated in

Page 48 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


49

ProCOLD study without considering dyspnea.

Page 49 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


50

Table 4. Characteristics of 7 prognostic models for outcome prediction in COPD patients that presented an overall low risk of bias.

Reference Model name Clinical setting Outcome Predictors Shrinkage

methods

Handling of

continuous predictors

EPVOptimism-

corrected C-statistic (95% CI)

Puhan, 2009 [42] ADO index Outpatient Mortality

(3 years) Age, FEV1, mMRC Uniform Continuous 26.3 0.63*

Puhan, 2009 [42] Updated BODE index Outpatient Mortality

(3 years)BMI, FEV1, mMRC, 6MWD Uniform Continuous 19.8 0.61*

Puhan, 2012 [22] Updated ADO index Outpatient Mortality

(3 years) Age, FEV1, mMRC Uniform Continuous 311 0.73 (0.70 to 0.76)

Puhan, 2012 [22] Extended ADO index Outpatient Mortality

(3 years)Age, FEV1, mMRC, BMI, CVD, sex Uniform Continuous 155.5 0.74 (0.71 to 0.77)

Bertens, 2013 [88] NR Outpatient AECOPD

(2 years)

FEV1, previous exacerbations, smoking, vascular disease

Uniform Continuous 17.5 0.66 (0.61 to 0.71)

Boeck, 2016 [50] B-AE-D index Outpatient Mortality

(2 years)BMI, previous exacerbations, mMRC Lasso Continuous 18 0.63 (0.61 to 0.66)

Boeck, 2016 [50] B-AE-D-C index Outpatient Mortality

(2 years)

BMI, previous exacerbations, mMRC, serum copeptin

Lasso Continuous 13.5 0.65 (0.57 to 0.72)

Footnotes:

* Confidence intervals were not reported.

The c-statistic presented in the table is the optimism-corrected metric as reported in internal validation.

Page 50 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


51

Abbreviations: 6MWD, 6 minute walk distance test; AECOPD, acute exacerbation of chronic obstructive pulmonary disase; BMI, body mass index; CI, confidence interval; CVD, cardiovascular disease; EPV, events per variable; FEV1, forced expiratory volume in 1 second; mMRC, modified Medical Research Council dyspnea scale; NR, not reported.

Page 51 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


52

Table 5. Predictors included in the prognostic models for outcome prediction in COPD patients with low risk of bias.

mMRC FEV1 Age BMI Previous AECOPD Additional predictors

Updated BODE ✓ ✓ ✓ 6MWDADO ✓ ✓ ✓ -

Updated ADO ✓ ✓ ✓ -Extended ADO ✓ ✓ ✓ ✓ Sex, CVDBertens, 2013 ✓ ✓ Smoking, vascular disease

B-AE-D ✓ ✓ ✓ -B-AE-D-C ✓ ✓ ✓ Copeptin

Abbreviations: 6MWD, 6 minute walk distance test; AECOPD, acute exacerbation of chronic obstructive pulmonary disease; BMI,

body mass index; CVD, cardiovascular disease; FEV1, forced expiratory volume in 1 second; mMRC, modified Medical Research

Council dyspnea scale.

Page 52 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


53

Table 6. Model equations of prognostic models for outcome prediction in COPD patients with low risk of bias.

Prediction

model

Model equation

ADO 𝑦 = ―0.012 × 𝐹𝐸𝑉1(% 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑) + 0.193 × 𝑚𝑀𝑅𝐶 + 0.027 × 𝑎𝑔𝑒 ― 3.436

Updated

ADO

𝑦 = ―0.288 × 𝐹𝐸𝑉1(% 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑) + 0.2585 × 𝑚𝑀𝑅𝐶 + 0.0703 × 𝑎𝑔𝑒 ― 5.640

Updated

BODE

𝑦 = ―0.013 × 𝐵𝑀𝐼 ― 0.005 × 𝐹𝐸𝑉1(% 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑) + 0.146 × 𝑚𝑀𝑅𝐶 ― 0.005 × 6𝑀𝑊𝐷 + 1.483

B-AE-D 𝑦= 0.97 × (18.5 ≤ 𝐵𝑀𝐼 < 21) + 1.45 × (𝐵𝑀𝐼 < 18.5) + 0.45 × (𝑝𝑟𝑒𝑣𝑖𝑜𝑢𝑠 𝑠𝑒𝑣𝑒𝑟𝑒 𝑒𝑥𝑎𝑐𝑒𝑟𝑏𝑎𝑡𝑖𝑜𝑛𝑠 = 1) + 1.22 × (𝑝𝑟𝑒𝑣𝑖𝑜𝑢𝑠 𝑠𝑒𝑣𝑒𝑟𝑒 𝑒𝑥𝑎𝑐𝑒𝑟𝑏𝑎𝑡𝑖𝑜𝑛𝑠 ≥ 2)+ 0.97 × (𝑚𝑀𝑅𝐶 = 3) + 1.67 × (𝑚𝑀𝑅𝐶 = 4) + 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡

B-AE-D-C 𝑦= 0.97 × (18.5 ≤ 𝐵𝑀𝐼 < 21) + 1.45 × (𝐵𝑀𝐼 < 18.5) + 0.45 × (𝑝𝑟𝑒𝑣𝑖𝑜𝑢𝑠 𝑠𝑒𝑣𝑒𝑟𝑒 𝑒𝑥𝑎𝑐𝑒𝑟𝑏𝑎𝑡𝑖𝑜𝑛𝑠 = 1) + 1.22 × (𝑝𝑟𝑒𝑣𝑖𝑜𝑢𝑠 𝑠𝑒𝑣𝑒𝑟𝑒 𝑒𝑥𝑎𝑐𝑒𝑟𝑏𝑎𝑡𝑖𝑜𝑛𝑠 ≥ 2)+ 0.97 × (𝑚𝑀𝑅𝐶 = 3) + 1.67 × (𝑚𝑀𝑅𝐶 = 4) + 0.50 × (20 ≤ 𝑐𝑜𝑝𝑒𝑝𝑡𝑖𝑛 < 40) + 1.58 × (𝑐𝑜𝑝𝑒𝑝𝑡𝑖𝑛 ≥ 40) + 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡

Bertens et

al.

𝑦 = 1.62 × (𝑝𝑟𝑒𝑠𝑒𝑛𝑐𝑒 𝑜𝑓 𝑝𝑟𝑒𝑣𝑖𝑜𝑢𝑠 𝑒𝑥𝑎𝑐𝑒𝑟𝑏𝑎𝑡𝑖𝑜𝑛) ― 0.05 × 𝐹𝐸𝑉1(% 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑, 𝑝𝑒𝑟 5% 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 𝑖𝑛𝑐𝑟𝑒𝑎𝑠𝑒) + 0.12 × (2 × log (packyears)) + 0.65× (𝑝𝑟𝑒𝑠𝑒𝑛𝑐𝑒 𝑜𝑓 𝑣𝑎𝑠𝑐𝑢𝑙𝑎𝑟 𝑑𝑖𝑠𝑒𝑎𝑠𝑒) ― 1.33

For B-AE-D and B-AE-D-C models, the constant of regression equation was not reported.

Abbreviations: 6MWD, 6 minute walk distance test; BMI, body mass index; FEV1, forced expiratory volume in 1 second; mMRC,

modified Medical Research Council dyspnea scale.

Page 53 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


54

Table 7. Results of meta-analyses of c-statistics for prognostic models in COPD patients.

Model Outcome Number of datasetsNumber of

events/participants

Summary c-statistic (95%

CI)I2

ADO index Mortality 11 11,258/72,850 0.731 (0.692 to 0.766) 95

ADO indexRe-admission or

mortality3 936/2417 0.630 (0.513 to 0.734) 81

APACHE II NIV failure 3 121/550 0.718 (0.647 to 0.780) 0

APACHE II Mortality 7 NA/NA* 0.769 (0.681 to 0.838) 84

BOD index Mortality 4 NA/NA* 0.665 (0.578 to 0.742) 63

BODE index Mortality 8 847/6124 0.663 (0.624 to 0.701) 36

BODE index AECOPD 3 156/428 0.686 (0.442 to 0.857) 71

BODEx indexRe-admission or

mortality3 936/2417 0.636 (0.598 to 0.674) 0

BODEx index Mortality 4 1505/4963 0.730 (0.597 to 0.831) 93

CODEX Mortality 3 8359/53,975 0.720 (0.500 to 0.869) 96

CODEXRe-admission or

mortality3 936/2417 0.657 (0.566 to 0.737) 67

COTE index Mortality 4 8303/53,737 0.655 (0.616 to 0.692) 57

CURB-65 Mortality 6 451/4250 0.730 (0.690 to 0.767) 24

DOSE index AECOPD 3 NA/NA* 0.615 (0.291 to 0.861) 93

DOSE indexRe-admission or

mortality3 936/2417 0.611 (0.562 to 0.658) 0

Page 54 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


55

Model Outcome Number of datasetsNumber of

events/participants

Summary c-statistic (95%

CI)I2

DOSE index Mortality 3 9390/56,546 0.624 (0.552 to 0.691) 85

LACE indexRe-admission or

mortality4 1601/5079 0.632 (0.612 to 0.652) 0

Updated ADO index Mortality 4 NA/NA* 0.699 (0.624 to 0.764) 91

Updated BODE index Mortality 3 149/723 0.647 (0.456 to 0.800) 48

Abbreviations:

CI, confidence interval; NA, not available

Footnotes:

* For at least one dataset, the number of events and/or participants was not reported.

Page 55 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

Confidential: For Review OnlyA systematic review and critical appraisal of prognostic

prediction models for patients with chronic obstructive

pulmonary disease

Supplementary material

Supplementary Table 1. Summary of predicted outcomes for the 408 prognostic models for patients with COPD.

Supplementary Table 2. Top predictors in the 408 prognostic models for COPD patients stratified by clinical setting.

Supplementary Table 3. Articles describing the development of prognostic models for patients with COPD in outpatient settings.

Supplementary Table 4. Articles describing the development of prognostic models for hospitalized COPD patients.

Supplementary Table 5. Articles describing the development of prognostic model for COPD patients attending the emergency department.

Supplementary Table 6. Articles describing the validation of prognostic models developed for diseases other than COPD.

Page 56 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

Confidential: For Review OnlySupplementary Table 1. Summary of predicted outcomes for the 408 prognostic models for patients with COPD.

OutcomeEmergency department

(n = 14 models)

Hospitalized patients

(n = 155 models)

Outpatients(n = 239)

Overall(n = 408)

Mortality 7 78 124 209Change in physical activity 2 0 0 2

Composite outcome 2 13 9 24Hospitalization 1 0 16 17ICU admission 1 0 0 1Readmission 0 36 0 36NIV failure 0 14 0 14Length of stay 0 9 0 9Weaning success 0 2 0 2Treatment failure 1 0 8 9AECOPD 0 2 40 42Prolonged MV 0 1 0 1Spirometric indices 0 0 25 25Cost 0 0 2 2Adherence to rehab 0 0 2 2Late recovery 0 0 1 1COPD-related disability 0 0 3 36-MWD decline 0 0 4 4Pneumonia 0 0 1 1Hypoxic RF 0 0 3 3BODE improvement 0 0 1 1

Abbreviations: 6-MWD, 6-minute walking distance test; AECOPD, acute exacerbation of chronic obstructive pulmonary disease; COPD, chronic obstructive pulmonary disease; ICU, intensive care unit; MV, mechanical ventilation; NIV, non-invasive ventilation; RF, respiratory failure

Page 57 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

Confidential: For Review OnlySupplementary Table 2. Top predictors in the 408 prognostic models for COPD patients stratified by clinical setting.

Overall (n = 408 models) Outpatients (n = 239 models)

Hospitalized patients (n = 155 models)

Patients attending emergency department

(n = 14 models)

Predictor N of models Predictor N of

models Predictor N of models Predictor N of

models

Age 166 Age 105 Age 56 LTOT/NIV at home 8

FEV1 85 FEV1 69 Sex 30 Age 5Sex 74 Smoking 54 PaCO2 24 mMRC scale 5

BMI 66 BMI 51 Previous hospitalizations 20

Charlson comorbidity

index4

Smoking 65 Sex 43 Length of stay 20 PaCO2 3

Previous exacerbations 53 Previous

exacerbations 43Charlson

comorbidity index

19

Use of inspiratory accessory

muscle and Paradoxical breathing

3

Previous hospitalizations 50 BODE index 43 pH 18

Glasgow (or Japan) coma

scale3

BODE index 43 Previous hospitalizations 28 Heart failure 16 FEV1 2

mMRC scale 42 Diabetes mellitus 24 BMI 15 pH 2

Charlson comorbidity

index35 mMRC scale 23 Serum albumin 15 Cardiovascular

diseases 2

Serum CRP 34 Cardiovascular diseases 23 FEV1 14 Previous

hospitalizations 2

Serum CRP 23 mMRC scale 14Anemia 14

Page 58 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Supplementary Table 3. Articles describing the development of prognostic models for patients with COPD in outpatient settings.

References Number of models developedDompeling, 1992 [1] 2Campbell, 1985 [2] 1Smit, 1983 [3] 3Daughton, 1984 [4] 1Ball, 1995 [5] 1Ashutosh, 1997 [6] 1Kessler, 1999 [7] 1Dewan, 2000 [8] 1Miravitlles, 2000 [9] 2Miravitlles, 2001 [10] 1Fan, 2002 [11] 1Celli, 2004 [12] 1Bloch, 2004 [13] 1Miravitlles, 2005 [14] 2Mapel, 2005 [15] 1Miravitlles, 2006 [16] 2Man, 2006 [17] 1Esteban, 2006 [18] 1Niewoehner, 2007 [19] 2Fan, 2007 [20] 3Briggs, 2008 [21] 1Blanchette, 2008 [22] 1Fan, 2008 [23] 1Soler-Cataluña, 2009 [24] 2Omachi, 2008 [25] 2Schembri, 2009 [26] 1Puhan, 2009 [27] 2Mehrotra, 2010 [28] 2Benzo, 2010 [29] 1Eisner, 2011 [30] 3Brusse, 2011 [31] 1Simon-Tuval, 2011 [32] 1Waschki, 2011 [33] 1Lee, 2011 [34] 3Esteban, 2011 [35] 1Zhang, 2011 [36] 1

Page 59 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Esteban, 2011 [37] 1Gale, 2011 [38] 3Williams, 2012 [39] 1Yoo, 2011 [40] 3Ozgur, 2012 [41] 1Celli, 2012 [42] 9Divo, 2012 [43] 2Jensen, 2013 [44] 2Puhan, 2012 [45] 2Mannino, 2013 [46] 1Ryynanen, 2013 [47] 1Thomsen, 2013 [48] 2Stolz, 2014 [49] 8Roberts, 2013 [50] 4Heerema, 2013 [51] 1Bertens, 2013 [52] 1Moy, 2014 [53] 2Bowler, 2014 [54] 1Koskela, 2014 [55] 1Stolz, 2014 [56] 3Boutou, 2014 [57] 1Suzuki, 2014 [58] 7Make, 2015 [59] 1Wilson, 2015 [60] 2Montserrat-Capdevila, 2015 [61] 1Shin, 2015 [62] 2Abascal-Bolado, 2015 [63] 1Kerkhof, 2015 [64] 1Montserrat-Capdevila, 2015 [65] 1Miravitlles, 2016 [66] 1Ramon, 2016 [67] 3Park, 2016 [68] 5Blumenthal, 2016 [69] 1Beijers, 2016 [70] 1Boeck, 2016 [71] 12Horita, 2016 [72] 1Zafari, 2016 [73] 1Chan, 2017 [74] 1

Page 60 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Sand, 2016 [75] 17Ansari, 2016 [76] 2Barton, 2017 [77] 1Crook, 2017 [78] 4Russo, 2017 [79] 2Sundh, 2017 [80] 3Kurashima, 2016 [81] 1Villalobos, 2017 [82] 2Strassman, 2017 [83] 17Dal Negro, 2017 [84] 1Hoogendoorn, 2017 [85] 1Mendy, 2018 [86] 14Kang, 2017 [87] 6Fijačko, 2018 [88] 6Bafadhel, 2018 [89] 1Stanford, 2018 [90] 1Samp, 2018 [91] 1Miravitlles, 2018 [92] 1Vela, 2018 [93] 1Morales, 2018 [94] 2Hwang, 2019 [95] 9Annavarapu, 2018 [96] 1

Page 61 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Supplementary Table 4. Articles describing the development of prognostic models for hospitalized COPD patients.

References Number of models developedMenzies, 1989 [97] 2Fuso, 1995 [98] 2Nava, 1994 [99] 2Dubois, 1994 [100] 1Rieves, 1993 [101] 2Vitacca, 1996 [102] 1Connors, 1996 [103] 1Szekely, 1997 [104] 1Antonelli Incalzi, 1997 [105] 1Anton, 2000 [106] 1Putinati, 2000 [107] 1Antonelli Incalzi, 2001 [108] 1Roberts, 2002 [109] 3Patil, 2003 [110] 6Goel, 2003 [111] 3Scala, 2004 [112] 2Khilnani, 2004 [113] 1Confalonieri, 2005 [114] 1Ucgun, 2006 [115] 3Yohannes, 2005 [116] 2Almagro, 2006 [117] 1Chen, 2006 [118] 1Wildman, 2007 [119] 2Liu, 2007 [120] 1Ruiz-Gonzalez, 2008 [121] 1Mohan, 2008 [122] 2Wildman, 2009 [123] 1Chakrabarti, 2009 [124] 4Tsimogianni, 2009 [125] 1Tabak, 2009 [126] 1Terzano, 2010 [127] 3Roca, 2011 [128] 1Aburto, 2011 [129] 1Asiimwe, 2011 [130] 1Abrams, 2011 [131] 1Matkovic, 2012 [132] 3

Page 62 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Steer, 2012 [133] 3Slenter, 2013 [134] 1Jurado Gamez, 2013 [135] 1Tabak, 2013 [136] 1Haja Mydin, 2013 [137] 1Amalakuhan, 2012 [138] 1Lindenauer, 2013 [139] 1Almagro, 2014 [140] 1Wang, 2014 [141] 1Batzlaff, 2014 [142] 1Sharif, 2014 [143] 2Diamantea, 2014 [144] 1Cheng, 2014 [145] 2Roche, 2014 [146] 1Quintana, 2014 [147] 1Grolimund, 2015 [148] 5Crisafulli, 2015 [149] 1Fan, 2014 [150] 1Quintana, 2015 [151] 1Quintana, 2014 [152] 3Yu, 2015 [153] 1Roberts, 2015 [154] 5Liu, 2015 [155] 1Kon, 2015 [156] 2Glaser, 2015 [157] 1Crisafulli, 2016 [158] 1Garcia Sidro, 2015 [159] 1Ramaraju, 2016 [160] 2Sainaghi, 2016 [161] 1He, 2018 [162] 1Garcia-Rivero, 2017 [163] 2Echevarria, 2017 [164] 1Sakamoto, 2017 [165] 1Feng, 2017 [166] 5Almagro, 2017 [167] 1Lau, 2017 [168] 1Vanasse, 2017 [169] 1Duenk, 2017 [170] 1

Page 63 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Yao, 2017 [171] 4Bernabeu-Mora, 2017 [172] 2Winther, 2017 [173] 2Zhong, 2017 [174] 1Pavlisa, 2017 [175] 4Rezaee, 2017 [176] 3Hakim, 2018 [177] 5Esteban, 2018 [178] 1Serra-Pimacal, 2018 [179] 3Cerezo Lajas, 2018 [180] 1Epstein, 2018 [181] 2Bottle, 2018 [182] 2Schuler, 2018 [183] 2Madkour, 2013 [184] 2

Page 64 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Supplementary Table 5. Articles describing the development of prognostic model for COPD patients attending the emergency department.

References Number of modelsMurata, 1992 [185] 1Kim, 2006 [186] 1Roche, 2008 [187] 1Stiell, 2014 [188] 1Quintana, 2014 [189] 1Quintana, 2014 [147] 1Quintana, 2014 [152] 2Esteban, 2015 [190] 2Esteban, 2016 [191] 4

Page 65 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Supplementary Table 6. Articles describing the validation of prognostic models developed for diseases other than COPD.

Model name ReferencesAPACHE II [113,122,124,133,150,192–197]APACHE III [194]CHA2DS2-VASC [198,199]Charlson comorbidity index [200,201]CRB-65 [202]CREWS [203]CURB-65 [133,196,204–207]Elixhauser comorbidity index [200]GRACE [208]HOSPITAL score [209]LACE [164,177]MDA [210]MODS [194]NEWS [203]NRS 2002 [211]PSI [205]Salford-NEWS [203]SAPS II [122,184,194]SOFA [194]

Page 66 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Page 67 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Supplementary References

1 Dompeling E, van Grunsven PM, Molema J, et al. Early detection of patients with fast progressive asthma or chronic bronchitis in general practice. Scand J Prim Health Care 1992;10:143–50.

2 Campbell AH, Barter CE, O’Connell JM, et al. Factors affecting the decline of ventilatory function in chronic bronchitis. Thorax 1985;40:741–8.

3 Smit JM, Burema J, May JF, et al. Prognosis in severe chronic obstructive pulmonary disease with regard to the electrocardiogram. J Electrocardiol 1983;16:77–86.

4 Daughton DM, James Fix A, Kass I, et al. Three-year survival rates of pulmonary rehabilitation patients with chronic obstructive pulmonary disease. J Natl Med Assoc 1984;76:265–8.

5 Ball P, Harris JM, Lowson D, et al. Acute infective exacerbations of chronic bronchitis. QJM 1995;88:61–8.

6 Ashutosh K, Haldipur C, Boucher ML. Clinical and personality profiles and survival in patients with COPD. Chest 1997;111:95–8.

7 Kessler R, Faller M, Fourgaut G, et al. Predictive factors of hospitalization for acute exacerbation in a series of 64 patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 1999;159:158–64. doi:10.1164/ajrccm.159.1.9803117

8 Dewan NA, Rafique S, Kanwar B, et al. Acute exacerbation of COPD: factors associated with poor treatment outcome. Chest 2000;117:662–71.

9 Miravitlles M, Guerrero T, Mayordomo C, et al. Factors associated with increased risk of exacerbation and hospital admission in a cohort of ambulatory COPD patients: a multiple logistic regression analysis. The EOLO Study Group. Respiration 2000;67:495–501. doi:10.1159/000067462

10 Miravitlles M, Murio C, Guerrero T. Factors associated with relapse after ambulatory treatment of acute exacerbations of chronic bronchitis. DAFNE Study Group. Eur Respir J 2001;17:928–33.

11 Fan VS, Curtis JR, Tu S-P, et al. Using quality of life to predict hospitalization and mortality in patients with obstructive lung diseases. Chest 2002;122:429–36.

12 Celli BR, Cote CG, Marin JM, et al. The body-mass index, airflow obstruction, dyspnea, and exercise capacity index in chronic obstructive pulmonary disease. N Engl J Med 2004;350:1005–12. doi:10.1056/NEJMoa021322

13 Bloch KE, Weder W, Bachmann LM, et al. Model-Based versus Clinical Prediction of the Spirometric Response to Lung Volume Reduction Surgery. Respiration 2004;71:611–8. doi:10.1159/000081762

14 Miravitlles M, Llor C, Naberan K, et al. Variables associated with recovery from acute exacerbations of chronic bronchitis and chronic obstructive pulmonary disease. Respir Med 2005;99:955–65. doi:10.1016/j.rmed.2005.01.013

15 Mapel DW, McMillan GP, Frost FJ, et al. Predicting the costs of managing patients with chronic obstructive pulmonary disease. Respir Med 2005;99:1325–33. doi:10.1016/j.rmed.2005.03.001

16 Miravitlles M, Calle M, Alvarez-Gutierrez F, et al. Exacerbations, hospital admissions and impaired health status in chronic obstructive pulmonary disease. Qual Life Res 2006;15:471–80. doi:10.1007/s11136-005-3215-y

17 Man SFP, Connett JE, Anthonisen NR, et al. C-reactive protein and mortality in mild to moderate chronic obstructive pulmonary disease. Thorax 2006;61:849–53. doi:10.1136/thx.2006.059808

18 Esteban C, Quintana JM, Aburto M, et al. A simple score for assessing stable chronic obstructive pulmonary disease. QJM 2006;99:751–9. doi:10.1093/qjmed/hcl110

19 Niewoehner DE, Lokhnygina Y, Rice K, et al. Risk indexes for exacerbations and hospitalizations due to COPD. Chest 2007;131:20–8. doi:10.1378/chest.06-1316

20 Fan VS, Ramsey SD, Make BJ, et al. Physiologic variables and functional status independently predict COPD hospitalizations and emergency department visits in patients with severe COPD. COPD 2007;4:29–

Page 68 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


39. doi:10.1080/15412550601169430

21 Briggs A, Spencer M, Wang H, et al. Development and validation of a prognostic index for health outcomes in chronic obstructive pulmonary disease. Arch Intern Med 2008;168:71–9. doi:10.1001/archinternmed.2007.37

22 Blanchette CM, Gutierrez B, Ory C, et al. Economic burden in direct costs of concomitant chronic obstructive pulmonary disease and asthma in a Medicare Advantage population. J Manag Care Pharm 2008;14:176–85. doi:10.18553/jmcp.2008.14.2.176

23 Fan VS, Giardino ND, Blough DK, et al. Costs of pulmonary rehabilitation and predictors of adherence in the National Emphysema Treatment Trial. COPD 2008;5:105–16. doi:10.1080/15412550801941190

24 Soler-Cataluña JJ, Martínez-García MA, Sánchez LS, et al. Severe exacerbations and BODE index: two independent risk factors for death in male COPD patients. Respir Med 2009;103:692–9. doi:10.1016/j.rmed.2008.12.005

25 Omachi TA, Yelin EH, Katz PP, et al. The COPD severity score: a dynamic prediction tool for health-care utilization. COPD 2008;5:339–46. doi:10.1080/15412550802522700

26 Schembri S, Anderson W, Morant S, et al. A predictive model of hospitalisation and death from chronic obstructive pulmonary disease. Respir Med 2009;103:1461–7. doi:10.1016/j.rmed.2009.04.021

27 Puhan MA, Garcia-Aymerich J, Frey M, et al. Expansion of the prognostic assessment of patients with chronic obstructive pulmonary disease: the updated BODE index and the ADO index. Lancet (London, England) 2009;374:704–11. doi:10.1016/S0140-6736(09)61301-5

28 Mehrotra N, Freire AX, Bauer DC, et al. Predictors of mortality in elderly subjects with obstructive airway disease: the PILE score. Ann Epidemiol 2010;20:223–32. doi:10.1016/j.annepidem.2009.11.005

29 Benzo RP, Chang C-CH, Farrell MH, et al. Physical activity, health status and risk of hospitalization in patients with severe chronic obstructive pulmonary disease. Respiration 2010;80:10–8. doi:10.1159/000296504

30 Eisner MD, Iribarren C, Blanc PD, et al. Development of disability in chronic obstructive pulmonary disease: beyond lung function. Thorax 2011;66:108–14. doi:10.1136/thx.2010.137661

31 Brusse-Keizer M, van der Palen J, van der Valk P, et al. Clinical predictors of exacerbation frequency in chronic obstructive pulmonary disease. Clin Respir J 2011;5:227–34. doi:10.1111/j.1752-699X.2010.00234.x

32 Simon-Tuval T, Scharf SM, Maimon N, et al. Determinants of elevated healthcare utilization in patients with COPD. Respir Res 2011;12:7. doi:10.1186/1465-9921-12-7

33 Waschki B, Kirsten A, Holz O, et al. Physical activity is the strongest predictor of all-cause mortality in patients with COPD: a prospective cohort study. Chest 2011;140:331–42. doi:10.1378/chest.10-2521

34 Lee JS, Huh JW, Chae EJ, et al. Predictors of pulmonary function response to treatment with salmeterol/fluticasone in patients with chronic obstructive pulmonary disease. J Korean Med Sci 2011;26:379–85. doi:10.3346/jkms.2011.26.3.379

35 Esteban C, Arostegui I, Moraza J, et al. Development of a decision tree to assess the severity and prognosis of stable COPD. Eur Respir J 2011;38:1294–300. doi:10.1183/09031936.00189010

36 Zhang J, Rutten FH, Cramer MJ, et al. The importance of cardiovascular disease for mortality in patients with COPD: a prognostic cohort study. Fam Pract 2011;28:474–81. doi:10.1093/fampra/cmr024

37 Esteban C, Quintana JM, Aburto M, et al. The health, activity, dyspnea, obstruction, age, and hospitalization: prognostic score for stable COPD patients. Respir Med 2011;105:1662–70. doi:10.1016/j.rmed.2011.05.005

38 Gale CP, White JES, Hunter A, et al. Predicting mortality and hospital admission in patients with COPD: significance of NT pro-BNP, clinical and echocardiographic assessment. J Cardiovasc Med (Hagerstown) 2011;12:613–8. doi:10.2459/JCM.0b013e3283491780

Page 69 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


39 Williams JEA, Green RH, Warrington V, et al. Development of the i-BODE: validation of the incremental shuttle walking test within the BODE index. Respir Med 2012;106:390–6. doi:10.1016/j.rmed.2011.09.005

40 Yoo J-W, Hong Y, Seo JB, et al. Comparison of clinico-physiologic and CT imaging risk factors for COPD exacerbation. J Korean Med Sci 2011;26:1606–12. doi:10.3346/jkms.2011.26.12.1606

41 Ozgür ES, Nayci SA, Özge C, et al. An integrated index combined by dynamic hyperinflation and exercise capacity in the prediction of morbidity and mortality in COPD. Respir Care 2012;57:1452–9. doi:10.4187/respcare.01440

42 Celli BR, Locantore N, Yates J, et al. Inflammatory biomarkers improve clinical prediction of mortality in chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2012;185:1065–72. doi:10.1164/rccm.201110-1792OC

43 Divo M, Cote C, de Torres JP, et al. Comorbidities and risk of mortality in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2012;186:155–61. doi:10.1164/rccm.201201-0034OC

44 Jensen MT, Marott JL, Lange P, et al. Resting heart rate is a predictor of mortality in COPD. Eur Respir J 2013;42:341–9. doi:10.1183/09031936.00072212

45 Puhan MA, Hansel NN, Sobradillo P, et al. Large-scale international validation of the ADO index in subjects with COPD: an individual subject data analysis of 10 cohorts. BMJ Open 2012;2. doi:10.1136/bmjopen-2012-002152

46 Mannino DM, Diaz-Guzman E, Pospisil J. A new approach to classification of disease severity and progression of COPD. Chest 2013;144:1179–85. doi:10.1378/chest.12-2674

47 Ryynänen O-P, Soini EJ, Lindqvist A, et al. Bayesian predictors of very poor health related quality of life and mortality in patients with COPD. BMC Med Inform Decis Mak 2013;13:34. doi:10.1186/1472-6947-13-34

48 Thomsen M, Ingebrigtsen TS, Marott JL, et al. Inflammatory biomarkers and exacerbations in chronic obstructive pulmonary disease. JAMA 2013;309:2353–61. doi:10.1001/jama.2013.5732

49 Stolz D, Kostikas K, Blasi F, et al. Adrenomedullin refines mortality prediction by the BODE index in COPD: the ‘BODE-A’ index. Eur Respir J 2014;43:397–408. doi:10.1183/09031936.00058713

50 Roberts MH, Mapel DW, Bruse S, et al. Development of a modified BODE index as a mortality risk measure among older adults with and without chronic obstructive pulmonary disease. Am J Epidemiol 2013;178:1150–60. doi:10.1093/aje/kwt087

51 Heerema-Poelman A, Stuive I, Wempe JB. Adherence to a Maintenance Exercise Program 1 Year After Pulmonary Rehabilitation. J Cardiopulm Rehabil Prev 2013;33:419–26. doi:10.1097/HCR.0b013e3182a5274a

52 Bertens LCM, Reitsma JB, Moons KGM, et al. Development and validation of a model to predict the risk of exacerbations in chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis 2013;8:493–9. doi:10.2147/COPD.S49609

53 Moy ML, Teylan M, Danilack VA, et al. An index of daily step count and systemic inflammation predicts clinical outcomes in chronic obstructive pulmonary disease. Ann Am Thorac Soc 2014;11:149–57. doi:10.1513/AnnalsATS.201307-243OC

54 Bowler RP, Kim V, Regan E, et al. Prediction of acute respiratory disease in current and former smokers with and without COPD. Chest 2014;146:941–50. doi:10.1378/chest.13-2946

55 Koskela J, Kilpeläinen M, Kupiainen H, et al. Co-morbidities are the key nominators of the health related quality of life in mild and moderate COPD. BMC Pulm Med 2014;14:102. doi:10.1186/1471-2466-14-102

56 Stolz D, Meyer A, Rakic J, et al. Mortality risk prediction in COPD by a prognostic biomarker panel. Eur Respir J 2014;44:1557–70. doi:10.1183/09031936.00043814

57 Boutou AK, Nair A, Douraghi-Zadeh D, et al. A combined pulmonary function and emphysema score prognostic index for staging in Chronic Obstructive Pulmonary Disease. PLoS One 2014;9:e111109.

Page 70 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


doi:10.1371/journal.pone.0111109

58 Suzuki M, Makita H, Östling J, et al. Lower leptin/adiponectin ratio and risk of rapid lung function decline in chronic obstructive pulmonary disease. Ann Am Thorac Soc 2014;11:1511–9. doi:10.1513/AnnalsATS.201408-351OC

59 Make BJ, Eriksson G, Calverley PM, et al. A score to predict short-term risk of COPD exacerbations (SCOPEX). Int J Chron Obstruct Pulmon Dis 2015;10:201–9. doi:10.2147/COPD.S69589

60 Wilson R, Anzueto A, Miravitlles M, et al. Prognostic factors for clinical failure of exacerbations in elderly outpatients with moderate-to-severe COPD. Int J Chron Obstruct Pulmon Dis 2015;10:985–93. doi:10.2147/COPD.S80926

61 Montserrat-Capdevila J, Godoy P, Marsal JR, et al. Predictive Model of Hospital Admission for COPD Exacerbation. Respir Care 2015;60:1288–94. doi:10.4187/respcare.04005

62 Shin TR, Oh Y-M, Park JH, et al. The Prognostic Value of Residual Volume/Total Lung Capacity in Patients with Chronic Obstructive Pulmonary Disease. J Korean Med Sci 2015;30:1459–65. doi:10.3346/jkms.2015.30.10.1459

63 Abascal-Bolado B, Novotny PJ, Sloan JA, et al. Forecasting COPD hospitalization in the clinic: optimizing the chronic respiratory questionnaire. Int J Chron Obstruct Pulmon Dis 2015;10:2295–301. doi:10.2147/COPD.S87469

64 Kerkhof M, Freeman D, Jones R, et al. Predicting frequent COPD exacerbations using primary care data. Int J Chron Obstruct Pulmon Dis 2015;10:2439–50. doi:10.2147/COPD.S94259

65 Montserrat-Capdevila J, Godoy P, Marsal JR, et al. Risk of exacerbation in chronic obstructive pulmonary disease: a primary care retrospective cohort study. BMC Fam Pract 2015;16:173. doi:10.1186/s12875-015-0387-6

66 Miravitlles M, García-Sidro P, Fernández-Nistal A, et al. The chronic obstructive pulmonary disease assessment test improves the predictive value of previous exacerbations for poor outcomes in COPD. Int J Chron Obstruct Pulmon Dis 2015;10:2571–9. doi:10.2147/COPD.S91163

67 Ramon MA, Ferrer J, Gimeno-Santos E, et al. Inspiratory capacity-to-total lung capacity ratio and dyspnoea predict exercise capacity decline in COPD. Respirology 2016;21:476–82. doi:10.1111/resp.12723

68 Park HY, Lee H, Koh W-J, et al. Association of blood eosinophils and plasma periostin with FEV1 response after 3-month inhaled corticosteroid and long-acting beta2-agonist treatment in stable COPD patients. Int J Chron Obstruct Pulmon Dis 2016;11:23–30. doi:10.2147/COPD.S94797

69 Blumenthal JA, Smith PJ, Durheim M, et al. Biobehavioral Prognostic Factors in Chronic Obstructive Pulmonary Disease. Psychosom Med 2016;78:153–62. doi:10.1097/PSY.0000000000000260

70 Beijers RJHCG, van den Borst B, Newman AB, et al. A Multidimensional Risk Score to Predict All-Cause Hospitalization in Community-Dwelling Older Individuals With Obstructive Lung Disease. J Am Med Dir Assoc 2016;17:508–13. doi:10.1016/j.jamda.2016.01.007

71 Boeck L, Soriano JB, Brusse-Keizer M, et al. Prognostic assessment in COPD without lung function: the B-AE-D indices. Eur Respir J 2016;47:1635–44. doi:10.1183/13993003.01485-2015

72 Horita N, Koblizek V, Plutinsky M, et al. Chronic obstructive pulmonary disease prognostic score: A new index. Biomed Pap Med Fac Univ Palacky Olomouc Czech Repub 2016;160:211–8. doi:10.5507/bp.2016.030

73 Zafari Z, Sin DD, Postma DS, et al. Individualized prediction of lung-function decline in chronic obstructive pulmonary disease. CMAJ 2016;188:1004–11. doi:10.1503/cmaj.151483

74 Chan HP, Mukhopadhyay A, Chong PLP, et al. Role of BMI, airflow obstruction, St George’s Respiratory Questionnaire and age index in prognostication of Asian COPD. Respirology 2017;22:114–9. doi:10.1111/resp.12877

75 Sand JMB, Leeming DJ, Byrjalsen I, et al. High levels of biomarkers of collagen remodeling are associated

Page 71 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


with increased mortality in COPD - results from the ECLIPSE study. Respir Res 2016;17:125. doi:10.1186/s12931-016-0440-6

76 Ansari K, Keaney N, Kay A, et al. Body mass index, airflow obstruction and dyspnea and body mass index, airflow obstruction, dyspnea scores, age and pack years-predictive properties of new multidimensional prognostic indices of chronic obstructive pulmonary disease in primary care. Ann Thorac Med 2016;11:261. doi:10.4103/1817-1737.191866

77 Barton CA, Bassett KL, Buckman J, et al. The predictive value of an adjusted COPD assessment test score on the risk of respiratory-related hospitalizations in severe COPD patients. Chron Respir Dis 2017;14:72–84. doi:10.1177/1479972316687099

78 Crook S, Frei A, Ter Riet G, et al. Prediction of long-term clinical outcomes using simple functional exercise performance tests in patients with COPD: a 5-year prospective cohort study. Respir Res 2017;18:112. doi:10.1186/s12931-017-0598-6

79 Russo AN, Sathiyamoorthy G, Lau C, et al. Impact of a Post-Discharge Integrated Disease Management Program on COPD Hospital Readmissions. Respir Care 2017;62:1396–402. doi:10.4187/respcare.05547

80 Sundh J, Ekström M. Risk factors for developing hypoxic respiratory failure in COPD. Int J Chron Obstruct Pulmon Dis 2017;12:2095–100. doi:10.2147/COPD.S140299

81 Kurashima K, Takaku Y, Nakamoto K, et al. Risk Factors for Pneumonia and the Effect of the Pneumococcal Vaccine in Patients With Chronic Airflow Obstruction. Chronic Obstr Pulm Dis (Miami, Fla) 2016;3:610–9. doi:10.15326/jcopdf.3.3.2015.0167

82 Villalobos N, Davidson R, Ghori UK, et al. External Validation of the COmorbidity Test. COPD 2017;14:513–7. doi:10.1080/15412555.2017.1354981

83 Strassmann A, Frei A, Haile SR, et al. Commonly Used Patient-Reported Outcomes Do Not Improve Prediction of COPD Exacerbations: A Multicenter 4½ Year Prospective Cohort Study. Chest 2017;152:1179–87. doi:10.1016/j.chest.2017.09.003

84 Dal Negro RW, Celli BR. Patient Related Outcomes-BODE (PRO-BODE): A composite index incorporating health utilization resources predicts mortality and economic cost of COPD in real life. Respir Med 2017;131:175–8. doi:10.1016/j.rmed.2017.08.019

85 Hoogendoorn M, Feenstra TL, Boland M, et al. Prediction models for exacerbations in different COPD patient populations: comparing results of five large data sources. Int J Chron Obstruct Pulmon Dis 2017;12:3183–94. doi:10.2147/COPD.S142378

86 Mendy A, Forno E, Niyonsenga T, et al. Blood biomarkers as predictors of long-term mortality in COPD. Clin Respir J 2018;12:1891–9. doi:10.1111/crj.12752

87 Kang J, Kim KT, Lee J-H, et al. Predicting treatable traits for long-acting bronchodilators in patients with stable COPD. Int J Chron Obstruct Pulmon Dis 2017;12:3557–65. doi:10.2147/COPD.S151909

88 Fijačko V, Labor M, Fijačko M, et al. Predictors of short-term LAMA ineffectiveness in treatment naïve patients with moderate to severe COPD. Wien Klin Wochenschr 2018;130:247–58. doi:10.1007/s00508-017-1307-7

89 Bafadhel M, Peterson S, De Blas MA, et al. Predictors of exacerbation risk and response to budesonide in patients with chronic obstructive pulmonary disease: a post-hoc analysis of three randomised trials. Lancet Respir Med 2018;6:117–26. doi:10.1016/S2213-2600(18)30006-7

90 Stanford RH, Nag A, Mapel DW, et al. Claims-based risk model for first severe COPD exacerbation. Am J Manag Care 2018;24:e45–53.

91 Samp JC, Joo MJ, Schumock GT, et al. Predicting Acute Exacerbations in Chronic Obstructive Pulmonary Disease. J Manag care Spec Pharm 2018;24:265–79. doi:10.18553/jmcp.2018.24.3.265

92 Miravitlles M, Sliwinski P, Rhee CK, et al. Evaluation of criteria for clinical control in a prospective, international, multicenter study of patients with COPD. Respir Med 2018;136:8–14. doi:10.1016/j.rmed.2018.01.019

Page 72 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


93 Vela E, Tényi Á, Cano I, et al. Population-based analysis of patients with COPD in Catalonia: a cohort study with implications for clinical management. BMJ Open 2018;8:e017283. doi:10.1136/bmjopen-2017-017283

94 Morales DR, Flynn R, Zhang J, et al. External validation of ADO, DOSE, COTE and CODEX at predicting death in primary care patients with COPD using standard and machine learning approaches. Respir Med 2018;138:150–5. doi:10.1016/j.rmed.2018.04.003

95 Hwang J, Oh Y-M, Lee M, et al. Low morphometric complexity of emphysematous lesions predicts survival in chronic obstructive pulmonary disease patients. Eur Radiol 2019;29:176–85. doi:10.1007/s00330-018-5551-7

96 Annavarapu S, Goldfarb S, Gelb M, et al. Development and validation of a predictive model to identify patients at risk of severe COPD exacerbations using administrative claims data. Int J Chron Obstruct Pulmon Dis 2018;13:2121–30. doi:10.2147/COPD.S155773

97 Menzies R, Gibbons W, Goldberg P. Determinants of weaning and survival among patients with COPD who require mechanical ventilation for acute respiratory failure. Chest 1989;95:398–405.

98 Fuso L, Incalzi RA, Pistelli R, et al. Predicting mortality of patients hospitalized for acutely exacerbated chronic obstructive pulmonary disease. Am J Med 1995;98:272–7.

99 Nava S, Rubini F, Zanotti E, et al. Survival and prediction of successful ventilator weaning in COPD patients requiring mechanical ventilation for more than 21 days. Eur Respir J 1994;7:1645–52.

100 Dubois P, Jamart J, Machiels J, et al. Prognosis of severely hypoxemic patients receiving long-term oxygen therapy. Chest 1994;105:469–74.

101 Rieves RD, Bass D, Carter RR, et al. Severe COPD and acute respiratory failure. Correlates for survival at the time of tracheal intubation. Chest 1993;104:854–60.

102 Vitacca M, Clini E, Porta R, et al. Acute exacerbations in patients with COPD: predictors of need for mechanical ventilation. Eur Respir J 1996;9:1487–93.

103 Connors AF, Dawson N V, Thomas C, et al. Outcomes following acute exacerbation of severe chronic obstructive lung disease. The SUPPORT investigators (Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments). Am J Respir Crit Care Med 1996;154:959–67. doi:10.1164/ajrccm.154.4.8887592

104 Szekely LA, Oelberg DA, Wright C, et al. Preoperative predictors of operative morbidity and mortality in COPD patients undergoing bilateral lung volume reduction surgery. Chest 1997;111:550–8.

105 Antonelli Incalzi R, Fuso L, De Rosa M, et al. Co-morbidity contributes to predict mortality of patients with chronic obstructive pulmonary disease. Eur Respir J 1997;10:2794–800.

106 Antón A, Güell R, Gómez J, et al. Predicting the result of noninvasive ventilation in severe acute exacerbations of patients with chronic airflow limitation. Chest 2000;117:828–33.

107 Putinati S, Ballerin L, Piattella M, et al. Is it possible to predict the success of non-invasive positive pressure ventilation in acute respiratory failure due to COPD? Respir Med 2000;94:997–1001. doi:10.1053/rmed.2000.0883

108 Incalzi RA, Pedone C, Onder G, et al. Predicting length of stay of older patients with exacerbated chronic obstructive pulmonary disease. Aging (Milano) 2001;13:49–57.

109 Roberts CM, Lowe D, Bucknall CE, et al. Clinical audit indicators of outcome following admission to hospital with acute exacerbation of chronic obstructive pulmonary disease. Thorax 2002;57:137–41.

110 Patil SP, Krishnan JA, Lechtzin N, et al. In-hospital mortality following acute exacerbations of chronic obstructive pulmonary disease. Arch Intern Med 2003;163:1180–6. doi:10.1001/archinte.163.10.1180

111 Goel A, Pinckney RG, Littenberg B. APACHE II predicts long-term survival in COPD patients admitted to a general medical ward. J Gen Intern Med 2003;18:824–30.

112 Scala R, Bartolucci S, Naldi M, et al. Co-morbidity and acute decompensations of COPD requiring non-

Page 73 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


invasive positive-pressure ventilation. Intensive Care Med 2004;30:1747–54. doi:10.1007/s00134-004-2368-4

113 Khilnani GC, Banga A, Sharma SK. Predictors of mortality of patients with acute respiratory failure secondary to chronic obstructive pulmonary disease admitted to an intensive care unit: A one year study. BMC Pulm Med 2004;4:12. doi:10.1186/1471-2466-4-12

114 Confalonieri M, Garuti G, Cattaruzza MS, et al. A chart of failure risk for noninvasive ventilation in patients with COPD exacerbation. Eur Respir J 2005;25:348–55. doi:10.1183/09031936.05.00085304

115 Ucgun I, Metintas M, Moral H, et al. Predictors of hospital outcome and intubation in COPD patients admitted to the respiratory ICU for acute hypercapnic respiratory failure. Respir Med 2006;100:66–74. doi:10.1016/j.rmed.2005.04.005

116 Yohannes AM, Baldwin RC, Connolly MJ. Predictors of 1-year mortality in patients discharged from hospital following acute exacerbation of chronic obstructive pulmonary disease. Age Ageing 2005;34:491–6. doi:10.1093/ageing/afi163

117 Almagro P, Barreiro B, Ochoa de Echaguen A, et al. Risk factors for hospital readmission in patients with chronic obstructive pulmonary disease. Respiration 2006;73:311–7. doi:10.1159/000088092

118 Chen Y-J, Narsavage GL. Factors related to chronic obstructive pulmonary disease readmission in Taiwan. West J Nurs Res 2006;28:105–24. doi:10.1177/0193945905282354

119 Wildman MJ, Harrison DA, Welch CA, et al. A new measure of acute physiological derangement for patients with exacerbations of obstructive airways disease: the COPD and Asthma Physiology Score. Respir Med 2007;101:1994–2002. doi:10.1016/j.rmed.2007.04.002

120 Liu H, Zhang T, Ye J. Determinants of prolonged mechanical ventilation in patients with chronic obstructive pulmonary diseases and acute hypercapnic respiratory failure. Eur J Intern Med 2007;18:542–7. doi:10.1016/j.ejim.2007.04.017

121 Ruiz-González A, Lacasta D, Ibarz M, et al. C-reactive protein and other predictors of poor outcome in patients hospitalized with exacerbations of chronic obstructive pulmonary disease. Respirology 2008;13:1028–33. doi:10.1111/j.1440-1843.2008.01403.x

122 Mohan A, Bhatt SP, Mohan C, et al. Derivation of a prognostic equation to predict in-hospital mortality and requirement of invasive mechanical ventilation in patients with acute exacerbation of chronic obstructive pulmonary disease. Indian J Chest Dis Allied Sci 2008;50:335–42.

123 Wildman MJ, Sanderson C, Groves J, et al. Predicting mortality for patients with exacerbations of COPD and Asthma in the COPD and Asthma Outcome Study (CAOS). QJM 2009;102:389–99. doi:10.1093/qjmed/hcp036

124 Chakrabarti B, Angus RM, Agarwal S, et al. Hyperglycaemia as a predictor of outcome during non-invasive ventilation in decompensated COPD. Thorax 2009;64:857–62. doi:10.1136/thx.2008.106989

125 Tsimogianni AM, Papiris SA, Stathopoulos GT, et al. Predictors of outcome after exacerbation of chronic obstructive pulmonary disease. J Gen Intern Med 2009;24:1043–8. doi:10.1007/s11606-009-1061-2

126 Tabak YP, Sun X, Johannes RS, et al. Mortality and need for mechanical ventilation in acute exacerbations of chronic obstructive pulmonary disease: development and validation of a simple risk score. Arch Intern Med 2009;169:1595–602. doi:10.1001/archinternmed.2009.270

127 Terzano C, Conti V, Di Stefano F, et al. Comorbidity, hospitalization, and mortality in COPD: results from a longitudinal study. Lung 2010;188:321–9. doi:10.1007/s00408-009-9222-y

128 Roca B, Almagro P, López F, et al. Factors associated with mortality in patients with exacerbation of chronic obstructive pulmonary disease hospitalized in General Medicine departments. Intern Emerg Med 2011;6:47–54. doi:10.1007/s11739-010-0465-7

129 Aburto M, Esteban C, Moraza FJ, et al. COPD exacerbation: mortality prognosis factors in a respiratory care unit. Arch Bronconeumol 2011;47:79–84. doi:10.1016/j.arbres.2010.10.012

Page 74 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


130 Asiimwe AC, Brims FJH, Andrews NP, et al. Routine laboratory tests can predict in-hospital mortality in acute exacerbations of COPD. Lung 2011;189:225–32. doi:10.1007/s00408-011-9298-z

131 Abrams TE, Vaughan-Sarrazin M, Vander Weg MW. Acute Exacerbations of Chronic Obstructive Pulmonary Disease and the Effect of Existing Psychiatric Comorbidity on Subsequent Mortality. Psychosomatics 2011;52:441–9. doi:10.1016/j.psym.2011.03.005

132 Matkovic Z, Huerta A, Soler N, et al. Predictors of adverse outcome in patients hospitalised for exacerbation of chronic obstructive pulmonary disease. Respiration 2012;84:17–26. doi:10.1159/000335467

133 Steer J, Gibson J, Bourke SC. The DECAF Score: predicting hospital mortality in exacerbations of chronic obstructive pulmonary disease. Thorax 2012;67:970–6. doi:10.1136/thoraxjnl-2012-202103

134 Slenter RHJ, Sprooten RTM, Kotz D, et al. Predictors of 1-year mortality at hospital admission for acute exacerbations of chronic obstructive pulmonary disease. Respiration 2013;85:15–26. doi:10.1159/000342036

135 Jurado Gámez B, Feu Collado N, Jurado García JC, et al. Home intervention and predictor variables for rehospitalization in chronic obstructive pulmonary disease exacerbations. Arch Bronconeumol 2013;49:10–4. doi:10.1016/j.arbres.2012.08.003

136 Tabak YP, Sun X, Johannes RS, et al. Development and validation of a mortality risk-adjustment model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med Care 2013;51:597–605. doi:10.1097/MLR.0b013e3182901982

137 Haja Mydin H, Murphy S, Clague H, et al. Anemia and performance status as prognostic markers in acute hypercapnic respiratory failure due to chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis 2013;8:151–7. doi:10.2147/COPD.S39403

138 Amalakuhan B, Kiljanek L, Parvathaneni A, et al. A prediction model for COPD readmissions: catching up, catching our breath, and improving a national problem. J Community Hosp Intern Med Perspect 2012;2:9915. doi:10.3402/jchimp.v2i1.9915

139 Lindenauer PK, Grosso LM, Wang C, et al. Development, validation, and results of a risk-standardized measure of hospital 30-day mortality for patients with exacerbation of chronic obstructive pulmonary disease. J Hosp Med 2013;8:428–35. doi:10.1002/jhm.2066

140 Almagro P, Soriano JB, Cabrera FJ, et al. Short- and medium-term prognosis in patients hospitalized for COPD exacerbation: the CODEX index. Chest 2014;145:972–80. doi:10.1378/chest.13-1328

141 Wang Y, Stavem K, Dahl FA, et al. Factors associated with a prolonged length of stay after acute exacerbation of chronic obstructive pulmonary disease (AECOPD). Int J Chron Obstruct Pulmon Dis 2014;9:99–105. doi:10.2147/COPD.S51467

142 Batzlaff CM, Karpman C, Afessa B, et al. Predicting 1-year mortality rate for patients admitted with an acute exacerbation of chronic obstructive pulmonary disease to an intensive care unit: an opportunity for palliative care. Mayo Clin Proc 2014;89:638–43. doi:10.1016/j.mayocp.2013.12.004

143 Sharif R, Parekh TM, Pierson KS, et al. Predictors of early readmission among patients 40 to 64 years of age hospitalized for chronic obstructive pulmonary disease. Ann Am Thorac Soc 2014;11:685–94. doi:10.1513/AnnalsATS.201310-358OC

144 Diamantea F, Kostikas K, Bartziokas K, et al. Prediction of hospitalization stay in COPD exacerbations: the AECOPD-F score. Respir Care 2014;59:1679–86. doi:10.4187/respcare.03171

145 Cheng Y, Borrego ME, Frost FJ, et al. Predictors for mortality in hospitalized patients with chronic obstructive pulmonary disease. Springerplus 2014;3:359. doi:10.1186/2193-1801-3-359

146 Roche N, Chavaillon J-M, Maurer C, et al. A clinical in-hospital prognostic score for acute exacerbations of COPD. Respir Res 2014;15:99. doi:10.1186/s12931-014-0099-9

147 Quintana JM, Esteban C, Garcia-Gutierrez S, et al. Predictors of hospital admission two months after emergency department evaluation of COPD exacerbation. Respiration 2014;88:298–306. doi:10.1159/000365996

Page 75 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


148 Grolimund E, Kutz A, Marlowe RJ, et al. Long-term Prognosis in COPD Exacerbation: Role of Biomarkers, Clinical Variables and Exacerbation Type. COPD 2015;12:295–305. doi:10.3109/15412555.2014.949002

149 Crisafulli E, Torres A, Huerta A, et al. C-Reactive Protein at Discharge, Diabetes Mellitus and ≥ 1 Hospitalization During Previous Year Predict Early Readmission in Patients with Acute Exacerbation of Chronic Obstructive Pulmonary Disease. COPD 2015;12:306–14. doi:10.3109/15412555.2014.933954

150 Fan L, Zhao Q, Liu Y, et al. Semiquantitative cough strength score and associated outcomes in noninvasive positive pressure ventilation patients with acute exacerbation of chronic obstructive pulmonary disease. Respir Med 2014;108:1801–7. doi:10.1016/j.rmed.2014.10.001

151 Quintana JM, Unzurrunzaga A, Garcia-Gutierrez S, et al. Predictors of Hospital Length of Stay in Patients with Exacerbations of COPD: A Cohort Study. J Gen Intern Med 2015;30:824–31. doi:10.1007/s11606-014-3129-x

152 Quintana JM, Esteban C, Unzurrunzaga A, et al. Prognostic severity scores for patients with COPD exacerbations attending emergency departments. Int J Tuberc Lung Dis 2014;18:1415–20. doi:10.5588/ijtld.14.0312

153 Yu T-C, Zhou H, Suh K, et al. Assessing the importance of predictors in unplanned hospital readmissions for chronic obstructive pulmonary disease. Clinicoecon Outcomes Res 2015;7:37–51. doi:10.2147/CEOR.S74181

154 Roberts M, Mapel D, Von Worley A, et al. Clinical factors, including All Patient Refined Diagnosis Related Group severity, as predictors of early rehospitalization after COPD exacerbation. Drugs Context 2015;4:1–15. doi:10.7573/dic.212278

155 Liu D, Peng S-H, Zhang J, et al. Prediction of short term re-exacerbation in patients with acute exacerbation of chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis 2015;10:1265–73. doi:10.2147/COPD.S83378

156 Kon SSC, Jones SE, Schofield SJ, et al. Gait speed and readmission following hospitalisation for acute exacerbations of COPD: a prospective study. Thorax 2015;70:1131–7. doi:10.1136/thoraxjnl-2015-207046

157 Glaser JB, El-Haddad H. Exploring Novel Medicare Readmission Risk Variables in Chronic Obstructive Pulmonary Disease Patients at High Risk of Readmission within 30 Days of Hospital Discharge. Ann Am Thorac Soc 2015;12:1288–93. doi:10.1513/AnnalsATS.201504-228OC

158 Crisafulli E, Torres A, Huerta A, et al. Predicting In-Hospital Treatment Failure (≤ 7 days) in Patients with COPD Exacerbation Using Antibiotics and Systemic Steroids. COPD 2016;13:82–92. doi:10.3109/15412555.2015.1057276

159 García-Sidro P, Naval E, Martinez Rivera C, et al. The CAT (COPD Assessment Test) questionnaire as a predictor of the evolution of severe COPD exacerbations. Respir Med 2015;109:1546–52. doi:10.1016/j.rmed.2015.10.011

160 Ramaraju K, Kaza AM, Balasubramanian N, et al. Predicting Healthcare Utilization by Patients Admitted for COPD Exacerbation. J Clin Diagn Res 2016;10:OC13-7. doi:10.7860/JCDR/2016/17721.7216

161 Sainaghi PP, Colombo D, Re A, et al. Natural history and risk stratification of patients undergoing non-invasive ventilation in a non-ICU setting for severe COPD exacerbations. Intern Emerg Med 2016;11:969–75. doi:10.1007/s11739-016-1473-z

162 He H, Sun Y, Sun B, et al. Application of a parametric model in the mortality risk analysis of ICU patients with severe COPD. Clin Respir J 2018;12:491–8. doi:10.1111/crj.12549

163 García-Rivero JL, Esquinas C, Barrecheguren M, et al. Risk Factors of Poor Outcomes after Admission for a COPD Exacerbation: Multivariate Logistic Predictive Models. COPD 2017;14:164–9. doi:10.1080/15412555.2016.1260538

164 Echevarria C, Steer J, Heslop-Marshall K, et al. The PEARL score predicts 90-day readmission or death after hospitalisation for acute exacerbation of COPD. Thorax 2017;72:686–93. doi:10.1136/thoraxjnl-2016-209298

Page 76 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


165 Sakamoto Y, Yamauchi Y, Yasunaga H, et al. Development of a nomogram for predicting in-hospital mortality of patients with exacerbation of chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis 2017;12:1605–11. doi:10.2147/COPD.S129714

166 Feng Z, Wang T, Liu P, et al. Efficacy of Various Scoring Systems for Predicting the 28-Day Survival Rate among Patients with Acute Exacerbation of Chronic Obstructive Pulmonary Disease Requiring Emergency Intensive Care. Can Respir J 2017;2017:3063510. doi:10.1155/2017/3063510

167 Almagro P, Yun S, Sangil A, et al. Palliative care and prognosis in COPD: a systematic review with a validation cohort. Int J Chron Obstruct Pulmon Dis 2017;12:1721–9. doi:10.2147/COPD.S135657

168 Lau CS, Siracuse BL, Chamberlain RS. Readmission After COPD Exacerbation Scale: determining 30-day readmission risk for COPD patients. Int J Chron Obstruct Pulmon Dis 2017;12:1891–902. doi:10.2147/COPD.S136768

169 Vanasse A, Courteau J, Couillard S, et al. Predicting One-year Mortality After a ‘First’ Hospitalization for Chronic Obstructive Pulmonary Disease: An Eight-Variable Assessment Score Tool. COPD 2017;14:490–7. doi:10.1080/15412555.2017.1343814

170 Duenk RG, Verhagen C, Bronkhorst EM, et al. Development of the ProPal-COPD tool to identify patients with COPD for proactive palliative care. Int J Chron Obstruct Pulmon Dis 2017;12:2121–8. doi:10.2147/COPD.S140037

171 Yao C, Liu X, Tang Z. Prognostic role of neutrophil-lymphocyte ratio and platelet-lymphocyte ratio for hospital mortality in patients with AECOPD. Int J Chron Obstruct Pulmon Dis 2017;12:2285–90. doi:10.2147/COPD.S141760

172 Bernabeu-Mora R, García-Guillamón G, Valera-Novella E, et al. Frailty is a predictive factor of readmission within 90 days of hospitalization for acute exacerbations of chronic obstructive pulmonary disease: a longitudinal study. Ther Adv Respir Dis 2017;11:383–92. doi:10.1177/1753465817726314

173 Winther JA, Brynildsen J, Høiseth AD, et al. Prognostic and diagnostic significance of copeptin in acute exacerbation of chronic obstructive pulmonary disease and acute heart failure: data from the ACE 2 study. Respir Res 2017;18:184. doi:10.1186/s12931-017-0665-z

174 Zhong X, Lee S, Zhao C, et al. Reducing COPD readmissions through predictive modeling and incentive-based interventions. Health Care Manag Sci 2019;22:121–39. doi:10.1007/s10729-017-9426-2

175 Pavliša G, Labor M, Puretić H, et al. Anemia, hypoalbuminemia, and elevated troponin levels as risk factors for respiratory failure in patients with severe exacerbations of chronic obstructive pulmonary disease requiring invasive mechanical ventilation. Croat Med J 2017;58:395–405.

176 Rezaee ME, Ward CE, Nuanez B, et al. Examining 30-day COPD readmissions through the emergency department. Int J Chron Obstruct Pulmon Dis 2018;13:109–20. doi:10.2147/COPD.S147796

177 Hakim MA, Garden FL, Jennings MD, et al. Performance of the LACE index to predict 30-day hospital readmissions in patients with chronic obstructive pulmonary disease. Clin Epidemiol 2018;10:51–9. doi:10.2147/CLEP.S149574

178 Esteban C, Castro-Acosta A, Alvarez-Martínez CJ, et al. Predictors of one-year mortality after hospitalization for an exacerbation of COPD. BMC Pulm Med 2018;18:18. doi:10.1186/s12890-018-0574-z

179 Serra-Picamal X, Roman R, Escarrabill J, et al. Hospitalizations due to exacerbations of COPD: A big data perspective. Respir Med 2018;145:219–25. doi:10.1016/j.rmed.2018.01.008

180 Cerezo Lajas A, Gutiérrez González E, Llorente Parrado C, et al. Readmission Due to Exacerbation of COPD: Associated Factors. Lung 2018;196:185–93. doi:10.1007/s00408-018-0093-y

181 Epstein D, Nasser R, Mashiach T, et al. Increased red cell distribution width: A novel predictor of adverse outcome in patients hospitalized due to acute exacerbation of chronic obstructive pulmonary disease. Respir Med 2018;136:1–7. doi:10.1016/j.rmed.2018.01.011

182 Botle A, Honeyford K, Chowdhury F, et al. Factors associated with hospital emergency readmission and mortality rates in patients with heart failure or chronic obstructive pulmonary disease: a national

Page 77 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


observational study. Southampton: : Health Services and Delivery Research 2018.

183 Schuler M, Wittmann M, Faller H, et al. Including changes in dyspnea after inpatient rehabilitation improves prediction models of exacerbations in COPD. Respir Med 2018;141:87–93. doi:10.1016/j.rmed.2018.06.027

184 Madkour AM, Adly NN. Predictors of in-hospital mortality and need for invasive mechanical ventilation in elderly COPD patients presenting with acute hypercapnic respiratory failure. Egypt J Chest Dis Tuberc 2013;62:393–400. doi:10.1016/j.ejcdt.2013.07.003

185 Murata GH, Gorby MS, Kapsner CO, et al. A multivariate model for the prediction of relapse after outpatient treatment of decompensated chronic obstructive pulmonary disease. Arch Intern Med 1992;152:73–7.

186 Kim S, Clark S, Camargo CA. Mortality after an emergency department visit for exacerbation of chronic obstructive pulmonary disease. COPD 2006;3:75–81.

187 Roche N, Zureik M, Soussan D, et al. Predictors of outcomes in COPD exacerbation cases presenting to the emergency department. Eur Respir J 2008;32:953–61. doi:10.1183/09031936.00129507

188 Stiell IG, Clement CM, Aaron SD, et al. Clinical characteristics associated with adverse events in patients with exacerbation of chronic obstructive pulmonary disease: a prospective cohort study. CMAJ 2014;186:E193-204. doi:10.1503/cmaj.130968

189 Quintana JM, Esteban C, Unzurrunzaga A, et al. Predictive score for mortality in patients with COPD exacerbations attending hospital emergency departments. BMC Med 2014;12:66. doi:10.1186/1741-7015-12-66

190 Esteban C, Arostegui I, Garcia-Gutierrez S, et al. A decision tree to assess short-term mortality after an emergency department visit for an exacerbation of COPD: a cohort study. Respir Res 2015;16:151. doi:10.1186/s12931-015-0313-4

191 Esteban C, Garcia-Gutierrez S, Legarreta MJ, et al. One-year Mortality in COPD After an Exacerbation: The Effect of Physical Activity Changes During the Event. COPD 2016;13:718–25. doi:10.1080/15412555.2016.1188903

192 Liu H, Zhang T, Ye J. Analysis of risk factors for hospital mortality in patients with chronic obstructive pulmonary diseases requiring invasive mechanical ventilation. Chin Med J (Engl) 2007;120:287–93.

193 Afessa B, Morales IJ, Scanlon PD, et al. Prognostic factors, clinical course, and hospital outcome of patients with chronic obstructive pulmonary disease admitted to an intensive care unit for acute respiratory failure. Crit Care Med 2002;30:1610–5.

194 Xiao K, Guo C, Su L, et al. Prognostic value of different scoring models in patients with multiple organ dysfunction syndrome associated with acute COPD exacerbation. J Thorac Dis 2015;7:329–36. doi:10.3978/j.issn.2072-1439.2014.11.27

195 Conti V, Paone G, Mollica C, et al. Predictors of outcome for patients with severe respiratory failure requiring non invasive mechanical ventilation. Eur Rev Med Pharmacol Sci 2015;19:3855–60.

196 Echevarria C, Steer J, Heslop-Marshall K, et al. Validation of the DECAF score to predict hospital mortality in acute exacerbations of COPD. Thorax 2016;71:133–40. doi:10.1136/thoraxjnl-2015-207775

197 Ferrer M, Ioanas M, Arancibia F, et al. Microbial airway colonization is associated with noninvasive ventilation failure in exacerbation of chronic obstructive pulmonary disease. Crit Care Med 2005;33:2003–9.

198 Hu W-S, Lin C-L. Use of CHA2DS2-VASc Score to Predict New-Onset Atrial Fibrillation in Chronic Obstructive Pulmonary Disease Patients - Large-Scale Longitudinal Study. Circ J 2017;81:1792–7. doi:10.1253/circj.CJ-17-0130

199 Ooi H, Chen L-H, Ni Y-L, et al. CHA2DS2-VASc scores predict major adverse cardiovascular events in patients with chronic obstructive pulmonary disease. Clin Respir J 2018;12:1038–45. doi:10.1111/crj.12624

Page 78 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


200 Austin PC, Stanbrook MB, Anderson GM, et al. Comparative ability of comorbidity classification methods for administrative data to predict outcomes in patients with chronic obstructive pulmonary disease. Ann Epidemiol 2012;22:881–7. doi:10.1016/j.annepidem.2012.09.011

201 Golpe R, Mengual-Macenlle N, Sanjuán-López P, et al. Prognostic Indices and Mortality Prediction in COPD Caused by Biomass Smoke Exposure. Lung 2015;193:497–503. doi:10.1007/s00408-015-9731-9

202 Edwards L, Perrin K, Wijesinghe M, et al. The value of the CRB65 score to predict mortality in exacerbations of COPD requiring hospital admission. Respirology 2011;16:625–9. doi:10.1111/j.1440-1843.2011.01926.x

203 Hodgson LE, Dimitrov BD, Congleton J, et al. A validation of the National Early Warning Score to predict outcome in patients with COPD exacerbation. Thorax 2017;72:23–30. doi:10.1136/thoraxjnl-2016-208436

204 Chang CL, Sullivan GD, Karalus NC, et al. Predicting early mortality in acute exacerbation of chronic obstructive pulmonary disease using the CURB65 score. Respirology 2011;16:146–51. doi:10.1111/j.1440-1843.2010.01866.x

205 Hu G, Zhou Y, Wu Y, et al. The Pneumonia Severity Index as a Predictor of In-Hospital Mortality in Acute Exacerbation of Chronic Obstructive Pulmonary Disease. PLoS One 2015;10:e0133160. doi:10.1371/journal.pone.0133160

206 Steer J, Norman EM, Afolabi OA, et al. Dyspnoea severity and pneumonia as predictors of in-hospital mortality and early readmission in acute exacerbations of COPD. Thorax 2012;67:117–21. doi:10.1136/thoraxjnl-2011-200332

207 Shorr AF, Sun X, Johannes RS, et al. Predicting the need for mechanical ventilation in acute exacerbations of chronic obstructive pulmonary disease: comparing the CURB-65 and BAP-65 scores. J Crit Care 2012;27:564–70. doi:10.1016/j.jcrc.2012.02.015

208 Rothnie KJ, Smeeth L, Pearce N, et al. Predicting mortality after acute coronary syndromes in people with chronic obstructive pulmonary disease. Heart 2016;102:1442–8. doi:10.1136/heartjnl-2016-309359

209 Burke RE, Schnipper JL, Williams M V, et al. The HOSPITAL Score Predicts Potentially Preventable 30-Day Readmissions in Conditions Targeted by the Hospital Readmissions Reduction Program. Med Care 2017;55:285–90. doi:10.1097/MLR.0000000000000665

210 Pedone C, Scarlata S, Forastiere F, et al. BODE index or geriatric multidimensional assessment for the prediction of very-long-term mortality in elderly patients with chronic obstructive pulmonary disease? a prospective cohort study. Age Ageing 2014;43:553–8. doi:10.1093/ageing/aft197

211 Chen R, Xing L, You C, et al. Prediction of prognosis in chronic obstructive pulmonary disease patients with respiratory failure: A comparison of three nutritional assessment methods. Eur J Intern Med 2018;57:70–5. doi:10.1016/j.ejim.2018.06.006

Page 79 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


Page 80 of 80


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

· Confidential: For Review Only A systematic review and critical appraisal of prognostic models...

Documents

Transcript of · Confidential: For Review Only A systematic review and critical appraisal of prognostic models...