A decision tree to assess short-term mortality after an emergency department visit for an exacerbation of COPD: a cohort study
© Esteban et al. 2015
Received: 11 June 2015
Accepted: 9 December 2015
Published: 22 December 2015
Creating an easy-to-use instrument to identify predictors of short-term (30/60-day) mortality after an exacerbation of chronic obstructive pulmonary disease (eCOPD) could help clinicians choose specific measures of medical care to decrease mortality in these patients. The objective of this study was to develop and validate a classification and regression tree (CART) to predict short term mortality among patients evaluated in an emergency department (ED) for an eCOPD.
We conducted a prospective cohort study including participants from 16 hospitals in Spain. COPD patients with an exacerbation attending the emergency department (ED) of any of the hospitals between June 2008 and September 2010 were recruited. Patients were randomly divided into derivation (50 %) and validation samples (50 %). A CART based on a recursive partitioning algorithm was created in the derivation sample and applied to the validation sample.
Two thousand four hundred eighty-seven patients, 1252 patients in the derivation sample and 1235 in the validation sample, were enrolled in the study. Based on the results of the univariate analysis, five variables (baseline dyspnea, cardiac disease, the presence of paradoxical breathing or use of accessory inspiratory muscles, age, and Glasgow Coma Scale score) were used to build the CART. Mortality rates 30 days after discharge ranged from 0 % to 55 % in the five CART classes. The lowest mortality rate was for the branch composed of low baseline dyspnea and lack of cardiac disease. The highest mortality rate was in the branch with the highest baseline dyspnea level, use of accessory inspiratory muscles or paradoxical breathing upon ED arrival, and Glasgow score <15. The area under the receiver-operating curve (AUC) in the derivation sample was 0.835 (95 % CI: 0.783, 0.888) and 0.794 (95 % CI: 0.723, 0.865) in the validation sample. CART was improved to predict 60-days mortality risk by adding the Charlson Comorbidity Index, reaching an AUC in the derivation sample of 0.817 (95 % CI: 0.776, 0.859) and 0.770 (95 % CI: 0.716, 0.823) in the validation sample.
We identified several easy-to-determine variables that allow clinicians to classify eCOPD patients by short term mortality risk, which can provide useful information for establishing appropriate clinical care.
Outcomes as long-term mortality and short-term mortality are frequently used in studies of mortality following hospitalization for an exacerbation of COPD (eCOPD) [1, 2]. In long-term follow-up, mortality is likely more influenced by the general characteristics of the patient’s COPD than the severity of the eCOPD that triggered the admission. Short-term mortality, usually defined as mortality occurring less than 90 days after presentation to a hospital with an eCOPD, usually includes in-hospital mortality, and factors related to the severity of the exacerbation likely play more important role in the short term rather than in the mid or long term follow-up.
CART models have been used in various disciplines and diseases [3, 4]. CART analysis has also been used to predict 5-year mortality among patients with stable COPD using five easily obtained parameters in clinical practice with good predictive capacity compared to other prognostic multidimensional based instruments . CART has also been used to predict short-term or long-term mortality and the need for mechanical ventilation among patients with acute exacerbations of COPD .
Many studies evaluating short-term outcomes in eCOPD patients have focused on the 90-day period following the episode. We focused on a shorter period after the index event (30/60 days) in an effort to find factors more closely related to the eCOPD event and differences in predictive factors for both mortality periods. Few studies have taken this approach, even though important outcomes such as mortality and readmissions occur during this period. The aim of our study was to develop and validate a CART to predict 30 and 60-days mortality following an emergency department (ED) evaluation for an eCOPD. Identifying factors related to mortality during this period could provide valuable information about the appropriate clinical care for these patients.
We used data collected as part of the Investigacion en Resultados y Servicios de Salud COPD Appropriateness (IRYSS-COPD) for this investigation. Methods of the IRYSS-COPD Study have been described in detail elsewhere . In brief, this prospective cohort study included patients with an eCOPD attending the emergency departments (ED) of 16 hospitals in Spain between June 2008 and September 2010. All patients were informed of the goals of the study and invited to voluntarily participate in it. Patients who agreed to participate provided written consent. All information was kept confidential.
Patients were eligible for the study if they presented to the ED with symptoms consistent of an eCOPD. COPD was confirmed if the patient had a forced expiratory volume in 1 second/forced vital capacity (FEV1/FVC) quotient <70 %. Exacerbation was defined as an event in the natural course of a patient’s COPD characterized by a change in baseline dyspnea, cough, and/or sputum that was beyond normal day-to-day variations and that may have warranted a change in medication or treatment . For patients already known to have been diagnosed of COPD, the closest sprirometry data to the ED index visit performed at a time when the patient was stable, and not longer than 6 months, was taken as reference. When COPD was newly diagnosed during the ED visit, to be included in the study, the patient had to have COPD confirmed by spirometry within 60 days after the index episode at a time when he or she was stable . In those cases, that spirometry was taken as reference. No spirometry data was recorded from the ED nor during the admission to the hospital.
Gold  patients were excluded from the study if, at the time they were seen in the ED, the eCOPD was complicated by a comorbidity such as pneumonia, pneumothorax, pulmonary embolism, lung cancer, or left cardiac failure. Other exclusion criteria included a diagnosis of asthma, extensive bronchiectasis, sequelae of tuberculosis, pleural thickening, or restrictive diseases. Patients who did not wish to participate were also excluded.
Data collected upon arrival in the ED included socioeconomic information, clinical data at baseline, presence of pathologies recorded in the Charlson Comorbidity Index , and information about the eCOPD event such as arterial blood gases, respiratory rate, dyspnea and consciousness level measured by the Glasgow Coma Scale .
For patients admitted to the hospital, we collected additional data from the medical record and from a direct interview with the patient on the first day after admission and also on the day of discharge. We asked all patients to tell us about their physical activity, general health, and dyspnea level when they were in a stable condition before the eCOPD and 24 hours after being admitted to the hospital or discharged from the ED to home. We used the Medical Research Council (MRC) breathlessness scale  to measure baseline dyspnea.
For all patients with known COPD, additional variables collected from medical records included baseline severity of COPD as measured by FEV1; hospital admissions for eCOPD during the previous 12 months; baseline therapy and the presence of conditions needed to determine the Charlson Comorbidity Index.
Reviewers were trained to identify pertinent data and a precise manual was developed to help reviewers collect the data.
Mortality at 30/60 days after the index ED index was determined by consulting medical records, regional electronic databases and the national registry of mortality.
The outcomes variables were defined as mortality within 30 days or 60 days of the index ED visit for the eCOPD. Patients were randomly divided into a derivation sample (50 %) and a validation sample (50 %). Derivations sample was used in order to develop the CART, whereas validation sample was used to validate the results obtained from the previously derived CART. Both samples were described using means and the standard deviations for continuous variables and the number of cases and percentages for categorical variables. Differences between the derivation and the validation samples were tested for the distribution of each variable using the t-test for continuous variables and the chi-square test for categorical variables. Missing data on baseline level of dyspnea (MRC breathlessness scale) were considered as a separate category from other data. Justification for the inclusion of missing dyspnea data was based on the fact that patients without a measurement of baseline dyspnea were significantly different from patients with lower levels of breathlessness (MRC categories 1 to 4 [p < 0.0001]) but not from those with the most severe breathlessness (MRC category 5 [p = 0.63]).
A CART based on a recursive partitioning algorithm was created in the derivation sample  to identify 30-day mortality risk factors with the highest discriminative power. The goal was to identify the variables and partition point that optimally separate low-risk patients from high-risk patients. To internally validate the risk of 30-day mortality derived from the regression tree, we used bootstrap resampling with N = 2,000 repetitions and estimated 95 % confidence intervals (95 % CI) . We report the median of these 2,000 repetitions as the parameter estimate and the 2.5 and 97.5 percentiles as the 95 % CIs.
To make the CART more user friendly, we simplified the resulting algorithm into a manageable number of risk classes based mainly on the estimated risk of 30-day mortality. We applied the risk classification derived from the derivation sample to the validation sample. The Cochran-Armitage trending statistic was performed to assess whether classification provided by the CART could differentiate low-risk patients from high-risk patients in a fashion of graded response based on the level of risk present.
The 30-day mortality derived and validated classification tree was applied to 60-day mortality. Additional variables to improve the 30-day mortality risk prediction to 60-day mortality risk prediction were selected from the univariate analysis and added to the CART analysis in order to get the best prediction tree for 60-day mortality.
Model discrimination of the trees and the risk categories was assessed by the area under the receiver operating curves (AUC).
Effects were considered statistically significant at α = 0.05. All statistical analyses were performed using SAS for Windows© version 9.1, except for the development and validation of the regression tree, which was built using R version 2.14.
Univariate analysis performed in the derivation sample (n = 1252)
1206 (96.3 %)
46 (3.7 %)
1173 (93.7 %)
79 (6.3 %)
Baseline FEV1 %
30 < FEV1 % < 50
Charlson Comorbidity Indexa
Previous LT-DOT or NIMV at home
Number of previous admissions due to eCOPD
Glasgow Coma Score – <15
Heart rate upon ED arrival (≥120)
Use of inspiratory accessory muscle upon ED arrival
Paradoxical breathing upon ED arrival
pH upon ED arrival
PCO2 upon ED arrival
MRC breathlessness scale
Distribution of subjects by estimated 30/60-day mortality and stratified risk group in both samples
Derivation sample (1252)
Validation sample (1235)
Risk group (Fig. 1)
0.808 (0.742 – 0.873)
0.767 (0.686 – 0.847)
Risk group (Fig. 2)
0.798 (0.757 – 0.838)
0.744 (0.691 – 0.898)
Dyspnea at baseline was the main variable associated with 30-day mortality after an ED evaluation for eCOPD. It provided the first branch of the CART. The next branches included the presence of cardiac disease, age, UAIM/PB, and the Glasgow Coma Scale score.
In our cohort, 3.6 % of patients died within 30 days of the index eCOPD. In similar studies, mortality during this period was 1 %  and 2.1 % . At 60-day after the event mortality rate almost doubled the rate.
The variable that established separation in the first branch of the CART was baseline dyspnea. Among patients with stable COPD, dyspnea has been identified as a more important prognostic factor than pulmonary function (FEV1) [17, 18]. In two studies of in-hospital mortality among eCOPD patients, baseline dyspnea was independently associated with mortality [19, 20]. In contrast, a study evaluating long-term mortality (4 years) following hospitalization for an eCOPD found that dyspnea at baseline was associated with mortality in the univariate analysis but not in the multivariate analysis . Therefore one variable reflecting patients’ baseline situation, dyspnea, was a very important predictor of in-hospital and very short-term mortality after an eCOPD requiring hospital evaluation. In our study, patients with a level 5 on the MRC breathlessness scale had higher mortality (18 %–55 % depending on the absence or presence of neurological impairment). In contrast, patients with level 1–2 dyspnea at baseline had low rates of mortality (0 %–2 %).
Cardiac comorbidity was the second-level key factor in the CART. In analyzing mortality 90 days after discharge for an eCOPD, Almagro et al. identified age, FEV1 according the GOLD classification, number of previous hospitalizations for COPD, use of home oxygen therapy, higher functional dependence, and comorbidities according Charlson comorbidity index as factors related to mortality . In our study, cardiac diseases had a greater impact on 30-day mortality than comorbidities in general. Donaldson et al. showed that in a short period of time (1–5 days) after an eCOPD treated with antibiotics and systemic steroids, the risk of myocardial infarction significantly increased . In our cohort, 2 % of patients with little breathlessness at baseline (MRC levels of 1–2) and cardiac disease died within 30 days compared to 0 % who did not have cardiac disease. A similar pattern was seen among patients with more severe breathlessness (MRC level 3–4), with mortality rates of 5 % among those with cardiac disease and 1 % among those without it. Cardiac disease and COPD share smoking and low-grade chronic inflammation as potential factors that could explain their coexistence.
In a prospective study of patients admitted for an eCOPD, 10 % had increases in their troponin T levels . In this group, even a modest increase in this biomarker was associated with an increase in long-term mortality, especially in the first year after the episode . The mortality rate increases further when troponin T elevations are associated with tachycardia (>100 beats/min) . Thirty-day mortality among patients hospitalized for an eCOPD has also been independently associated with elevated levels of NT-proBNP or troponin T; when both NT-proBNP and troponin T were elevated, mortality was 15-fold higher than among patients with normal values .
The design of our study did not modify the usual clinical practice, so troponin T and NT-proBNP were measured at the discretion of the treating clinician. Thus, we were not able to measure the impact of these markers.
Roche et al. , using a similar study as ours but focused on in-hospital mortality, established a model that included age, baseline dyspnea, and a severity index that include UAIM and neurological impairment. This model is very similar to the variables included in our CART. However, there are some differences. First, our CART included cardiac disease as a key comorbidity in the final outcome. Second, the variables included in our CART did not affect every patient. In fact, neurologic impairment (Glasgow Coma Scale score <15) was included in only one branch which, added to the UAIM/PB and with a dyspnea grade of 5, was associated with the highest mortality rate of the whole cohort, 55 %.
A low Glasgow Coma Scale score has been associated with increased mortality in ICU-admitted eCOPD patients , but not in short-term (30-day) follow-up.
Variables such as severity of airway obstruction and previous hospitalizations for eCOPD have been cited as factors related to eCOPD mortality. In our study, though, they were not identified as prognostic variables for 30-day mortality following ED evaluation for an eCOPD.
Once the CART developed for 30-day mortality risk prediction was applied to 60-day mortality risk prediction predictive ability decreases. However, a new variable took a relevant role in almost every branch of the decision tree, number of comorbidities as measured by the Charlson Comorbidity–index, improving the predictive ability of the tree for 60-day mortality risk prediction. Comorbidities have been considered an important mortality predicting factor  even has been included in some multidimensional prognosis index. [16, 28] This implies that factors related to mortality after such a hospitalization change in a short -time, reaching progressively more importance some aspect of the general clinical condition of the patient as comorbidities are. This is a key issue when considering the global treatment and follow-up for these patients.
Our decision tree was built by recursive partitioning using CART. One important advantage of CART over linear and additive models is that it does not require parametric specification of the nature of the relationship between predictors and outcome. In practice, this means that the assumption of linearity, which is frequently made in conventional regression models, is not required. In addition, the CART method allows for naturally incorporating interactions between predictors beyond what had previously been known, and these predictors can be easily interpreted by researchers.
These results highlight the ease with which CART models incorporate complex interactions. The interactions describe above would not have been detected by regression models, even with large sample sizes as the one in our study. Our CART was developed with 46 events and 5 predictors. When developing clinical prediction models with a binary outcome, a recommended sample size of 10 events per predictor is an extended rule [29–31]. Moreover, the internal validation of the CART provided by the bootstrap analysis showed that the results were very stable even with less than 4 % of events in the sample. In addition, CART models can handle missing values in a more natural way than more traditional techniques, in which subjects with missing values are eliminated from the analysis. In our decision tree, a clinician can extrapolate the likely mortality risk for a patient with missing information.
Our study has several strengths. The large sample and number of hospitals included reflect the general population of eCOPD patients evaluated in EDs in Spain. The decision tree uses measures generally gathered by physicians in the evaluation of eCOPD patients, and allows clinicians to easily establish prognosis without having to memorize the scores of different variables. It is much easier to use than complex and often cumbersome models. A similar CART model has been developed to evaluate the long-term (5 year) prognosis of patients with stable COPD.  Finally, our study points out an important issue as it is that mortality after 30 days after an ED visit is importantly conditioned by the comorbidities of the patient. This is important when planning the care and follow up of these patients.
Limitations of our study must also be noted. First, did not identify the causes of mortality. This could be important because cardiac events frequently occur after an eCOPD and could be related to previous cardiac disease rather than the eCOPD. The main limitation of decision tree models is that including higher-order interactions without considering the main effects could lead to spurious relations between predictors and overestimate the effect of some predictors. This is sometimes referred to as estimation bias. However, we believe that the use of combined split and bootstrap validation techniques provides internally and externally validated results that minimize spurious relations, as has been shown in previous studies . Our tree is proposed to be used at the ED, or even before at the primary care level when seeing a eCOPD patient, as a decision making tool. Nevertheless, since our algorithm include either admitted to the hospital and discharge to home patients a bias should be taken into account since more fatalities were obviously found among patients admitted to the hospital. Therefore, caution must be taken if the algorithm is used exclusively with patients already admitted or discharged home. As an additional limitation of the study we must include that no inter or intra observer reliability studies were performed. Nevertheless, reviewers were trained by the principal investigators at each site and were provided with a manual for the collection of data.
In summary, a CART model based on measures commonly collected in the ED evaluation of patients experiencing an eCOPD created a simple decision tree that identifies patients’ risk of short-term (30/60-day) mortality. Use of this decision could provide valuable information about the appropriate clinical care for these patients.
We are grateful for the support of the 16 participating hospitals and the ED physicians, other clinicians, and staff members of the various services, research, quality units, and medical records sections of these hospitals. We also gratefully acknowledge the patients who participated in the study. The authors also acknowledge the editorial assistance provided by Patrick Skerrett.
This work was supported in part by grants from the Fondo de Investigación Sanitaria (PI 06\1010, PI06\1017, PI06\714, PI06\0326, PI06\0664); Department of Health of the Basque Government (2012111008), Department of Education, Language Policy and Culture of the Basque Government (IT620-13); the Research Committee of the Hospital Galdakao; and the thematic networks - REDISSEC (Red de Investigación en Servicios de Salud en Enfermedades Crónicas) - of the Instituto de Salud Carlos III.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Steer J, Gibson GJ, Bourke SC. Predicting outcomes following hospitalization for acute exacerbations of COPD. QJM. 2010;103:817–29.View ArticlePubMedGoogle Scholar
- Singanayagam A, Schembri S, Chalmers JD. Predictors of mortality in hospitalized adults with acute exacerbation of chronic obstructive pulmonary disease. Ann Am Thorac Soc. 2013;10:81–9.View ArticlePubMedGoogle Scholar
- Kaji AH, Hanif AM, Bosson N, Ostermayer D, Niemann JT. Predictors of Neurologic Outcome in Patients Resuscitated from Out-of-Hospital Cardiac Arrest Using Classification and Regression Tree Analysis. Am J Cardiol. 2014;114:1024–8.View ArticlePubMedGoogle Scholar
- Pouliakis A, Margari C, Margari N, Chrelias C, Zygouris D, Meristoudis C, et al. Using classification and regression trees, liquid-based cytology and nuclear morphometry for the discrimination of endometrial lesions. Diagn Cytopathol. 2014;42:582–91.View ArticlePubMedGoogle Scholar
- Esteban C, Arostegui I, Moraza J, Aburto M, Quintana JM, Pérez-Izquierdo J, et al. Development of a decision tree to assess the severity and prognosis of stable COPD. Eur Respir J. 2011;38:1294–300.View ArticlePubMedGoogle Scholar
- Tabak YP, Sun X, Johannes RS, Gupta V, Shorr AF. Mortality and need for mechanical ventilation in acute exacerbations of chronic obstructive pulmonary disease. Development and validation of a simple risk score. Arch Intern Med. 2009;169:1595–602.View ArticlePubMedGoogle Scholar
- Quintana JM, Esteban C, Barrio I, Garcia-Gutierrez S, Gonzalez N, Arostegui I, et al. The IRYSS-COPD appropriateness study: objectives, methodology, and description of the prospective cohort. BMC Health Serv Res. 2011;11:322.View ArticlePubMedPubMed CentralGoogle Scholar
- Rabe KF, Hurd S, Anzueto A, Barnes PJ, Buist SA, Calverley P, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease: GOLD executive summary. Am J Respir Crit Care Med. 2007;176(6):532–55.View ArticlePubMedGoogle Scholar
- Gold PM. The 2007 GOLD Guidelines: a comprehensive care framework. Respir Care. 2009;54:1040–9.PubMedGoogle Scholar
- Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83.View ArticlePubMedGoogle Scholar
- Teasdale G, Jennett B. Assessment of coma and impaired consciousness. A practical scale. Lancet. 1974;2:81–4.View ArticlePubMedGoogle Scholar
- Fletcher CM, Elmes PC, Wood CH. The significance of respiratory symptoms and the diagnosis of chronic bronchitis in a working population. BMJ. 1959;2:257–66.View ArticlePubMedPubMed CentralGoogle Scholar
- Zhang H, Singer B. Recursive Partitioning in the Health Sciences. New York: Springer-Verlag; 1999.View ArticleGoogle Scholar
- Efron B, Tibshirani R. An Introduction to the Bootstrap. London, England: Chapman & Hall; 1993.View ArticleGoogle Scholar
- Stiell IG, Clement CM, Aaron SD, Rowe BH, Perry JJ, Brison RJ, et al. Clinical characteristics associated with adverse events in patients with exacerbation of chronic obstructive pulmonary disease: a prospective cohort study. CMAJ. 2014;186:E193–204.View ArticlePubMedPubMed CentralGoogle Scholar
- Almagro P, Soriano JB, Cabrera FJ, Boixeda R, Alonso-Ortiz MB, Barreiro B, et al. Short- and medium-term prognosis in patients hospitalized for COPD exacerbation: the CODEX index. Chest. 2014;145:972–80.View ArticlePubMedGoogle Scholar
- Nishimura K, Takateru I, Tsukino M, Oga T. Dyspnea is a better predictor of 5 year survival than airway obstruction in patients with COPD. Chest. 2002;121:1434–40.View ArticlePubMedGoogle Scholar
- Esteban C, Quintana JM, Aburto M, Moraza J, Egurrola M, España PP, et al. Predictors of mortality in patients with stable COPD. J Gen Intern Med. 2008;23:1829–34.View ArticlePubMedPubMed CentralGoogle Scholar
- Roche N, Zureik M, Soussan D, Neukirch F, Perrotin D. Urgence BPCO (COPD Emergency) Scientific Committee. Predictors of outcomes in COPD exacerbation cases presenting to the emergency department. Eur Respir J. 2008;32:953–61.View ArticlePubMedGoogle Scholar
- Steer J, Norman EM, Afolabi OA, Gibson GJ, Bourke SC. Dyspnoea severity and pneumonia as predictors of in-hospital mortality and early readmission in acute exacerbations of COPD. Thorax. 2012;67:117–21.View ArticlePubMedGoogle Scholar
- Piquet J, Chavaillon JM, David P, Martin F, Blanchon F, Roche N. French College of General Hospital Respiratory Physicians (CPHG). High-risk patients following hospitalisation for an acute exacerbation of COPD. Eur Respir J. 2013;42:946–55.View ArticlePubMedGoogle Scholar
- Almagro P, Cabrera FJ, Diez J, Boixeda R, Alonso Ortiz MB, Murio C, et al. Comorbidities and short-term prognosis in patients hospitalized for acute exacerbation of COPD: the EPOC en Servicios de medicina interna (ESMI) study. Chest. 2012;142:1126–33.View ArticlePubMedGoogle Scholar
- Donaldson GC, Hurst JR, Smith CJ, Hubbard RB, Wedzicha JA. Increased risk of myocardial infarction and stroke following exacerbation of COPD. Chest. 2010;137:1091–7.View ArticlePubMedGoogle Scholar
- McAllister DA, Maclay JD, Mills NL, Leitch A, Reid P, Carruthers R, et al. Diagnosis of myocardial infarction following hospitalisation for exacerbation of COPD. Eur Respir J. 2012;39:1097–103.View ArticlePubMedGoogle Scholar
- Høiseth AD, Neukamm A, Karlsson BD, Omland T, Brekke PH, Søyseth V. Elevated high-sensitivity cardiac troponin T is associated with increased mortality after acute exacerbation of chronic obstructive pulmonary disease. Thorax. 2011;66:775–81.View ArticlePubMedGoogle Scholar
- Chang CL, Robinson SC, Mills GD, Sullivan GD, Karalus NC, McLachlan JD, et al. Biochemical markers of cardiac dysfunction predict mortality in acute exacerbations of COPD. Thorax. 2011;66:764–8.View ArticlePubMedGoogle Scholar
- Tabak YP, Sun X, Johannes RS, Hyde L, Shorr AF, Lindenauer PK. Development and validation of a mortality risk-adjustment model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med Care. 2013;51:597–605.View ArticlePubMedGoogle Scholar
- Divo M, Cote C, De Torres JP, Casanova C, Marin JM, Pinto-Plata V, et al. Comorbidities and risk of mortality in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2012;186:155–61.View ArticlePubMedGoogle Scholar
- Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277:488–94.View ArticlePubMedGoogle Scholar
- Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. New York: Springer-Verlag; 2009.View ArticleGoogle Scholar
- Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol. 2007;165:710–8.View ArticlePubMedGoogle Scholar