Developing a scoring tool to estimate the risk of deterioration for normotensive patients with acute pulmonary embolism on admission

Background It is important to identify deterioration in normotensive patients with acute pulmonary embolism (PE). This study aimed to develop a tool for predicting deterioration among normotensive patients with acute PE on admission. Methods Clinical, laboratory, and computed tomography parameters were retrospectively collected for normotensive patients with acute PE who were treated at a Chinese center from January 2011 to May 2020 on admission into the hospital. The endpoint of the deterioration was any adverse outcome within 30 days. Eligible patients were randomized 2:1 to derivation and validation cohorts, and a nomogram was developed and validated by the aforementioned cohorts, respectively. The areas under the curves (AUCs) with 95% confidence intervals (CIs) were calculated. A risk-scoring tool for predicting deterioration was applied as a web-based calculator. Results The 845 eligible patients (420 men, 425 women) had an average age of 60.05 ± 15.43 years. Adverse outcomes were identified for 81 patients (9.6%). The nomogram for adverse outcomes included heart rate, systolic pressure, N-terminal-pro brain natriuretic peptide, and ventricle/atrial diameter ratios at 4-chamber view, which provided AUC values of 0.925 in the derivation cohort (95% CI 0.900–0.946, p < 0.001) and 0.900 in the validation cohort (95% CI 0.883–0.948, p < 0.001). A risk-scoring tool was published as a web-based calculator (https://gaoyzcmu.shinyapps.io/APE9AD/). Conclusions We developed a web-based scoring tool that may help predict deterioration in normotensive patients with acute PE.


Background
In 2019, the European Society of Cardiology (ESC) revised the risk stratification system for patients with acute pulmonary embolism (PE) based on the 2014 ESC guideline [1,2]. However, 2019 ESC guidelines still emphasized the critical role of identifying patients with poor prognosis from normotensive patients by the Bova and Fast scores [1]. Furthermore, there are patients with poor prognosis who have shown a false sense of security with normotension at their admission, which might mask the risk of rapid deterioration and death [3,4]. Unfortunately, there is no universally recognized tool for distinguishing these patients and guiding clinical decision-making to define the appropriate treatment strategy [1].
Computed tomography (CT) pulmonary angiography can be used to diagnose PE and also can be used for identifying patients who have poor prognosis [5][6][7]. However, most models for predicting a poor short-term prognosis among normotensive patients with acute PE have not incorporated CT parameters or only used a CT parameter for identifying right ventricle (RV) dysfunction [3,[8][9][10]. Different measurement methods and different thresholds in the evaluation of right-toleft heart size by CT caused this discrepancy [11,12]. Therefore, redefining the ratio of right-to-left heart size parameters from CT and then combining clinical and laboratory parameters might help promote predictive ability [1].
Although CT parameters regarding cardiac size can reflect an increased impedance in the pulmonary circulation [13]. Individual responses to the right ventricular afterload increase varies broadly depending on comorbidities, associated vasoconstriction, degree of proximal obstruction, and RV straining. These broad variations can be manifested in clinical and laboratory parameters, such as hypotension, tachycardia, myocardial markers and ultrasound RV dysfunction [1]. Based on the 2019 ESC guideline, the present study aimed to develop a semi-quantitative tool that combined clinical, laboratory, and simple CT parameters to promote predictive ability in normotensive patients with acute PE poor prognosis.

Study design
This retrospective study evaluated clinical, laboratory, and simple CT parameters of normotensive patients with acute PE from admission. The Bova score and 2019 ESC algorithm were used for risk stratification. The outcomes of interest were defined as the occurrence of adverse outcomes within 30 days after admission into hospital. Eligible patients were randomized 2:1 into derivation and validation cohorts. The derivation cohort was used to develop and evaluate a multivariable logistic regression model for predicting the outcomes of interest. The discriminatory power was evaluated by comparing the nomogram to the established risk stratification systems. The consistency of the nomogram was evaluated using the validation cohort. The investigators independently collected the data regarding clinical, laboratory, and CT parameters as well as data regarding the risk stratification scores and outcomes of interest. This research was approved by the Institutional Review Board of the Shengjing Hospital of China Medical University (No. 2020PS522K), and informed consent was exempted due to the absence of treatment intervention in patients.

Patient selection
Normotensive patients with acute PE were evaluated if they were treated at the Shengjing Hospital of China Medical University between January 2011 and May 2020. The diagnosis and management of acute PE was based on the 2019 ESC guidelines [1]. The inclusion criteria were an age of ≥ 18 years and a PE diagnosis based on CT pulmonary angiography. The exclusion criteria were pregnancy, reception of reperfusion treatment before admission, and missing data regarding CT parameters, echocardiography, cardiac troponin I (c-Tn I), and N-terminal-pro brain natriuretic peptide (NT-pro BNP) levels.

Clinical data
The patients' medical records were reviewed to collect their demographic characteristics and baseline data from their admission regarding heart rate, systolic pressure, history of disease, arterial oxyhemoglobin saturation, c-Tn I concentration (μg/L), and NT-pro BNP concentration (pg/mL).

Assessing RV dysfunction
Within 24 h after admission, RV dysfunction determined a transthoracic echocardiography using an IE Elite ultrasound machine (Philips) equipped with an S 5-1 transducer (frequency conversion 1-5 MHz) by ultrasound specialist as following criteria: RV dilation (end-diastolic diameter > 30 mm, evaluated at 4-chamber view or parasternal view), an increased RV/left ventricle(LV) end-diastolic diameter ratio > 0.9 at 4-chamber view, hypokinesia of the free RV wall, increased velocity of the jet of tricuspid regurgitation at apical 4-chamber view, decreased tricuspid annulus plane systolic, anyone or combinations of the condition above [2,14].

Outcomes of interest
The outcomes of interest were defined as the occurrence of adverse outcomes within 30 days after admission. Adverse outcomes were defined as PE-related deaths, the need for mechanical ventilation, the need for cardiopulmonary resuscitation, and the need for life-saving vasopressor and reperfusion treatment [9,15].

Measurement of CT parameters
Three simple CT parameters were selected for the analysis. The first factor was thrombus location, which was categorized as within the central pulmonary artery (CPA embolism) [15,16], spanning both sides of the bifurcation (saddle-CPA embolism) [17], and outside the CPA (non-CPA embolism) (Additional file 3: Figure S1 a-c). The second factor was the RV and LV diameters in the short-axis plane, which were measured as the maximal diameter from the cardiac intima to the interventricular septum [18], as well as the relative ratio of the RV/LV short-axis diameters (Additional file 4: Figure S2). The third factor was the maximum chamber diameters, which were measured using a 4-chamber view perpendicular to the atrial and interventricular septum (Additional file 4: Figure S2) [6], as well as the relative ratios of the RV/ LV and right atrium (RA)/left atrium (LA) 4-chamber diameters. All CT parameters were measured using the Mimics Medical software (version 19.0, Mimics Medical software, Leuven, Belgium).

Development of the model and risk-scoring tool
The model was developed based on three steps: (a) identifying relevant prognostic factors; (b) developing and validating the model; (c) evaluating the model's discriminatory power relative to the 2019 ESC algorithm and Bova score.
In the first step, eligible patients were randomized 2:1 into derivation and validation cohorts based on the TRIPOD standard [19]. All clinical, laboratory, and CT parameters were included into a classification and regression tree (CART) to identify relevant prognostic factors with importance [20]. All potential decisional factors for adverse outcomes were evaluated and chosen into splits providing the optimal separations by binomial data until the splits reached a minimum size or no improvement could be made [21]. All the chosen binomial parameters from CART were used to develop the model.
In the second step, univariate and multivariate logistic regression analyses were used to investigate binomial prognostic factors using the derivation cohort, and a nomogram was created by converting each regression coefficient from the multivariate logistic regression onto a scale of 0 points (low) to 100 points (high). The total scores for all variables were summed [22], and the different risk groups were separated based on their total nomogram scores via another CART analysis. A validation cohort was used to evaluate the model's consistency relative to the observed outcomes [21]. A calibration curve was used to assess the consistency between actual incidence and predicted incidence of the nomogram in the derivation and validation cohorts.
In the third step, the models' abilities to predict adverse outcomes were compared to the 2019 ESC algorithm [1] and the Bova score [10] based on the receiver-operating characteristic curve (ROC) and decision curve analysis (DCA). The final risk-scoring tool was published as a free web-based calculator.

Statistical analysis
Continuous variables were expressed as mean ± standard deviation and compared using the Student's t test. Categorical variables were presented as numbers (%) and compared using the χ 2 test. By a recursive partitioning analysis, CART was used to dichotomize each variable while controlling for confounders and divide the derivation cohort into different risk groups according to the total nomogram score [23]. Univariate and multivariate logistic regression analyses were used to evaluate the different factors, and the results were expressed as odds ratios (ORs) with corresponding 95% confidence intervals (CIs). The nomograms' predictive performances were evaluated based on the concordance index (C-index) and calibration with 1000 bootstrap resampling [22]. The ROC curves were used to evaluate sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the area under the curve (AUC). A calibration curve was used to assess the consistency between the actual incidence and predicted incidence of the nomogram [24]. Clinical utility was evaluated based on net benefit from the DCA. DeLong's test was used to compare AUC values [25]. Differences were considered significant at p-values of < 0.05, and all analyses were performed using R software (version 4.0.1; R Foundation, https ://www.r-proje ct.org).

Demographics and baseline characteristics
We evaluated 902 normotensive patients with acute PE, although 57 patients were excluded; a total of 3 patients were excluded due to pregnancy, 5 patients were excluded due to reception of reperfusion treatment, and 49 patients were excluded due to the absence of data on CT parameters, echocardiography, c-Tn I, or NT-pro BNP. Finally, a total of 845 eligible patients were included (Fig. 1), including 420 male patients and 425 female patients with an average age of 60.05 ± 15.43 years. Adverse outcomes were identified for 81 patients (42 male and 39 female) who had an average age of 59.36 ± 15.74 years (Table 1). No adverse outcomes were identified for 764 patients (378 male and 386 female) who had an average age of 60.12 ± 15.40 years. Patients with adverse outcomes had significantly higher values for heart rate, c-Tn I, and NT-pro BNP (all p < 0.001) and had significantly lower value of systolic pressure (p < 0.001). Patients with adverse outcomes were more likely to have RV dysfunction (p < 0.001). Among CT parameters, patients with adverse outcomes were more likely to have CPA saddle-CPA embolisms (both p < 0.001). Furthermore, patients with adverse outcomes had high values of the RV short-axis diameter, RV 4-chamber diameter, RA 4-chamber diameter, RV/LV short-axis diameter ratio, RV/LV 4-chamber diameter ratio, and RA/LA 4-chamber diameter ratio (all p < 0.001). However, patients with adverse outcomes also had lower values for the LV shortaxis diameter, LV 4-chamber diameter, and LA 4-chamber diameter (all p < 0.001).
The Bova score for patients with adverse outcomes revealed low risk (

Comparison between derivation and validation cohorts
After a random grouping with a ratio of 2:1, a total of 554 and 291 patients were divided into derivation and validation cohorts, respectively. In the derivation cohort, the average age was 60.43 ± 14.95 years including 267 men and 287 women. In the validation cohort, the average age was 59.32 ± 16.30 years including 153 men and 138 women. There was no statistical difference between derivation and validation cohorts (all p > 0.05) ( Table 2).

Performance of the nomograms in the derivation and validation cohorts
Nomograms were developed using the two multivariate logistic regression models (Fig. 2). The nomogram for predicting adverse outcomes incorporated five variables and provided good C-index values in the derivation (C-index: 0.925, 95% CI 0.900-0.946) and validation cohorts (C-index: 0.900, 95% CI 0.883-0.948) (Fig. 3). The calibration curve also revealed good agreement between the nomogram's predictions and the actual outcomes (Fig. 4).
The DCA revealed that the nomogram had greater net benefit than the 2019 ESC algorithm or the Bova score for predicting adverse outcomes. Using the nomogram for predicting adverse outcomes added a net benefit of 0.03-0.98 (Fig. 5).

Development of the risk-scoring tool
The nomogram for predicting adverse outcomes was used to develop a web-based calculator (https ://gaoyz cmu.shiny apps.io/APE9A D/), which assigned patients to a high-risk group (≥ 145 points) or a low-risk group (< 145 points). The QR code in the lower right corner of each calculator in Fig. 6 can be used to publish the results to mobile electronic equipment.

Discussion
This study developed a tool for estimating the risk of deterioration among normotensive patients with acute PE. Results revealed that the risk of adverse outcomes within the first 30 days after admission could be predicted using a nomogram that incorporated the RV/ LV and RA/LA 4-chamber diameter ratios, NT-pro BNP concentration, systolic pressure, and heart rate. Furthermore, this risk-scoring tool had better discriminatory power and a greater net benefit than did the 2019 ESC  algorithm and the Bova score. Finally, this tool was converted into convenient web-based calculators that could be used in clinical practice.
Our results are consistent with the Bova score as a decreased systolic pressure (90-100 mmHg) and an elevated heart rate (≥ 110 beats/min) were risk factors for adverse outcomes. In normotensive patients with acute PE that is deteriorating, the presentation of lower systolic pressure together with more tachycardia are warning of overt RV dysfunction [26] and are associated with the poor short-term prognosis in acute PE [1,27]. However, we found some differences that elevated NT-pro BNP concentration was included in the final scoring tool for predicting adverse outcomes, and c-Tn I concentration was not included. c-Tn I was still one of the important predictors of adverse outcomes in patients with acute PE [1,10]. Based on the CART analysis that selected variables that were most likely to identify adverse outcomes [28], c-Tn I might not have had enough importance compared to the other parameters in our study. In some previous studies, the elevated c-Tn I was also not included after a multifactor analysis [29,30]. NT-pro BNP reflected RV strain, which was complementary and not mutually exclusive to CT parameters [29]. This might be the reason that the NT-pro BNP, not c-Tn I, was included into the final model in our study.
Interestingly, the increased predictive value of our risk-scoring tool (vs. the 2019 ESC algorithm and the Bova score) was mainly related to the RV/LV and RA/ LA 4-chamber diameter ratios. A previous report had described the interaction between the RV and LV via the interventricular septum [31], and the RV and pulmonary circulation are characterized by low impedance and high output [11]. Furthermore, the size of the LV is larger than the size of the RV. Thus, severe PE leads to increased pressure in the RV, which compresses the LV via the interventricular septum, and clear RV-to-LV compression can be observed in cases with severe chronic pulmonary hypertension (PH) [32]. Severe PH is also the main pathophysiological mechanism underlying the occurrence of adverse outcomes in acute PE [1]. Decreased blood return to the LV also further reduces LV size, which  leads to an increased right-to-left heart size ratio. The four-chamber view is defined as the plane perpendicular to the atrial and interventricular septum [6], which can be used to accurately evaluate the increased size of the right heart, the decreased size of the left heart, and the interaction between these factors. The threshold value in our study was an RV/LV 4-chamber diameter ratio of 1.25, rather than previously reported cutoffs of 0.9, 1.0, or 1.1 [33,34], although differences in the definition of this threshold may explain the selection of different Decision curve analysis for the nomogram. A comparison of net benefit between the nomogram, the 2019 ESC algorithm, and the Bova score, which revealed that the nomogram was better in terms of net benefit (net benefit: 0.03-0.98). The nomogram is shown using the red line, Bove score is shown using the green line, and 2019 ESC algorithm is shown using green line. ESC European Society of Cardiology   Fig. 6 The web-based calculator for estimating the risk of adverse outcomes (https ://gaoyz cmu.shiny apps.io/APE9A D/). The QR code in the lower right corner can be used to publish the result to mobile electronic equipment. NT-pro BNP N-terminal pro-brain natriuretic peptide; RV right ventricle; LV left ventricle; RA right atrium; LA left atrium parameters between our study and previous studies. The RA/LA 4-chamber diameter ratio was another factor in our risk-scoring tool. The membranous structure [31] and greater pressure sensitivity of the atrium (vs. the ventricle) may explain the lower weighting of the RA/LA 4-chamber diameter ratio relative to the weighting of the RV/LV 4-chamber diameter ratio in our scoring tool.

Limitations
The present study has several limitations that should be considered. First, although we evaluated data from normotensive patients with acute PE who were treated during an approximately 10-year period, the retrospective analysis is prone to bias. Second, although this scoring tool was developed using randomized derivation and validation cohorts, external validation is also required. Third, we did not have access to data regarding cardiac troponin T, and heart type fatty acid binding protein, which precluded comparisons of our tools to the fast prognostic score [9] for predicting adverse outcomes.

Conclusion
We developed a scoring tool that was published as webbased calculators for predicting adverse outcomes among normotensive patients with acute PE. This risk-scoring tool may help improve the management of patients with acute PE by predicting deterioration.