The prognostic value of pulmonary embolism severity index in acute pulmonary embolism: a meta-analysis

Background Prognostic assessment is important for the management of patients with acute pulmonary embolism (APE). Pulmonary Embolism Severity Index (PESI) and simple PESI (sPESI) are new emerged prognostic assessment tools for APE. The aim of this meta-analysis is to assess the accuracy of the PESI and the sPESI to predict prognostic outcomes (all-cause and PE-related mortality, serious adverse events) in APE patients, and compare between these two PESIs. Methods MEDLINE and EMBASE database were searched up to June 2012 using the terms “Pulmonary Embolism Severity Index” and “pulmonary embolism”. Summary odds ratio (OR) with 95% confidence intervals (CIs) for prognostic outcomes in low risk PESI versus high risk PESI were calculated. Summary receiver operating characteristic curve (SROC) used to estimate overall predicting accuracies of prognostic outcomes. Results Twenty-one studies were included in this meta-analysis. The results showed low-risk PESI was significantly associated with lower all-cause mortality (OR 0.13; 95% CI 0.12 to 0.15), PE-related mortality (OR 0.09; 95% CI 0.05 to 0.17) and serious adverse events (OR 0.34; 95% CI 0.29 to 0.41), with no homogeneity across studies. In sPESI subgroup, the OR of all-cause mortality, PE-related mortality, and serious adverse events was 0.10 (95% CI 0.08 to 0.14), 0.09 (95% CI 0.03 to 0.26) and 0.40 (95% CI 0.31 to 0.51), respectively; while in PESI subgroup, the OR was 0.14 (95% CI 0.13 to 0.16), 0.09 (95% CI 0.04 to 0.21), and 0.30 (95% CI 0.23 to 0.38), respectively. For accuracy analysis, the pooled sensitivity, the pooled specificity, and the overall weighted AUC for PESI predicting all-cause mortality was 0.909 (95% CI: 0.900 to 0.916), 0.411 (95% CI: 0.407 to 0.415), and 0.7853±0.0058, respectively; for PE-related mortality, it was 0.953 (95% CI: 0.913 to 0.978), 0.374 (95% CI: 0.360 to 0.388), and 0.8218±0.0349, respectively; for serious adverse events, it was 0.821 (95% CI: 0.795 to 0.845), 0.389 (95% CI: 0.384 to 0.394), and 0.6809±0.0208, respectively. In sPESI subgroup, the AUC for predicting all-cause mortality, PE-related mortality, and serious adverse events was 0.7920±0.0117, 0.8317±0.0547, and 0.6454±0.0197, respectively. In PESI subgroup, the AUC was 0.7856±0.0075, 0.8158±0.0451, and 0.6609±0.0252, respectively. Conclusions PESI has discriminative power to predict the short-term death and adverse outcome events in patients with acute pulmonary embolism, the PESI and the sPESI have similar accuracy, while sPESI is easier to use. However, the calibration for predicting prognosis can’t be calculated from this meta-analysis, some prospective studies for accessing PESI predicting calibration can be recommended.


Background
Epidemiological studies showed the prevalence of acute pulmonary embolism (APE) among hospitalized patients in the United States, according to data collected between 1979 and 1999, was 0.4% [1]. In Europe, a total of 67,351 (in 2005), 69,234 (in 2006) and 71,223(in 2007) APE were coded either as principal or secondary diagnosis in German hospitals according to the data from Federal Statistical Office [2]. In Asia, from a multi-center registration study of Chinese hospitals between 1997 and 2008, a total of 18,206 patients were confirmed with APE from 16,972,182 hospital admissions, the annual incidence was 0.1% (95% CI: 0.1% to 0.2%) [3]; while in Korean, incidence of established APE was 88 (0.17%) of 50,882 identified retrospectively from patients hospitalized during a 2-year period from 2005 to 2007 [4]. APE has become a relatively common cardiovascular emergency.
The key consequences of a pulmonary thromboembolic episode are haemodynamic. Large and multiple emboli might abruptly increase pulmonary vascular resistance to a level of after load which cannot be matched by the right ventricle (RV), and sudden death may occur [5]. In the ICOPER registry, the 14-day mortality of APE was above 20% [6]. In the RIETE registry, at 3 months, the cumulative rates of overall mortality and fatal PE were 8.65% and 1.68%, respectively [7]. While in the EMPEROR registry, the all-cause 30-day mortality rate was 5.4% (95% CI: 4.4% to 6.6%) [8]. APE has become one of the leading causes of preventable hospital deaths.
The mortality of APE can be predicted by haemodynamic status, right ventricular dysfunction, myocardial injury, and other clinical and routine laboratory tests. Shock and hypotension are principal markers of high risk of early death in APE. RV dysfunction which is detected by echocardiography, computed tomography, and brain natriuretic peptide (BNP) is related to intermediate risk of short-term mortality in APE. Myocardial injury in patients with APE is related to an intermediate risk of short-term mortality, which can be detected by troponin T or I testing [5]. While the Pulmonary Embolism Severity Index (PESI) was recently found to be a well validated and highly reliable clinical prognostic model for patients with APE.
The PESI was designed by Aujesky D and his colleague in 2005, which comprises 11 routinely available clinical predictor variables with different prognostic weights [9]. On the basis of the PESI score, each patient is classified into one of five classes (I-V), with 30day mortality ranging from 1.1% to 24.5%. Patients in risk classes I and II are categorized as low-risk and in risk classes III-V are categorized as high-risk. While the PESI is hard to memorizes for based on 11 items, and it's difficult to calculate in busy emergency departments. The simplified PESI (sPESI) was developed in 2010, which includes six of the 11 original PESI variables [10]. Patients with none of the variables (0 points) are categorized as low-risk; with one to six the variables (1-6 points) are categorized as high-risk. The PESI and sPESI are listed in the Additional file 1: Appendix 1.
The prognostic value of the PESI and sPESI has been assessed by some studies, but has not been systematic reviewed. We conduct this meta-analysis to assess the accuracy of the PESI and the sPESI to predict prognostic outcomes in APE patients. In particular, we also compared the predicting accuracy between these two PESIs.

Eligibility criteria
We defined studies as being eligible for inclusion in this analysis if they met the following criteria: (1) types of studies: observational studies ( cohort studies and casecontrol studies ) were included; (2) types of participants: population of objective diagnosis of APE with shortterm prognosis outcome were included; (3) types of outcomes: studies included at least one of three prognostic outcomes: all-cause mortality, PE-related mortality, serious adverse events; and a 2×2 table of outcome results could be constructed based on low-risk PESIs or highrisk PESIs.

Search strategy and study selection
We searched MEDLINE (PubMed, http://www.ncbi.nlm. nih.gov/pubmed/), and EMBASE (http://www.embase. com) up to June 2012. The search strategy included terms of "Pulmonary Embolism Severity Index" and "Pulmonary Embolism". The search was limited to English-language articles. We did not appoint limit in country, race, or publication year. To identify any additional relevant studies, we also hand-searched conference proceedings and scanned references of retrieved articles.
Study selection was initially performed by review of titles and abstracts. When there was any possibility that it might be relevant, the full text were downloaded and then reviewed for data retrieval.
Two reviewers independently judged study eligibility while screening the citations. Disagreements were resolved by a third reviewer.

Quality assessment
We used Newcastle-Ottawa Scale (NOS) to assess the quality of the included observational studies [11]. The NOS contains eight items, categorized into three dimensions including Selection (4), Comparability (1), and Exposure (3). A high quality study can be awarded a maximum of one star for each numbered item within the Selection and Exposure categories. A maximum of two stars can be given for Comparability. The NOS ranges between zero up to nine stars. The NOS is listed in the Additional file 2: Appendix 2.
Two reviewers independent assessed the quality of studies. Disagreements were resolved by a third reviewer.

Data extraction
We collected information about study characteristics (study year, country, study design, number of patients, mean age, gender distribution, type of PESI, methods for diagnosis of APE, haemodynamic status, length of follow-up) and number of patients with the available outcomes of all-cause death, PE-related death, serious adverse events among low-risk or high-risk PESIs.
In this meta-analysis, all-cause mortality was considered as the primary outcome, PE-related death, serious adverse events were considered as the secondary outcome. Serious adverse events were defined as any of the following: nonfatal recurrent PE, nonfatal recurrent DVT, nonfatal bleeding, or delayed homodynamic instability.
Data were extracted independently by two reviewers. Any differences of opinion were resolved by discussion and consensus reached by discussion with a third reviewer.

Statistical analysis
First, we considered high-risk PESI as a risk factor for APE prognostic adverse outcomes. Summary OR with 95% CIs for three prognostic outcomes in low-risk PESI versus high-risk PESI was calculated respectively. Statistical heterogeneity was explored by χ 2 and inconsistency (I 2 ) statistics; an I 2 value of 50 percent or more represented heterogeneity [12]. If there was no heterogeneity, fixed effects model was used for meta-analysis; otherwise, a random effect model based on the DerSimonian and Laird estimator was used [13]. Overall effects were determined using the Z test. Potential publication bias was evaluated by the funnel plot. Subgroup meta-analyses were performed by PESI subgroup and sPESI subgroup. Sensitivity analysis was performed by only pooled prospective studies. The first step meta-analysis was performed with Review Manager (RevMan) software (version 5.0.21; Update Software Ltd, Oxford, Oxon, UK).
Second, we assess the accuracy (discriminative power) of PESI to predict APE prognostic outcomes. If we predict low-risk PESI as good prognostic outcomes, highrisk PESI as worse prognostic outcomes, the sensitivity (true positive rate) and specificity (true negative rate) of each included study was calculated compared to actual prognostic outcomes (the so-called gold standard). And the overall pooled sensitivity and specificity with 95% CIs was estimated by DerSimonian and Laird's randomeffects model [13]. In addition, summary receiver operator characteristic (SROC) analysis was performed to examine the interaction between sensitivity and (1-specificity) [14], and to quantify test performance using the area under the curve (AUC) and Q* value [15]. The second step metaanalyses were performed with Meta DiSc 1.4 (version 0.6; By Joseph Lau) [16].

Description of included studies
Overall, 21 studies [10,[17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36] were selected for this meta-analysis described in Figure 1. Among those studies, 10 studies were prospective cohorts, 9 studies were retrospective cohorts or case-control studies, the remaining 2 studies didn't mention about the study design type. Eleven studies were multi-center studies. Countries involved in these studies were Germany, Spain, Poland, Greece, Switzerland, USA, Belgium, Italy, Greece, UK, Israel, and Korea. Most of the studies (20 studies) came from Europe and North America, only a study came form Asia. All of the 21 included studies used the same diagnostic criteria of PE, which included high clinical probability of PE, and confirmed by contrast-enhanced multi detector CT or ventilationperfusion lung scan, or confirmed lower limb deep vein thrombosis (DVT) by venous ultrasound. A total of 5 studies included haemodynamic stable patients, a study included haemodynamic stable and unstable patients, but most of the studies didn't mention about the haemodynamic status of the included patients. The length of follow-up ranged from hospital discharge to 6 month following admission to hospital. The NOS for methodological quality assessment of these 21 studies all scored at 7 of 8 star, indicating good quality. The main characteristics of the selected studies are reported in Table 1.

Meta-analysis
High-risk PESI as a risk factor for adverse outcomes The association between PESI and PE adverse prognosis outcome risk of every included study are shown in Table 2. Data on all-cause mortality were reported in 19 studies. Five studies assessed the sPESI, 11 studies assessed the PESI, 3 studies assessed sPESI and PESI in the same study. A total of 50,021 patients were analyzed in the studies, including 4,991 patients with allcause death. There was no heterogeneity in the studies (I 2 = 0%). The pooled all-cause mortality was 2.0% (456/22978) in patients with low-risk PESI Vs 16.7% (4535/27043) in patients with high-risk PESI. The summary OR of patients with low-risk PESI compared with patients with high-risk PESI was 0.13 (95% CI 0.12 to 0.15; Z=33.76, P < 0.00001) ( Figure 2). The funnel plot showed no asymmetry, which indicated no evidence of publication bias ( Figure 3A). While in subgroup analysis for sPESI and PESI, the pooled all-cause mortality was 1.8% (180/9995) in patients with low-risk sPESI Vs 25.2%(2040/8096) in patients with high-risk sPESI, the summary OR of patients with low-risk sPESI compared with patients with high-risk sPESI was 0.10 (95% CI 0.08 to 0.14; Z=15.04, P < 0.00001) (Figure 2 Above); the pooled all-cause mortality was 2.1% (276/12983) in patients with low-risk PESI Vs 13.2% (2495/18947) in patients with high-risk PESI, the summary OR of patients with low-risk PESI compared with patients with high-risk PESI was 0.14 (95% CI 0.13 to 0.16; Z=30.10, P < 0.00001) (Figure 2 Below).
Data on PE-related mortality were reported in 7 studies, 3 studies assessed the sPESI, and 4 studies assessed the PESI. A total of 4,794 patients were analyzed in the studies, including 192 patients with PE-related death. There was no heterogeneity in the studies (I 2 = 0%). The pooled PE-related mortality was 0.5% (9/1731) in patients with low-risk PESI Vs 6.0% (183/3063) in patients with high-risk PESI. The summary OR of patients with low-risk PESI compared with patients with high-risk PESI was 0.09 (95% CI 0.05 to 0.17; Z=7.14, P < 0.00001) ( Figure 4). The funnel plot showed no asymmetry, which indicated no evidence of publication bias ( Figure 3B). While in subgroup analysis for sPESI and PESI, the pooled PE-related mortality was 0.4% (3/770) in patients with low-risk sPESI Vs 5.1% (72/1404) in patients with high-risk sPESI, the summary OR of patients with low-risk sPESI compared with patients with high-risk sPESI was 0.09 (95% CI 0.03 to 0.26; Z=4.39, P < 0.00001) (Figure 4 Above); the pooled PErelated mortality was 0.6% (6/961) in patients with low-risk PESI Vs 6.7% (111/1659) in patients with high-risk PESI, the summary OR of patients with lowrisk PESI compared with patients with high-risk PESI was 0.09 (95% CI 0.04 to 0.21; Z=5.62, P < 0.00001) (Figure 4 Below).
Data on serious adverse events were reported in 8 studies, 4 studies assessed the sPESI, 3 studies assessed the PESI, a study assessed sPESI and PESI in the same study. A total of 50,021 patients were analyzed in the studies, including 4,991 patients with serious adverse events. There was no heterogeneity in the studies (I 2 = 36%). The pooled serious adverse events rate was 1.2% (166/13355) in patients with low-risk PESI Vs 3.5% (761/21621) in patients with high-risk PESI. The summary OR of patients with low-risk PESI compared with patients with high-risk PESI was 0.34 (95% CI 0.29 to 0.41; Z=12.32, P < 0.00001) ( Figure 5). The funnel plot showed no asymmetry, which indicated no evidence of publication bias ( Figure 3C). While in subgroup analysis for sPESI and PESI, the pooled serious adverse events   (Figure 5 Below). Sensitivity analysis of primary outcome (all-cause mortality) by removal the retrospective studies showed there was no heterogeneity in the studies (I 2 = 1%); the summary OR of patients with low-risk PESI compared with patients with high-risk PESI was 0.13 (95% CI 0.10 to 0.19; Z=11.81, P < 0.00001) ( Figure 6); the funnel plot showed no asymmetry, which indicated no evidence of publication bias ( Figure 3D). Sensitivity analyses showed the robust result of the pooled meta-analyses.

Accuracy of the PESI to predict prognostic outcomes
The sensitivity and specificity for PESI in PE prognosis diagnosis of each included studies are also shown in Table 2.

Discussion
Some meta-analysis and systematic reviews have provided sufficient evidence that the prognostic outcomes of APE can be affected by haemodynamic status, right ventricular dysfunction, myocardial injury, and other clinical features [37][38][39][40][41]. PESI was designed based on some clinical features. Our meta-analysis showed highrisk PESI is a risk factor for all-cause death, PE-related death, and serious adverse events. Pooled sensitivity, specificity, and SROC were used for predicting PE prognosis. An AUC value of 0.5 indicates that the test has no discriminatory ability, whereas an AUC value of 1.0 indicates perfect capability [42]. While to demonstrate excellent accuracy, the AUC should be in the region of 0.97 or above; an AUC of 0.93 to 0.96 is very good; 0.75 to 0.92 is good; but an AUC less than 0.75 has obvious deficiencies in its accuracy [43]. Our meta-analysis showed the overall weighted AUC for all-cause mortality was 0.78 (95% CI: 0.77 to 0.80); and AUC for PE-related mortality was 0.82 (95% CI: 0.75 to 0.89). PESI has good accuracy for predicting PE prognosis. The main reason is that the PESI was derived form multi-center large sample studies with appropriate research methods. In the PESI study, 15,531 discharged patients with PE treated at 186 hospitals were included, on the basis of the β-coefficients of the stepwise logistic regression model, PESI with a point score was generated [9].
Our meta-analysis showed in sPESI subgroup, the OR of all-cause mortality, PE-related mortality, and serious adverse events was similar to that in PESI subgroup. Most important, in sPESI subgroup, the AUC for predicting all-cause mortality, PE-related mortality, and serious adverse events was also similar to that in PESI subgroup. It has the same accuracy between PESI and sPESI. The sPESI is also derived form logistic regression analysis [10]. In the derivation data set, univariate logistic regression of the original 11 PESI variables led to the removal of variables that did not reach statistical significance and subsequently produced the simplified PESI that contained the variables of cancer, chronic cardiopulmonary   Meta-analysis of serious adverse events in PE with PESI low risk versus PESI high risk. Serious adverse events were defined as any of the following: nonfatal recurrent PE, nonfatal recurrent DVT, nonfatal bleeding, or delayed homodynamic instability. disease, heart rate, systolic blood pressure, and oxyhemoglobin saturation levels. 11 variables in PESI reduced to 6 variables in sPESI. The sPESI does not decrease prognostic accuracy compared with the original PESI, but sPESI is easier to use.
In general, the performance of prognostic models should be assessed in two ways: its discrimination and calibration aspects. Discrimination is the ability of the model to correctly separate the subjects into different groups. Calibration is the degree of correspondence between the estimated probability produced by the model and the actual observed probability [6]. Discrimination is also called predicting accuracy, which can be assessed by sensitivity (true positive rate), and specificity (true negative rate), ROC, and AUC. Our meta-analysis has already proved the PESI has good accuracy for predicting APE outcomes. The calibration is also assessed by the Hosmer-Lemeshow test [44]. But the data of the calibration was not found in the included studies. So we can't calculate the calibration for predicting prognosis in this meta-analysis. Some prospective studies for accessing PESI predicting calibration can be recommended.
Except we can't calculated the calibration for predicting prognosis in this meta-analysis, there are some other limitations. First, Lankeit M maybe used same participants in two included studies [17] and [19], as well as Singanayagam A's studies [26,29], and Jiménez D's studies [34,35]. Repeated include will increase the weight of the studies, and maybe result bias. Second, different PE haemodynamic status of patients included in our metaanalysis, some included haemodynamic stable patients, some included haemodynamic stable and unstable patients. But this bias is small, because evaluation of hemodynamic status is included in the PESI and sPESI. Third, the meta-analysis can't test the prognostic accuracy against other clinical score such as the Geneva prognostic score. Finally, most of the included studies were came from Europe and North America, only a study came from Asia, no study came from Africa, South America and Oceania. Some high-quality studies should