The value of ROX index in predicting the outcome of high flow nasal cannula: a systematic review and meta-analysis

Background High flow nasal cannula (HFNC) therapy is widely employed in acute hypoxemic respiratory failure (AHRF) patients. However, the techniques for predicting HFNC outcome remain scarce. Methods PubMed, EMBASE, and Cochrane Library were searched until April 20, 2021. We included the studies that evaluated the potential predictive value of ROX (respiratory rate-oxygenation) index for HFNC outcome. This meta-analysis determined sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic score, diagnostic odds ratio (DOR), and pooled area under the summary receiver operating characteristic (SROC) curve. Results We assessed nine studies with 1933 patients, of which 745 patients experienced HFNC failure. This meta-analysis found that sensitivity, specificity, PLR, NLR, diagnostic score, and DOR of ROX index in predicting HFNC failure were 0.67 (95% CI 0.57–0.76), 0.72 (95% CI 0.65–0.78), 2.4 (95% CI 2.0–2.8), 0.46 (95% CI 0.37–0.58), 1.65(95% CI 1.37–1.93), and 5.0 (95% CI 4.0–7.0), respectively. In addition, SROC was 0.75 (95% CI 0.71–0.79). Besides, our subgroup analyses revealed that ROX index had higher sensitivity and specificity for predicting HFNC failure in COVID-19 patients, use the cut-off value > 5, and the acquisition time of other times after receiving HFNC had a greater sensitivity and specificity when compared to 6 h. Conclusions This study demonstrated that ROX index could function as a novel potential marker to identify patients with a higher risk of HFNC failure. However, the prediction efficiency was moderate, and additional research is required to determine the optimal cut-off value and propel acquisition time of ROX index in the future. PROSPERO registration number: CRD42021240607.


Introduction
High flow nasal cannula oxygen therapy (HFNC) is designed to supply constantly inspired oxygen concentration (21-100%), temperature (31-37 °C), and humidity through high flow (8-80 L/min) nasal prongs. Although it is applied in clinics for only a short time, it has become a critical oxygen therapy tool for hypoxemic acute respiratory failure (ARF) patients nowadays. Several recently published studies have confirmed that HFNC is safe and effective for ARF patients [1][2][3], particularly for decreasing the intubation risk. Roca et al. [4] identified that HFNC was the only variable associated with a lower risk of subsequent mechanical ventilation (MV) (odds ratio 0.11, 95% CI 0.02-0.69; P = 0.02) in ARF patients. Another study demonstrated that patients with PaO 2 / FIO 2 ≤ 200 mmHg exhibited a lower risk of intubation Open Access *Correspondence: lilihbch@163.com 1 Department of Critical Care Medicine, Zhejiang Hospital, Hangzhou 310013, Zhejiang, China Full list of author information is available at the end of the article than standard oxygen therapy or non-invasive ventilation (NIV) [5].
As a previous study demonstrated that delaying intubation increases mortality [6], patients with a higher risk of HFNC failure must be carefully monitored, and intubation indications should be dynamically evaluated. However, identifying patients with a high risk of HFNC failure is a significant challenge. Higher respiratory rate and lower PaO 2 /FIO 2 were two predictors of intubation in ARF patients treated with non-invasive oxygenation strategy [7], and since a great linear correlation is present between PaO 2 /FiO 2 and SpO 2 /FiO 2 parameters [8], SpO 2 /FiO 2 can be an effective substitute for PaO 2 /FiO 2 . Based on the above principles, a new index called ROX (respiratory rate-oxygenation, calculated by the ratio of SpO 2 /FiO 2 to respiratory rate) was developed in recent years. Several studies thought that ROX index had a good predictive capacity for HFNC outcome [9][10][11][12][13][14], whereas others demonstrated that ROX index should not be routinely utilized to predict HFNC outcome [15]. As a result, we conducted a systemic review and meta-analysis to systematically investigate the predictive capacity of ROX index for HFNC outcome.

Protocol and guidance
Our meta-analysis was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) [16]. The protocol of this study was registered, with PROSPERO registration number of CRD42021240607.

Literature search strategy
According to the criteria of literature retrieval strategies, two authors independently searched MEDLINE, EMBASE, and Cochrane Library electronic databases. Since the first relevant article was published in 2016, the retrieval time was from 2016 to April 20, 2021. Without language restrictions, we conducted a search using the following terms: respiratory rate-oxygenation/ROX/ROX index/ROXI and HFNC/high-flow nasal cannula/nasal high-flow oxygen therapy/high-flow oxygen therapy. The study research disagreements were resolved by discussions, and when discussions failed to address the disagreements, a third author was consulted.

Eligibility criteria
Literatures were selected based on the following eligibility criteria: (1) Studies evaluated the ability of ROX index in predicting HFNC outcome among adult patients (age over 18 years). (2) Number of true-positive (TP), truenegative (TN), false-positive (FP), and false-negative (FN) could be found or calculated by other data from the study. (3) No restrictions were set regarding the races or locations of patients and the types of diseases for which HFNC was employed. We excluded systematic reviews, case reports, pediatric studies, and repeated reports.

Data extraction
Two data collectors independently read the literatures according to inclusion and exclusion criteria and extracted the associated data elements from the enrolled papers using a self-made data extraction form. Inconsistencies were solved by discussion, and when discussion failed to address the disagreements, a third author was consulted. The extracted contents were as follows: (1) The basic characteristics of the included studies: name of the first author, publication year, locations of the study, the diseases that require HFNC use, number of patients, the study design, acquisition time of ROX index and definition of HFNC failure in each enrolled study. (2) Other essential data of each study include TP, TN, FP, FN, cutoff value of ROX index, as well as the sensitivity and specificity to predict HFNC failure. We also contacted the corresponding authors by email when necessary data were not included in the article.

Methodological quality assessment
Based on quality assessment of diagnostic accuracy studies 2 (QUADAS-2) guidelines [17], two authors independently assessed the risk of bias in enrolled studies, and a third author was invited to resolve disagreements during methodological quality assessment. The two main domains included risk of bias and applicability concerns. Each study evaluated the domain of risk of bias by evaluating patient selection, index test, reference standard, flow, and timing, whereas the domain of applicability concerns evaluated patient selection, index test, and reference standard. For each item, the risk of bias was classified as high, low, or unclear.

Statistical analysis
The data analysis was conducted using STATA version 1.4 (Stata Corporation, College Station, TX). This meta-analysis determined pooled sensitivity, specificity, negative likelihood ratio (NLR), positive likelihood ratio (PLR), diagnostic score, and diagnostic odds ratio (DOR). When statistical heterogeneity was not found (P > 0.05, I 2 < 50), the inconsistency factor (I 2 ) statistics and Cochran-Q test were employed to analyze the potential heterogeneity, and the fixed-effect model was applied to calculate the pooled effect size; otherwise, the random-effects model was utilized. Subgroup analyses were employed to explain source of heterogeneity, and they included the types of diseases associated with HFNC use, cut-off value, and the acquisition time of ROX index. In addition, Deeks' funnel plot was performed to estimate the potential publication bias.

Literature search results
A total of 371 literatures were searched, including 260 from PubMed, 65 from EMBASE, and 46 from the Cochrane Library databases, 3 from other sources (such as abstracts from conferences). There were 279 articles left after excluding the reduplicated articles. After reviewing the abstracts, some articles were excluded, including systematic reviews, case reports, and coverage not matched, and 17 articles remained, which we read in full. A total of 8 studies were excluded due to the lack of data required for analysis, whereas 9 articles met the full inclusion criteria finally. The detailed PRISMA flowchart is depicted in Fig. 1.

Results of methodological quality evaluation
After careful evaluation of the methodological quality of all nine enrolled studies, we found that seven studies [9,10,12,13,15,18,19] had a low risk of bias in patient selection, three trials [12,13,15] had a low risk of bias in index test, all the included studies exhibited low risk of bias in the item of reference standard, and eight studies [9-13, 15, 18, 19] had low risk in flow and timing. To sum up, the included seven studies were of good quality ( Table 3).

Subgroup analyses for the predicting value of ROX stratified by different conditions
To determine heterogeneity between studies, we conducted subgroup analyses based on the types of diseases correlated with HFNC use, the areas of enrolled studies, and the acquisition time of ROX index (Table 4).

Publication bias analysis
We performed Deeks' funnel plot to evaluate the potential publication bias, and the funnel plot demonstrated no statistically significant publication bias in this meta-analysis (P = 0.74) (Fig. 4).

Discussion
The accurate prediction of HFNC outcome ensures timely detection of patients with a higher risk of intubation. This study evaluated ROX index for sensitivity and specificity in predicting HFNC failure. Our metaanalysis revealed that pooled sensitivity and specificity of ROX index were 0.67 (95% CI 0.57-0.76) and 0.72 (95% CI 0.65-0.78), respectively, the overall value  This meta-analysis demonstrated that ROX index could be a viable tool for clinicians to assess HFNC progress and outcome, as confirmed by many previous studies. However, because there is certain degree of heterogeneity in patients with respiratory failure caused by different types of diseases, in subgroup analysis, we conducted subgroup analysis on patients in different populations, divided them into COVID-19 group and other population group (pneumonia and ARF), and the result showed higher discriminatory accuracy among COVID-19 group compared to other population group (pneumonia and ARF). However, the number of studies is too small for non-COVID-19 patients with only two studies about pneumonia patients and three studies about ARF patients. Furthermore, most ARF patients were caused by pneumonia (but the specific data are not explained in detail in some studies, we also tried to contact the researchers to ask for specific data, but in vain), Therefore, we did not conduct further subgroup analysis on other population (pneumonia and ARF), which may disturb the result of our meta-analysis. We thought more studies are needed to further analyze the predictive value of ROX among non-COVID-19 patients.
There is no universal agreement on the cut-off value of the ROX index, so the cut-off values used in the included studies varied from 2.7 to 9.2, we further conducted the subgroup meta-analysis stratified by cut-off value, and our subgroup analysis demonstrated higher discriminatory accuracy including studies used cut-off value > 5 compared to ≤ 5 cut-off value, thus, we thought > 5 cut-off value maybe close to the optimal cut-off value. For the acquisition time of the ROX index, most studies [9-12, 15, 19] reported 6 h as the acquisition time, however, our subgroup meta-analysis demonstrated good performance of ROX index was found in settings with other acquisition times, for the number of studies enrolled was small about the other acquisition times of the ROX index, we thought the ideal time to acquire the index remains unknown, and further studies are required to find the proper time.
This meta-analysis illustrated that prediction efficiency of ROX index was moderate, and several Table 3 Methodological quality assessment of studies included LR low risk; HR high risk; UR unclear risk; LC low concern; HC high concern; UC unclear concern

Study
Risk of bias  reasons may explain this. On the one hand, the cutoff and the acquisition time of ROX index varied in different studies, and differences in study patients may cause variation in the ideal cut-off value and the acquisition time. On the other hand, the ROX index could only reflect the work of breathing, and other factors such as tachycardia were associated with HFNC failure [7]. The prediction ability of ROX index may be improved when combined with other parameters. In 2020, Goh et al. [10] reported a new index called ROX-HR index (the ratio of ROX index over HR), and their study indicated that ROX-HR index higher than eight was significantly associated with HFNC success at 6 and 10 h, and ROX-HR index outperformed ROX index in predicting HFNC failure among postintubation patients. However, this was the only study that evaluated ROX-HR index value in predicting HFNC failure. Finally, the challenging problem is when to conduct intubation when ROX index is lower than the cut-off value. Should we act immediately or wait until the criteria to intubate are met completely? Obviously, studies are required to identify an optimal intubation time in the future. However, this meta-analysis also has several limitations. Firstly, most studies included had a small sample size; this may distort the results to some extent. Secondly, only one study was multicenter RCT, whereas the most were single-center prospective observational cohort studies. Thirdly, there was significant heterogeneity among the enrolled studies. Although we conducted subgroup analysis to identify the source of heterogeneity, other factors like bias in HFNC failure definition, the lack of common intubation criteria between studies may also be possible heterogeneity sources, which may cause practice variation. As a consequence, since findings and interpretations of this meta-analysis are limited by shortcomings showed above, it is crucial to be prudent when referencing some results of this study and must consider personal experiences and practical situations of patients.
Our meta-analysis has the following strengths. Firstly, this was the first meta-analysis to systematically assess ROX index value for predicting HFNC failure among different patients. Secondly, we conducted several meaningful subgroup analyses to comprehensively evaluate ROX index value in predicting HFNC failure, this may  help guide clinical practice. Thirdly, our study strictly conformed to the broad EQUATOR guidelines-a tool to ensure the value and reliability of research literature [20].

Conclusion
In summary, our meta-analysis reveals that ROX index is an easy-to-use and promising tool for clinicians to identify patients with a higher risk of HFNC failure, and those with a lower value of ROX index must be carefully monitored, accompanied by dynamic evaluation of intubation indications. Additional studies are required to determine the best cut-off value and the proper acquisition time of ROX index in the future.

Learn more biomedcentral.com/submissions
Ready to submit your research Ready to submit your research ? Choose BMC and benefit from: ? Choose BMC and benefit from: