Measuring respiratory symptoms of COPD: performance of the EXACT- Respiratory Symptoms Tool (E-RS) in three clinical trials

Respiratory symptoms, including breathlessness, cough, and sputum production/expectoration, are defining features of chronic obstructive pulmonary disease (COPD) that can adversely affect patient functioning and quality of life [1]-[10]. Symptomatic relief is often the patient's primary concern, an important treatment goal for clinicians, and a key outcome in clinical intervention trials. With no known cure for COPD, effective symptom palliation and exacerbation prevention are paramount.

The EXACT - Respiratory Symptoms (E-RS) scale was designed to serve as a primary, secondary, or exploratory endpoint in clinical trials evaluating the effect of treatment on respiratory symptoms of COPD. The E-RS is based on the 11 respiratory symptom items from the 14-item EXACT, a daily diary used to measure exacerbations of COPD [11]-[15] (see Additional file 1: Table E7). The E-RS yields a total score, quantifying respiratory symptom severity overall, and 3 subscale scores assessing breathlessness; cough and sputum; and chest symptoms. This permits 2 validated uses for a single diary: quantification of respiratory symptoms in stable COPD using E-RS total and subscale scores and the assessment of acute exacerbations (frequency, severity, duration of symptom-defined events, and change in exacerbation symptoms with medically-treated events) using the EXACT total score [15],[16].

Good research practice and Food and Drug Administration (FDA) guidelines [17]-[19] for patient-reported outcome (PRO) measures were followed during E-RS development [15],[17]-[19]. Qualitative research was performed to assess content validity and reliability, validity, and sensitivity to change were tested in data from a prospective naturalistic study in the United States (US) [20]. To date, performance properties of E-RS scores within the context of international randomized controlled trials (RCTs) had not been established.

The objective of this study was to test the performance of E-RS scores in each of 3 independent, international RCTs evaluating 2 experimental drugs for the treatment of COPD.

Determining the extent to which interventions provide respiratory symptom relief requires randomized trials with precise endpoint measurement. Comparing treatment effects across studies, e.g., meta-analysis, requires comparable metrics. To date, there has been no standardized, reliable, and valid diary for evaluating the cardinal respiratory symptoms of COPD developed with regulatory standards for drug development tools in mind [17]-[19]. This paper describes the performance of the E-RS in 3 independent, international RCTs of stable, symptomatic patients with moderate to severe airway obstruction receiving maintenance therapy.

RS-Total and subscale scores exhibited evidence of reliability, validity, and responsiveness in each trial. Reliability was estimated using tests of internal consistency, a measure of scale coherence, and reproducibility over time. Across studies, estimates were strong, exceeding the 0.70 target for use in clinical trials [32],[35] and the more conservative 0.80 standard [33], suggesting E-RS scores are precise, with relatively low levels of measurement error. With one exception, values were similar to those reported previously [20]. The exception was the internal consistency estimates for the RS-Cough and Sputum in the Mpex sample (ICC for daily measurements 0.40; 0.52 weekly). This provides an interesting case study for reliability estimation. It is well known, and a function of the formula for coefficient alpha, that reliability is a characteristic of the scale scores in a study population. An inclusion criterion for the Mpex trial was mucopurulent sputum on most days. During baseline and over the study period, 6%-11% of this patient sample reported that it was extremely difficult to bring up mucus (phlegm); for comparison purposes, the ceiling effect for this item was observed in less than 2% of the other 2 samples. Ceiling effects in one variable will attenuate correlations that include this variable and hence reduce reliability coefficients. Thus, the lower coefficient for internal consistency in the RS-Cough and Sputum scale in the Mpex study sample is consistent the sputum severity characteristics of this sample. A drug that eased difficulty coughing up sputum should lead to a downward shift (improvement) in this aspect of cough and sputum, reducing the proportion of patients with extreme difficulties. The subscale should also show higher reliability levels with effective treatment, as the ceiling effect for this item is reduced or eliminated and equilibrated across the items comprising the scale.

The 2-day test-retest estimates in the Mpex data indicate consecutive daily scores are reproducible in patients who report no change in their lung condition over 2 days. The lower estimates observed over a 6-day interval in all subjects during the pre-randomization baseline run-in suggest some degree of variability in patients assumed to be stable over this period. Similar patterns were observed in the initial testing of E-RS scores [20]. Together, these results indicate a diary capturing symptom severity each day and averaged over time would be more accurate than periodic symptom assessments with longer recall periods commonly used in health status questionnaires, such as the SGRQ or Chronic Respiratory Questionnaire (CRQ) [36]. This not only enhances the precision of symptom severity estimates and treatment effects, but permits study of day-to-day symptom variability, an area in need of further research.

The magnitude and pattern of correlations and known-group differences were consistent with what would be expected of a valid patient-reported measure of respiratory symptoms. Coefficients were highest in tests of concurrent validity, i.e., between E-RS and BCSS scores, since both measure respiratory symptoms. In tests of convergent validity, E-RS and SGRQ health status scores were also strongly and consistently correlated across the 3 trials, although somewhat weaker than in the initial development study (0.75, 0.69, 0.58, 0.52 for RS-Total and subscale scores, respectively [20]). Another widely used measure of health status, the COPD Assessment Test (CAT) [37] was not administered in these trials; given the strong relationship between the SGRQ and CAT (e.g., r = 0.69 to 0.87) [38], one would expect the E-RS and CAT to be highly correlated as well. The E-RS should be complementary to heath status questionnaires such as the SGRQ, CAT, and CRQ in clinical trials since it captures day-to-day severity and variability of the cardinal respiratory symptoms of COPD with minimal recall bias and with content, subscale structure, and scores that capture these symptoms. To optimize data quality, particularly over lengthy trials, the E-RS should be completed as part of the EXACT, on a pretested, user-friendly electronic device programmed with reminders; subjects should be trained on the device and monitored for compliance during the course of the study [39].

Patients with more severe symptoms reported poorer functional performance, as measured by the FPI-SF. In keeping with divergent validity, correlations between respiratory symptoms and airway obstruction (FEV₁% predicted) were weak, although statistically significant. Of the 3 subscales, RS-Breathlessness was consistently the strongest correlate of FEV₁% predicted, mMRC, rescue medication use, and functional performance, all indicators or effects of dyspnoea. This is consistent with convergent validity for RS-Breathlessness and divergent validity for RS-Cough and Sputum and RS-Chest Symptoms.

E-RS scores were sensitive to change in patients showing improvement in health status (3 of 3 studies), symptoms (2 of 2 studies) or exercise tolerance (3 of 3 studies) from baseline to month 3. Exploratory analyses suggest E-RS scores are also sensitive to symptomatic worsening over 12 weeks.

In the initial testing of the E-RS, only distribution-based methods (½ standard deviation of baseline values) were available for responder definitions of symptomatic improvement; these gave values of: RS-Total: 3.35; RS-Breathlessness: 1.85; RS-Cough and Sputum: 1.15; and RS-Chest Symptoms: 1.05 [20]. Such methods are largely unvalidated in terms of their relationship to the patient's actual experience and may give values higher than the "true" responder threshold [40]. The current set of analyses support this caveat; the initial estimates should be interpreted as moderate to large, and not as minimal. Based on results from these 3 trials, across criterion-variable and distribution-based methods, variable and distribution-based methods, the following responder definitions for symptomatic improvement are proposed:

RS-Total ≥ −2.0 (scale range: 0-40)

RS-Breathlessness ≥ −1.0 (scale range: 0-17)

RS-Cough and Sputum ≥ −0.70 (scale range 0-11)

RS-Chest Symptoms ≥ −0.70 (scale range: 0-12)

Descriptive statistics and figures showing E-RS change scores over 12 weeks for responders and non-responders are provided in Additional file 1: Table E4 and Figures E1a–E1d. Results of the exploratory analyses suggest symmetric thresholds for symptomatic improvement and decline (i.e., applying ≥ +2.0; ≥ + 1.0; ≥ + 0.70; ≥ + 0.70 for Total and subscale scores, respectively for symptom worsening) (see Additional file 1: Tables E5 and E6). Given the magnitude of symptomatic improvement in responders and symptomatic decline in those whose symptoms worsened, these definitions may also be conservative estimates. Further research is needed in studies with global ratings of change, compounds showing treatment effects in several efficacy endpoints, including respiratory symptoms, and longer studies in patients at risk of decline.

Although the results presented here provide evidence that E-RS scores are reliable, valid, and sensitive to change in international RCT settings, several limitations should be noted. First, because the experimental drugs showed no therapeutic effects, responsiveness to treatment relative to placebo could not be evaluated. Tests of sensitivity to change and responder definitions were based on changes observed in criterion variables generally associated with symptomatic change. Results reported by Beier et al. [41] indicate E-RS scores are sensitive to symptomatic improvement with effective treatment over 6 weeks. In that trial, significant within- and between- group treatment effects were observed for aclidinium bromide and tiotropium versus placebo (E-RS-Total, RS-Breathlessness, and RS Chest Symptoms; significant effects for RS-Cough and Sputum for aclidinium only).

A second limitation is related to the study samples. Each trial enrolled symptomatic and clinically stable patients, consistent with the target population for the measure. In addition, the inclusion criteria specified a history of 1 or more clinic visits or hospitalizations for COPD exacerbation the prior 12 months. Although it is reasonable to expect the E-RS to perform similarly in patients without this history, further study should be undertaken to test this assumption. Finally, these trials were limited to 3 to 6 months; longer studies would permit an evaluation of the patterns and persistence of symptomatic improvement and worsening, including further study of thresholds for meaningful improvement and deterioration.

The E-RS provides a reliable, valid, and responsive method for quantifying respiratory symptom severity in clinical studies of COPD. Because the 11-item E-RS is embedded in the 14-item EXACT, a single diary can be used to evaluate the effects of treatment on day-to-day symptom severity in stable disease, using E-RS scores, and on acute exacerbations of COPD using the EXACT scoring algorithms.