Study design and treatment setting
Data were collected as an additional, informal part of a prospective clinical study with a 12 months follow-up period on the effects of a 12-week personalized multidisciplinary pulmonary rehabilitation program, either at a high-altitude (Dutch Asthma Centre in Davos, Switzerland) or at sea-level (Merem Asthma Centre in Hilversum, The Netherlands) [15]. The clinical study was originally set up as a trial and was registered at The Netherlands Trial Register (www.trialregister.nl; NTR5182), but it fell back to an observational design because randomization turned out to be not feasible. Nevertheless, patients were assessed and evaluated during and after their pulmonary rehabilitation in accordance with a systematic protocol.
Both asthma centres supplied structured, quality-controlled personalized treatment and pulmonary rehabilitation for adults with severe asthma. The rehabilitation included attempts to achieve optimal asthma control and to reduce (oral) corticosteroids to the lowest effective level, exercise training, asthma education including self-management, and psychological support. Both centres provided standardized treatment by using a modular approach with nine basic modules: medication and inhalation; exacerbation; self-management; physical fitness; daily physical activity; functional-ADL-training, dyspnea management; food and diet; coping; psychological support.
After the 12-week rehabilitation period, patients were followed for an additional 12 months with follow-up visits every 3 months at the asthma centre in The Netherlands. During the follow-up, patients were treated by their referring pulmonologist according to the (international) guidelines.
Patients
This validation study includes 83 adults (18–75 years) with severe refractory asthma who were living in The Netherlands and who filled out the CaReQoL Asthma at follow-up at 6 and 12 months after the pulmonary rehabilitation. All selected patients had a diagnosis of severe refractory asthma according to the ERS/ATS criteria [1], and they were referred by their pulmonologist to a tertiary asthma clinic between October 2015 and February 2018 [15].
At baseline, all patients were symptomatic and had uncontrolled asthma, they used long-acting bronchodilators and high dose inhaled corticosteroids with or without oral corticosteroid, and they were either non-smokers or ex-smokers for > 6 months [15]. Treatment of comorbidity was optimized before taking part in the study [15]. Uncontrolled asthma was defined by the presence of at least two of the following criteria: (1) poor symptom control defined as an ACQ-score ≥ 1.5 or an ACT-score < 20; (2) frequent severe exacerbations defined as two or more bursts of systemic corticosteroids (> 3 days) in the previous year; (3) serious exacerbations defined as at least one hospitalization or intensive care unit stay or mechanical ventilation in the previous year because of an asthma exacerbation; and/or (4) persistent airflow limitation (post-bronchodilator FEV1 < 80% predicted or a FEV1/forced vital capacity (FCV) z-score < 1.64). More details on the recruitment and selection procedure and the exclusion criteria are described in a previous paper on the clinical study by De Nijs et al. [15].
Patients voluntarily completed the CaReQoL Asthma in addition to the primary and secondary outcome measures of the clinical study. All patients provided their written informed consent on taking part in the clinical study which was approved by the Ethics Committee of the Academic Medical Center of the University of Amsterdam (Amsterdam, The Netherlands) [15]. The CaReQoL Asthma was not officially part of the clinical study, and an amendment about the additional informal data collection with this instrument was handed over to the Ethics Committee after the start of the clinical study.
Data collection and instruments
Data collection with the CaReQoL Asthma was an additional, informal part of the clinical study. Measurement points of the clinical study itself were at baseline or entry of the study (t0) and two additional follow-up measurements after completing the 12-week pulmonary rehabilitation trajectory: after 6 months (38 weeks after entry; t38) and at 12 months (64 weeks after entry; t64).
Apart from the CaReQoL Asthma, two other asthma-specific self-report questionnaires were used: the Asthma Quality of Life Questionnaire (AQLQ) and Asthma Control Questionnaire (ACQ), and a clinical measure for lung functioning (FEV1). The CaReQoL Asthma was only assessed retrospectively at t38 and t64 (i.e. 38 and 64 weeks after the start of the rehabilitation), whereas the other instruments (AQLQ, ACQ) and the FEV1 were assessed at all three points including baseline (t0, t38 and t64).
CaReQoL asthma
The Care Related Quality of Life in Asthma (CaReQoL Asthma) is a 26-item questionnaire about the perceived effects of pulmonary rehabilitation on the quality of life of asthma patients [13]. This PROM aims to retrospectively evaluate the ‘Care Related Quality of Life’, by asking asthma patients directly to rate the effects of rehabilitation on various health aspects and their overall quality of life. The face and content validity of this questionnaire have already been pre-tested in 15 cognitive interviews and a psychometric test based on a survey among 195 patients [13]. See Appendix for the content of the questionnaire.
The CaReQoL Asthma comprises five domains or scales: Physical functioning (8 items), Social functioning (4 items), Coping with asthma (5 items), Knowledge about asthma (3 items), and Medication (3 items). The 23 items measuring these five domains state to what extent the pulmonary rehabilitation contributed to the patients’ treatment goals and health aspects, with a 5-point answering scale: 1 = completely disagree, 2 = partly disagree, 3 = neutral (not agree, nor disagree), 4 = partly agree, 5 = completely agree. The escape option ‘not applicable’ was coded as missing. For each domain, an average scale score was calculated from the item scores (range 1–5), only for patients who completed more than half of the items within the corresponding domain. All five scales previously showed to have good internal consistency, with Cronbach’s alpha’s between 0.75 and 0.96 [13].
The last three items of the CaReQoL Asthma reflect on the overall effect of pulmonary rehabilitation on the quality of life (1 item; 1 = completely disagree, to 5 = completely agree), the global perceived effect (GPE [16]) of the rehabilitation on the patient’s health (1 item (How is your health, compared to your health prior to rehabilitation?); 1 = very much deterioration, 2 = much deterioration; 3 = little deterioration; 4 = no change; 5 = little improvement; 6 = much improvement; 7 = very much improvement), and the respondent’s age (in years), respectively.
Asthma quality of life questionnaire
The Asthma Quality of Life Questionnaire (AQLQ) is an asthma-specific 32-item questionnaire that measures the health-related quality of life of adults with asthma for four domains: Symptoms; Activity limitations; Emotional functioning; and Environmental stimuli [9]. Four domain scores and a total score were calculated (means of all item-scores). The Minimal Important Difference (MID) for the AQLQ is 0.5, which means that a change in score of 0.5 on the 7-point scale is the smallest change that can be considered clinically important [17].
Asthma control questionnaire
The Asthma Control Questionnaire (ACQ) measures both the adequacy of asthma control and change in asthma control, which occurs either spontaneously or as a result of treatment. In this study, a 6-item version of the ACQ questionnaire was used (without the clinic staff score of the FEV1 predicted) [18]. The ACQ score was calculated as the mean of the 6 symptom-related items (0 = totally controlled; 6 = severely uncontrolled; with scores < 0.5 indicating ‘controlled asthma’ and > 1.5 ‘uncontrolled asthma’). A change or difference in ACQ score of at least 0.5 can be considered as clinically important [18].
Forced expiratory volume in one second
Pulmonary function was measured according to international recommendations and included the forced vital capacity (FVC) and the forced expiratory volume in one second (FEV1), assessed after inhaled administration of 400 μg salbutamol and expressed as percentage of the predicted value [19]. The FEV1 at baseline (t0), t38 and t64 was used as an indicator of the pulmonary function in this paper.
Statistical analyses
The analyses focused on the construct validity (factor structure, internal consistency, convergent and divergent validity), criterion validity, responsiveness and the minimal important change (MIC). Analyses included: factor analyses, internal consistency (Cronbach’s alpha), Spearman's rank correlation coefficients (r), mean scores and 95% confidence intervals, paired t-tests and Student–Newman–Keuls (SNK) tests. All analyses were performed in Stata/SE (version 15) and a p-value of < 0.05 was considered as statistically significant.
Since the response rate for the CaReQoL Asthma was the highest at t38, this measurement point was mainly used to investigate the psychometric properties of the CaReQoL Asthma and to compare the CaReQol scores with change scores of the AQLQ, ACQ and FEV1 (∆ t0–t38). Except for assessing the responsiveness, for which the scores at t64 were used as well.
Construct validity
Defining the five domains of the CaReQoL Asthma was based on psychometric analyses, including factor analyses, conducted in a previous study (factor loadings range: 0.35–0.93, inter-scale correlations range: 0.59–0.90, n = 195) [13]. To confirm the psychometric properties of the questionnaire in the current study sample, factor analyses were conducted for each subscale separately (Principal Component Analyses, with oblimin rotations; Eigenvalue > 1 and factor loadings > 0.40). Then the internal consistency of the subscales was assessed by calculating the Cronbach’s alpha for each scale. An alpha of > 0.70 is generally considered as a good reliability [8, 20].
Subsequently, Spearman’s rank correlation coefficients (r) were calculated to see whether the scales or items of the CaReQoL Asthma do significantly correlate with other similar measures (i.e. convergent validity), or do not (i.e. divergent or discriminant validity). Convergent/divergent validity was assessed by calculating Spearman’s rank correlations between CaReQoL Asthma scores (t38) and change scores (Δ t0–t38) of similar domains of the AQLQ and the ACQ.
Based on the questionnaires’ content and more or less similar items, moderate correlations (r = 0.40–0.59) were expected between the following CaReQoL scores and related domains or items in the AQLQ and ACQ questionnaires:
-
Physical functioning vs. changes in AQLQ Total score, AQLQ Symptoms, AQLQ Activity limitations, and ACQ;
-
Social functioning vs. changes in AQLQ Total score, AQLQ Activity limitations and ACQ;
-
Coping with asthma vs. changes in AQLQ Emotional functioning and ACQ;
-
Medication vs. changes in AQLQ Symptoms and ACQ;
-
Overall quality of life (1 item) vs. changes in AQLQ Total score, AQLQ Symptoms and ACQ;
-
Global perceived effect (1 item) vs. changes in AQLQ Total score, AQLQ Symptoms and ACQ.
Weak (r < 0.40) or insignificant (p > 0.05) correlations were expected between all other combinations of total and domain scores, which would confirm divergent validity. See Table 3 for the hypothesized relationships. Good construct validity means that at least 75% of the hypotheses are correct [20].
Criterion validity
Criterion validity refers to a comparison between the measure in question and an outcome assessed at the same time (concurrent validity), and the extent to which the measure is related to or predicts a concrete outcome or criterion (predictive validity) [20]. In this study the GPE (global perceived effect) was chosen as a criterion, because it is a concrete and global measure for the perceived effect of the pulmonary rehabilitation at t38. The criterion validity was assessed by first subdividing respondents into three groups based on the self-reported global perceived effect (GPE) at t38, and by comparing the three subgroups with respect to the five CaReQoL domain scores and the change scores of the other health measures since the start of the rehabilitation trajectory (Δ t0–t38).
The three subgroups according to the criterion variable (GPE) are categorized as follows: (1) patients who reported a deterioration (a little, much or very much); (2) patients who reported ‘no change’ or ‘little improvement’; and (3) patients who reported ‘(very) much improvement’. The reason for including ‘little improvement’ in group 2 was the low number of patients in the category ‘no improvement’ (n = 6 at t38) and to create a more substantial reference group. Student–Newman–Keuls tests were performed to test the null hypotheses of ‘no differences between subgroups’. It was hypothesized that the CaReQoL Asthma very well reflects the GPE-scores at t38 (by showing significant differences in mean scores between the three groups at t38), thus proving the ability of this instrument to discriminate between patients with a different level of perceived effect, whereas changes in other scores (Δ t0–t38 AQLQ, ACQ, FEV1) are less likely to do so.
Responsiveness
Responsiveness is either defined as the effect of treatment, or as a measure of longitudinal validity defined by the correlation of changes in the instrument with changes in other measures [21]. In this study, the responsiveness of the CaReQoL Asthma, which aims to directly reflect the effects of pulmonary rehabilitation on several domains of health, was assessed in three steps. In step one, to investigate statistical significance of the longitudinal changes in care related aspects of health, the average scores on subsequent time points were compared. For the CaReQoL Asthma, the scores at t38 and t64 were compared by conducting one-sample paired t-tests, with the null hypotheses being ‘no difference between t38 and t64’. In step two, the CaReQoL Asthma scores at t38 were correlated with the change scores (Δ t0–t38) for other instruments, with the calculations and hypotheses being similar to those of the construct validity. In step three, the CaReQoL Asthma scores at t64 were correlated with the change scores for other instruments (Δ t0–t64), again with similar hypotheses as those formulated for the construct validity. See Table 3 for the hypothesized relationships.
Minimal important change
The minimal important change (MIC) was calculated for the CaReQoL Asthma at t38. The MIC is defined as the minimal change in a scale score that is considered to be important [20]. Firstly, three subgroups of the GPE were used (see "Criterion validity"). For each scale score, the average score of patients in group 2 (no or little improvement, see also "Criterion validity" for the consideration of combining these two groups) was subtracted from the average score of patients in group 3 (much or very much improvement). In addition, relevant changes or MICs of the AQLQ and ACQ scores (i.e. changes of at least 0.5 point) were also considered as external criteria for computing the MICs of the CaReQoL Asthma domains [17, 18].
