Open Access

Development of the Pulmonary Arterial Hypertension-Symptoms and Impact (PAH-SYMPACT®) questionnaire: a new patient-reported outcome instrument for PAH

  • Deborah McCollister1Email author,
  • Shannon Shaffer2,
  • David B. Badesch1, 3,
  • Arthur Filusch4,
  • Elke Hunsche5,
  • René Schüler5,
  • Ingela Wiklund6 and
  • Andrew Peacock7
Respiratory Research201617:72

https://doi.org/10.1186/s12931-016-0388-6

Received: 12 September 2015

Accepted: 7 June 2016

Published: 14 June 2016

Abstract

Background

Regulators and clinical experts increasingly recognize the importance of incorporating patient-reported outcomes (PROs) in clinical studies of therapies for pulmonary arterial hypertension (PAH). No PAH-specific instruments have been developed to date in accordance with the 2009 FDA guidance for the development of PROs as endpoints in clinical trials. A qualitative research study was conducted to develop a new instrument assessing PAH symptoms and their impacts following the FDA PRO guidance.

Methods

A cross-sectional study was conducted at 5 centers in the US in symptomatic PAH patients aged 18–80 years. Concept elicitation was based on 5 focus group discussions, after which saturation of emergent concepts was reached. A PRO instrument for PAH symptoms and their impacts was drafted. To assess the appropriateness of items, instructions, response options, and recall periods, 2 rounds of one-on-one cognitive interviews were conducted, with instrument revisions following each round. Additional interviews tested the usability of an electronic version (ePRO). PRO development considered input from an international Steering Committee, and translatability and lexibility assessments.

Results

Focus groups comprised 25 patients (5 per group); 20 additional patients participated in cognitive interviews (10 per round); and 10 participated in usability interviews. Participants had a mean ± SD age of 53.1 ± 15.8 years, were predominantly female (93 %), and were diverse in race/ethnicity, WHO functional class (FC I/II: 56 %, III/IV: 44 %), and PAH etiology (idiopathic: 56 %, familial: 2 %, associated: 42 %). The draft PRO instrument (PAH-SYMPACT®) was found to be clear, comprehensive, and relevant to PAH patients in cognitive interviews. Items were organized in a draft conceptual framework with 16 symptom items in 4 domains (respiratory symptoms, tiredness, cardiovascular symptoms, other symptoms) and 25 impact items in 5 domains (physical activities, daily activities, social impact, cognition, emotional impact). The recall period is the past 24 h for symptoms, and the past 7 days for impacts.

Conclusions

The PAH-SYMPACT® was shown to capture symptoms and their impacts relevant to PAH patients, demonstrating content saturation, concept validity, and ePRO usability. Final content and psychometric validation of the instrument will be based on the results of an ongoing Phase IIIb clinical trial in PAH patients.

Keywords

Pulmonary arterial hypertensionPatient-reported outcomesHealth-related quality of lifeSymptomsActivities of daily living

Background

Pulmonary arterial hypertension (PAH) is a rare and debilitating chronic disease of the pulmonary vasculature, characterized by vascular proliferation and remodeling of the small pulmonary arteries [1, 2]. If not treated, it ultimately leads to right heart failure and premature death [3].

The most recent clinical classification scheme for pulmonary hypertension (PH) from the 5th World Symposium on PH distinguishes Group 1 PAH from other forms of PH (Groups 2–5) [4]. Although all PH Groups share symptoms in common [5], there may be differences between types of PH in the relative importance of symptoms and their impacts on patients [6], and potentially also in the types of symptoms.

The most common symptoms reported by patients with PAH are shortness of breath with exertion and fatigue [7]. Symptoms of PAH progress in severity if untreated [3], and may have a major impact on patients’ functioning and physical, psychological, and social well-being [8, 9]; higher rates of depression have been found among PAH patients [10].

Recommendations for PAH trial endpoints from the 4th and 5th World Symposia on PH stress the importance of measuring patient-reported outcomes (PROs) as a secondary endpoint in clinical trials [11, 12]. These developments reflect the rising importance accorded to the patient voice in the drug development and approval process across diseases [13]. Notably, the US Food and Drug Administration (FDA) has implemented a new Patient-Focused Drug Development (PFDD) initiative, intended to bring patient perspectives into early stages of product development [14]. As part of the PFDD, the FDA conducted a public meeting to capture perspectives from patients living with PAH about their disease, its impact on their daily life, and currently available therapies [15].

The FDA has established guidance providing clear scientific standards for clinical outcome assessment [16]. The 2009 FDA guidance for PROs defines a PRO assessment as a report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else [17]. The PRO guidance and the FDA’s 2013 roadmap to patient-focused outcome measurement in clinical trials both emphasize the need to document the content validity of a PRO in patients with the disease [17, 18]. Establishing content validity involves documenting that the structure and content (i.e., items) of the PRO instrument capture the connection between the intended measurement concept and the way patients from the target population understand and discuss that concept [19].

Most PRO questionnaires used in previous PAH clinical trials are generic quality-of-life (QoL) measures that do not adequately reflect the clinical status and symptoms, changes in health status and symptoms, or prognosis of patients with PAH [8, 20]. Other disease-specific PROs that have been used in PAH were either developed in patient populations other than PAH, such as the Minnesota Living with Heart Failure Questionnaire (MLHFQ) [21], or in mixed samples that included PAH patients and patients with other forms of PH. Accordingly, there was a need for a PRO questionnaire developed to assess PAH symptoms and impacts, which can serve as an efficacy endpoint in future PAH studies.

Methods

Study design

In accordance with the FDA roadmap to patient-focused clinical outcome assessment [18], a search was undertaken for existing questionnaires that would meet the PRO guidance requirements for use in PAH studies. No existing PRO measure developed according to the guidance requirements was found, and it was deemed not feasible to modify existing PRO measures. To develop a new disease-specific PRO, a multi-center, cross-sectional, qualitative PRO research study was conducted in patients with PAH in the US between November 3, 2011 and January 9, 2013. The overall study comprised 3 phases (Fig. 1): a concept elicitation phase leading to a draft PRO, a second phase comprising 2 rounds of cognitive interviews, and a final cognitive and usability interview phase based on the electronic version of the instrument (ePRO).
Fig. 1

Study flow

Interviewers used semi-structured interview guides, and interviews were transcribed by a third-party professional transcription service.

The study was conducted in consultation with a Steering Committee (SC) consisting of 3 expert clinicians from the US, the UK, and Germany, and a research instructor with a nursing background from the US. The SC provided clinical expertise and assisted with study design, interpretation of patient responses, item development and item modification, taking patients’ feedback into account. The SC also provided guidance on whether elicited symptoms were generally related to PAH rather than being due to comorbidities or side effects of treatment. The research protocol was approved by the institutional review board of each participating institution (Ethical Review Committee, Inc., Colorado Multiple Institutional Review Board, Legacy Health IRB and Intermountain Healthcare IRB). All participants provided written informed consent.

Each phase of the study included a translatability assessment to ensure that the items were appropriate and relevant across cultures, in order to facilitate future translations of the new PRO instrument. Lexibility assessment using the Flesch-Kincaid Grade Level Scale (scored to correspond to school grade level) was conducted after Phases 1 and 3 to ensure that the instructions, items, and response options were easy to understand by individuals with a range of reading levels. A 6th to 9th Grade reading level was considered acceptable, in terms of being inclusive of patients with lower levels of education [22].

Phase 1

The concept elicitation phase of the study involved discussions in focus groups, each with 5 PAH patients, using open-ended questions to elicit all concepts regarding PAH symptoms and their impacts on patients’ lives that were considered by patients to be relevant and important. Focus-group discussions were chosen as the appropriate format for concept elicitation as they allow respondents to use the ideas of others as cues to express their own views, and also to compare with their own experiences [19]. In this phase, potential item recall periods and response scale options for the questionnaire were also assessed.

The FDA requires evidence that concept saturation has been reached in the item-generation phase of PRO development [17]. In this context, saturation is defined as the point at which no new relevant or important information emerges and collecting additional data will not likely add to the understanding of how patients perceive the symptoms and impacts of their disease [17]. Accordingly, focus groups in the present study were continued until the point at which no substantially new information or concepts continued to emerge beyond what had been mentioned in previous focus-group discussions [23]. All concepts were summarized in saturation grids for symptoms and for impacts.

Once saturation was reached for symptoms and impacts, a draft PRO questionnaire was developed following generally accepted procedures [24], incorporating feedback from the SC, translatability assessment, and lexibility assessment. The saturation grid was used as the basis for identifying the items to be included in the draft questionnaire. Symptoms due to adverse events were not included, as the objective was to develop a PRO to serve as an efficacy endpoint in future PAH studies. For symptoms of PAH, the threshold for including an item was that at least 20 % of the patients in the total sample (i.e., ≥5 patients) had mentioned it. For impacts, the threshold was emergence of a concept in at least 2 of the 5 focus groups. The rationale for the different thresholds for including symptom vs. impact items was that symptoms, being typically expressed in specific terms (e.g., breathlessness), could be compared across individual patients. In contrast, impacts were often expressed in general terms (e.g., interference with daily activities) and individuals varied in how they described impacts.

For the draft PRO, a draft conceptual framework was developed, which is an organizing tool to depict the relationship of the items to hypothesized domains (i.e., categories of items) [19] for both PAH symptoms and impacts.

Phase 2

The overall objective of Phase 2 was to assess the appropriateness of several aspects of the draft PRO instrument, including confirming that PAH patients understood its items as intended and that they considered its items both relevant and comprehensive (i.e., no new concepts emerged in discussions with patients). In addition, Phase 2 assessed the appropriateness of the instructions, response options, and recall periods for symptoms and impacts. The formatting of the draft PRO was also evaluated.

Phase 2 consisted of 2 rounds of semi-structured one-on-one cognitive interviews with different patients from those participating in Phase 1, conducted either face-to-face or by telephone for those patients unable to easily travel to the study centers. Ten patients were included in each round of interviews (10 interviews are generally considered to be sufficient to reach saturation in cognitive interviewing [23]). Inclusion of a second round of cognitive interviews enabled confirmation of revisions to the draft PRO implemented after the first round. Questionnaire revisions followed each interview round, taking into account patient responses, as well as SC input and translatability assessment of any changes.

Phase 3

Phase 3 comprised 10 additional face-to-face cognitive interviews (following the same sample-size considerations as for Phase 2) with different patients from those participating in Phases 1 and 2, to confirm the content validity and appropriate patient interpretation of the PRO following limited changes implemented after Phase 2. Phase 3 also included usability testing by the 10 respondents of an electronic version of the questionnaire (ePRO) administered on a tablet computer. Prior to completing the ePRO, participants were provided with instructions and hands-on training on using the tablet. Final revisions of the ePRO followed Phase 3, with continued translatability assessment and lexibility assessment.

Patient population

Patients meeting study eligibility criteria were recruited from 5 clinical sites in different regions of the US (Table 1). All participants were required to meet the following inclusion criteria: age 18–80 years, inclusive; a definite diagnosis of PAH confirmed by right-heart catheterization; symptomatic in World Health Organization (WHO) Functional Class (FC) I–IV as documented within the past 6 months; and the ability to speak, read, and understand English.
Table 1

Study sites

Site number

Phase of study

1

2

3

1

X

X

X

2

 

X

X

3

X

X

X

4

 

X

 

5

X

  

To exclude patients with other diseases associated with similar symptoms to those of patients with PAH and/or patients likely unable to participate in interviews, patients meeting the following criteria were not eligible for the study: forms of PH other than PAH; moderate-to-severe obstructive (forced expiratory volume in 1 s/forced vital capacity [FEV1/FVC] < 70 %, and FEV1 < 65 % of predicted value after bronchodilator administration [25]) or restrictive lung disease (total lung capacity < 60 % of predicted value [26]) at diagnosis; diagnosis of obstructive sleep apnea; any other known concomitant life-threatening disease with a life expectancy <12 months; any other clinically relevant and/or serious chronic medical condition which, in the opinion of the investigator, would interfere with the patient’s ability to participate in an interview; and current participation in a randomized double-blinded clinical research trial that includes the use of investigational medications for any condition. Participants interviewed in one study phase were ineligible for subsequent study phases.

Recruitment aimed to achieve diversity of patients and to be generally representative of the PAH population seen in clinical practice and likely to be recruited in future clinical trials (Table 2). The primary source for developing these recruitment targets was the distribution of patient characteristics at enrolment in the REVEAL registry [27, 28]. Diverse disease severity was targeted to ensure that the PRO would be suitable for use across a wide range of PAH patients. Patients on oxygen therapy were intentionally over-recruited to ensure applicability of the instrument in these patients. Recruitment in Phase 3 aimed to enroll 50 % of patients aged 65 years or older, to ensure that the ability of patients in this age group to use the ePRO could be assessed.
Table 2

Patient demographic and clinical characteristics for targeted recruitment and enrolled cohort

 

Target

Phase 1: focus groups (n = 25)

Phase 2: cognitive interviews (n = 20)

Phase 3: cognitive/usability interviews (n = 10)

Total sample (n = 55)

Age (years)

     

 <65

Phase 1 & 2: No exact target aside from diversity;

Phase 3: 50 %

18 (72 %)

13 (68 %)

8 (80 %)

40 (73 %)

 ≥65

Phase 1 & 2: No exact target aside from diversity;

Phase 3: 50 %

7 (18 %)

6 (32 %)

2 (20 %)

15 (27 %)

 Mean ± SD

 

53.6 ± 15.5

54.3 ± 17.7

49.6 ± 13.1

53.1 ± 15.8

 Range

 

28–76

20–79

28–69

20–79

Gender (n, %)

     

 Female

75 %–80 %

23 (92 %)

18 (90 %)

10 (100 %)

51 (93 %)

 Male

20 %–25 %

2 (8 %)

2 (10 %)

0 (0 %)

4 (7 %)

Ethnicity (n, %)a

     

 Caucasian

70 %–75 %

17 (68 %)

12 (60 %)

8 (80 %)

37 (68 %)

 African American

7 %–10 %

4 (16 %)

4 (20 %)

1 (10 %)

9 (16 %)

 Hispanic or Latino

7 %–10 %

1 (4 %)

3 (15 %)

2 (20 %)

6 (11 %)

 Asian

3 %–5 %

0

0

0

0

 American Indian/Alaskan Native

 

3 (12 %)

0 (0 %)

0 (0 %)

3 (5 %)

 Otherb

 

0 (0 %)

1 (5 %)

0 (0 %)

1 (2 %)

Highest level of education (n, %)

No exact target aside from diverse educational background

    

 Elementary/primary school

 

0 (0 %)

0 (0 %)

1 (10 %)

1 (2 %)

 Secondary/high school

 

8 (32 %)

5 (25 %)

1 (10 %)

14 (25 %)

 Some college

 

12 (48 %)

7 (35 %)

6 (60 %)

25 (45 %)

 College degree

 

2 (8 %)

4 (20 %)

1 (10 %)

7 (13 %)

 Postgraduate degree

 

3 (12 %)

3 (15 %)

1 (10 %)

7 (13 %)

 Other

 

0 (0 %)

1 (5 %)

0 (0 %)

1 (2 %)

Incident vs. prevalent patients

     

 Newly diagnosed (incident)

10 %–15 %

2 (8 %)

5 (25 %)

2 (20 %)

9 (16 %)

 Previously diagnosed (prevalent)

85 %–90 %

23 (92 %)

15 (75 %)

8 (80 %)

46 (84 %)

Patient’s most recent FC

     

 FC I

5 %–8 %

2 (8 %)

2 (10 %)

0 (0 %)

4 (7 %)

 FC II

35 %–40 %

11 (44 %)

8 (40 %)

8 (80 %)

27 (49 %)

 FC III

45 %–50 %

11 (44 %)

9 (45 %)

2 (20 %)

22 (40 %)

 FC IV

3 %–5 %

1 (4 %)

1 (5 %)

0 (0 %)

2 (4 %)

PAH etiology

     

 APAH

50 %–55 %

11 (56 %)

9 (45 %)

3 (30 %)

23 (42 %)

 IPAH

45 %–50 %

14 (44 %)

10 (50 %)

7 (70 %)

31 (56 %)

 FPAH

3 %–5 %

0 (0 %)

1 (5 %)

0 (0 %)

1 (2 %)

Shortness of breath at its worst (numerical rating scale, 0–10), mean ± SD

N/A

    

 Walking on flat surface

 

4.0 ± 2.2

4.0 ± 2.9

4.8 ± 3.1

4.1 ± 2.7

 Climbing up 1 flight of stairs

 

6.6 ± 2.5

5.3 ± 3.2c

8.1 ± 2.0

6.4 ± 2.9c

aNot mutually exclusive; bParticipant wrote “Jamaican” (n = 1); c1 participant did not answer this question (Phase 2 n = 19, Total n = 54); APAH PAH associated with other conditions, FC functional class, FPAH familial PAH, IPAH idiopathic PAH, N/A not applicable, PAH pulmonary arterial hypertension, SD standard deviation

Assessments

To track patients characteristics vs. the recruitment targets and to characterize patients’ health, assessments administered in each study phase included a participant-completed sociodemographic form, a clinical questionnaire completed by site clinicians, as well as 2 numerical rating scales for shortness-of-breath while walking on a flat surface and climbing up 1 flight of stairs (0–10; 0 = “No shortness of breath at all”; 10 = “Most shortness of breath possible”) and the Medical Outcomes Study 36-item Short Form questionnaire (SF-36, version 2 [29]) which were completed by participants. Participants in Phase 3 also completed a Device Usability Questionnaire assessing the navigation, readability, screen layout, font size, ease of use, ease of selecting an answer, and overall usability of the ePRO.

Statistical analysis

Content analyses were performed on the qualitative data (i.e., the audio recordings and transcripts) from interviews. Interviews were analyzed using ATLAS.ti v5.0 qualitative data analysis software (ATLAS.ti Scientific Software Development GmbH), to assist with the organization of data and the identification of major concepts. The qualitative data analyses evaluated the saturation of concepts, clarity of instructions, item clarity, interpretation of items, ease of completion, appropriateness of response options and recall periods, and comprehensiveness of the questionnaire (i.e., no missing items).

An item tracking matrix for the PRO was developed to document the item generation and modification process. The item tracking matrix included the rationale for decisions to add, drop, retain, or modify items.

Quantitative data analyses were performed to describe patient demographic and clinical characteristics, as well as the results of the QoL and numeric rating scale assessments. Summary statistics for the quantitative analyses are reported as mean and standard deviation (SD) for continuous variables, and as number and percentage for categorical variables. SF-36 scores were normalized to the adult US population [30, 31].

Results

Patient characteristics

A total of 55 patients were enrolled across all 3 study phases: 25 in Phase 1, 20 in Phase 2, and 10 in Phase 3. Patients were diverse in key baseline characteristics (Table 2 and Additional file 1). The overwhelming majority of patients (84 %) were previously diagnosed (i.e., prevalent), and 45 % were using daily oxygen therapy. Nearly all patients (96 %) were taking at least 1 PAH-specific medication, and all were taking some medication to treat PAH (including PAH-specific and nonspecific medications).

Participants had similar SF-36 scores across study phases, which revealed poorer health status in all SF-36 domains in comparison with general population norms in the US, which by definition are 50 for each domain and summary component score (Fig. 2). The greatest health impairments were seen in the domains of physical functioning, role physical, general health, and social functioning. The importance of shortness of breath in PAH was confirmed by results of the numerical rating scales (Table 2).
Fig. 2

SF-36 results (mean ± standard deviation). Dashed line shows the general population norm score (50) for females and males in the US

Phase 1

Concept saturation was reached within 5 focus groups for all emergently reported PAH symptoms and within 4 focus groups for impacts (i.e., no previously unmentioned symptoms emerged in the fifth focus group and no new impacts emerged in the fourth and fifth focus groups). Symptoms reported by participants in all 5 focus groups were: shortness of breath, tiredness/fatigue, weakness/lack of energy, swelling (in ankles/legs/hands), rapid heartbeat/heart fluttering, chest pain/tightness, dizziness/lightheadedness, and fainting/passing out. Common impacts reported in at least 4 focus groups included: taking stairs, doing exercise, walking slowly or with difficulty, doing housework, running errands, sleeping difficulties, depression/sadness, frustration/anger, needing help from others, and effects on work or school.

The symptoms and impacts selected for inclusion in the draft PRO are listed in Table 3. To ensure the PRO reflects symptoms and impacts as experienced by patients, care was taken to review the comments actually expressed by the patients in their own words. Examples of comments spontaneously expressed by patients are shown in Table 4.
Table 3

Spontaneously reported PAH symptoms and impacts selected for inclusion in the draft PRO

 

n (%)

Symptoms (n = 25 patients)

 

 Shortness of breath

23 (92 %)

 Tiredness/fatigue/weakness/lack of energy

21 (84 %)

 Swelling in ankles or legs

19 (76 %)

 Rapid heartbeat/heart fluttering

19 (76 %)

 Chest pain/tightness

16 (64 %)

 Dizziness/lightheadedness

14 (56 %)

 Cough/dry cough

13 (52 %)

 Loss of appetite

10 (40 %)

 Swelling in stomach area

6 (24 %)

Impacts (n = 5 focus groups)a

 

Physical

 

 Stairs/exercise

5 (100 %)

 Carrying/lifting things

4 (80 %)

 Walking slower or with difficulty

3 (60 %)

Activities of Daily Living

 

 Household chores

5 (100 %)

 Errands

5 (100 %)

 Work (effects on career or school/reducing hours)

5 (100 %)

 Washing/dressing

3 (60 %)

Social Activities

 

 Needing help from others

4 (80 %)

 Going out to social activities

3 (60 %)

 Maintaining relationships

3 (60 %)

 Feeling embarrassed

2 (40 %)

Cognition

 

 Concentration/memory/articulating thoughts/slow thinking

3 (60 %)

Emotions

 

 Sad

4 (80 %)

 Frustrated/angry/mad

4 (80 %)

 Anxious/worried/stressed/loss of control

3 (60 %)

aRefers to difficulties in the listed areas

Table 4

Example quotations from participants about their PAH symptoms and impacts

Symptom/Impact

Example quotation

Shortness of breath

I’ve just always had shortness of breath and can hardly breathe sometimes in daily activities …

Tiredness

I’m always tired. …Always. I could just get up from a 4-hour nap, and still want to go back to bed because I’m tired.

Swelling in ankles or legs

… when I used to have it in my legs and my feet, my legs were so heavy that I could hardly even walk, and my feet were so swollen.

Rapid heartbeat

I’ve had my heart rate go up with just sitting. I’m just sitting there and all of a sudden it starts beating real, real fast, and I have to take a few deep breaths to try to get it to slow down.

Physical activities

You know they say walk-how far can you walk. [laughter] Oh, about two blocks and I have to sit down or something, you know.

Activities of daily living

…I’m the cook, and [laughter] used to I’d go in and prepare everything all at one time. Well, now I go in and peel my potatoes and cut them up and put them on the stove or whatever, and I go back and sit down for 20 minutes, and then I go back and prepare something else, until finally I have it all going on the stove cooking, but you have to stretch things out. …

Cognition

Um, yeah, your thinking sometimes just isn’t—how can I—how can I describe it? You’re not able to think things through clearly like you had in the past. Um, there’s a hesitation. You know, you might be asked a question, and there’s a hesitation because you—you’re trying to put the pieces together to respond.

Emotional impacts

I mean, I think the biggest impact is just the worry about it, you know. I worry about getting sicker. I worry about what could happen. You know, I know the stories and how it can go. And you know in the beginning, you read all this stuff, and you think you’re not going to live a year so you start thinking about that.

Based on the saturation grid from the focus-group results and with input from the SC, a draft questionnaire was developed with 16 symptom and 23 impact questions. Activities applicable to both genders were selected for inclusion. Emergent impacts were not included in the draft questionnaire if they were deemed to be infrequently experienced (less than weekly) or unlikely to be shared by most patients (e.g., pregnancy, sexual relations), and would therefore likely lead to missing values.

Similar items potentially measuring the same concepts (e.g., “cough” and “dry cough”) were included in the draft PRO developed after Phase 1; in Phase 2, patients were asked whether these items measure the same concept and if yes, which item should be kept.

A standard 5-point Likert response scale was chosen to assess each item. For symptoms, the response options were graded in terms of severity (no symptom to very severe symptom experience). For impacts, response options were graded either in terms of severity (not at all to very much) or the level of difficulty associated with the performance of the activity (none at all to extreme). Scale selection involved a consideration of both evidence on performance of different response scales described in the literature [32] and participant comments during the interviews: the 0–4 Likert scale with numbers and verbal anchors is commonly used in research practice for rating symptoms in pulmonary disorders and has been found to provide a sufficient range of answer choices to detect changes in symptom severity [33]. The 5-point Likert scale was well received by Phase 1 focus-group participants.

The past 24 h was chosen as an appropriate recall period for symptoms because some patients reported their symptoms changed day-to-day. The past 7 days was chosen as the recall period for impacts because patients may not have the opportunity to carry out impact activities on every day of the week. These recall periods were selected after a review of recall periods that were identified in the literature as being appropriate for assessment of symptoms and impacts of other chronic and debilitating diseases [3335], and with patient confirmation of their appropriateness.

Phase 2

The PRO instrument developed in Phase 1 was named the Pulmonary Arterial Hypertension-Symptoms and Impact (PAH-SYMPACT) Questionnaire®. The draft PAH-SYMPACT® was generally found to be clear, comprehensive, and relevant by PAH patients in the Phase 2 cognitive interviews. No additional concepts beyond those already mentioned in Phase 1 were uncovered in the new sample of patients interviewed in Phase 2.

In the first round of interviews, all participants (n = 10, 100 %) reported that the symptom questions applied to their current experiences with PAH and considered the formatting of the questions appropriate. Nearly all participants reported that the impact questions applied to their current experiences (n = 9, 90 %). All of the participants that were asked said that the symptom instructions were clear and easy to understand but some (4/9, 44 %) were unsure about whether they should answer impact questions based on whether or not they were using oxygen during the activity. Modifications made to the draft PAH-SYMPACT® after round 1 of the cognitive interviews, following additional input from the translation expert, included:
  • A question about oxygen use was added to allow for reporting of symptoms and impacts depending on oxygen use

  • One impact question was deleted because participants said it was not applicable

  • One impact question was made more specific to avoid variability in item interpretation

  • The question stem and response options for several impact items were reworded slightly

No changes were made to the number of symptom items, symptom question phrasing, instructions, or recall period.

In the second round of cognitive interviews using the modified PAH-SYMPACT®, all 10 participants said that the symptom questions applied to their PAH experience, considered the general phrasing of the symptom and impact questions to be appropriate, and confirmed that the instructions were clear and easy to understand. Nine participants (90 %) considered the impact questions applied to their experience. Additional modifications after the round-2 cognitive interviews, made with additional SC and translation-expert input, included:
  • One impact question was added and 1 deleted

  • Two impact questions were each separated into 2 distinct questions

  • Wording of some impact questions was modified slightly

The revised PAH-SYMPACT® after Phase 2 comprised 1 question about oxygen use, 16 symptom questions, and 25 impact questions.

Phase 3

In Phase 3, no participant reported difficulty with any feature of the tablet device; all had highly positive overall impressions of using it, and considered it an easy and convenient way to complete the questionnaire.

Based on participant responses in Phase 3, no relevant PAH symptoms or impacts were identified as missing. After patient feedback and input from the translation expert, the following minor modifications were made:
  • One symptom item and 1 impact item were reworded slightly

  • The example given for 1 impact item was changed to be more relevant

  • Some symptom- and impact-item instructions were modified slightly

The resulting current PAH-SYMPACT® incorporates 4 hypothesized symptom domains comprising 16 items, and 5 hypothesized impact domains comprising 25 items (Fig. 3).
Fig. 3

Conceptual framework

Lexibility analysis indicated that the mean ± SD reading grade level is 1.9 ± 1.7 for symptom items and 3.6 ± 2.8 for impact items. The reason for the higher average reading level for impact items is the need for longer questions to clarify the intended meaning of some impacts; nevertheless, impact items were easily understood by patients. For PAH-SYMPACT® instructions, the reading grade level was 11.0 for symptoms and 12.2 for impacts. The higher scores for instructions are accounted for by the need to identify the disease by its long and complex name, “pulmonary arterial hypertension”. Instructions were easily understood by participants since PAH patients are familiar with the name of their disease and use this term regularly.

Discussion

The PAH-SYMPACT® questionnaire is the first PRO instrument to be developed for PAH by following strictly the FDA’s PRO guidance. The qualitative research study described here supports the content validity of the PAH-SYMPACT® in a general PAH population in the US, thereby ensuring that the instrument is relevant to patients.

The PAH-SYMPACT® is designed to be a practical tool both for use in general practice and in future trials to assess the effect of PAH and PAH-specific therapies on patients’ symptoms and their impacts. To ensure the potential for its use as an efficacy endpoint in future PAH clinical trials, the PAH-SYMPACT® development process was rigorous with respect to the requirements of the FDA guidance [17]. A stepwise approach was followed, starting with a literature review to confirm the need for a new disease-specific PRO. Patient interviews were conducted in participants representative of both the broader PAH population and the intended population in PAH clinical trials, with recruitment targets being generally met. Patient responses were used as the basis for the development of all questionnaire items, with expert clinician input regarding interpretation of the elicited PAH symptoms and their impacts. The draft PRO was tested with additional PAH patients, whose cognitive interview transcripts demonstrated that no important items were missing and that patients understood the questionnaire’s wording as intended.

Migration of the questionnaire to a tablet device allowing greater convenience in terms of data capture and improved data quality (e.g., avoidance of missing data, automatic date and time stamp) is another strength of the PAH-SYMPACT®. As required by the FDA, a usability study was performed to provide evidence of the feasibility of PAH patients using the ePRO.

The input of a translation expert at each step of questionnaire revision should help to facilitate the adaptation of the PRO to different languages. Lexibility assessment results suggest the PAH-SYMPACT® will be understood even by respondents with relatively basic education. Interview participants were able to correctly restate the instructions in their own words, demonstrating that the questionnaire instructions were well understood.

Previously, only 3 disease-specific PROs have been developed for patients with PH: the Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR) [36], the Living with Pulmonary Hypertension questionnaire (LPH) [37], and the emPHasis-10 questionnaire [38]. However, none of these instruments meet the strict FDA PRO guidance criteria for use in PAH patients. The populations used to develop the CAMPHOR and the emPHasis-10 included patients with some other forms of PH, primarily chronic thromboembolic pulmonary hypertension, and there is no evidence of any discussion of differences of item relevancy or interpretation by PH etiology for either tool [36, 38]. Whether or not the same QoL measure can be used in patients with different forms of PH remains to be determined [20].

In addition, no evidence was reported that a saturation analysis was conducted for CAMPHOR and emPHasis-10 to confirm that all concepts important to patients were captured. Indeed, CAMPHOR is missing concepts such as dizziness, chest pain and palpitations [36], which emerged as important concepts during PAH-SYMPACT® development and which were described by participants in the 2014 FDA PFDD meeting with PAH patients [15]. The emPHasis-10 was developed primarily as a simple scoring system for QoL in PH patients intended for use in clinical practice, not clinical trials, which might explain why the instrument does not cover the full spectrum of PAH symptoms [38].

There was also no demonstration that saturation of symptoms was reached for the LPH, which was an adaptation of an existing measure, the MLHFQ, for use in PAH populations [37]. Although the LPH was developed based on interviews of PAH patients exclusively, the process did not incorporate the potential for addition of new symptoms and impacts beyond those already covered by the MLHFQ. The main modification was a revision to the recall period from 4 weeks to 1 week. Although the latter change was described as an important modification to ensure the instrument met the FDA preference for “short recall periods” [37], a 1-week recall period could introduce recall bias and a daily diary is likely preferable for symptoms, as patients might not be able to average symptom burden over several days.

While the PAH-SYMPACT® development process followed the FDA PRO guidance, there are some considerations when judging its applicability to the intended patient population. No individuals of Asian descent could be enrolled in the qualitative research, despite efforts to meet the recruitment target for this group. Nevertheless, the patient sample may still be considered representative of a clinical-practice population, since the proportion of Asians among US patients with PAH is very low [28]. Based on SC input, no racial differences in symptoms or impacts are anticipated in PAH; thus the lack of Asians in the study population is not expected to diminish the applicability of the questionnaire to this population. Another consideration is that more women were enrolled than seen in clinical practice (93 % vs. 80 % in REVEAL [27]), but since only symptoms and impacts that are generally experienced by both genders were included in the instrument, this should not have impacted the development process. Very few patients with familial PAH (2 %) or in FC IV (4 %) were included, reflecting the difficulty of recruiting patients from these uncommon subgroups.

Although the patients participating in the qualitative interviews were asked about their PAH symptoms and impacts, it may be difficult for patients to attribute symptoms and impacts to their disease. For example, certain PAH-specific therapies are associated with fluid retention or lightheadedness, both of which are also symptoms of PAH. To address this issue, instrument development incorporated feedback from the SC on whether symptoms mentioned by patients were generally PAH-related or potentially side-effects of treatment, guided by analyses of symptoms reported by patients receiving and not receiving different PAH-specific therapies.

Conclusions

The current PAH-SYMPACT® questionnaire was shown to be a practical and convenient tool for comprehensively capturing PAH symptoms and their impacts relevant to patients. Prior to its use in PAH clinical trials or clinical practice, the PRO needs to undergo final validation steps, including identification of a scoring algorithm, and quantitative testing to determine the psychometric measurement properties of the instrument. This final content and psychometric validation of the PAH-SYMPACT® will be based on the results of the SYMPHONY trial (Study AC-055-401; ClinicalTrials.gov No. NCT01841762), an ongoing Phase IIIb clinical trial in patients with PAH in the US.

Abbreviations

APAH, PAH associated with other conditions; CAMPHOR, Cambridge Pulmonary Hypertension Outcome Review; EMA, European Medicines Agency; FC, Functional class; FDA, Food and Drug Administration; FPAH, familial pulmonary arterial hypertension; IPAH, Idiopathic PAH; LPH, Living with Pulmonary Hypertension questionnaire; MLHFQ, Minnesota Living with Heart Failure Questionnaire; NYHA, New York Heart Association; PAH, Pulmonary arterial hypertension; PAH-SYMPACT®, Pulmonary Arterial Hypertension-Symptoms and Impact Questionnaire®; PH, Pulmonary hypertension; PRO, Patient-reported outcome; QoL, Quality of life; SC, Steering committee; SD, Standard deviation; SF-36, Medical Outcomes Study 36-item Short Form questionnaire; WHO, World Health Organization

Declarations

Acknowledgements

Medical writing support was provided by W. Mark Roberts, PhD, Montréal, Québec, Canada, funded by Actelion Pharmaceuticals Ltd.

Authors’ contributions

EH, RS and IW conceived of the study. SS, DBB, EH, RS, IW and AP contributed to study design. DM, SS and DBB were involved in data acquisition. All authors contributed to data analysis/interpretation and manuscript drafting/revision, and approved the final manuscript.

Competing interests

This study was sponsored by Actelion Pharmaceuticals Ltd, Allschwil, Switzerland. Patient recruitment, data collection, and data analysis were conducted by Evidera, funded by Actelion. D McCollister has received consulting fees from Actelion, Ikaria, and Novartis, and has served on boards or advisory committees for Bayer, Gilead, and United Therapeutics. S Shaffer and I Wiklund are salaried employees of Evidera. DB Badesch has received grant/research support from Actelion/CoTherix, AiRES, Bayer, Gilead, Lilly, the NIH, Novartis, Pfizer, and United Therapeutics/Lung Rx; has served as a consultant to Actelion/CoTherix, Arena, Bayer, Gilead, Ikaria, Lilly, MondoBiotech/MondoGEN, Pfizer, and United Therapeutics/Lung Rx; and has provided consultation to legal counsel for Actelion. A Filusch has received speaker fees and honoraria for consultations from Actelion, GSK, and Lilly. E Hunsche and R Schüler are employees of Actelion Pharmaceuticals Ltd. and have stock options or bond holdings in Actelion. A Peacock has received unrestricted grants for research, honoraria for speaking (non-promotional) at meetings, and assistance with travel from Actelion, Bayer, GSK, Lilly, Pfizer, and United Therapeutics.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Division of Pulmonary Sciences and Critical Care Medicine, University of Colorado Denver
(2)
Outcomes Research, Evidera
(3)
Division of Cardiology, University of Colorado Denver
(4)
Department of Pneumology and Cardiology, HPK – Heidelberg Private Clinic
(5)
Global Market Access and Pricing, Actelion Pharmaceuticals Ltd
(6)
Outcomes Research, Evidera, 1 Butterwick
(7)
Scottish Pulmonary Vascular Unit, Regional Heart and Lung Centre

References

  1. Eddahibi S, Morrell N, d’Ortho MP, Naeije R, Adnot S. Pathobiology of pulmonary arterial hypertension. Eur Respir J. 2002;20:1559–72.View ArticlePubMedGoogle Scholar
  2. Morrell NW, Adnot S, Archer SL, Dupuis J, Jones PL, MacLean MR, et al. Cellular and molecular basis of pulmonary arterial hypertension. J Am Coll Cardiol. 2009;54:S20–31.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Galiè N, Hoeper MM, Humbert M, Torbicki A, Vachiery JL, Barbera JA, et al. Guidelines for the diagnosis and treatment of pulmonary hypertension: the Task Force for the Diagnosis and Treatment of Pulmonary Hypertension of the European Society of Cardiology (ESC) and the European Respiratory Society (ERS), endorsed by the International Society of Heart and Lung Transplantation (ISHLT). Eur Heart J. 2009;30:2493–537.View ArticlePubMedGoogle Scholar
  4. Simonneau G, Gatzoulis MA, Adatia I, Celermajer D, Denton C, Ghofrani A, et al. Updated clinical classification of pulmonary hypertension. J Am Coll Cardiol. 2013;62:D34–41.View ArticlePubMedGoogle Scholar
  5. McLaughlin VV, Archer SL, Badesch DB, Barst RJ, Farber HW, Lindner JR, et al. ACCF/AHA 2009 expert consensus document on pulmonary hypertension. A report of the American College of Cardiology Foundation Task Force on Expert Consensus Documents and the American Heart Association developed in collaboration with the American College of Chest Physicians, American Thoracic Society, Inc., and the Pulmonary Hypertension Association. J Am Coll Cardiol. 2009;53:1573–619.View ArticlePubMedGoogle Scholar
  6. Cenedese E, Speich R, Dorschner L, Ulrich S, Maggiorini M, Jenni R, et al. Measurement of quality of life in pulmonary hypertension and its significance. Eur Respir J. 2006;28:808–15.View ArticlePubMedGoogle Scholar
  7. Matura LA, McDonough A, Carroll DL. Cluster analysis of symptoms in pulmonary arterial hypertension: a pilot study. Eur J Cardiovasc Nurs. 2012;11:51–61.View ArticlePubMedGoogle Scholar
  8. Chen H, Taichman DB, Doyle RL. Health-related quality of life and patient-reported outcomes in pulmonary arterial hypertension. Proc Am Thorac Soc. 2008;5:623–30.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Guillevin L, Armstrong I, Aldrighetti R, Howard LS, Ryftenius H, Fischer A, et al. Understanding the impact of pulmonary arterial hypertension on patients‘ and carers’ lives. Eur Respir Rev. 2013;22:535–42.View ArticlePubMedGoogle Scholar
  10. McCollister DH, Beutz M, McLaughlin V, Rumsfeld J, Masoudi FA, Tripputi M, et al. Depressive symptoms in pulmonary arterial hypertension: prevalence and association with functional status. Psychosomatics. 2010;51:339–e8.View ArticlePubMedGoogle Scholar
  11. McLaughlin VV, Badesch DB, Delcroix M, Fleming TR, Gaine SP, Galiè N, et al. End points and clinical trial design in pulmonary arterial hypertension. J Am Coll Cardiol. 2009;54:S97–107.View ArticlePubMedGoogle Scholar
  12. Gomberg-Maitland M, Bull TM, Saggar R, Barst RJ, Elgazayerly A, Fleming TR, et al. New trial designs and potential therapies for pulmonary artery hypertension. J Am Coll Cardiol. 2013;62:D82–91.View ArticlePubMedPubMed CentralGoogle Scholar
  13. DeMuro C, Clark M, Doward L, Evans E, Mordin M, Gnanasakthy A. Assessment of PRO label claims granted by the FDA as compared to the EMA (2006–2010). Value Health. 2013;16:1150–5.View ArticlePubMedGoogle Scholar
  14. Perfetto EM, Burke L, Oehrlein EM, Epstein RS. Patient-focused drug development: a new direction for collaboration. Med Care. 2015;53:9–17.View ArticlePubMedGoogle Scholar
  15. Food and Drug Administration. The voice of the patient: pulmonary arterial hypertension. http://www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrugUserFee/UCM429382.pdf. Accessed 14 June 2015.
  16. Food and Drug Administration. Clinical Outcome Assessment Qualification Program. http://www.fda.gov/Drugs/DevelopmentApprovalProcess/DrugDevelopmentToolsQualificationProgram/ucm284077.htm. Accessed 14 June 2014.
  17. Food and Drug Administration. Guidance for Industry. Patient-reported outcome measures: use in medical product development to support labeling claims. http://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf. Accessed 16 June 2014.
  18. Food and Drug Administration. Roadmap to patient-focused outcome measurement in clinical trials. http://www.fda.gov/Drugs/DevelopmentApprovalProcess/DrugDevelopmentToolsQualificationProgram/ucm370177.htm. Accessed 18 June 2014.
  19. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1--eliciting concepts for a new PRO instrument. Value Health. 2011;14:967–77.View ArticlePubMedGoogle Scholar
  20. Rubenfire M, Lippo G, Bodini BD, Blasi F, Allegra L, Bossone E. Evaluating health-related quality of life, work ability, and disability in pulmonary arterial hypertension: an unmet need. Chest. 2009;136:597–603.View ArticlePubMedGoogle Scholar
  21. Rector TS, Kubo SH, Cohn JN. Patients’ self-assessment of their congestive heart failure. Part 2: content, reliability and validity of a new measure, the Minnesota Living with Heart Failure Questionnaire. Heart Failure. 1987;3:198–209.Google Scholar
  22. Goldfarb NM, DuBay WH. Writing good at a seventh-grade reading level. J Clin Res Best Pract. 2006;2:1–4.Google Scholar
  23. Leidy NK, Vernon M. Perspectives on patient-reported outcomes: content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics. 2008;26:363–70.View ArticlePubMedGoogle Scholar
  24. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and use. New York: Oxford University Press; 1991.Google Scholar
  25. Pauwels RA, Buist AS, Calverley PM, Jenkins CR, Hurd SS, Committee GS. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease. NHLBI/WHO Global Initiative for Chronic Obstructive Lung Disease (GOLD) Workshop summary. Am J Respir Crit Care Med. 2001;163:1256–76.View ArticlePubMedGoogle Scholar
  26. American Thoracic Society. Lung function testing: selection of reference values and interpretative strategies. Am Rev Respir Dis. 1991;144:1202–18.View ArticleGoogle Scholar
  27. Badesch DB, Raskob GE, Elliott CG, Krichman AM, Farber HW, Frost AE, et al. Pulmonary arterial hypertension: baseline characteristics from the REVEAL Registry. Chest. 2010;137:376–87.View ArticlePubMedGoogle Scholar
  28. Frost AE, Badesch DB, Barst RJ, Benza RL, Elliott CG, Farber HW, et al. The changing picture of patients with pulmonary arterial hypertension in the United States: how REVEAL differs from historic and non-US Contemporary Registries. Chest. 2011;139:128–37.View ArticlePubMedGoogle Scholar
  29. Ware Jr JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30:473–83.View ArticlePubMedGoogle Scholar
  30. Ware JE, Kosinski M, Keller SD. SF-36® Physical and Mental Health Summary Scales: A User’s Manual. Boston: The Health Institute, New England Medical Center; 1994.Google Scholar
  31. Ware JE, Snow KK, Kosinski M, Gandek B. SF-36® Health Survey Manual and Interpretation Guide. Boston: The Health Institute, New England Medical Center; 1993.Google Scholar
  32. Rudell K, Vernon MK, Gries KS, Safikhani S, DeLozier AM, McQuarrie K, et al. What type of response scale is the most responsive? A comprehensive review of response scale options for patient-reported outcome measures. Value Health. 2015;18:A26.View ArticleGoogle Scholar
  33. Leidy NK, Wilcox TK, Jones PW, Murray L, Winnette R, Howard K, et al. Development of the EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT): a patient-reported outcome (PRO) measure. Value Health. 2010;13:965–75.View ArticlePubMedGoogle Scholar
  34. Howard K, Berry P, Petrillo J, Wiklund I, Roberts L, Watkins M, et al. Development of the Shortness of Breath with Daily Activities questionnaire (SOBDA). Value Health. 2012;15:1042–50.View ArticlePubMedGoogle Scholar
  35. Deal LS, Williams VS, Fehnel SE. Development of an electronic daily uterine fibroid symptom diary. Patient. 2011;4:31–44.View ArticlePubMedGoogle Scholar
  36. McKenna SP, Doughty N, Meads DM, Doward LC, Pepke-Zaba J. The Cambridge Pulmonary Hypertension Outcome Review (CAMPHOR): a measure of health-related quality of life and quality of life for patients with pulmonary hypertension. Qual Life Res. 2006;15:103–15.View ArticlePubMedGoogle Scholar
  37. Bonner N, Abetz L, Meunier J, Sikirica M, Mathai SC. Development and validation of the living with pulmonary hypertension questionnaire in pulmonary arterial hypertension patients. Health Qual Life Outcomes. 2013;11:161.View ArticlePubMedPubMed CentralGoogle Scholar
  38. Yorke J, Corris P, Gaine S, Gibbs JS, Kiely DG, Harries C, et al. emPHasis-10: development of a health-related quality of life measure in pulmonary hypertension. Eur Respir J. 2014;43:1106–13.View ArticlePubMedGoogle Scholar

Copyright

© The Author(s). 2016

Advertisement