Validation of a brief telephone battery for neurocognitive assessment of patients with pulmonary arterial hypertension
© Taichman et al. 2005
Received: 26 January 2005
Accepted: 25 April 2005
Published: 25 April 2005
The effects of pulmonary arterial hypertension on brain function are not understood, despite patients' frequent complaints of cognitive difficulties. Using clinical instruments normally administered during standard in-person assessment of neurocognitive function in adults, we assembled a battery of tests designed for administration over the telephone. The purpose was to improve patient participation, facilitate repeated test administration, and reduce the cost of research on the neuropsychological consequences of acute and chronic cardiorespiratory diseases. We undertook this study to validate telephone administration of the tests.
23 adults with pulmonary arterial hypertension underwent neurocognitive assessment using both standard in-person and telephone test administration, and the results of the two methods compared using interclass correlations.
For most of the tests in the battery, scores from the telephone assessment correlated strongly with those obtained by in-person administration of the same tests. Interclass correlations between 0.5 and 0.8 were observed for tests that assessed attention, memory, concentration/working memory, reasoning, and language/crystallized intelligence (p ≤ 0.05 for each). Interclass correlations for the Hayling Sentence Completion test of executive function approached significance (p = 0.09). All telephone tests were completed within one hour.
Administration of this neurocognitive test battery by telephone should facilitate assessment of neuropsychological deficits among patients with pulmonary arterial hypertension living across broad geographical areas, and may be useful for monitoring changes in neurocognitive function in response to PAH-specific therapy or disease progression.
Pulmonary arterial hypertension (PAH) is a devastating disease characterized by progressive shortness of breath and the eventual development of life-threatening heart failure [1–4]. While its effects on cardiovascular function have been well documented, little is known about effects of this disease on other organ systems, notably the brain. Patients frequently complain of changes in memory, concentration and judgment in association with the development of cardiopulmonary symptoms. Objective assessments of cognitive function, however, have not been performed.
The "gold standard" for comprehensive assessment of neurocognitive function is a comprehensive battery of individually validated tests that are administered in-person by an experienced interviewer. Comprehensive, standard testing often takes four hours or longer and may require two or more separately scheduled sessions. The experience can be stressful and fatiguing, particularly for chronically ill or physically disabled patients. The stress of additional travel to a testing site adds to the burden. Consequently, many patients decline to participate in clinical research that uses formal in-person neurocognitive testing, especially if the protocol includes repeated testing over time [6, 7].
To address these concerns, we developed a focused battery of neurocognitive tests for administration over the telephone. The battery is comprised of individually validated neurocognitive test instruments appropriate for administration to adults with cardiopulmonary disease who are fluent in English and capable of communicating verbally. Telephone administration of the test battery has been evaluated for feasibility and validity in survivors of the acute respiratory distress syndrome . The present study establishes the validity of telephone administration of the test battery by comparing the results of telephone testing against "gold standard" in-person administration in adult ambulatory patients with moderately to severely symptomatic PAH.
Consecutive patients diagnosed with pulmonary arterial hypertension according to standard criteria [2, 9] were prospectively recruited for neurocognitive testing from the Pulmonary Hypertension Clinic at LDS Hospital. Written informed consent was obtained from the patients for both the in-person and telephone neuropsychological testing. Inclusion criteria were age 18 years or older and the ability to give informed consent. Exclusion criteria included non-fluency in English, a history of major psychiatric illness (e.g. schizophrenia, schizoaffective disorder, bipolar disorder, psychoses requiring medication and/or hospitalization, and major depression requiring hospitalization), known learning disability, prior traumatic brain injury, diagnosis of dementia, cerebral vascular accident, neurologic disorder (e.g. multiple sclerosis, Huntington's Disease, etc.), prior cardiac surgery, or current alcohol or drug abuse.
Patients were recruited for neurocognitive assessment from a group who had consented to in-person testing. Sixty-seven patients were screened. Seventeen patients declined, two were excluded due to non-fluency in English, and two were medically unstable at the time of evaluation and died during the recruitment period. Of the 46 patients who consented to in-person neurocognitive testing, 25 consented to additional testing by telephone.
Patient demographic, medical and laboratory data were collected for all enrolled patients. This study was approved by the University of Pennsylvania and LDS Hospital Institutional Review Boards and conformed to institutional and federal guidelines for the protection of human subjects.
Neurocognitive Battery for Telephone Administration
WMS-III: Digits Forward
WMS-III: Digits Backward
WMS-III: Letter-Number Sequencing
Hayling Sentence Completion Test
Language / Crystallized Intelligence
WMS-III: Logical Memory I & II
The in-person assessment was carried out in a private office at LDS Hospital. The identical tests were administered subsequently by telephone at a prearranged time when patients were at home and free from distraction. To minimize potential learning effects, telephone testing was performed at least 2.5 months following in-person assessment, except with one patient who was tested 57 days after in-person evaluation. A Ph.D. neuropsychologist (ROH) administered the in-person tests and a neuropsychology doctoral student administered the telephone tests with no knowledge of the results of the previous in-person testing. During both the in-person and telephone tests, patients were instructed not to write down information and to answer questions without assistance. The in-person and telephone assessments were both conducted in single sessions, and each took 35 to 45 minutes to complete.
All neuropsychological tests were scored according to the published guidelines. Each test yields a raw score that was converted into a scaled score (mean = 10; SD = 3), which was used for statistical analyses, except for Logical Memory where the raw scores are used.
Descriptive statistics were carried out for demographic and medical data. The neuropsychological test scores from the in-person administration were compared to telephone test scores using interclass correlations. To facilitate interpretation of significant correlations (p ≤ 0.05) and because traditional significance levels for correlations coefficients are influenced by factors such as group size, range of scores, and multiple comparisons, we used the following conservative classification: fair correlation with coefficients between 0.21 and 0.40; moderate correlation 0.41 to 0.60; substantial correlation 0.61 to 0.80; and almost perfect correlation 0.81 to 1.00 .
To assess potential learning effects, systematic differences between first and second administrations for each of the tests were assessed using paired sample t-tests. The differences between the in-person and telephone test results are expressed as standardized effects sizes (T2-T1 differences divided by T1 standard deviation) .
Demographic and Medical Data
Mean ± SD
Gender (% female) (n = 25)
13.6 ± 3.0
6 to 20
49.7 ± 13.9
20 to 69
Time Since Diagnosis (years)
1.8 ± 1.5
0.8 to 5.3
62.6 ± 13.5
38 to 97
Most recent 6 minute walk (meters)
455 ± 132
227 to 877
New York Heart Functional Class (N)
Supplemental Oxygen (N)
2 Liters per minute
3 Liters per minute
4 Liters per minute
In-person and telephone neuropsychological test scores.
5 to 14
5 to 15
14 to 33
17 to 44
10 to 30
6 to 34
6 to 18
5 to 15
Hayling Sentence Completion Test
4 to 8
6 to 10
4 to 16
5 to 16
6 to 16
7 to 17
Reliability of the in-person and telephone neuropsychological test scores.
Learning Effect (Improvement T1 to T2 expressed in SDs)
.78 (0 to 3.1)
-.28 to .52
1.3 (0 to 5.0)
-.20 to .72
1.2 (.18 to 3.4)
-.24 to .82
.41 to .90
-.24 to .84
.83 (0 to 1.9)
.29 to .83
.98 (0 to 3.8)
-.14 to .61
.65 (0 to 2.0)
.62 to .92
.53 (0 to 1.5)
.38 to .85
Stability over time was greatest for the Similarities and Vocabulary tests. The effects of learning showed that test scores tended to increase between the in-person and telephone tests, with the most improvement for verbal memory (e.g. logical memory immediate and delayed recall).
We found the battery of well-established neurocognitive tests to be amenable to administration by telephone and valid for the identification of neurocognitive deficits in patients with PAH. Testing was readily completed in a single, 30–60 minute session, and required neither specialized testing facilities nor travel by physically debilitated patients spread across a broad geographic area.
Our study was designed to validate the administration by telephone of a battery of neurocognitive tests against the in-person ("gold standard") performance of these same assessments. Each of the tests in the battery has been previously validated for the identification of neurocognitive deficits in various populations, including those with cardiopulmonary disease-. As such, our subjects' scores during in-person testing served as matched controls for comparison with the results obtained upon application of these same tests over the telephone.
The scores from telephone and in-person assessments correlated strongly for the majority of tests. Overall, the strengths of the correlations with in-person testing found here are comparable to those that we reported previously for the same test battery applied to ARDS survivors  and the correlations reported for other telephone neurocognitive test batteries [21–25]. Two items in the test battery did not correlate as well as the others. The interclass correlations for the Hayling Sentence Completion test only approached significance (p = 0.09). The in-person and telephone administrations of the Letter-Number-Sequencing component of the WMS III (a test of concentration/working memory) did not correlate well. Some subjects appeared to have difficulty discriminating phonetically similar sounds (e.g. the letters 'm' and 'n') when presented during telephone sessions; visual cues may have alleviated such issues at in-person sessions. In contrast, another component of the WMS III (Digits Backward) evaluating the same cognitive domain (concentration/working memory) had substantial correlations.
Although the correlations we found between in-person testing and subsequent telephone administration of the same test battery were moderate or higher, they were not perfect. The effects of learning or practice suggest that test scores increase between the in-person and telephone tests, with the most improvement in verbal memory. Thus, the improvement in test scores on the telephone administration of the test likely reflects practice effects. An alternative explanation for the tendency of subjects to perform better on the telephone test battery may be environmental factors. For example subjects scored higher on certain tasks when assessed at home as compared to similar tasks performed in a clinic setting . Improved orientation to time and place have been found when patients were tested in their own residence . Further, patients report less anxiety and prefer telephone testing compared to in-person evaluation . In addition to the pragmatic advantages, telephone testing may provide a better assessment of patients' cognitive function in their normal environment. Finally, it is possible that neurocognitive function improved during the interval between the two test sessions (mean 122 days). The reason for a potential improvement in neurocognitive performance will be important in future studies that use repeated test administration to determine the effect of drugs for PAH or other interventions on neurocognitive function.
An important limitation of our study was our inability to reverse the order of administration (in-person and telephone) . Our subjects were enrolled in another ongoing study, which required in-person assessments prior to enrollment in this study of telephone testing . An alternative would be to repeat in-person and telephone assessments in random order following the initial interview. Such additional testing, however, might have increased the potential for learning or practice affects, and the further time and travel commitments for patients likely impacting study participation. Future studies should counterbalance the order of in-person and telephone administration. Due to pragmatic limitations the time interval between in-person and telephone testing was somewhat longer than the minimum time necessary to minimize recall and learning effects.
Telephone testing has been used successfully in the assessment of neruocognitive impairment in other patient populations. Further, the ease in application and relative low cost of telephone testing have enabled assessments in several large clinical studies of cognitive function: screening of 4,932 elderly patients for Alzheimer's Disease using the modified Mini-Mental State Examination for telephone administration ; cognitive function in 4,023 patients with cardiovascular risk factors ; 466 patients with coronary artery bypass graft surgery , and in a self-referred ARDS patient group . In addition to neurocognitive testing, telephone-based assessments have provided accurate determinations of quality of life, medication usage, 24-hour physical activity and dietary recall [33–35].
Cognitive function has not been studied in pulmonary arterial hypertension, despite the frequent reports of problems with memory and concentration . Cognitive impairments are important complications of other chronic and life-threatening illnesses, and are associated with a significantly worse prognosis [36–38]. Cognitive impairments can also profoundly reduce quality of life [39–42]. Therapies that improve physical function in PAH may have important (but as yet unknown) effects on neurocognitive function, positive or negative. For these reasons, research on neurocognitive function is warranted. The brief neurocognitive telephone test battery is valid for assessment of cognitive function in this population and provides the means to pursue further, larger studies to assess the frequency and risk factors of cognitive sequelae in patients with PAH.
This study has demonstrated that scores on a battery of neurocognitive tests obtained by telephone administration correlated well with in-person testing in patients with pulmonary arterial hypertension. The strong correlations observed are comparable to previous studies that assessed in-person and telephone versions of neurocognitive tests. With minor modification, the telephone neuropsychological test battery described here provides an economical and reliable method for assessing cognitive function in patients with pulmonary arterial hypertension.
List of Abbreviations Used
Pulmonary Arterial Hypertension
Acute Respiratory Distress Syndrome
Wechsler Memory Scale
Wechsler Adult Intelligence Scale
These studies were supported by a Development Partner's Junior Faculty Award from GSK Pharmaceuticals (DBT). ROH is a recipient of grant support from the National Institute of Mental Health (R01 MH065406-01A1).
- Rubin LJ: Primary pulmonary hypertension. N Engl J Med 1997,336(2):111–7.View ArticlePubMedGoogle Scholar
- Barst RJ, et al.: Diagnosis and differential assessment of pulmonary arterial hypertension. J Am Coll Cardiol 2004,43(12 Suppl S):40S-47S.View ArticlePubMedGoogle Scholar
- Rich S, et al.: Primary pulmonary hypertension. A national prospective study. Ann Intern Med 1987,107(2):216–23.View ArticlePubMedGoogle Scholar
- D'Alonzo GE, et al.: Survival in patients with primary pulmonary hypertension. Results from a national prospective registry. Ann Intern Med 1991,115(5):343–9.View ArticlePubMedGoogle Scholar
- Hopkins RO, et al.: Cognitive dysfunction in patients with pulmonary arterial hypertension. Am J Respir Cell Mol Biol 2003,167(7):A273.Google Scholar
- Rothenhausler HB, et al.: The relationship between cognitive performance and employment and health status in long-term survivors of the acute respiratory distress syndrome: results of an exploratory study. Gen Hosp Psychiatry 2001,23(2):90–6.View ArticlePubMedGoogle Scholar
- Jackson JC, et al.: Six-month neuropsychological outcome of medical intensive care unit patients. Crit Care Med 2003,31(4):1226–34.View ArticlePubMedGoogle Scholar
- Christie J, et al.: Long-term cognitive, mood, and quality of life impairments in a select population of ARDS survivors from an internet-based ARDS support center. Am J Respir Cell Mol Biol 2002,165(8):A220.Google Scholar
- McGoon M, et al.: Screening, early detection, and diagnosis of pulmonary arterial hypertension: ACCP evidence-based clinical practice guidelines. Chest 2004,126(1 Suppl):14S-34S.View ArticlePubMedGoogle Scholar
- Gale SD, Hopkins RO: Effects of hypoxia on the brain: neuroimaging and neuropsychological findings following carbon monoxide poisoning and obstructive sleep apnea. J Int Neuropsychol Soc 2004,10(1):60–71.View ArticlePubMedGoogle Scholar
- Gale SD, Hopkins RO, Weaver LK, Walker JM, Bigler ED, Cloward TV: Hippocampal atrophy following sleep apnea and carbon monoxide poisoning: simialrities and differences. J Int Neuropsychol Soc 2000, 6:154.Google Scholar
- Hopkins RO, et al.: Neuropsychological sequelae and impaired health status in survivors of severe acute respiratory distress syndrome. Am J Respir Crit Care Med 1999,160(1):50–6.View ArticlePubMedGoogle Scholar
- Weaver LK, et al.: Hyperbaric oxygen for acute carbon monoxide poisoning. N Engl J Med 2002,347(14):1057–67.View ArticlePubMedGoogle Scholar
- Wechsler D: Wechsler Memory Scale. Volume 3. San Antonio: The Psychological Corporation; 1997.Google Scholar
- Wechsler D: Wechsler Adult Intelligence Scale. San Antonio: The Psychological Corporation; 1997.Google Scholar
- Kiernan RJ, et al.: The Neurobehavioral Cognitive Status Examination: a brief but quantitative approach to cognitive assessment. Ann Intern Med 1987,107(4):481–5.View ArticlePubMedGoogle Scholar
- Justice AC, Covinsky KE, Berlin JA: Assessing the generalizability of prognostic information. Ann Intern Med 1999,130(6):515–24.View ArticlePubMedGoogle Scholar
- Engelhart C, Eisenstein N, Meininger J: Psychometric properties of the neurobehavioral cognitive status exam. Clin Neuropsychol 1994,8(4):405–415.View ArticleGoogle Scholar
- Mitrushina M, Abara J, Blumenfeld A: Aspects of validity and reliability of the Neurobehavioral Cognitive Status Examination (NCSE) in assessment of psychiatric patients. J Psychiatr Res 1994,28(1):85–95.View ArticlePubMedGoogle Scholar
- Kukull WA, et al.: Interrater reliability of Alzheimer's disease diagnosis. Neurology 1990,40(2):257–60.View ArticlePubMedGoogle Scholar
- Prince MJMA, Richards N, Quarishi 5, Horn I: The development and initial validation of a telephone-administered cognitive test battery (TACT). International journal of methods in psychiatric research 1999, 8:49–57.View ArticleGoogle Scholar
- Roccaforte WH, et al.: Validation of a telephone version of the mini-mental state examination. J Am Geriatr Soc 1992,40(7):697–702.View ArticlePubMedGoogle Scholar
- Ferrucci L, et al.: Is the telephone interview for cognitive status a valid alternative in persons who cannot be evaluated by the Mini Mental State Examination? Aging (Milano) Ferrucci 1998,10(4):332–8.Google Scholar
- Desmond DW, Tatemichi TK, Hanzawa L: The telephone interview for cognitive status (TICS): reliability and validity in stroke sample. International journal of geriatric psychiatry 1994, 9:803–807.View ArticleGoogle Scholar
- Debanne SM, et al.: Validation of a Telephone Cognitive Assessment Battery. J Am Geriatr Soc 1997,45(11):1352–9.View ArticlePubMedGoogle Scholar
- Ward HW: Cognitive function testing in comprehensive geriatric assessment. A comparison of cognitive test performance in residential and clinic settings. J Am Geriatr Soc 1997,38(10):1088–92.View ArticleGoogle Scholar
- Kent J, Plomin R: Testing specific cognitive abilities by telephone and mail. Intelligence 1987, 11:391–400.View ArticleGoogle Scholar
- Robins LN: Reflections on testing validity of psychiatric interviews. Arch Gen Psychiatry 1985, 42:918–924.View ArticlePubMedGoogle Scholar
- White J, et al.: Relationship between cognitive and Emotional Function, asnd Quality of Life in Patients with Pulmoanry Arterial Hypertension (PAH). Am J Respir Crit Care Med 2004,169(7):A174.Google Scholar
- Breitner JC, et al.: APOE-epsilon4 count predicts age when prevalence of AD increases, then declines: the Cache County Study. Neurology 1999,53(2):321–31.View ArticlePubMedGoogle Scholar
- Garrett KD, et al.: The relationship between cardiovascular risk factors and cognitive decline. J Int Neuropsychol Soc 2003, 9:243.Google Scholar
- Potter GG, et al.: The effects of coronary artery bypass graft on cognitive status change among elderly male twins. J Int Neuropsychol Soc 2003, 9:243–244.Google Scholar
- Glasso KD, et al.: Relative validity of multiple telephone versus face-to-face 24-hour dietary recalls. Ann Epidemiol 1994, 4:332–336.View ArticleGoogle Scholar
- Matthews CE, et al.: Evaluation of computerized 24 hour physical activity recall. Med Sci Sports Exerc 2002,34(suppl S):236.Google Scholar
- Matthews CE, et al.: Comparing physical activity assessment methods in the Seasonal Variation of Blood Cholesterol Study. Med Sci Sports Exerc 2000,32(5):976–84.View ArticlePubMedGoogle Scholar
- Cohen RA, et al.: Neurocognitive functioning and improvement in quality of life following participation in cardiac rehabilitation. Am J Cardiol 1999,83(9):1374–8.View ArticlePubMedGoogle Scholar
- Andersen K, et al.: Cognitive impairment and mortality among nonagenarians: the Danish 1905 cohort survey. Dement Geriatr Cogn Disord 2002,13(3):156–63.View ArticlePubMedGoogle Scholar
- Zuccala G, et al.: The effects of cognitive impairment on mortality among hospitalized patients with heart failure. Am J Med 2003,115(2):97–103.View ArticlePubMedGoogle Scholar
- Tozzi V, et al.: Neurocognitive performance and quality of life in patients with HIV infection. AIDS Res Hum Retroviruses 2003,19(8):643–52.View ArticlePubMedGoogle Scholar
- Tozzi V, et al.: Neurocognitive impairment influences quality of life in HIV-infected patients receiving HAART. Int J STD AIDS 2004,15(4):254–9.View ArticlePubMedGoogle Scholar
- Weitzner MA: Psychosocial and neuropsychiatric aspects of patients with primary brain tumors. Cancer Invest 1999,17(4):285–91. discussion 296–7View ArticlePubMedGoogle Scholar
- Harder H, et al.: Cognitive functioning and quality of life in long-term adult survivors of bone marrow transplantation. Cancer 2002,95(1):183–92.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.