Conners Continuous Performce Test Reliability and Validitdy
Abstract
Objective
The Conners' Continuous Performance Test II (CCPT-II) is one of the most commonly used tests of sustained attention in schizophrenia. To manage and monitor sustained attention deficits in schizophrenia, clinicians have to reliably assess the degree of attention impairment. The purpose of our study was to determine the relative and absolute reliability of the CCPT-II in schizophrenia.
Method
The final sample was a total of 108 patients with schizophrenia. The primary measures in this study were the CCPT-II and the Clinical Global Impression-Severity (CGI-S). The CCPT-II was administered twice, 1 month apart, by a specially trained occupational therapist. We assessed two types of reliability: relative and absolute reliability. Relative reliability was determined with the intra-class correlation coefficient (ICC). Absolute reliability was quantified with standard error of measurement (SEM), minimal detectable change (MDC), and Bland–Altman's 95% limits of agreement (LOA).
Results
The ICCs for the five CCPT-II indexes (Omissions, Commissions, Hit RT, Hit RT Std Error, and Variability) ranged from 0.66 to 0.79. The MDCs (MDC%) of the five indices were 13.53 (156.78), 10.67 (47.85), 122.10 (34.16), 8.15 (106.82), and 25.81 (162.63), respectively.
Conclusions
According to the results, the CCPT-II has limited reliability in monitoring the sustained attention function of patients with schizophrenia. Our results can be used as a reference for the measurement error of CCPT-II to help clinicians and researchers determine the true change between successive assessments of patients with schizophrenia.
Introduction
Attention refers to being able to focus on a specific thing without being distracted (Gillig & Sanders, 2011) and is the most fundamental cognitive function in everyday life (Fett et al., 2011; Lysaker, Vohs, and Tsai, 2009; Okahashi et al., 2013). Attention includes sustained attention, selective attention, and divided attention (Chen et al., 2015). Sustained attention is the basic guarantee of complete selective attention, divided attention, and other cognitive functions (Chen et al., 2015). Cognitive deficit, one of the core symptoms of schizophrenia, is associated with impaired quality of life and poor functional outcomes (Green, Kern, Braff, and Mintz, 2000; Green, Kern, and Heaton, 2004; Koide et al., 2012). Among the variety of cognitive deficits in schizophrenia, attentional dysfunction is the most common (Shen et al., 2014). The measurement of attention (especially sustained attention) has been identified as one of the key outcomes for assessing cognitive impairment in psychological studies of schizophrenia (Rapisarda et al., 2014). In clinical settings, to detect and monitor the sustained attention status of patients with schizophrenia, this attention function is measured periodically. Thus, measures assessing sustained attention with sound reliability are crucial in the clinical setting.
Reliability refers to the consistency of assessment scores (Atkinson & Nevill, 1998; Downing, 2004). Test–retest reliability was the highest rated test criterion among surveyed experts in a recent NIH consensus symposium concerning neuropsychological tests for assessment of schizophrenia (Green et al., 2004). Two other types of reliability are relative and absolute reliability (Liaw et al., 2008). Relative reliability can be determined with the intra-class correlation coefficient (ICC) (Shrout & Fleiss, 1979). The ICC does not provide information about the accuracy of the scores of an individual (Overend, Anderson, Sawant, Perryman, and Locking-Cusolito, 2010). The ICC is a reliability parameter that is commonly used to represent the degree of reproducibility and examines the relationship between repeated measurements (Bland & Altman, 1990). However, the ICC cannot be inferred clinically because it provides no indication of the magnitude of disagreement between measurements (Liaw et al., 2012). Absolute reliability is quantified through standard error of measurement (SEM), minimal detectable change (MDC), and Bland–Altman's 95% limits of agreement (LOA) (Atkinson & Nevill, 1998; Bruton, Conway, and Holgate, 2000; Goldsmith, Boers, Bombardier, and Tugwell, 1993). SEM is used for measurement error to determine the precision of the total scores of both measurements (Heine, van den Akker, Verschuren, Visser-Meily, and Kwakkel, 2015; Stratford & Goldsmith, 1997). SEM can be used to calculate the MDC score at different confidence levels (Beckerman et al., 2001). The MDC is defined as the smallest amount of change in score that is not due to measurement error (Chuang et al., 2013; Haley & Fragala-Pinkham, 2006; Hesseberg, Bentzen, and Bergland, 2014; Schuck & Christian, 2003). If the difference between measurements from two test occasions exceeds the MDC value, the change is likely due not to measurement error but to real change (Haley & Fragala-Pinkham, 2006). The smaller the SEM and the MDC are, the more reliable the measure (Atkinson & Nevill, 1998).
The Conners' Continuous Performance Test II (CCPT-II) has a long history of use for measuring processes related to vigilance, response inhibition, signal detection, and other aspects of performance (Conners, Epstein, Angold, and Klaric, 2003). The CCPT-II measures a person's ability to discriminate a target stimulus from other stimuli (Sanz, Gomez, Vargas, and Marin, 2012). The CCPT 1.0 was a DOS-based research version that utilized the same properties as the later version. Version 1.0 was used to pilot the paradigm, obtain preliminary research results, and collect data. To the best of our knowledge, no reliability data related to persons with schizophrenia exists. The CCPT-II is generally considered a single overall measure of sustained attention, and the 5 CCPT-II indexes show the most power in discriminating patients with schizophrenia from healthy participants (Sanz, Gomez, Vargas, and Marin, 2012).
In terms of the reliability of the CCPT-II, acceptable test–retest reliability (0.55–0.84) was reported (Conners, 2004). However, the sample size of the study (Conners, 2004) that investigated the test–retest reliability of the CCPT-II was relatively small and included participants with various characteristics. These factors threaten the robustness of the results and generalizability of the aforementioned study (Conners, 2004). In addition, the absolute reliability of the CCPT-II for patients with schizophrenia is undetermined, limiting the interpretability and applicability of this measure for clinical settings. Thus, the purpose of our study was to determine the relative reliability (reflected by ICC values) and absolute reliability (reflected by SEM and MDC) of the CCPT-II in people with schizophrenia.
Methods
Participants
Participants were recruited from a clinical psychiatric hospital, and all were Taiwanese. Participants with the following characteristics were included: (A) diagnosis of schizophrenia according to the International Classification of Disease, 9th Revision, Clinical Modification (ICD-9-CM) diagnostic criteria; (B) clinical stability with a stable dose of antipsychotic medication for at least 3 months prior to the research testing sessions; (C) sufficient reading or listening comprehension to complete the CCPT-II; (D) absence of substance abuse or other neurological deficits (e.g., mental retardation, dementia, or developmental disability); (E) age 18–65 years; and (F) provision of informed consent. Criterion (B) was used to support the assumption for examining test–retest reliability (i.e., the attention performance of participants was assumed to be stable between two measurements).
Patients were excluded from the study if they: (A) were medically unstable, had other major diseases, or had secondary diagnoses of mental retardation, dementia, or neurological impairment; (B) were suffering from a severe medical or neurological condition or another psychiatric disorder that required treatment; (C) were participating in another clinical trial; and/or (D) were suffering an episode of major depression or exhibited difficulties in recognizing the letters of the English alphabet. The study protocol was reviewed and approved by the Institutional Review Board of the study hospital.
Originally, a total of 150 patients were approached for recruitment in the study, and 31 of them did not meet the criteria. In the end, 119 patients who were eligible for the study were recruited. Of these, 11 of the original participants did not complete the second session because they were discharged from the hospital, withdrew, or were otherwise lost to follow-up. Thus, the final sample included 108 patients.
Measures
The primary measures in the present study were the CCPT-II and Clinical Global Impression-Severity (CGI-S).
The Conners' Continuous Performance Test II (CCPT-II)
The CCPT-II has long been used in related studies (Lysaker, Tsai, Henninger, Vohs, and Viverito, 2010; Sanz, Gomez, Vargas, and Marin, 2012). The CCPT-II is a computerized test for sustained attention and impulsive control (Chen, Koh, Hsieh, and Hsueh, 2009). The CCPT-II has two important features: first, the participants are expected to respond to most of the presented stimuli when the computer program displays letters of the alphabet on the screen. This establishes a response mode that makes it more difficult to inhibit responses to a non-target. The CCPT-II is more sensitive to errors of commission, which together with a rapid response style are considered to measure hyperactivity-impulsivity (Egeland & Kovalik-Gran, 2010). Second, during the testing procedure, participants are required to sustain their attention on stimuli presented on a blank screen.
There are six blocks in the CCPT-II, each with three sub-blocks of 20 trials. Each trial is a letter presentation, either the letter X or others (Conners, 2004; Hodge et al., 2010). The CCPT-II involves the presentation of target and non-target stimuli (Conners, 1985). The measure runs for 14 min. Briefly, participants are required to respond to the stimuli on a computer screen by pressing a space bar for every letter except the letter "X". Slow reaction time combined with a large number of Omissions and Commissions indicates inattention; fast reaction combined with many Commissions but few Omissions reflects impulsivity. High scores of Hit Reaction Time (Hit RT) and Hit Reaction Time Standard Error (Hit RT Std Error) indicate highly variable reactions, usually related to inattention; a high score of Variability of Standard Error (Variability) is linked to the inability to sustain the optimal performance level throughout the test (Chen et al., 2009). Regarding the reliability of the CCPT-II, the split-half reliability is 0.66–0.95, and test–retest reliability within 3 months is 0.55–0.84, as shown in a study of 23 participants (10 non-clinical and 13 with a variety of clinical diagnoses) (Conners, 2004).
In summary, five indexes of the CCPT-II were analyzed in the present study, including two error indexes (i.e., Omissions and Commissions) and three time indexes (i.e., Hit RT, Hit RT Std Error, and Variability).
Clinical Global Impression-Severity (CGI-S)
The CGI-S, one of the most widely used measures in schizophrenia research and clinical practice, is a 1-item observer-rated scale (Clark et al., 2013). The scale is a clinician-rated instrument used to assess global severity of symptoms (Guy, 1976). The CGI-S is a standardized assessment to rate the severity of psychiatric illness (Huang et al., 2012). The ratings of illness severity are 1, not at all; 2, borderline; 3, mild; 4, moderate; 5, marked; 6, severe; and 7, extremely ill (Otto et al., 2010). The CGI-S was used to assess the change in clinical status (Leucht, Davis, Engel, Kissling, and Kane, 2009; Patrick et al., 2009; Wu et al., 2013).
Procedure
The CCPT-II was administered twice at an interval of 1 month by a specially trained occupational therapist to the participants. The occupational therapist was masked to the participants' status and severity. Participants completed the test within about 14 min. Patients' demographic data were collected from medical records. The psychopathological stability of the participants with schizophrenia was assessed with the CGI-S (Guy, 1976). The patients with stable CGI-S (i.e., the CGI-S category was exactly the same at the 2 time points) were included for further analysis. The CGI-S was administered by the same rater who is the therapist administering the CCPT-II at both time points (Nasrallah, Morosini, and Gagnon, 2008). Participants who had unstable CGI-S scores were excluded from this study. All participants confirmed that they had continued with their regular therapeutic activities during the interval between the two sessions in this study.
Statistical Analyses
Relative reliability was assessed through the calculation of ICC(2,1) (Shrout & Fleiss, 1979). The ICC is calculated using a 2-way mixed-effect model with an agreement coefficient (Prince, Makrides, and Richman, 1980). Because the same rater assessed each participant, the rater was considered a fixed-effect, and the participants were considered a random effect. Therefore, the ICC used the mixed model. The 2-way indicates a two-dimensional analysis, involving two independent variables (Portney & Watkins, 2009). An ICC value below 0.69 indicates poor reproducibility; that from 0.70 to 0.79, fair reproducibility; that from 0.80 to 0.89, good reproducibility; that greater than or equal to 0.90, high reproducibility (Arnall, Koumantakis, Oldham, and Cooper, 2002).
In regard to the absolute reliability, we quantified measurement errors through the computation of the SEM (i.e., standard deviation (SD) of all test–retest scores × √ (1 – ICC)) (Goldsmith et al., 1993). The MDC (1.96 × SEM × √2) was used to determine whether the change score of an individual patient was real (Beckerman et al., 2001; Lexell & Downham, 2005). The MDC% is the MDC divided by the mean of all scores for the sample. An MDC% of less than 30 is considered acceptable, and an MDC% of less than 10 is considered excellent (Smidt et al., 2002). Generally, the smaller the SEM is, the more reliable the measure (Atkinson & Nevill, 1998). The MDC exhibits the smallest change that indicates a real improvement for a single individual (Beckerman et al., 2001; Hopkins, 2000; Liaw et al., 2008; Schuck & Christian, 2003). A measure for monitoring change over time should have low SEM and MDC.
Schizophrenia has a number of subtypes, the most common of which is the paranoid subtype. We also calculated the values of ICC, SEM, and MDC for Omissions, Commissions, Hit RT, Hit RT Std Error, and Variability, respectively, in participants with the paranoid subtype.
We used a paired t-test to examine whether systematic biases existed. The differences between test and retest were considered significant if p-values were smaller than.05.
We also conducted Bland–Altman analysis, which included a scatter plot of the differences between test and retest values against their mean with 95% LOA (i.e., mean difference ±1.96 × SDs of the difference) (Bland & Altman, 1986). The plot shows the difference between test sessions 2 and 1 (2−1) against the mean of the two test sessions for each participants (Bland & Altman, 2003). Thus, it provides visual interpretation for ease of inspection of the size and range of differences between the two sessions (Liaw et al., 2008). In addition, the Pearson's r was used to examine the heterogeneity of the sample. A correlation between the mean and absolute value of difference of two successive sessions of more than 0.3 was viewed as evidence of heteroscedasticity (Bland & Altman, 1990). SPSS version 18.0 (SPSS Inc., Chicago, IL) was used for data analysis. The alpha level was set at 0.05 for all statistical tests, and all p-values are two-tailed. The study also analyzed the effect sizes of the five indexes between the test and retest measures. The effect size was computed by using Cohen's d.
The split-half reliabilities were computed from 18 subblocks (six blocks each with three different inter-stimulus intervals, ISIs) split into two groups of nine subblocks, such that each half of the test was equally represented in a counterbalanced fashion.
The correlation between clinical variables (i.e., age of onset, duration of illness, CGI-S level, and different gender) and each index of the CCPT-II was analyzed by Pearson correlation test and one-way ANOVA. The study also analyzed the effect sizes in the mean differences between test–retest measurements of each index of the CCPT-II related to clinical variables (CGI-S level and different gender). The effect size was computed by using Cohen's d.
In addition to raw scores, standardized scores were also calculated for the reliabilities of the CCPT-II by ICC, SEM, and MDC.
Results
Demographic and Clinical Characteristics
We recruited 119 patients who were eligible for the study. Of these, 11 of the original participants did not complete the second session because they were discharged from the hospital, withdrew, or were otherwise lost to follow-up. Thus, the final sample included 108 patients. All of these participants tested into the same CGI category at both sessions. Their CGI-S scores were mostly mild (40.7%) or borderline (30.6%). Table 1 displays demographic data for the entire sample of 108 participants.
Table 1.
Demographic characteristics of the sample (n = 108)
| Variable | M | SD |
|---|---|---|
| Age | 43.6 | 9.4 |
| Onset age | 22.9 | 6.6 |
| Psychiatric history in years | 20.7 | 9.2 |
| Variable | N | % |
| Gender | ||
| Female | 46 | 42.6% |
| Male | 62 | 57.4% |
| Education status | ||
| College | 12 | 11.1% |
| Senior high school | 57 | 52.8% |
| Junior high school | 34 | 31.5% |
| Elementary school | 3 | 2.8% |
| None | 2 | 1.9% |
| Schizophrenia subtypes | ||
| Disorganized | 9 | 8.3% |
| Catatonic | 1 | 0.9% |
| Paranoid | 85 | 78.7% |
| Residual | 11 | 10.2% |
| Undifferentiated | 2 | 1.9% |
| CGI-S scores | ||
| Not at all (1) | 31 | 28.7 |
| Mild (2) | 44 | 40.7 |
| Borderline (3) | 33 | 30.6 |
| Variable | M | SD |
|---|---|---|
| Age | 43.6 | 9.4 |
| Onset age | 22.9 | 6.6 |
| Psychiatric history in years | 20.7 | 9.2 |
| Variable | N | % |
| Gender | ||
| Female | 46 | 42.6% |
| Male | 62 | 57.4% |
| Education status | ||
| College | 12 | 11.1% |
| Senior high school | 57 | 52.8% |
| Junior high school | 34 | 31.5% |
| Elementary school | 3 | 2.8% |
| None | 2 | 1.9% |
| Schizophrenia subtypes | ||
| Disorganized | 9 | 8.3% |
| Catatonic | 1 | 0.9% |
| Paranoid | 85 | 78.7% |
| Residual | 11 | 10.2% |
| Undifferentiated | 2 | 1.9% |
| CGI-S scores | ||
| Not at all (1) | 31 | 28.7 |
| Mild (2) | 44 | 40.7 |
| Borderline (3) | 33 | 30.6 |
Notes: M = mean; SD = standard deviation; CGI-S = Clinical Global Impression-Severity.
Table 1.
Demographic characteristics of the sample (n = 108)
| Variable | M | SD |
|---|---|---|
| Age | 43.6 | 9.4 |
| Onset age | 22.9 | 6.6 |
| Psychiatric history in years | 20.7 | 9.2 |
| Variable | N | % |
| Gender | ||
| Female | 46 | 42.6% |
| Male | 62 | 57.4% |
| Education status | ||
| College | 12 | 11.1% |
| Senior high school | 57 | 52.8% |
| Junior high school | 34 | 31.5% |
| Elementary school | 3 | 2.8% |
| None | 2 | 1.9% |
| Schizophrenia subtypes | ||
| Disorganized | 9 | 8.3% |
| Catatonic | 1 | 0.9% |
| Paranoid | 85 | 78.7% |
| Residual | 11 | 10.2% |
| Undifferentiated | 2 | 1.9% |
| CGI-S scores | ||
| Not at all (1) | 31 | 28.7 |
| Mild (2) | 44 | 40.7 |
| Borderline (3) | 33 | 30.6 |
| Variable | M | SD |
|---|---|---|
| Age | 43.6 | 9.4 |
| Onset age | 22.9 | 6.6 |
| Psychiatric history in years | 20.7 | 9.2 |
| Variable | N | % |
| Gender | ||
| Female | 46 | 42.6% |
| Male | 62 | 57.4% |
| Education status | ||
| College | 12 | 11.1% |
| Senior high school | 57 | 52.8% |
| Junior high school | 34 | 31.5% |
| Elementary school | 3 | 2.8% |
| None | 2 | 1.9% |
| Schizophrenia subtypes | ||
| Disorganized | 9 | 8.3% |
| Catatonic | 1 | 0.9% |
| Paranoid | 85 | 78.7% |
| Residual | 11 | 10.2% |
| Undifferentiated | 2 | 1.9% |
| CGI-S scores | ||
| Not at all (1) | 31 | 28.7 |
| Mild (2) | 44 | 40.7 |
| Borderline (3) | 33 | 30.6 |
Notes: M = mean; SD = standard deviation; CGI-S = Clinical Global Impression-Severity.
Relative Reliability
Table 2 shows the test–retest reliability for the CCPT-II. The ICCs for the five CCPT-II indexes ranged from 0.66 to 0.79. The Omissions, Commissions, and Hit RT Std Error indicated fair test–retest agreement. The Hit RT and Variability were poor. In the study, most of the participants had paranoid type schizophrenia (78.7%). The ICCs for the five CCPT-II indexes for the paranoid subtype participants were 0.75, 0.77, 0.65, 0.67, and 0.65 for the Omissions, Commissions, Hit RT, Hit RT Std Error, and Variability, respectively.
Table 2.
Test–retest reliability of the Conners' Continuous Performance Test II (raw scores)
| Agreement | ||||||
|---|---|---|---|---|---|---|
| Measure | 1st testing M (SD) | 2nd testing M (SD) | Difference M (SD) | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 8.90 (9.59) | 8.37 (9.60) | −0.53 (6.87) | 0.74 (0.65−0.82) | 4.88 | 13.53 (156.78) |
| Commissions (error) | 22.08 (8.34) | 22.51 (8.50) | 0.43 (5.44) | 0.79 (0.71−0.85) | 3.85 | 10.67 (47.85) |
| Hit reaction time (ms) | 355.74 (72.92) | 359.03 (74.40) | 3.29 (62.54) | 0.66 (0.54−0.76) | 44.05 | 122.10 (34.16) |
| Hit reaction time standard error (ms) | 7.24 (4.61) | 8.03 (6.04) | 0.78 (4.12) | 0.70 (0.59−0.79) | 2.94 | 8.15 (106.82) |
| Variability of standard error (ms) | 14.52 (13.73) | 17.22 (18.75) | 2.70 (13.05) | 0.68 (0.56−0.77) | 9.31 | 25.81 (162.63) |
| Agreement | ||||||
|---|---|---|---|---|---|---|
| Measure | 1st testing M (SD) | 2nd testing M (SD) | Difference M (SD) | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 8.90 (9.59) | 8.37 (9.60) | −0.53 (6.87) | 0.74 (0.65−0.82) | 4.88 | 13.53 (156.78) |
| Commissions (error) | 22.08 (8.34) | 22.51 (8.50) | 0.43 (5.44) | 0.79 (0.71−0.85) | 3.85 | 10.67 (47.85) |
| Hit reaction time (ms) | 355.74 (72.92) | 359.03 (74.40) | 3.29 (62.54) | 0.66 (0.54−0.76) | 44.05 | 122.10 (34.16) |
| Hit reaction time standard error (ms) | 7.24 (4.61) | 8.03 (6.04) | 0.78 (4.12) | 0.70 (0.59−0.79) | 2.94 | 8.15 (106.82) |
| Variability of standard error (ms) | 14.52 (13.73) | 17.22 (18.75) | 2.70 (13.05) | 0.68 (0.56−0.77) | 9.31 | 25.81 (162.63) |
Notes: M = mean; SD = standard deviation; ICC = intra-class correlation coefficients; SEM = standard error of measurement; MDC = minimal detectable change; CI = confidence interval; ms = millisecond.
Table 2.
Test–retest reliability of the Conners' Continuous Performance Test II (raw scores)
| Agreement | ||||||
|---|---|---|---|---|---|---|
| Measure | 1st testing M (SD) | 2nd testing M (SD) | Difference M (SD) | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 8.90 (9.59) | 8.37 (9.60) | −0.53 (6.87) | 0.74 (0.65−0.82) | 4.88 | 13.53 (156.78) |
| Commissions (error) | 22.08 (8.34) | 22.51 (8.50) | 0.43 (5.44) | 0.79 (0.71−0.85) | 3.85 | 10.67 (47.85) |
| Hit reaction time (ms) | 355.74 (72.92) | 359.03 (74.40) | 3.29 (62.54) | 0.66 (0.54−0.76) | 44.05 | 122.10 (34.16) |
| Hit reaction time standard error (ms) | 7.24 (4.61) | 8.03 (6.04) | 0.78 (4.12) | 0.70 (0.59−0.79) | 2.94 | 8.15 (106.82) |
| Variability of standard error (ms) | 14.52 (13.73) | 17.22 (18.75) | 2.70 (13.05) | 0.68 (0.56−0.77) | 9.31 | 25.81 (162.63) |
| Agreement | ||||||
|---|---|---|---|---|---|---|
| Measure | 1st testing M (SD) | 2nd testing M (SD) | Difference M (SD) | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 8.90 (9.59) | 8.37 (9.60) | −0.53 (6.87) | 0.74 (0.65−0.82) | 4.88 | 13.53 (156.78) |
| Commissions (error) | 22.08 (8.34) | 22.51 (8.50) | 0.43 (5.44) | 0.79 (0.71−0.85) | 3.85 | 10.67 (47.85) |
| Hit reaction time (ms) | 355.74 (72.92) | 359.03 (74.40) | 3.29 (62.54) | 0.66 (0.54−0.76) | 44.05 | 122.10 (34.16) |
| Hit reaction time standard error (ms) | 7.24 (4.61) | 8.03 (6.04) | 0.78 (4.12) | 0.70 (0.59−0.79) | 2.94 | 8.15 (106.82) |
| Variability of standard error (ms) | 14.52 (13.73) | 17.22 (18.75) | 2.70 (13.05) | 0.68 (0.56−0.77) | 9.31 | 25.81 (162.63) |
Notes: M = mean; SD = standard deviation; ICC = intra-class correlation coefficients; SEM = standard error of measurement; MDC = minimal detectable change; CI = confidence interval; ms = millisecond.
Absolute Reliability
The current study revealed that the MDCs (MDC %) of the Omissions, Commissions, Hit RT, Hit RT Std Error, and Variability were 13.53 (156.78), 10.67 (47.85), 122.10 (34.16), 8.15 (106.82), and 25.81 (162.63), respectively. The SEM scores for the five CCPT-II indexes for the paranoid subtype participants were 4.71, 3.90, 44.58, 3.07, and 9.59. The MDC scores were 13.06, 10.81, 123.57, 8.51, and 26.58.
The results of paired t-test showed that only the p-value of Variability was smaller than .05. The effect sizes of the five indexes between the test and retest measures were 0.08, 0.08, 0.05, 0.19, and 0.21, respectively, indicating that minimal changes occurred between the test and retest measurements.
Five Bland–Altman plots (Fig. 1) that were representative of the CCPT-II showed the LOA ranges to be −14.00 to 12.94, −10.23 to 11.09, −119.29 to 125.87, −7.30 to 8.86, and −22.88 to 28.28 for the Omissions, Commissions, Hit RT, Hit RT Std Error, and Variability respectively. In addition, trends in the plots of "Omission", "Hit RT", "Hit RT SE" and "Variability" were noted, presenting that the data were heteroscedasticity (Pearson's r were 0.52, 0.35, 0.68 and 0.73, respectively).
Fig. 1.
Bland–Altman method for plotting the difference in scores against the mean scores of CCPT-II. The two bold lines define the limits of agreement (mean of difference ± 1.96 SD).
The split-half reliability of the Omissions, Commissions, Hit RT, and Hit RT Std Error were 0.72, 0.85, 0.87, and 0.67, respectively.
For the validation of the effects of clinical characteristics of the participant to the reliabilities of the CCPT-II, we used Pearson correlation to examine the correlations between the age of onset and duration of illness with the mean difference between test–retest measurements of each index of the CCPT-II. The results showed that the correlations between the age of onset and the mean differences of five indexes of the CCPT-II between test–retest measurements ranged from −0.07 to 0.12, and that the duration of illness was −0.11 to 0.03, respectively. We also used one-way Analysis of Variance (ANOVA) to compare the mean differences between test–retest measurements of each index of the CCPT-II for various CGI-S levels and different genders. The participants were divided into three groups by the CGI-S level (i.e., not at all, mild, and borderline) in this study. The results showed that the differences in the mean differences between test–retest measurements of each index of the CCPT-II in various CGI-S levels (p = .29) and different genders (p = .82) were not significant. The effect sizes between the various CGI-S levels in the mean differences between test–retest measurements of each index of the CCPT-II and various CGI-S levels in the level of "not at all" to the level of "mild" were 0.18, 0.17, 0.36, 0.31, and 0.28. The effect sizes of the level of "not at all" to the level of "borderline" were 0.08, 0.11, 0.33, 0.43, and 0.38. The effect sizes of the level of "mild" to the level of "borderline" were 0.18, 0.05, 0.03, 0.10, and 0.06. The effect sizes between different genders in the mean differences between test–retest measurements of each index of the CCPT-II and different gender were 0.35, 0.14, 0.01, 0.20, and 0.18.
On the other hand, the ICCs of the five indices of the CCPT-II using standardized scores ranged from 0.66 to 0.79 (in Table 3). The SEMs ranged from 4.57 to 5.82. The MDCs (MDC%) ranged from 12.67 (126.95) to 16.13 (161.62%).
Table 3.
Test–retest reliability of the Conners' Continuous Performance Test II (standardized scores)
| Agreement | |||
|---|---|---|---|
| Measure | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 0.74 (0.65−0.82) | 5.09 | 14.11 (141.38) |
| Commissions (error) | 0.79 (0.71−0.85) | 4.57 | 12.67 (126.95) |
| Hit reaction time (ms) | 0.66 (0.54−0.76) | 5.82 | 16.13 (161.62) |
| Hit reaction time standard error (ms) | 0.73 (0.63−0.81) | 5.19 | 14.39 (144.19) |
| Variability of standard error (ms) | 0.72 (0.61−0.80) | 5.28 | 14.64 (146.69) |
| Agreement | |||
|---|---|---|---|
| Measure | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 0.74 (0.65−0.82) | 5.09 | 14.11 (141.38) |
| Commissions (error) | 0.79 (0.71−0.85) | 4.57 | 12.67 (126.95) |
| Hit reaction time (ms) | 0.66 (0.54−0.76) | 5.82 | 16.13 (161.62) |
| Hit reaction time standard error (ms) | 0.73 (0.63−0.81) | 5.19 | 14.39 (144.19) |
| Variability of standard error (ms) | 0.72 (0.61−0.80) | 5.28 | 14.64 (146.69) |
Notes: ICC = intra-class correlation coefficients; SEM = standard error of measurement; MDC = minimal detectable change; CI = confidence interval.
Table 3.
Test–retest reliability of the Conners' Continuous Performance Test II (standardized scores)
| Agreement | |||
|---|---|---|---|
| Measure | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 0.74 (0.65−0.82) | 5.09 | 14.11 (141.38) |
| Commissions (error) | 0.79 (0.71−0.85) | 4.57 | 12.67 (126.95) |
| Hit reaction time (ms) | 0.66 (0.54−0.76) | 5.82 | 16.13 (161.62) |
| Hit reaction time standard error (ms) | 0.73 (0.63−0.81) | 5.19 | 14.39 (144.19) |
| Variability of standard error (ms) | 0.72 (0.61−0.80) | 5.28 | 14.64 (146.69) |
| Agreement | |||
|---|---|---|---|
| Measure | ICC (95% CI) | SEM | MDC (%) |
| Omissions (error) | 0.74 (0.65−0.82) | 5.09 | 14.11 (141.38) |
| Commissions (error) | 0.79 (0.71−0.85) | 4.57 | 12.67 (126.95) |
| Hit reaction time (ms) | 0.66 (0.54−0.76) | 5.82 | 16.13 (161.62) |
| Hit reaction time standard error (ms) | 0.73 (0.63−0.81) | 5.19 | 14.39 (144.19) |
| Variability of standard error (ms) | 0.72 (0.61−0.80) | 5.28 | 14.64 (146.69) |
Notes: ICC = intra-class correlation coefficients; SEM = standard error of measurement; MDC = minimal detectable change; CI = confidence interval.
Discussion
The study firstly examined the CCPT-II by assessing the relative and absolute reliabilities for the persons with schizophrenia.
Relative Reliability
We found that the ICC values of the CCPT-II were fair to poor. For the repeated use of the CCPT-II, the scores of Omissions, Commissions, and Hit Std Error would be more reliable than Hit RT and Variability. In the published CCPT-II technical guide and software manual, test–retest reliability is reported from a sample of 23 individuals, 10 non-clinical and 13 with a variety of clinical diagnoses (Conners, 1985, 2004). The test–retest reliability interval was approximately 3 months, and correlations for the variables ranged from 0.55 (Hit RT) to 0.84 (Omissions). These results were close to ours, implying the test–retest reliability of the CCPT-II in patients with schizophrenia is not clearly different from that in the aforementioned sample. But due to the limited test–retest reliability of the CCPI-II, it is suggested that clinicians and researchers who use this instrument to portray the sustained attention of patients with schizophrenia should take into account the fair agreement, especially in the Hit RT and Variability indices. On the other hand, the ICC values of Omissions, Commissions, and Hit reaction time indices in the current study, which ranged from 0.70 (Hit RT Std Error) to 0.79 (Commissions), indicate that the CCPT-II is somewhat reliable in patients with schizophrenia. Our results provide robust psychometric information of the CCPT-II for potential users in clinical and research settings.
Absolute Reliability
The current study revealed that the MDC%s were above 100%, except the Commissions and Hit reaction time. All of the MDC%s were greater than the acceptable criterion (30%), (Chen et al., 2014; Huang et al., 2011), suggesting that when the CCPT-II is used in patients with schizophrenia, caution must be taken in explaining the change score in attentional function. A large MDC means large random measurement error exists in the successive measurements. In other words, a large MDC represents a large threshold for determining whether a change score is real. According to our results, a score of 14 or higher is needed for a change score of "Omissions" to be considered real (i.e., beyond measurement error). This score is relatively high as compared with the mean of "Omissions". In other words, it is very difficult for a patient with schizophrenia to improve a score by 14 and thereby demonstrate real improvement. Thus, our results reveal that the CCPT-II has larger random measurement error and imply that the CCPT-II needs higher amount of difference to identify real change in individual patients with schizophrenia.
Bland and Altman plots can be used to visualize systematic variations around the zero line, the LOA, and to illustrate whether a trend (heteroscedasticity) exists in the sample in a study (Flansbjer, Holmback, Downham, Patten, and Lexell, 2005). Heteroscedasticity means that the individuals who score the highest values on a particular test also show the greatest amount of difference between two sessions. In addition, a trend (heteroscedasticity) appeared in the plots of the "Omission", "Hit RT", "Hit RT SE" and "Variability" indices. When heteroscedasticity exists, the MDC% is more appropriate than the MDC for use as a threshold to determine a real change. That is to say, a relative value (i.e., the initial value × MDC%) is more suitable than an absolute value (i.e., MDC) for use as a threshold (Atkinson & Nevill, 1998). As shown in Fig. 1, the LOAs of all indices were large, revealing that the CCPT-II has limited practicality in clinical settings in 1-month retest assessment in persons diagnosed with schizophrenia.
To evaluate the performance of the participants in this study, we compared our results with those of prior studies. The results of the study were compared to those of children aged from 9 to 18 from a normative epidemiological sample in 2003. The participants with schizophrenia in our study had higher raw scores in Omissions and lower scores in Commissions (Conners, Epstein, Angold, and Klaric, 2003). Compared to 158 persons with traumatic brain injury (TBI), the participants in this study had higher raw scores in Commissions but lower raw scores in Omissions (Lange et al., 2013). Compared to 76 persons with chronic stroke, the participants in this study had higher raw scores in Commissions and lower raw scores in Omissions (Chen et al., 2009). Therefore, persons with schizophrenia tend to have better accuracy in attention than persons with TBI and chronic stroke. Also, the participants with schizophrenia had better impulsivity than the young children.
To investigate the effects of the clinical variables of the participants (e.g., age of onset, duration of illness, CGI-S level, and gender) on the reliabilities of the CCPT-II, we analyzed the correlations between age of onset and duration of illness with the mean difference between test and retest measurements of each index of the CCPT-II by Pearson correlation test, and the correlations between the CGI-S level and different gender with the mean difference between test and retest measurements of each index of the CCPT-II by ANOVA. The results revealed no significant effects of the clinical variables to the reliabilities of the CCPT-II. The low correlations between these clinical variables and the mean differences between test–retest measurements of the five indexes of the CCPT-II indicated that these clinical variables did not threaten the robustness of our results.
The results of ICCs, SEM, and MDC for the participants with the paranoid subtype were almost the same as those of the five indexes of the CCPT-II focused on the whole schizophrenia population in this study. This finding strengthens the robustness of our results.
In terms of the split-half reliabilities of the CCPT-II, the previous study (n = 520) reported split-half reliabilities of the CCPT-II of 0.94, 0.83, 0.95, and 0.87 for the Omissions, Commissions, Hit RT, and Hit RT Std Error indexes, respectively. Those in our study (0.72, 0.85, 0.87, and 0.67 for the Omissions, Commissions, Hit RT, and Hit RT Std Error) were mostly lower than those in the 1994 study. Thus, the split-half reliabilities of the CCPT-II for persons with schizophrenia were not quite good enough.
The Test of Everyday Attention (TEA) is another measure for testing clients' attention (Robertson et al., 1994). The TEA is an ecological measure to be generalized and applied in everyday life. The TEA includes more ecologically valid tasks and incorporates relatively familiar materials or concepts, such as maps and telephone directories (Bate, Mathias, and Crawford, 2001). Unlike the CCPT-II, the TEA needs the content validity to be re-established for use in different cultures. Therefore, the CCPT-II is more popular and easier to administer than the TEA in research and clinic settings for persons with schizophrenia.
More trials per session would increase the stability of the measurements and reduce the MDC and the MDC% (Huang et al., 2011). However, due to safety and time considerations, the participants were well instructed and allowed one practice trial before the formal CCPT-II in this study. Future research designed to reduce the MDC or measurement error of the CCPT-II is warranted.
The Conners' Continuous Performance Test 3rd Edition (CCPT-3) was published in 2014. The new version consists of 1400 cases representative of the latest United States (U.S.) population census. There is a revised paradigm that includes a new ratio of non-targets (the letter X) to targets (all other letters) to improve the sensitivity of the test to impulsivity problems. However, the CCPT-II is still more popular than the CCPT-3 in clinical settings at present. Thus, our results still have value in that they provide an importance reference for clinicians and researchers who are accustomed to the administration of the CCPT-II.
This study is the first to report the relative reliability and absolute reliability of the CCPT-II based on raw scores rather than standardized scores. Values of reliability based on standardized scores are easier to compare between various indices. From the results of calculating standardized scores for the reliability of the CCPT-II, the highest relative reliability of the five indices was Commissions. The smallest SEM and MDC (MDC%) were also for Commissions. Therefore, the standardized score of Commissions has the smallest threshold for determining whether a change score is real. The results revealed that the standardized score of Commissions of CCPT-II needs the smallest amount of difference to identify real change in individual patients with schizophrenia. The MDC of Commissions based on raw scores is the second smallest of the five indices of CCPT-II. Generally speaking, the raw and standardized scores of Commissions have relatively small thresholds for showing a real change in the CCPT-II.
Limitations
Schizophrenia is a heterogeneous disorder with fluctuating symptom severity, even in stable patients. The CGI-S is a single-item global assessment on a 7-point scale. Relying only on the CGI-S as a basis for the assumption of stability might entail limitations. The extent of subtle changes in symptoms is of concern as a source of variability in assessing test–retest reliability in this sample. Using CGI-S scores to show that a stable condition was maintained may lead to unsatisfactory results. In the future, related studies may consider more precise assessment tools for monitoring stability in such patients.
This study was followed the release of the CCPT-3 in 2014. Psychometric data on the CCPT-II in schizophrenia for assessing attentional functions is still helpful to clinicians and researchers who have yet to adopt the CCPT-3. In addition, as with all measures, this technique requires extensive examination to further clarify its particular strengths and limitations. On the other hand, medication effects are an important issue to address in future research.
Conclusion
In summary, our results indicate that the CCPT-II has limited reliability in monitoring the sustained attention function of patients with schizophrenia. However, further studies with larger sample sizes or modified inclusion and exclusion criteria are warranted to validate our results.
The results of the present study can be used as a reference for the measurement error of CCPT-II to help clinicians and researchers determine true change between successive assessments of patients with schizophrenia.
Conflict of Interest
None declared.
References
Arnall
,
F. A.
, Koumantakis G. A. Oldham J. A. Cooper R. G.
2002
).
Between-days reliability of electromyographic measures of paraspinal muscle fatigue at 40, 50 and 60% levels of maximal voluntary contractile force
.
Clinical Rehabilitation
,
16
,
761
–
771
.
Atkinson
,
G.
, & Nevill A. M.
1998
).
Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine
.
Sports Medicine
,
26
,
217
–
238
.
Bate
,
A. J.
, Mathias J. L. Crawford J. R.
2001
).
Performance on the Test of Everyday Attention and standard tests of attention following severe traumatic brain injury
.
The Clinical Neuropsychologist
,
15
(3)
,
405
–
422
.
.
Beckerman
,
H.
, Roebroeck M. E. Lankhorst G. J. Becher J. G. Bezemer P. D. Verbeek A. L.
2001
).
Smallest real difference, a link between reproducibility and responsiveness
.
Quality of Life Research
,
10
,
571
–
578
.
Bland
,
J. M.
, & Altman D. G.
1986
).
Statistical methods for assessing agreement between two methods of clinical measurement
.
Lancet
,
1
,
307
–
310
.
Bland
,
J. M.
, & Altman D. G.
1990
).
A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two methods of measurement
.
Computers in Biology and Medicine
,
20
,
337
–
340
.
Bland
,
J. M.
, & Altman D. G.
2003
).
Applying the right statistics: analyses of measurement studies
.
Ultrasound in Obstetrics and Gynecology
,
22
,
85
–
93
.
Bruton
,
A.
, Conway J. H. Holgate S. T.
2000
).
Reliability: What is it, and how is it measured?
.
Physiotherapy
,
86
,
94
–
99
.
Chen
,
C. H.
, Lin S. F. Yu W. H. Lin J. H. Chen H. L. Hsieh C. L.
2014
).
Comparison of the test–retest reliability of the balance computerized adaptive test and a computerized posturography instrument in patients with stroke
.
Archives of Physical Medicine and Rehabilitation
,
95
,
1477
–
1483
.
.
Chen
,
H. C.
, Koh C. L. Hsieh C. L. Hsueh I. P.
2009
).
Test–re-test reliability of two sustained attention tests in persons with chronic stroke
.
Brain Injury
,
23
,
715
–
722
.
.
Chen
,
X.
, Zhang Z. H. Song Y. Yuan W. Liu Z. X. Tang M. Q.
2015
).
A paired case-control comparison of ziprasidone on visual sustained attention and visual selective attention in patients with paranoid schizophrenia
.
European Review for Medical and Pharmacological Sciences
,
19
,
16
,
2952
–
2956
.
Chuang
,
L. L.
, Lin K. C. Wu C. Y. Chang C. W. Chen H. C. Yin H. P.
2013
).
Relative and absolute reliabilities of the myotonometric measurements of hemiparetic arms in patients with stroke
.
Archives of Physical Medicine and Rehabilitation
,
94
,
459
–
466
.
.
Clark
,
S. L.
, Souza R. P. Adkins D. E. Aberg K. Bukszar J. McClay J. L.
2013
).
Genome-wide association study of patient-rated and clinician-rated global impression of severity during antipsychotic treatment
.
Pharmacogenet Genomics
,
23
,
69
–
77
.
.
Conners
,
C. K.
(
1985
).
The computerized continuous performance test
.
Psychopharmacology Bulletin
,
21
,
891
–
892
.
Conners
,
C. K.
(
2004
).
Conners' Continuous Performance Test (CPT II): Version 5 for Windows technical guide and software manual
.
North Tonawanda, NY
:
MHS
.
Conners
,
C. K.
, Epstein J. N. Angold A. Klaric J.
2003
).
Continuous performance test performance in a normative epidemiological sample
.
Journal of Abnormal Child Psychology
,
31
(
5
),
555
–
562
.
Downing
,
S. M.
(
2004
).
Reliability: On the reproducibility of assessment data
.
Medical Education
,
38
,
1006
–
1012
.
.
Egeland
,
J.
, & Kovalik-Gran I.
2010
).
Validity of the factor structure of Conners' CPT
.
Journal of Attention Disorders
,
13
,
347
–
357
.
.
Fett
,
A. K.
, Viechtbauer W. Dominguez M. D. Penn D. L. van Os J. Krabbendam L.
2011
).
The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis
.
Neuroscience and Biobehavioral Reviews
,
35
,
573
–
588
.
Flansbjer
,
U. B.
, Holmback A. M. Downham D. Patten C. Lexell J.
2005
).
Reliability of gait performance tests in men and women with hemiparesis after stroke
.
Journal of Rehabilitation Medicine
,
37
,
75
–
82
.
Gillig
,
P. M.
, & Sanders R. D.
2011
).
Higher cortical functions: attention and vigilance
.
Innovations in Clinical Neuroscience
,
8
(1)
,
43
–
46
.
Goldsmith
,
C. H.
, Boers M. Bombardier C. Tugwell P.
1993
).
Criteria for clinically important changes in outcomes: development, scoring and evaluation of rheumatoid arthritis patient and trial profiles. OMERACT Committee
.
Journal of Rheumatology
,
20
,
561
–
565
.
Green
,
M. F.
, Kern R. S. Braff D. L. Mintz J.
2000
).
Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the "right stuff"?
.
Schizophrenia Bulletin
,
26
,
119
–
136
.
Green
,
M. F.
, Kern R. S. Heaton R. K.
2004
).
Longitudinal studies of cognition and functional outcome in schizophrenia: implications for MATRICS
.
Schizophrenia Research
,
72
,
41
–
51
.
Green
,
M. F.
, Nuechterlein K. H. Gold J. M. Barch D. M. Cohen J. Essock S.
2004
).
Approaching a consensus cognitive battery for clinical trials in schizophrenia: the NIMH-MATRICS conference to select cognitive domains and test criteria
.
Biological Psychiatry
,
56
,
301
–
307
.
Guy
,
W.
(
1976
).
Assessment manual for psychopharmacology.
.
Washington, DC
:
US Government Printing Office
.
Haley
,
S. M.
, & Fragala-Pinkham M. A.
2006
).
Interpreting change scores of tests and measures used in physical therapy
.
Physical Therapy
,
86
,
735
–
743
.
Heine
,
M.
, van den Akker L. E. Verschuren O. Visser-Meily A. Kwakkel G.
2015
).
Reliability and responsiveness of cardiopulmonary exercise testing in fatigued persons with multiple sclerosis and low to mild disability
.
PLoS One
,
10
,
e0122260
.
.
Hesseberg
,
K.
, Bentzen H. Bergland A.
2014
).
Reliability of the Senior Fitness Test in community-dwelling older people with cognitive impairment
.
Physiotherapy Research International
,
20
,
37
–
44
.
.
Hodge
,
M. A.
, Siciliano D. Withey P. Moss B. Moore G. Judd G.
2010
).
A randomized controlled trial of cognitive remediation in schizophrenia
.
Schizophrenia Bulletin
,
36
,
419
–
427
.
Hopkins
,
W. G.
(
2000
).
Measures of reliability in sports medicine and science
.
Sports Medicine
,
30
,
1
–
15
.
Huang
,
R. R.
, Chen Y. S. Chen C. C. Chou F. H. Su S. F. Chen M. C.
2012
).
Quality of life and its associated factors among patients with two common types of chronic mental illness living in Kaohsiung City
.
Psychiatry and Clinical Neurosciences
,
66
,
482
–
490
.
.
Huang
,
S. L.
, Hsieh C. L. Wu R. M. Tai C. H. Lin C. H. Lu W. S.
2011
).
Minimal detectable change of the timed "up & go" test and the dynamic gait index in people with Parkinson disease
.
Physical Therapy
,
91
,
114
–
121
.
.
Koide
,
T.
, Aleksic B. Kikuchi T. Banno M. Kohmura K. Adachi Y.
2012
).
Evaluation of factors affecting continuous performance test identical pairs version score of schizophrenic patients in a Japanese clinical sample
.
Schizophrenia Research and Treatment
,
2012
,
1
–
5
.
Lange
,
R. T.
, Iverson G. L. Brickell T. A. Staver T. Pancholi S. Bhagwat A.
2013
).
Clinical utility of the Conners' Continuous Performance Test-II to detect poor effort in U.S. military personnel following traumatic brain injury
.
Psychological Assessment
,
25
(2)
,
339
–
352
.
.
Leucht
,
S.
, Davis J. M. Engel R. R. Kissling W. Kane J. M.
2009
).
Definitions of response and remission in schizophrenia: recommendations for their use and their presentation
.
Acta Psychiatrica Scandinavica. Supplementum
7
–
14
.
.
Lexell
,
J. E.
, & Downham D. Y.
2005
).
How to assess the reliability of measurements in rehabilitation
.
American Journal of Physical Medicine and Rehabilitation
,
84
,
719
–
723
.
[pii]
Liaw
,
L. J.
, Hsieh C. L. Hsu M. J. Chen H. M. Lin J. H. Lo S. K.
2012
).
Test–retest reproducibility of two short-form balance measures used in individuals with stroke
.
International Journal of Rehabilitation Research
,
35
(3)
,
256
–
262
.
.
Liaw
,
L. J.
, Hsieh C. L. Lo S. K. Chen H. M. Lee S. Lin J. H.
2008
).
The relative and absolute reliability of two balance performance measures in chronic stroke patients
.
Disability and Rehabilitation
,
30
,
656
–
661
.
Lysaker
,
P. H.
, Tsai J. Henninger L. L. Vohs J. L. Viverito K.
2010
).
Decrements in sustained attention across trials in a continuous performance test: associations with social functioning in schizophrenia
.
Journal of Nervous and Mental Disease
,
198
,
154
–
158
.
.
Lysaker
,
P. H.
, Vohs J. L. Tsai J.
2009
).
Negative symptoms and concordant impairments in attention in schizophrenia: associations with social functioning, hope, self-esteem and internalized stigma
.
Schizophrenia Research
,
110
,
165
–
172
. S0920-9964(09)00043-7 [pii]
.
Nasrallah
,
H.
, Morosini P. Gagnon D. D.
2008
).
Reliability, validity and ability to detect change of the Personal and Social Performance scale in patients with stable schizophrenia
.
Psychiatry Research
,
161
,
213
–
224
.
.
Okahashi
,
S.
, Seki K. Nagano A. Luo Z. Kojima M. Futaki T.
2013
).
A virtual shopping test for realistic assessment of cognitive function
.
Journal of Neuroengineering Rehabilitation
,
10
,
59
.
.
Otto
,
M. W.
, Tolin D. F. Simon N. M. Pearlson G. D. Basden S. Meunier S. A.
2010
).
Efficacy of d-cycloserine for enhancing response to cognitive-behavior therapy for panic disorder
.
Biological Psychiatry
,
67
,
365
–
370
.
.
Overend
,
T.
, Anderson C. Sawant A. Perryman B. Locking-Cusolito H.
2010
).
Relative and absolute reliability of physical function measures in people with end-stage renal disease
.
Physiotherapy Canada
,
62
,
122
–
128
.
.
Patrick
,
D. L.
, Burns T. Morosini P. Rothman M. Gagnon D. D. Wild D.
2009
).
Reliability, validity and ability to detect change of the clinician-rated Personal and Social Performance scale in patients with acute symptoms of schizophrenia
.
Current Medical Research and Opinion
,
25
,
325
–
338
.
.
Portney
,
L. G.
, & Watkins M. P.
2009
).
Foundations of clinical research: Applications to practice
(3rd ed.).
Upper Saddle River, NJ
:
Pearson/Prentice Hall
.
Prince
,
B.
, Makrides L. Richman J.
1980
).
Research methodology and applied statistics. Part 2: the literature search
.
Physiotherapy Canada
,
32
,
201
–
206
.
Rapisarda
,
A.
, Kraus M. Tan Y. W. Lam M. Eng G. K. Lee J.
2014
.
The continuous performance test, identical pairs: norms, reliability and performance in healthy controls and patients with schizophrenia in Singapore
.
Schizophrenia Research
,
156
,
233
–
240
.
.
Robertson
,
I. H.
, Ward T. Ridgeway V. Nimmo-Smith I.
1994
).
The test of everyday attention
.
Bury St. Edmunds
:
Thames Valley Test Company
.
Sanz
,
J. C.
, Gomez V. Vargas M. L. Marin J. J.
2012
).
Dimensions of attention impairment and negative symptoms in schizophrenia: a multidimensional approach using the conners continuous performance test in a Spanish population
.
Cognitive and Behavioral Neurology
,
25
,
63
–
71
.
.
Schuck
,
P.
, & Christian Z.
2003
).
The "smallest real difference" as a measure of sensitivity to change: a critical analysis
.
International Journal of Rehabilitation Research
,
26
,
85
–
91
.
Shen
,
C.
, Popescu F. C. Hahn E. Ta T. T. Dettling M. Neuhaus A. H.
2014
).
Neurocognitive pattern analysis reveals classificatory hierarchy of attention deficits in schizophrenia
.
Schizophrenia Bulletin
,
40
(4)
,
878
–
885
.
.
Shrout
,
P. E.
, & Fleiss J. L.
1979
).
Intraclass correlations: uses in assessing rater reliability
.
Psychological Bulletin
,
86
,
420
–
428
.
Smidt
,
N.
, van der Windt D. A. Assendelft W. J. Mourits A. J. Devill W. L. de Winter A. F.
2002
).
Interobserver reproducibility of the assessment of severity of complaints, grip strength, and pressure pain threshold in patients with lateral epicondylitis
.
Archives of Physical Medicine and Rehabilitation
,
83
,
1145
–
1150
.
Stratford
,
P. W.
, & Goldsmith C. H.
1997
).
Use of the standard error as a reliability index of interest: an applied example using elbow flexor strength data
.
Physical Therapy
,
77
,
745
–
750
.
Wu
,
B. J.
, Lin C. H. Tseng H. F. Liu W. M. Chen W. C. Huang L. S.
2013
).
Validation of the Taiwanese Mandarin version of the Personal and Social Performance scale in a sample of 655 stable schizophrenic patients
.
Schizophrenia Research
,
146
,
34
–
39
.
.
© The Author 2016. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
© The Author 2016. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
shortridgeoursend.blogspot.com
Source: https://academic.oup.com/acn/article/31/7/769/2374081
0 Response to "Conners Continuous Performce Test Reliability and Validitdy"
Postar um comentário