Test-Retest Reliability and Reliable Change on the NIH Toolbox Cognition Battery

Objective: Researchers and practitioners can detect cognitive improvement or decline within a single examinee by applying a reliable change methodology. This study examined reliable change through test-retest data from the English-language National Institutes of Health Toolbox Cognition Battery (NIHTB-CB) normative sample. Method: Participants included adults (n = 138; age: M ± SD = 54.8 ± 20.0, range: 18-85; 51.4% men; 68.1% White) who completed test-retest assessments about a week apart on five fluid cognition tests, providing raw scores, age-adjusted standard scores (SS), and demographic-adjusted T-scores (T). Results: The Fluid Cognition Composite (SS: ICC = 0.87; T-score: ICC = 0.84) and the five fluid cognition tests had good test-retest reliability (SS: ICC range = 0.66-0.85; T-score: ICC range = 0.64-0.86). The lower and upper bounds of 70%, 80%, and 90% confidence intervals (CIs) were calculated around change scores, which serve as cutoffs for determining reliable change. Using T-scores, 90% CI, and adjustment for practice effects, 32.3% declined on one or more tests, 9.7% declined on two or more tests, 36.6% improved on one or more tests, and 5.4% improved on two or more tests. Conclusions: It was common for participants to show reliable change on at least one test score, but not two or more test scores. Per an 80% CI, test-retest difference scores beyond these cutoffs would indicate reliable change: Dimensional Change Card Sort (SS ≥ 14/T ≥ 10), Flanker (SS ≥ 12/T ≥ 8), List Sorting (SS ≥ 14/T ≥ 10), Picture Sequence Memory (SS ≥ 19/T ≥ 13), Pattern Comparison (SS ≥ 11/T ≥ 8), and Fluid Cognition Composite (SS ≥ 10/T ≥ 7). The reliable change cutoffs could be applied in research or practice to detect within-person change in fluid cognition at the individual level.