Measurement Error and Methodologic Issues in Analyses of the Proportion of Variance Explained in Cognition

doi:10.1007/s11065-024-09655-1

Measurement Error and Methodologic Issues in Analyses of the Proportion of Variance Explained in Cognition

Research output: Contribution to journal › Review article › peer-review

1 Scopus citations

Abstract

Existing studies examining the predictive ability of biomarkers for cognitive outcomes do not account for variance due to measurement error, which could lead to under-estimates of the proportion of variance explained. We used data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (N = 1084) to estimate the proportion of variance explained by Alzheimer’s disease (AD) imaging biomarkers in four cognitive outcomes: memory, executive functioning, language, and visuospatial functioning. We compared estimates from standard models that do not account for measurement error, and multilevel models that do account for measurement error. We also examined estimates across diagnostic subgroups (normal, MCI, AD). Estimates of the proportion of variance explained from multilevel models accounting for measurement error were larger (e.g., for language, 9–47% vs. 7–34% under standard modeling), with relatively greater differences between standard and multilevel measurement models for cognitive outcomes that have larger measurement error variance. Heterogeneity across subgroups also emphasized the importance of sample composition. Future studies should evaluate measurement error adjustments when considerable measurement error in cognitive outcomes is suspected.

Original language	English
Journal	Neuropsychology Review
DOIs	https://doi.org/10.1007/s11065-024-09655-1
State	Accepted/In press - 2024

Bibliographical note

Publisher Copyright:
© The Author(s) 2024.

Funding

Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health ( www.fnih.org ). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California. Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf . Open access funding provided by SCELC, Statewide California Electronic Library Consortium This work was supported by and occurred as part of the 2023 Advanced Psychometric Methods in Cognitive Aging Research conference funded by the National Institute on Aging (NIA) (R13 AG030995, D Mungas, PI). This work was also supported by the NIA grants 3R01 AG030153 to EN and ALG, R01 AG070953 to ALG, R01AG065359 to RMA, K99AG071837 to CBY, RF1AG082339 and R01AG082730 to DWF as well as the Alzheimer’s Association grant AARFD-21–849349 to CBY, and a Michael Smith Health Research BC Scholar Award SCH-2022–2664 to TSEP.

Funders	Funder number
National Institute of Biomedical Imaging and Bioengineering
SCELC
DoD Alzheimer's Disease Neuroimaging Initiative
DOD ADNI
U.S. Department of Defense	W81XWH-12-2-0012
National Institute on Aging	R01AG065359, R01 AG070953, K99AG071837, R01AG082730, RF1AG082339, R13 AG030995, 3R01 AG030153
National Institutes of Health (NIH)	U01 AG024904
Michael Smith Foundation for Health Research	SCH-2022–2664
Alzheimer's Association	AARFD-21–849349

Keywords

Bias
Biomarkers
Cognition
Dementia
Measurement

ASJC Scopus subject areas

Neuropsychology and Physiological Psychology

Access to Document

10.1007/s11065-024-09655-1

Cite this

@article{0d4c7918cf144d8c825e1c98836efc0f,

title = "Measurement Error and Methodologic Issues in Analyses of the Proportion of Variance Explained in Cognition",

abstract = "Existing studies examining the predictive ability of biomarkers for cognitive outcomes do not account for variance due to measurement error, which could lead to under-estimates of the proportion of variance explained. We used data from the Alzheimer{\textquoteright}s Disease Neuroimaging Initiative (ADNI) (N = 1084) to estimate the proportion of variance explained by Alzheimer{\textquoteright}s disease (AD) imaging biomarkers in four cognitive outcomes: memory, executive functioning, language, and visuospatial functioning. We compared estimates from standard models that do not account for measurement error, and multilevel models that do account for measurement error. We also examined estimates across diagnostic subgroups (normal, MCI, AD). Estimates of the proportion of variance explained from multilevel models accounting for measurement error were larger (e.g., for language, 9–47\% vs. 7–34\% under standard modeling), with relatively greater differences between standard and multilevel measurement models for cognitive outcomes that have larger measurement error variance. Heterogeneity across subgroups also emphasized the importance of sample composition. Future studies should evaluate measurement error adjustments when considerable measurement error in cognitive outcomes is suspected.",

keywords = "Bias, Biomarkers, Cognition, Dementia, Measurement",

author = "Emma Nichols and Vahan Aslanyan and Adrien, \{Tamare V.\} and Andrews, \{Ryan M.\} and Fardo, \{David W.\} and Gavett, \{Brandon E.\} and Paterson, \{Theone S.E.\} and Turney, \{Indira C.\} and Young, \{Christina B.\} and Uanhoro, \{James O.\} and Gross, \{Alden L.\}",

note = "Publisher Copyright: {\textcopyright} The Author(s) 2024.",

year = "2024",

doi = "10.1007/s11065-024-09655-1",

language = "English",

}

TY - JOUR

T1 - Measurement Error and Methodologic Issues in Analyses of the Proportion of Variance Explained in Cognition

AU - Nichols, Emma

AU - Aslanyan, Vahan

AU - Adrien, Tamare V.

AU - Andrews, Ryan M.

AU - Fardo, David W.

AU - Gavett, Brandon E.

AU - Paterson, Theone S.E.

AU - Turney, Indira C.

AU - Young, Christina B.

AU - Uanhoro, James O.

AU - Gross, Alden L.

PY - 2024

Y1 - 2024

N2 - Existing studies examining the predictive ability of biomarkers for cognitive outcomes do not account for variance due to measurement error, which could lead to under-estimates of the proportion of variance explained. We used data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (N = 1084) to estimate the proportion of variance explained by Alzheimer’s disease (AD) imaging biomarkers in four cognitive outcomes: memory, executive functioning, language, and visuospatial functioning. We compared estimates from standard models that do not account for measurement error, and multilevel models that do account for measurement error. We also examined estimates across diagnostic subgroups (normal, MCI, AD). Estimates of the proportion of variance explained from multilevel models accounting for measurement error were larger (e.g., for language, 9–47% vs. 7–34% under standard modeling), with relatively greater differences between standard and multilevel measurement models for cognitive outcomes that have larger measurement error variance. Heterogeneity across subgroups also emphasized the importance of sample composition. Future studies should evaluate measurement error adjustments when considerable measurement error in cognitive outcomes is suspected.

AB - Existing studies examining the predictive ability of biomarkers for cognitive outcomes do not account for variance due to measurement error, which could lead to under-estimates of the proportion of variance explained. We used data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (N = 1084) to estimate the proportion of variance explained by Alzheimer’s disease (AD) imaging biomarkers in four cognitive outcomes: memory, executive functioning, language, and visuospatial functioning. We compared estimates from standard models that do not account for measurement error, and multilevel models that do account for measurement error. We also examined estimates across diagnostic subgroups (normal, MCI, AD). Estimates of the proportion of variance explained from multilevel models accounting for measurement error were larger (e.g., for language, 9–47% vs. 7–34% under standard modeling), with relatively greater differences between standard and multilevel measurement models for cognitive outcomes that have larger measurement error variance. Heterogeneity across subgroups also emphasized the importance of sample composition. Future studies should evaluate measurement error adjustments when considerable measurement error in cognitive outcomes is suspected.

KW - Bias

KW - Biomarkers

KW - Cognition

KW - Dementia

KW - Measurement

UR - https://www.scopus.com/pages/publications/85210075764

UR - https://www.scopus.com/pages/publications/85210075764#tab=citedBy

U2 - 10.1007/s11065-024-09655-1

DO - 10.1007/s11065-024-09655-1

M3 - Review article

AN - SCOPUS:85210075764

SN - 1040-7308

JO - Neuropsychology Review

JF - Neuropsychology Review

ER -

Measurement Error and Methodologic Issues in Analyses of the Proportion of Variance Explained in Cognition

Abstract

Bibliographical note

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

S'ORCe Collaborative Group (RENEWED)

Cite this

Measurement Error and Methodologic Issues in Analyses of the Proportion of Variance Explained in Cognition

Abstract

Bibliographical note

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Projects

S'ORCe Collaborative Group (RENEWED)

Cite this