Evaluating the Accuracy of Remote Questionnaire and Speech Measures in Detecting Cognitive Impairment in the ADNI4 Remote Digital Cohort: Cross-Sectional Study
JMIR Neurotechnology · 2026-11-06 · 5 · e108372-e108372
DOI: 10.2196/108372 ↗Astratt
Abstract Background Scalable remote approaches are increasingly needed to identify cognitive impairment in clinical research. The Everyday Cognition (ECog) scale and its short form (12-item Everyday Cognition [ECog12]) capture subjective cognitive and functional decline, whereas automated speech-based story recall provides an objective measure of cognitive performance. Whether speech recall contributes information beyond remote questionnaire assessment remains unclear. Objective This study aimed to determine whether automated speech-based story recall measures provide incremental discrimination of cognitive impairment diagnoses beyond the self-reported ECog12 (Self-ECog12) scale and demographics, both without and with 2 additional subjective memory measures that assess current memory/thinking concerns and perceived recent decline. Methods This cross-sectional study included 7268 individuals from the Alzheimer’s Disease Neuroimaging Initiative 4 (ADNI4) Remote Digital cohort with complete baseline questionnaire, demographic, and Storyteller speech-recall data. The outcome was defined as a participant report of a prior diagnosis of mild cognitive impairment or memory problem, Alzheimer disease, or dementia (n=574, 7.9%). Participants completed a smartphone-based Storyteller assessment involving immediate and delayed spoken recall of standardized stories; recordings were automatically analyzed to derive 5 prespecified recall and language measures. Participants were stratified into training and independent natural-prevalence test cohorts. Logistic regression models evaluated Self-ECog12 alone; Self-ECog12 plus demographics; Storyteller added to that model; the 2 additional subjective memory measures; and the full model incorporating Storyteller. Model development and threshold selection were confined to training data. Discrimination, calibration, and threshold-dependent performance were evaluated in the held-out test cohort. Area under the receiver operating characteristic curve (AUC) comparisons used paired DeLong tests. Results The test cohort included 1817 participants, of whom 143 (7.9%) reported a prior diagnosis. Self-ECog12 alone achieved an AUC of 0.792 (95% CI 0.755-0.828), increasing to 0.827 (95% CI 0.792-0.859) with demographics. Adding Storyteller increased AUC to 0.853 (95% CI 0.821-0.883; ΔAUC 0.026, 95% CI 0.009‐0.043; paired DeLong P =.002). With the 2 subjective memory measures included, AUC was 0.857 (95% CI 0.830-0.882) and increased to 0.875 (95% CI 0.851-0.899) after adding Storyteller (ΔAUC 0.019, 95% CI 0.008‐0.030; P
EUVIMED