EUVIMEDEUVIMED entdecken
CrossrefInfokarte

A modernized picture description task enables detection of mild cognitive impairment through automated multimodal speech analysis in an open-source pipeline (SPEECH-MARC)

Cynthia Angela Nyongesa, Marcella Antoinette Barneclo, Kimberly Espejo, Yilei Dong, Anna Lisette Isenberg, Judy Pa

Digital Biomarkers · 2026-10-28

DOI: 10.1159/dib/acpag005 ↗
AbstractBibliografische Angaben

Abstract

Introduction: Mild cognitive impairment (MCI) affects an estimated 15 to 20 percent of adults over 65 and precedes Alzheimer’s disease in up to half of cases within five years, yet remains chronically underdiagnosed in primary care settings where brief, accessible screening tools are most needed. Speech produced during structured picture description tasks offers a non-invasive, scalable signal for early detection, but the field lacks open, reproducible pipelines validated in prospective clinical cohorts. We developed SPEECH-MARC, an open-source multimodal framework that extracts acoustic, linguistic, and scene-level contextual features from picture description speech to classify MoCA-defined cognitive status. Methods: In a prospective cohort of 127 community-dwelling older adults (cognitively unimpaired, CU: n=61; MCI: n=66) recruited through the UC San Diego Brain Health Virtual Reality Study, we extracted 192 candidate features spanning eGeMAPS acoustic parameters, lexical diversity indices, syntactic complexity measures, and an automated scoring framework for Information Content Units (ICUs) and Areas of Interest (AOIs) adapted for a culturally updated picture description stimulus. Classification used stratified 5-fold cross-validation with feature selection (k=20 features per fold) performed strictly within each training fold, maintaining an effective 1:5 feature-to-training-sample ratio appropriate for small-cohort machine learning. Results: Combined multimodal models achieved AUC=0.662 (95% CI: 0.563 to 0.755), outperforming acoustic-only (AUC=0.619), linguistic-only (AUC=0.603), and ICU/AOI-only (AUC=0.629) approaches. The strongest predictors were spectral flux variability, lexical diversity, and peripheral scene coverage. Mean word length showed the strongest continuous association with MoCA total score (r=0.31, p=0.0003). Multiple lexical and scene-description features showed FDR-corrected associations with category fluency, digit symbol, and design fluency scores measured independently in the same visit. Conclusion: Multimodal speech features extracted from a single picture description recording provide clinically interpretable discrimination of MoCA-defined cognitive status. Findings are cross-sectional and require longitudinal validation, but the feature patterns identified are consistent with known profiles of early cognitive decline. The framework is freely available as a Python package with a web-based interface requiring no programming expertise, intended to support research replication and pilot deployment in settings where access to formal neuropsychological testing is limited.

Wissen, das zu Ihrer Arbeit passt.

Vom kostenlosen Einstieg über die persönliche Recherche bis zur vollständigen Research Suite.

Tarife ansehen →

Sind alle Verlagsvolltexte enthalten?

Nein. Bibliografische Nachweise, Abstracts und Volltexte sind unterschiedliche Inhalte. Ob ein Volltext verfügbar ist, hängt von der Quelle, Open Access und Ihren vorhandenen Lizenzen ab. Ein EUVIMED-Tarif ersetzt keine zusätzlichen Verlagsrechte.

Tarife ansehen →