EUVIMEDEUVIMED entdecken
CrossrefInfokarte

Instability and Overinterpretation of Predictive Performance in a Small Multimodal Clinical Dataset: A Methodological Case Study of Adult ADHD Versus Other Psychiatric Disorders

Mikhail Kolev, Irina Naskinova, Meglena Lazarova, Hristo Kalinov, Velin Klisurov

Journal of Clinical Medicine · 2026-11-12 · 15 (22) · 8866

DOI: 10.3390/jcm15228866 ↗
AbstractBibliografische Angaben

Abstract

Small public multimodal datasets invite overstated claims that wearable signals classify attention-deficit/hyperactivity disorder (ADHD). The HYPERAKTIV dataset served as a methodological case study: 103 adult psychiatric outpatients, 51 with ADHD and 52 clinical controls, with actigraphy, heart-rate recordings, and cognitive testing. The task was differential classification, not screening. Background/Objectives: The objective is to quantify how far the reported performance depends on the evaluation protocol. Methods: The methods evaluated four nested feature blocks on 63 complete cases with one fixed model and one estimand: the mean out-of-fold area under the receiver operating characteristic curve (AUROC) over 20 fold assignments. Each block was compared with a permutation null from the identical protocol. Results: The results are that the fold assignment alone moved a single block’s AUROC by up to 0.235, and no block exceeded its null. The heart-rate-variability increment was −0.039 (percentile interval −0.153 to 0.155), inconclusive under every interval method. A bootstrap holding the fitted models fixed suggested a bounded null, and DeLong’s test made the cognitive increment significant; resampling the whole pipeline confirmed neither. On datasets with the heart-rate-variability block permuted, these two tests rejected in 18.3% and 14.7% of those datasets at a 5% level. The pre-scored ADHD Confidence Index alone reached an AUROC of 0.743, descriptively above every fitted model. Conclusions: The conclusions drawn are that small-sample studies should report a protocol-matched null and intervals that refit the whole pipeline.

Wissen, das zu Ihrer Arbeit passt.

Vom kostenlosen Einstieg über die persönliche Recherche bis zur vollständigen Research Suite.

Tarife ansehen →

Sind alle Verlagsvolltexte enthalten?

Nein. Bibliografische Nachweise, Abstracts und Volltexte sind unterschiedliche Inhalte. Ob ein Volltext verfügbar ist, hängt von der Quelle, Open Access und Ihren vorhandenen Lizenzen ab. Ein EUVIMED-Tarif ersetzt keine zusätzlichen Verlagsrechte.

Tarife ansehen →