EUVIMEDEUVIMED entdecken
Open Access

Participant-Aware Model Validation for Repeated-Measures Data: Comparative Cross-Validation Study

Abdolamir Karbalaie iD, Farhad Abtahi iD, Charlotte K Häger iD

JMIR AI · 30.04.2026 · 5 · e87728

DOI: 10.2196/87728 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR AI, online erschienen am 30.04.2026. Verfasst von Abdolamir Karbalaie und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Community Medicine and Rehabilitation, Umeå University, Umeå, Västerbotten, Sweden. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Repeated-measures datasets are common in biomechanics and digital health, where each participant contributes multiple correlated trials. If cross-validation (CV) ignores this structure, information can leak from training to test folds, inflating performance and undermining clinical credibility.

Objective

This study evaluates the impact of participant-aware validation strategies on model reliability in repeated-measures classification tasks, using fear of reinjury prediction following anterior cruciate ligament reconstruction (ACLR) as a case study.

Methods

We analyzed 623 hop trials from 72 individuals after ACLR to classify fear of reinjury based on biomechanical features. Four CV strategies were compared: stratified 10-fold CV, leave-one-participant-out cross-validation (LOPOCV), group 3-fold CV, and a nested framework combining LOPOCV (outer loop) with group 3-fold CV (inner loop). Ten supervised classifiers were benchmarked across classification accuracy, train-test generalization gap, model ranking consistency, and computational efficiency.

Results

Stratified 10-fold CV systematically overestimated model performance (eg, extra trees accuracy of 0.91 vs 0.66 under LOPOCV) due to participant-level data leakage. Group and nested CV strategies yielded more conservative and stable estimates. The nested LOPOCV + group CV framework achieved a good balance between generalization and participant-aware separation, with reduced bias and overfitting compared with nonnested alternatives.

Conclusions

Participant-aware validation strategies are essential for trustworthy machine learning (ML) evaluation in repeated-measures settings. Nested CV designs improve reproducibility, reduce selection bias, and align with regulatory expectations for clinical ML tools. These findings support best practices in model validation for biomechanics and digital health applications.

Schlagwörter

Stichwörter der Autor:innen

cross-validation benchmarkingdata leakage preventionhuman movement controlmachine learning validationmodel selection biastransparent AI evaluation

Autor:innen und Institutionen

  • Department of Community Medicine and Rehabilitation, Umeå University, Umeå, Västerbotten, Sweden.
  • Department of Biomedical Engineering and Health Systems, School of Engineering Sciences in Chemistry, Biotechnology and Health, KTH Royal Institute of Technology, Huddinge, Stockholm, Sweden.
  • Department of Clinical Science, Intervention and Technology, Karolinska Institutet, Huddinge, Stockholm, Sweden.
  • Department of Clinical Physiology, Karolinska University Hospital, Stockholm, Sweden.

ORCID-Profile: Abdolamir Karbalaie, Farhad Abtahi, Charlotte K Häger

Zitieren

APA

Karbalaie, A., Abtahi, F., & Häger, C. K. (2026). Participant-Aware Model Validation for Repeated-Measures Data: Comparative Cross-Validation Study. JMIR AI, 5, e87728. https://doi.org/10.2196/87728

Vancouver

Karbalaie A, Abtahi F, Häger CK. Participant-Aware Model Validation for Repeated-Measures Data: Comparative Cross-Validation Study. JMIR AI. 2026;5:e87728. doi:10.2196/87728

BibTeX
@article{Karbalaie2026,
  author = {Karbalaie, Abdolamir and Abtahi, Farhad and Häger, Charlotte K},
  title = {Participant-Aware Model Validation for Repeated-Measures Data: Comparative Cross-Validation Study},
  journal = {JMIR AI},
  year = {2026},
  volume = {5},
  pages = {e87728},
  doi = {10.2196/87728}
}
Kostenlos starten →