EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

Misleading Results in Posttraumatic Stress Disorder Predictive Models Using Electronic Health Record Data: Algorithm Validation Study

Thomas M Crow iD, Eric Lin iD, Kelly L Harper iD, Michael L Crowe iD, Terence M Keane iD, Brian P Marx iD

Journal of Medical Internet Research · 2025

DOI: 10.2196/63352 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift Journal of Medical Internet Research, 2025. Verfasst von Thomas M Crow und 5 weiteren Autor:innen. Beteiligte Einrichtung: Behavioral Science Division, National Center for Posttraumatic Stress Disorder, VA Boston Healthcare System, 150 S Huntington Ave, 13B, Boston, MA, 0…. Hauptthemen laut MEDLINE: Stress Disorders, Post-Traumatic, Electronic Health Records, Algorithms. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: CSRD VA, NIMH NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Electronic health record (EHR) data are increasingly used in predictive models of posttraumatic stress disorder (PTSD), but it is unknown how multivariable prediction of an EHR-based diagnosis might differ from prediction of a more rigorous diagnostic criterion. This distinction is important because EHR data are subject to multiple biases, including diagnostic misclassification and differential health care use resulting from factors such as illness severity.

Objective

This study aims to compare predictive models using the same predictors to predict an EHR-based versus semistructured interview-based PTSD diagnostic criterion, quantify model performance discrepancies, and examine potential mechanisms that account for performance differences.

Methods

We compared the performance of several machine learning models predicting EHR-based PTSD diagnosis to models predicting semistructured interview-based diagnosis in a nationwide sample of 1343 US veterans who completed Structured Clinical Interview for DSM-5 ( Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition ) (SCID-5) interviews and had clinic visit data extracted from the Veterans Affairs (VA) EHR. We developed 2 sets of predictive models using 3 algorithms (elastic net regression, random forest, and XGBoost), with a nested cross-validation scheme consisting of an initial train-test split and 10-fold cross-validation within the training set for each type of model. All models used a nearly identical set of predictors including 29 EHR-based visit count variables and 8 demographic variables.

Results

Diagnostic concordance between EHR-based PTSD diagnosis and SCID-5-based PTSD diagnosis was 73.3%, with 17.8% false negatives and 8.9% false positives for EHR-based diagnosis. Models predicting EHR-based PTSD performed very well (area under the receiver operating characteristic curve [AUC] .85-.9; Matthews correlation coefficient [MCC] .58-.69), whereas those predicting interview-based PTSD performed only moderately well overall (AUC .71-.76; MCC .24-.28). Sensitivity analyses showed that participants’ frequency of VA visits played a role in these differences, such that the density of EHR data (proportion of nonzero visit counts across EHR variables) was more associated with EHR-based PTSD diagnosis (b=−0.18, SE 0.02, P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Stress Disorders, Post-TraumaticElectronic Health RecordsAlgorithmsHumansMaleFemaleVeteransMiddle AgedAdultUnited StatesMachine Learning

Stichwörter der Autor:innen

PTSDclinicclinical informaticsclinical prediction modelsclinicselectronic health recordsmachine learningmental healthmisleading informationmisleading resultposttraumaticposttraumatic stress disordersemistructured interviewsensitivity analysesstress disorder

Autor:innen und Institutionen

  • Behavioral Science Division, National Center for Posttraumatic Stress Disorder, VA Boston Healthcare System, 150 S Huntington Ave, 13B, Boston, MA, 02130, United States, 1 6172329500.
  • Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), VA Boston Healthcare System, Boston, MA, United States.
  • McLean Institute for Technology in Psychiatry, McLean Hospital, Belmont, MA, United States.
  • Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, United States.

ORCID-Profile: Thomas M Crow, Eric Lin, Kelly L Harper, Michael L Crowe, Terence M Keane, Brian P Marx

Förderung

  • CSRD VA
  • NIMH NIH HHS

Zitieren

APA

Crow, T. M., Lin, E., Harper, K. L., Crowe, M. L., Keane, T. M., & Marx, B. P. (2025). Misleading Results in Posttraumatic Stress Disorder Predictive Models Using Electronic Health Record Data: Algorithm Validation Study. Journal of Medical Internet Research. https://doi.org/10.2196/63352

Vancouver

Crow TM, Lin E, Harper KL, Crowe ML, Keane TM, Marx BP. Misleading Results in Posttraumatic Stress Disorder Predictive Models Using Electronic Health Record Data: Algorithm Validation Study. Journal of Medical Internet Research. 2025. doi:10.2196/63352

BibTeX
@article{Crow2025,
  author = {Crow, Thomas M and Lin, Eric and Harper, Kelly L and Crowe, Michael L and Keane, Terence M and Marx, Brian P},
  title = {Misleading Results in Posttraumatic Stress Disorder Predictive Models Using Electronic Health Record Data: Algorithm Validation Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  doi = {10.2196/63352}
}
Kostenlos starten →