Association of Preexamination AI Virtual-Patient Trainer Scores With Scores on a Local Single-Station History-Taking Objective Structured Clinical Examination: Observational Cohort Study
JMIR Medical Education · 2026-11-03 · 12 · e109752
DOI: 10.2196/109752 ↗Astratt
Background Large language model–based virtual-patient systems score history-taking practice, but whether scores correspond to independently scored examination performance remains untested. Existing evaluations emphasize internal platform metrics or simulated-patient fidelity, leaving criterion-related evidence limited. Objective This study tests whether mean preexamination AI-trainer practice scores were associated with performance on 1 local objective structured clinical examination (OSCE) history-taking station, whether the association depended on the complaint-matched training case, and whether a model including the trainer score retained predictive performance in held-out students. Methods At 1 medical school, second-year students received a required 6-case assignment on the AI-Powered Medical History-Taking Training and Evaluation System (AMTES) and completed an end-of-course OSCE. Each student completed 1 station rated by 1 examiner; interrater and cross-station reliability could not be estimated. The 4 OSCE diagnoses differed from the training diagnoses: 3 shared a presenting-complaint category with an AMTES case, whereas 1 OSCE case had no presenting-complaint match. Platform-to-OSCE record linkage was 474 out of 480 (98.8%); 473 students with analyzable preexamination practice formed the analytic sample. The primary predictor was the mean of normalized scores across practice records time-stamped before each student’s examination-day cut-off. Analyses used Spearman correlations, OSCE-case fixed-effects regression, and repeated 5-fold cross-validation. Results OSCE scores (mean 81.5, SD 6.7) were associated with platform practice scores (Spearman ρ=0.388; P
EUVIMED