Generating Population-Linked Case Vignettes From Nationwide Triage Data: Framework Development and Validation Study
JMIR Medical Informatics · 2026-11-03 · 14 · e104801-e104801
DOI: 10.2196/104801 ↗Abstract
Abstract Background Digital decision-support tools such as triage systems and symptom checkers support millions of health-related decisions each year. Their quality and safety are commonly evaluated using textual patient cases, known as case vignettes. However, expert-written vignette sets cover only a limited spectrum of real-world patient presentations and lack population weights, which would allow extrapolating evaluation results to the underlying patient population. Objective This study aimed to develop a data-driven framework for automatically generating a human-manageable set of case vignettes from nationwide triage data that capture broad presentation diversity and link each vignette to a quantitative weight reflecting the number of underlying patient assessments. Methods From 3.2 million triage assessments conducted over 1 year using structured triage software in the German medical on-call service (telephone triage and online self-triage) and at joint contact points of the outpatient emergency care service and hospital emergency departments, we randomly sampled 50,000 cases. Triage questionnaires were converted into semantic embeddings using a German Sentence Transformer model and grouped by agglomerative clustering. For clusters containing sufficient assessments, we generated one representative assessment using a 2-phase simulated annealing optimization that minimized the distance from the cluster centroid while maximizing the number of answered triage questions. Each representative assessment was assigned the size of its source cluster as its sample-based weight. A similarity-based sensitivity analysis assessed whether these weights were preserved in the full 1-year population. Finally, question-answer pairs of the representative assessments were converted into textual case vignettes using controlled prompting of a large language model. Results The cluster analysis yielded 514 included clusters covering 96.8% of the sampled 50,000 assessments. The generated representatives showed strong agreement with the majority treatment urgency recommendation of their source cluster (Spearman ρ=0.78; P
EUVIMED