Development of large language model-generated multiple-choice questions and flashcards to enhance the Canadian undergraduate urology curriculum
Canadian Urological Association Journal · 2026-10-19 · 21 (2)
DOI: 10.5489/cuaj.9845 ↗Abstract
Introduction: The Canadian Undergraduate Urology Curriculum (CanUUC) is a free, online curriculum that covers important topics in urology for medical students. While multiple-choice questions (MCQs) and flashcards are widely used for medical education, they can be resource-intensive and time-consuming to produce. Therefore, to strengthen the CanUUC with such tools, we developed a customized large language model (LLM)-enabled system to generate MCQs and flashcards from CanUUC material and evaluated the quality of its outputs. Methods: We used OpenAI’s GPT-5 model and refined two system prompts to generate educational materials from two CanUUC topics (benign prostatic hyperplasia and urolithiasis). Six evaluators blinded to items’ origin were involved, and each item received two independent ratings for relevance, correctness, clarity, distractor quality, and suitability for education. Items with a mean score 1 plausible answer. Across all independent ratings, the proportion of top scores (4–5) ranged from 85.5−90.5% across all domains. Overall, 86.1% of independent ratings judged items suitable for medical education. When averaging the suitability score at the item-level, 12 items (13%) had a score
EUVIMED