Large Language Model Superiority in Guideline-Based Neurointerventional Multiple-Choice Questions: Performance Analysis of Radiologists vs LLMs Across Experience Levels
The European Research Journal · 2026-11-01 · 1263-1269
DOI: 10.18621/eurj.1236 ↗Abstract
Objective: To assess large language models’ (LLMs) mastery of contemporary consensus documents in guideline-based neurointerventional decision-making domains and to compare their performance with radiologists of different experience levels. Methods: In this cross-sectional study, 100 single-best-answer, 5-option MCQs were derived from 3 CIRSE guideline sources. A nonparticipating general radiologist authored the items (33 carotid stenting, 33 thrombectomy, 34 acute stroke EVT). One neurointerventional radiologist, 2 general radiologists, and 1 assistant radiologist answered independently, blinded to the key. The same set was posed in January 2026 to Claude 4.5 Opus, ChatGPT 5.2 Instant, and Gemini 3 Pro using default settings and a standardized prompt. Accuracy was compared with McNemar test and Bonferroni correction (P ≤ .002). Results: The highest accuracy was observed with Claude 4.5 Opus (mean [SD], .99 [.10]); ChatGPT 5.2 Instant (.96 [.19]) and Gemini 3 Pro (.95 [.21]) showed comparable performance. Among human participants, the neurointerventional radiologist achieved the highest accuracy (.85 [.35]), while general radiologists scored .67 to .70 and the assistant radiologist achieved .57. Each of the 3 LLMs significantly outperformed all human groups. Moreover, Claude 4.5 Opus also demonstrated significantly higher performance than the neurointerventional radiologist (P = .001). Conclusions: LLMs have reached a competitive level with neurointerventional radiologists in recalling and interpreting neurointerventional guideline knowledge. In particular, Claude 4.5 Opus demonstrated strong performance and appears to be a promising candidate for real-time decision support applications. Multimodal validation and verification studies using real clinical scenario–based designs are warranted.
EUVIMED