Performance of GPT-3.5 and GPT-4 on standardized urology knowledge assessment items in the United States: a descriptive study
Journal of Educational Evaluation for Health Professions · 2024-07-08 · 21 · 17
DOI: 10.3352/jeehp.2024.21.17 ↗Abstract
Purpose: This study aimed to evaluate the performance of Chat Generative Pre-Trained Transformer (ChatGPT) with respect to standardized urology multiple-choice items in the United States.Methods: In total, 700 multiple-choice urology board exam-style items were submitted to GPT-3.5 and GPT-4, and responses were recorded. Items were categorized based on topic and question complexity (recall, interpretation, and problem-solving). The accuracy of GPT-3.5 and GPT-4 was compared across item types in February 2024.Results: GPT-4 answered 44.4% of items correctly compared to 30.9% for GPT-3.5 (P
EUVIMED