EUVIMEDEUVIMED entdecken
CrossrefInfokarte

ChatGPT Generated Otorhinolaryngology Multiple‐Choice Questions: Quality, Psychometric Properties, and Suitability for Assessments

Cecilia Lotto, Sean C. Sheppard, Wilma Anschuetz, Daniel Stricker, Giulia Molinari, Sören Huwendiek, Lukas Anschuetz

OTO Open · 2024-07 · 8 (3)

DOI: 10.1002/oto2.70018 ↗
AbstractBibliografische Angaben

Abstract

Abstract Objective To explore Chat Generative Pretrained Transformer's (ChatGPT's) capability to create multiple‐choice questions about otorhinolaryngology (ORL). Study Design Experimental question generation and exam simulation. Setting Tertiary academic center. Methods ChatGPT 3.5 was prompted: “Can you please create a challenging 20‐question multiple‐choice questionnaire about clinical cases in otolaryngology, offering five answer options?.” The generated questionnaire was sent to medical students, residents, and consultants. Questions were investigated regarding quality criteria. Answers were anonymized and the resulting data was analyzed in terms of difficulty and internal consistency. Results ChatGPT 3.5 generated 20 exam questions of which 1 question was considered off‐topic, 3 questions had a false answer, and 3 questions had multiple correct answers. Subspecialty theme repartition was as follows: 5 questions were on otology, 5 about rhinology, and 10 questions addressed head and neck. The qualities of focus and relevance were good while the vignette and distractor qualities were low. The level of difficulty was suitable for undergraduate medical students (n = 24), but too easy for residents (n = 30) or consultants (n = 10) in ORL. Cronbach's α was highest (.69) with 15 selected questions using students' results. Conclusion ChatGPT 3.5 is able to generate grammatically correct simple ORL multiple choice questions for a medical student level. However, the overall quality of the questions was average, needing thorough review and revision by a medical expert to ensure suitability in future exams.

Wissen, das zu Ihrer Arbeit passt.

Vom kostenlosen Einstieg über die persönliche Recherche bis zur vollständigen Research Suite.

Tarife ansehen →

Sind alle Verlagsvolltexte enthalten?

Nein. Bibliografische Nachweise, Abstracts und Volltexte sind unterschiedliche Inhalte. Ob ein Volltext verfügbar ist, hängt von der Quelle, Open Access und Ihren vorhandenen Lizenzen ab. Ein EUVIMED-Tarif ersetzt keine zusätzlichen Verlagsrechte.

Tarife ansehen →