EUVIMEDEUVIMED entdecken
Randomisierte kontrollierte StudieStudienprotokollOpen Access

ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial

Christian Burisch iD, Abhav Bellary iD, Frank Breuckmann iD, Jan Ehlers iD, Serge C Thal iD, Timur Sellmann iD, Daniel Gödde iD

JMIR Research Protocols · 2025

DOI: 10.2196/63887 ↗

Kurz gefasst

Randomisierte kontrollierte Studie in der Zeitschrift JMIR Research Protocols, 2025. Verfasst von Christian Burisch und 6 weiteren Autor:innen. Beteiligte Einrichtung: State of North Rhine-Westphalia, Regional Government Düsseldorf, Leibniz-Gymnasium, Essen, Germany. Hauptthemen laut MEDLINE: Education, Medical, Continuing, Artificial Intelligence. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The increasing development and spread of artificial and assistive intelligence is opening up new areas of application not only in applied medicine but also in related fields such as continuing medical education (CME), which is part of the mandatory training program for medical doctors in Germany. This study aimed to determine whether medical laypersons can successfully conduct training courses specifically for physicians with the help of a large language model (LLM) such as ChatGPT-4. This study aims to qualitatively and quantitatively investigate the impact of using artificial intelligence (AI; specifically ChatGPT) on the acquisition of credit points in German postgraduate medical education.

Objective

Using this approach, we wanted to test further possible applications of AI in the postgraduate medical education setting and obtain results for practical use. Depending on the results, the potential influence of LLMs such as ChatGPT-4 on CME will be discussed, for example, as part of a SWOT (strengths, weaknesses, opportunities, threats) analysis.

Methods

We designed a randomized controlled trial, in which adult high school students attempt to solve CME tests across six medical specialties in three study arms in total with 18 CME training courses per study arm under different interventional conditions with varying amounts of permitted use of ChatGPT-4. Sample size calculation was performed including guess probability (20% correct answers, SD=40%; confidence level of 1–α=.95/α=.05; test power of 1–β=.95; P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Education, Medical, ContinuingArtificial IntelligenceHumansGermanyRandomized Controlled Trials as TopicEducational MeasurementGenerative Artificial Intelligence

Stichwörter der Autor:innen

ChatGPTartificial intelligencecontinuing medical educationlarge language modelpostgraduate educationself-assessment program

Autor:innen und Institutionen

  • State of North Rhine-Westphalia, Regional Government Düsseldorf, Leibniz-Gymnasium, Essen, Germany.
  • Department of Didactics and Education Research in the Health Sector, Faculty of Health, Witten/Herdecke University, Witten, Germany.
  • Faculty of Health, Witten/Herdecke University, Witten, Germany.
  • Department of Cardiology, Pneumology, Neurology and Intensive Care Medicine, Klinik Kitzinger Land, Kitzingen, Germany.
  • Department of Cardiology and Vascular Medicine, West German Heart and Vascular Center Essen, University Duisburg-Essen, Essen, Germany.
  • Department of Anesthesiology, HELIOS University Hospital, Wuppertal, Germany.

ORCID-Profile: Christian Burisch, Abhav Bellary, Frank Breuckmann, Jan Ehlers, Serge C Thal, Timur Sellmann, Daniel Gödde

Zitieren

APA

Burisch, C., Bellary, A., Breuckmann, F., Ehlers, J., Thal, S. C., Sellmann, T., & Gödde, D. (2025). ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial. JMIR Research Protocols. https://doi.org/10.2196/63887

Vancouver

Burisch C, Bellary A, Breuckmann F, Ehlers J, Thal SC, Sellmann T, et al. ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial. JMIR Research Protocols. 2025. doi:10.2196/63887

BibTeX
@article{Burisch2025,
  author = {Burisch, Christian and Bellary, Abhav and Breuckmann, Frank and Ehlers, Jan and Thal, Serge C and Sellmann, Timur and Gödde, Daniel},
  title = {ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial},
  journal = {JMIR Research Protocols},
  year = {2025},
  doi = {10.2196/63887}
}
Kostenlos starten →