EUVIMEDExplore EUVIMED
VergleichsstudieOpen access · CC BY-NC-ND 4.0

ChatGPT in urogynecology: Comparing large language model responses to human experts

Reut Rotem iD, Craven Simon, Misgav Rottenstreich iD, Barry A O'reilly iD, Adi Y Weintraub, Orfhlaith E O'Sullivan

Acta Obstetricia et Gynecologica Scandinavica · 2026-08 · 105 (8) · 1629-1635

DOI: 10.1111/aogs.70085 ↗

At a glance

Vergleichsstudie in Acta Obstetricia et Gynecologica Scandinavica, published online on 2025-10-23. Written by Reut Rotem and 5 co-authors. Institution: Department of Urogynaecology, Cork University Maternity Hospital, Cork, Ireland. Major MEDLINE topics: Large Language Models, Urology, Gynecology. Cited 9 times so far (Crossref); 24 references. The full text is freely available (open access, CC BY-NC-ND 4.0).

AbstractSubjectsBibliographic detailsCiteRelated studies

Abstract (original language)

Introduction

Large language models (LLMs) are increasingly used in healthcare, including urogynecology, where stigma may limit open discussion. LLM‐based chat platforms may provide a less intimidating and more accessible way for patients to obtain information, but their reliability requires evaluation. This study compared the quality of ChatGPT‐generated responses in urogynecology with those provided by a consultant urogynecologist, focusing on understandability, helpfulness, and reassurance. Material and Methods A cross‐sectional survey was conducted among urogynecology patients. After informed consent, participants reviewed responses to six common questions, each answered by ChatGPT and a single consultant. A blinded third‐party consultant verified clinical accuracy. Patients rated responses using a 5‐point Likert scale across three domains (maximum score 15 per response). Wilcoxon signed‐rank tests were used for comparison.

Results

A total of 203 patients participated (median age 56 years, interquartile range 46–66). ChatGPT responses received higher total ratings than consultant responses (76 [67–85] vs. 72 [63–80], p < 0.01). Scores were higher for understandability, helpfulness, and reassurance (all p < 0.01). ChatGPT was preferred in four of six questions, one showed no difference, and one favored the consultant. Subgroup analyses showed no significant variation based on patient characteristics.

Conclusions

In this exploratory study, women rated ChatGPT's responses as clearer and more reassuring than consultant answers. These findings reflect patient perceptions in a limited setting and should be interpreted with caution. While LLMs may have a supportive role in patient education, their use must remain secondary to expert clinical care and subject to careful oversight.

Subjects

MeSH terms (MEDLINE)

Large Language ModelsUrologyGynecologyHumansFemaleCross-Sectional StudiesMiddle AgedAgedGenerative Artificial IntelligenceSurveys and QuestionnairesReproducibility of Results

Author keywords

Urogynecologyartificial intelligencedecision‐makingnatural language processingpatient education

Authors and institutions

  • Department of Urogynaecology, Cork University Maternity Hospital, Cork, Ireland.
  • Department of Obstetrics and Gynecology, Shaare Zedek Medical Center, Affiliated with the Hebrew University School of Medicine, Jerusalem, Israel.
  • Department of Urogynecology, National Maternity Hospital, Dublin 2, Ireland.
  • Department of Obstetrics and Gynecology, Faculty of Health Sciences, Soroka University Medical Center, Ben-Gurion University of the Negev, Beer-Sheva, Israel.

ORCID profiles: Reut Rotem, Misgav Rottenstreich, Barry A O'reilly

Cite

APA

Rotem, R., Simon, C., Rottenstreich, M., O'reilly, B. A., Weintraub, A. Y., & O'Sullivan, O. E. (2026). ChatGPT in urogynecology: Comparing large language model responses to human experts. Acta Obstetricia et Gynecologica Scandinavica, 105(8), 1629-1635. https://doi.org/10.1111/aogs.70085

Vancouver

Rotem R, Simon C, Rottenstreich M, O'reilly BA, Weintraub AY, O'Sullivan OE. ChatGPT in urogynecology: Comparing large language model responses to human experts. Acta Obstetricia et Gynecologica Scandinavica. 2026;105(8):1629-1635. doi:10.1111/aogs.70085

BibTeX
@article{Rotem2026,
  author = {Rotem, Reut and Simon, Craven and Rottenstreich, Misgav and O'reilly, Barry A and Weintraub, Adi Y and O'Sullivan, Orfhlaith E},
  title = {ChatGPT in urogynecology: Comparing large language model responses to human experts},
  journal = {Acta Obstetricia et Gynecologica Scandinavica},
  year = {2026},
  volume = {105},
  number = {8},
  pages = {1629--1635},
  doi = {10.1111/aogs.70085}
}
Start for free →