Assessing the Quality of AI Explanations of Atrial Fibrillation: Comparative Analysis of ChatGPT and Google Gemini
JMIR Formative Research · 2026-11-05 · 10 · e73715
DOI: 10.2196/73715 ↗Abstract
Background Atrial fibrillation (AF) is one of the most common arrhythmias and is associated with multiple risk factors, including stroke and heart failure. Therefore, it is important that accurate and comprehensive educational resources on AF are available. Artificial intelligence (AI) platforms such as Google Gemini and ChatGPT are emerging tools for medical education. To our knowledge, this is the first study to assess the quality of these platforms for AF education. Objective This pilot study aimed to (1) assess the quality of ChatGPT and Google Gemini explanations of AF and its treatment, (2) compare responses from both platforms, and (3) analyze differences in interpretation between cardiologists and nonmedical professionals. Methods The prompt “Explain atrial fibrillation and how to treat it to a patient” was entered into both ChatGPT and Google Gemini. The responses, along with a survey created in Google Forms, were sent out to potential participants between September and October 2024. The survey questions were derived from the Patient Education Materials Assessment Tool for Printable Materials and DISCERN criteria. Each survey question was multiple choice and rated on a 5-point Likert scale, where 1 indicated strong disagreement and 5 indicated strong agreement. Averages of each answer from the survey were calculated and compared using the Wilcoxon signed-rank test. Results The survey was completed by 28 participants, including cardiologists and cardiology fellows and 17 nonmedical professionals (college graduates not employed in the medical field). No significant overall quality differences were observed between ChatGPT and Google Gemini. When comparing participant groups, cardiologists rated AI-generated responses higher than nonmedical professionals for perceived bias (mean 3.82 vs mean 4.33; P=.04) and for explaining consequences of no treatment (mean 2.85 vs mean 1.86; P=.005). In contrast, nonmedical professionals rated structural and organizational features more favorably than cardiologists, including visual cues (mean 3.97 vs mean 2.48; P
EUVIMED