Large language models with supervised fine-tuning for inflammatory bowel disease diagnosis and prognosis
World Journal of Gastroenterology · 2027-01-07 · 33 (1)
DOI: 10.3748/wjg.123087 ↗Abstract
BACKGROUND Inflammatory bowel disease (IBD) is a chronic, incurable condition characterized by complex complications. Effective management of IBD demands extensive physician expertise and time, presenting major challenges due to limited experience and healthcare resources. Large language models (LLMs) may serve as efficient auxiliary tools to alleviate workforce pressures and enhance the accessibility and efficiency of healthcare services. AIM To evaluate the capabilities of an LLM trained by supervised fine-tuning (SFT) in the diagnosis and treatment of IBD. METHODS In this observational study, the diagnostic and treatment capabilities of a general LLM for IBD were evaluated using cases derived from real-world electronic health records. High-quality medical records were filtered for SFT to construct a dedicated IBD-LLM, followed by a direct head-to-head comparison of IBD care performance between the IBD-LLM and resident physicians. RESULTS The general LLM demonstrated suboptimal performance in IBD diagnosis and treatment, although its response consistency and harmlessness were confirmed. In the head-to-head comparison, IBD-LLM vs resident physicians achieved scores of 2.88 vs 3.47 out of 5 in diagnostic accuracy (P = 0.001), 4.18 vs 3.47 out of 5 in humanistic care (P = 0.001), 4.39 vs 3.41 out of 5 in clinical judgment (P = 0.001), 3.88 vs 3.53 out of 5 in communication skills (P = 0.002), and 4.11 vs 3.74 out of 5 in organizational effectiveness (P = 0.003). Although the IBD-LLM demonstrated performance comparable to resident physicians in three parameters, it showed significantly lower diagnostic accuracy. No safety risks or artificial intelligence hallucinations were observed in outputs from either the general LLM or IBD-LLM. CONCLUSION A specialized LLM can be efficiently trained with limited samples using SFT, and demonstrated significant potential for assisting with medical tasks under the supervision of attending physicians.
EUVIMED