PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent
Journal of Medical Internet Research · 2026-11-09 · 28 · e101607-e101607
DOI: 10.2196/101607 ↗At a glance
Journal article in Journal of Medical Internet Research, published online on 2026-11-09. Written by Tianrui Kuang and 6 co-authors. Institution: Division of Pancreatic Surgery, Department of General Surgery and Laboratory of Pancreatic Cancer, West China Hospital of Sichuan University, No. 37,…. Major MEDLINE topics: Large Language Models, Pancreatic Neoplasms, Tomography, X-Ray Computed, Referral and Consultation. 35 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Accurate interpretation of pancreatic cancer status and resectability-related information, together with effective perioperative communication, remains a major challenge in pancreatic cancer care. In routine practice, unstructured computed tomography (CT) reports and complex clinical information may limit the accessibility and consistency of clinical information delivery and increase the communication burden on clinicians. Large language models (LLMs) offer new opportunities for clinical information support; however, existing applications are often task-specific and rarely combine CT report interpretation with perioperative consultation within a unified patient-oriented framework.
Objective
This study aimed to develop and retrospectively evaluate PancreaGuide, a knowledge-augmented LLM agent designed to support clinician-supervised CT report-based interpretation of pancreatic cancer resectability-related information and standardized perioperative consultation.
Methods
We developed PancreaGuide, an agent-based framework integrating structured prompting, retrieval-augmented generation, and a domain-specific pancreatic surgery knowledge base to support clinical information interpretation and perioperative consultation. A retrospective cohort of 444 patients who underwent contrast-enhanced CT at West China Hospital between 2023 and 2025 was analyzed. Three LLMs, ChatGPT, DeepSeek, and Gemini, were evaluated in both base and PancreaGuide-integrated configurations. CT report-based interpretation of resectability-related information was evaluated using accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F 1 -score, receiver operating characteristic curves (ROCs), and decision curve analysis (DCA) using a surgery-based proxy reference. Perioperative consultation quality was evaluated across 5 dimensions: accuracy, completeness, consistency, safety, and robustness.
Results
Higher classification metric point estimates were observed after integration of the PancreaGuide framework across all 3 backbone models. In CT report-based interpretation of resectability-related information, Gemini+ achieved the highest accuracy of 87.4% (95% CI 0.840‐0.902) and a sensitivity of 95.9% (95% CI 0.927‐0.978). All models showed improved sensitivity and specificity estimates after PancreaGuide integration, with improved agreement between model-generated interpretations and observed surgical management status. ROC and DCA suggested improved discrimination and estimated net benefit using the surgery-based proxy reference. In perioperative consultation, PancreaGuide-integrated models received higher expert-rated scores than their corresponding base models across all 5 evaluation dimensions. Gemini+ achieved the highest overall score (mean 3.88, SD 0.38), followed by ChatGPT+ (mean 3.8, SD 10.36) and DeepSeek+ (mean 3.75, SD 0.35). The greatest improvements were observed in completeness and robustness, with consistent gains across clinical consultation domains.
Conclusions
PancreaGuide provides a unified, knowledge-augmented framework for clinician-supervised CT report-based interpretation of resectability-related information and perioperative information support across multiple LLM backbones. The CT-based findings should be interpreted as agreement with observed surgical management status rather than validation against an independently adjudicated anatomical resectability. PancreaGuide may serve as an auxiliary information interpretation framework to improve the accessibility and consistency of clinical information delivery, while it should not replace formal radiological assessment, multidisciplinary evaluation, or clinician judgment.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Division of Pancreatic Surgery, Department of General Surgery and Laboratory of Pancreatic Cancer, West China Hospital of Sichuan University, No. 37, Guoxue Lane, Wuhou District, CHENGDU, China, 86 17786132270.
- Center of Gerontology and Geriatrics, National Clinical Research Center for Geriatrics, West China Hospital of Sichuan University, Chengdu, China.
Cite
APA
Kuang, T., Wang, R., Zhang, S., Sun, Y., Zhou, J., Wu, Z., & Peng, B. (2026). PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent. Journal of Medical Internet Research, 28, e101607-e101607. https://doi.org/10.2196/101607
Vancouver
Kuang T, Wang R, Zhang S, Sun Y, Zhou J, Wu Z, et al. PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent. Journal of Medical Internet Research. 2026;28:e101607-e101607. doi:10.2196/101607
BibTeX
@article{Kuang2026,
author = {Kuang, Tianrui and Wang, Rui and Zhang, Shaomin and Sun, Yunwei and Zhou, Jin and Wu, Zhong and Peng, Bing},
title = {PancreaGuide for Computed Tomography Report Interpretation and Perioperative Consultation in Pancreatic Cancer: Development and Retrospective Evaluation of a Knowledge-Augmented Large Language Model Agent},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e101607--e101607},
doi = {10.2196/101607}
}
EUVIMED