EUVIMEDEUVIMED entdecken
CrossrefInfokarte

Prompting Strategies for Readability Alignment Between User-Generated Ophthalmology Questions and Large Language Model Answers: Comparative Evaluation Study

Mário César Bulla, Daniel Lavinsky, Alexandre Antonio Marques Rosa, Pedro Carlos Carricondo

JMIR Formative Research · 2026-11-05 · 10 · e97408-e97408

DOI: 10.2196/97408 ↗
AbstractBibliografische Angaben

Abstract

Abstract Background Accessible health communication is essential for effective patient education, yet most medical information remains written at a level above the general population’s recommended reading level. Large language models (LLMs) can simplify complex medical information, but it remains unclear whether responses can be automatically adapted to match the linguistic complexity of user-generated questions seeking medical information. Objective To evaluate whether the LLM-generated responses to user-generated ophthalmology questions align with their readability and to compare different prompting strategies for readability control across multiple LLM architectures, including their impact on response quality as well as computational cost and efficiency. Methods A total of 400 user-generated ophthalmology questions collected from public YouTube comments were categorized into 3 readability tiers based on the Flesch Reading Ease Score. A controlled comparative evaluation of responses generated by 4 LLMs (GPT 5.2, Gemini 3, Claude Opus 4.5, and OSS-20b) was performed under 4 prompting conditions: Baseline (no readability control), Formula-Informed Readability Prompt, Semantic Readability Inference, and External Calculator–Guided Prompt. Agreement between question and response readability was assessed using quadratic-weighted Cohen kappa. Response quality was evaluated using BERTScore and LLM-as-a-judge assessments. Computational efficiency was evaluated through token usage, estimated cost per response, and latency. Results The external calculator–guided strategy produced the strongest alignment in readability across models, achieving the highest agreement (Gemini 3: κ=0.94; GPT 5.2: κ=0.90), while maintaining favorable computational efficiency. Formula-informed prompting showed intermediate performance, whereas semantic inference demonstrated poor agreement due to unreliable readability estimation. High-performing models (GPT 5.2 and Gemini 3) maintained very high factual accuracy (>98%) across strategies, indicating that readability control did not substantially degrade correctness. In contrast, the open-source OSS-20b model showed lower accuracy (83.75%‐90%) and greater sensitivity to readability constraints. Semantic similarity remained high across models (Bidirectional Encoder Representations from Transformers Score [BERTScore] F 1 >0.87). Longer questions were associated with slightly lower accuracy, particularly in lower-performing configurations. Conclusions Aligning LLM responses to the readability of user-generated questions is feasible and can be reliably achieved using externally guided readability signals. Hybrid pipelines that combine traditional readability metrics with LLM-generated content may therefore represent a practical approach for scalable, readability-adaptive health communication systems.

Wissen, das zu Ihrer Arbeit passt.

Vom kostenlosen Einstieg über die persönliche Recherche bis zur vollständigen Research Suite.

Tarife ansehen →

Sind alle Verlagsvolltexte enthalten?

Nein. Bibliografische Nachweise, Abstracts und Volltexte sind unterschiedliche Inhalte. Ob ein Volltext verfügbar ist, hängt von der Quelle, Open Access und Ihren vorhandenen Lizenzen ab. Ein EUVIMED-Tarif ersetzt keine zusätzlichen Verlagsrechte.

Tarife ansehen →