Building safer artificial intelligence mental health chatbots: a framework for transparency, evaluation, and shared accountability
Journal of the American Medical Informatics Association · 2026-08-01 · 33 (8) · 1538-1553
DOI: 10.1093/jamia/ocag078 ↗Abstract
Abstract Background Generative artificial intelligence (AI) chatbots built on large language models are rapidly entering mental-health care, offering human-like support without meeting evidentiary standards for safety or effectiveness. Objective To examine the risks, and outline a governance framework capable of supporting safe, accountable, and equitable deployment of AI mental-health chatbots. Methods We synthesized recent clinical, regulatory, and behavioral health literature on AI mental health chatbots, including reported harms and system failure modes, to identify governance gaps and develop a 3-stage safety framework. Conclusions Embedding transparency, standardized evaluation, and ongoing oversight across the chatbot lifecycle, with clear responsibilities shared among developers, regulators, clinicians, researchers, and professional societies, is essential to ensure that AI systems intended to support mental health do not inadvertently cause harm.
EUVIMED