Engagement, Feasibility, and Perceived Alliance With a Large Language Model–Enabled Mental Health Voice and Chatbot: Exploratory Longitudinal Observational Study
JMIR Formative Research · 2026-11-09 · 10 · e89708-e89708
DOI: 10.2196/89708 ↗Astratt
Abstract Background Conversational AI systems are increasingly used for web-based mental health support, yet evidence on sustained engagement, therapeutic alliance, and longitudinal outcomes remains limited. Baseline data from 527 self-referred users of the large language model–enabled chatbot Clare indicated high psychological distress and early development of a subjective working alliance. Objective This exploratory study examined the feasibility and descriptive outcome patterns associated with the use of the multimodal conversational agent Clare over 4 and 8 weeks in a real-world setting, focusing on engagement, perceived working alliance, and mental health measures. Methods We conducted a naturalistic, single-arm, repeated-measures observational study of self-referred, English-speaking adults recruited online via social media and search advertisements. Participants interacted with Clare over 8 weeks. Two analytic samples were defined by engagement and assessment completion: a 4-week sample (active engagement plus completed 4-week assessment) and an 8-week sample (additionally requiring continued engagement and completed 8-week assessment). Outcomes were assessed at baseline, 4, and 8 weeks: engagement metrics (calls, messages, and call duration), alliance measures (Working Alliance Inventory–Short Revised and Mobile Agnew Relationship Measure), and mental health measures (Patient Health Questionnaire–4, Patient Health Questionnaire–Distress, Mini–Social Phobia Inventory, UCLA Loneliness Scale, and Satisfaction With Life Scale). Changes were evaluated using 2-tailed paired-samples t tests with effect sizes; given substantial attrition and reliance on complete-case analyses, linear mixed-effects models were run as sensitivity analyses, and findings warrant cautious interpretation. Results Of 527 baseline participants, 369 enrolled and 348 actively engaged with Clare; 53 out of 348 (15.2%) met the 4-week criteria and 21 out of 348 (6.0%) met the 8-week criteria. Engagement peaked in week 1 (calls: mean 1.77, SD 1.52; messages: mean 10.02, SD 16.68) and declined thereafter. Among T1-T2 complete pairs (n=52), mean Patient Health Questionnaire–Distress scores decreased by 1.13 (95% CI 0.25 to 2.02) points, Patient Health Questionnaire–4 scores decreased by 0.69 (95% CI −0.01 to 1.40) points, Mini–Social Phobia Inventory scores decreased by 1.73 (95% CI 0.88 to 2.58) points, Satisfaction With Life Scale scores increased by 6.23 (95% CI 4.70 to 7.76) points, and UCLA scores increased by 0.48 (95% CI 0.00 to 0.96) points. Among participants with Working Alliance Inventory–Short Revised data (n=53), total scores increased by 0.20 points from T1b to T2 (95% CI −0.01 to 0.42); among the 8-week retained subgroup (n=21), Mobile Agnew Relationship Measure scores increased by 0.22 points from T2 to T3 (95% CI 0.03 to 0.42). Conclusions Sustained engagement was limited, and outcome estimates were obtained from small, highly selected follow-up samples. The observed changes cannot be attributed to Clare because the study had no control group and substantial, nonrandom attrition.
EUVIMED