Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 15.06.2026 · 10 (2) · 1-28
DOI: 10.1145/3810200 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 15.06.2026. Verfasst von Xinyue Gui und 7 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 88 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JST CRONOS.
Abstract (Originalsprache)
Autonomous vehicles (AVs) need to communicate with other road users in uncertain scenarios via external human-machine interfaces (eHMIs). Existing eHMIs are rule-based, relying on manually designed motion and communication patterns for predefined scenarios, which limits adaptability. To address this, we explore LLM-driven eHMIs, a new approach that leverages the human-like reasoning and communication abilities of Large Language Models (LLMs) for generating expressive communicative actions. We investigated two research questions: 1) How does a pre-trained LLM translate intended communicative messages into corresponding eHMI actions? and 2) How communication-efficient are these generated actions to humans? To answer these questions, we first constructed a prompt framework through iterative prototyping, followed by the development of an end-to-end physical pipeline. Finally, we conducted a 26-participant field study using a physical prototype to assess communication effectiveness by comparing human-designed and GPT-o3-mini-generated eHMI actions in terms of participants' correct reactions and message interpretation. Results show that participants' correct reactions do not equate to their correct interpretations. The current pre-trained LLMs can produce communicative actions with similar interpretation comparable to human-designed ones, but cannot enable the correct pedestrian reactions. Our discussion highlights different design patterns between humans and LLMs.
Förderung
- JST CRONOS
Zitieren
APA
Gui, X., Xia, D., Colley, M., Seo, S. H., Chang, C., Javanmardi, E., Tsukada, M., & Igarashi, T. (2026). Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 10(2), 1-28. https://doi.org/10.1145/3810200
Vancouver
Gui X, Xia D, Colley M, Seo SH, Chang C, Javanmardi E, et al. Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2026;10(2):1-28. doi:10.1145/3810200
BibTeX
@article{Gui2026,
author = {Gui, Xinyue and Xia, Ding and Colley, Mark and Seo, Stela Hanbyeol and Chang, Chia-Ming and Javanmardi, Ehsan and Tsukada, Manabu and Igarashi, Takeo},
title = {Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles},
journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
year = {2026},
volume = {10},
number = {2},
pages = {1--28},
doi = {10.1145/3810200}
}
EUVIMED