VoiceBench: Benchmarking LLM-Based Voice Assistants
Transactions of the Association for Computational Linguistics · 01.04.2026 · 14 · 378-398
DOI: 10.1162/tacl.a.628 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 01.04.2026. Verfasst von Yiming Chen und 5 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Recent advancements in large language models (LLMs) like GPT-4o have enabled real-time speech interactions through LLM-based voice assistants, offering an improved user experience over text-based interactions. However, a suitable benchmark to rigorously evaluate such speech interactions systems is currently lacking. To bridge this gap, we introduce VoiceBench, the first benchmark specifically designed to assess LLM-based voice assistants. VoiceBench comprises 6,783 synthetic and real spoken instructions recorded from diverse speakers across eight distinct tasks. These instructions are meticulously crafted to assess three crucial capability areas: general knowledge, instruction-following, and safety compliance. Furthermore, VoiceBench systematically incorporates realistic variations common in spoken interactions, including differences in speaker characteristics (e.g., accents), heterogeneous environmental conditions (e.g., reverberation), and content complexities such as mispronunciations. Extensive experiments reveal the limitations of current LLM-based voice assistant models and offer valuable insights for future research and development in this field.1
Zitieren
APA
Chen, Y., Yue, X., Zhang, C., Gao, X., Tan, R. T., & Li, H. (2026). VoiceBench: Benchmarking LLM-Based Voice Assistants. Transactions of the Association for Computational Linguistics, 14, 378-398. https://doi.org/10.1162/tacl.a.628
Vancouver
Chen Y, Yue X, Zhang C, Gao X, Tan RT, Li H. VoiceBench: Benchmarking LLM-Based Voice Assistants. Transactions of the Association for Computational Linguistics. 2026;14:378-398. doi:10.1162/tacl.a.628
BibTeX
@article{Chen2026,
author = {Chen, Yiming and Yue, Xianghu and Zhang, Chen and Gao, Xiaoxue and Tan, Robby T. and Li, Haizhou},
title = {VoiceBench: Benchmarking LLM-Based Voice Assistants},
journal = {Transactions of the Association for Computational Linguistics},
year = {2026},
volume = {14},
pages = {378--398},
doi = {10.1162/tacl.a.628}
}
EUVIMED