EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

VoiceBench: Benchmarking LLM-Based Voice Assistants

Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T. Tan, Haizhou Li

Transactions of the Association for Computational Linguistics · 01.04.2026 · 14 · 378-398

DOI: 10.1162/tacl.a.628 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 01.04.2026. Verfasst von Yiming Chen und 5 weiteren Autor:innen. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advancements in large language models (LLMs) like GPT-4o have enabled real-time speech interactions through LLM-based voice assistants, offering an improved user experience over text-based interactions. However, a suitable benchmark to rigorously evaluate such speech interactions systems is currently lacking. To bridge this gap, we introduce VoiceBench, the first benchmark specifically designed to assess LLM-based voice assistants. VoiceBench comprises 6,783 synthetic and real spoken instructions recorded from diverse speakers across eight distinct tasks. These instructions are meticulously crafted to assess three crucial capability areas: general knowledge, instruction-following, and safety compliance. Furthermore, VoiceBench systematically incorporates realistic variations common in spoken interactions, including differences in speaker characteristics (e.g., accents), heterogeneous environmental conditions (e.g., reverberation), and content complexities such as mispronunciations. Extensive experiments reveal the limitations of current LLM-based voice assistant models and offer valuable insights for future research and development in this field.1

Zitieren

APA

Chen, Y., Yue, X., Zhang, C., Gao, X., Tan, R. T., & Li, H. (2026). VoiceBench: Benchmarking LLM-Based Voice Assistants. Transactions of the Association for Computational Linguistics, 14, 378-398. https://doi.org/10.1162/tacl.a.628

Vancouver

Chen Y, Yue X, Zhang C, Gao X, Tan RT, Li H. VoiceBench: Benchmarking LLM-Based Voice Assistants. Transactions of the Association for Computational Linguistics. 2026;14:378-398. doi:10.1162/tacl.a.628

BibTeX
@article{Chen2026,
  author = {Chen, Yiming and Yue, Xianghu and Zhang, Chen and Gao, Xiaoxue and Tan, Robby T. and Li, Haizhou},
  title = {VoiceBench: Benchmarking LLM-Based Voice Assistants},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  volume = {14},
  pages = {378--398},
  doi = {10.1162/tacl.a.628}
}
Kostenlos starten →