Towards Practical Automata-Based Steering of Large Language Models for Diverse Structured Generation
Formal Aspects of Computing · 05.11.2026
DOI: 10.1145/3858629 ↗Kurz gefasst
Fachartikel in der Zeitschrift Formal Aspects of Computing, online erschienen am 05.11.2026. Verfasst von Xiaokun Luan und 4 weiteren Autor:innen. Die Arbeit nennt 37 Literaturangaben.
Abstract (Originalsprache)
Large language models are increasingly used to produce structured outputs, including JSON objects, API routes, and configuration fragments. Validity masks can enforce syntactic constraints but do not ensure that a sample set explores the constrained space broadly. We present DiverseGuide + , a configurable framework for diversity steering under regular expression constraints. Building on automata-based steering, DiverseGuide + tracks traversal histories in the guiding automaton and adjusts probabilities among valid tokens, preserving constrained-decoding validity while encouraging less visited structures. The framework adds soft path-frequency aggregation, which scores a token-induced path by its full frequency profile, and completion-proximity steering, which rewards progress toward acceptance under fixed token budgets. For efficiency, DiverseGuide + precomputes indexed token-transition records using a byte trie over the vocabulary. We evaluate DiverseGuide + against the original automata-based method and a standard structured generation baseline across three language models and six regular expression constraints. Automata-based steering consistently improves state, transition, and path coverage over the baseline, including at higher temperatures, and these gains carry over to library testing. Within this family, DiverseGuide + exposes configurable trade-offs among structural coverage, reject rate, and content diversity, while trading construction time and memory for lower logits processor latency.
Zitieren
APA
Luan, X., Wei, Z., Zhang, Y., Wang, K., & Sun, M. (2026). Towards Practical Automata-Based Steering of Large Language Models for Diverse Structured Generation. Formal Aspects of Computing. https://doi.org/10.1145/3858629
Vancouver
Luan X, Wei Z, Zhang Y, Wang K, Sun M. Towards Practical Automata-Based Steering of Large Language Models for Diverse Structured Generation. Formal Aspects of Computing. 2026. doi:10.1145/3858629
BibTeX
@article{Luan2026,
author = {Luan, Xiaokun and Wei, Zeming and Zhang, Yihao and Wang, Kailong and Sun, Meng},
title = {Towards Practical Automata-Based Steering of Large Language Models for Diverse Structured Generation},
journal = {Formal Aspects of Computing},
year = {2026},
doi = {10.1145/3858629}
}
EUVIMED