Towards Optimal Speculative Decoding: A Comprehensive Survey on Optimizations and Future Directions
ACM Computing Surveys · 31.01.2027 · 59 (1) · 1-35
DOI: 10.1145/3846171 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 29.10.2026. Verfasst von Yuyang Ji und 4 weiteren Autor:innen. Die Arbeit nennt 126 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: “Pioneer” and “Leading Goose” Research and Development Program of Zhejiang, Natural Science Foundation of Zhejiang University of Science and Technology, Yangtze River Delta Science and Technology Innovation Community Joint Research Project.
Abstract (Originalsprache)
Speculative decoding (SD) has emerged as a compelling approach to breaking the sequential bottleneck of autoregressive large language model (LLM) inference. By coupling lightweight drafting with parallel verification, SD reduces expensive target-model decoding while preserving generation fidelity. Yet its realized speedup often falls short of its theoretical potential, as inefficiencies persist throughout the speculative decoding pipeline. This survey presents a unified analysis of the factors that ultimately bound the acceleration of SD. We examine how draft effectiveness, verification efficiency, and system execution jointly determine end-to-end performance, and critically synthesize the tradeoffs underlying existing optimization directions. This perspective clarifies where speculative gains are created, where they are lost, and why isolated improvements do not necessarily translate into practical speedup. We further identify the unresolved challenges that continue to limit SD and outline promising directions toward more efficient and scalable LLM inference.
Förderung
- “Pioneer” and “Leading Goose” Research and Development Program of Zhejiang
- Natural Science Foundation of Zhejiang University of Science and Technology
- Yangtze River Delta Science and Technology Innovation Community Joint Research Project
Zitieren
APA
Ji, Y., Xu, W., Qian, X., Zhang, L., & Wan, J. (2027). Towards Optimal Speculative Decoding: A Comprehensive Survey on Optimizations and Future Directions. ACM Computing Surveys, 59(1), 1-35. https://doi.org/10.1145/3846171
Vancouver
Ji Y, Xu W, Qian X, Zhang L, Wan J. Towards Optimal Speculative Decoding: A Comprehensive Survey on Optimizations and Future Directions. ACM Computing Surveys. 2027;59(1):1-35. doi:10.1145/3846171
BibTeX
@article{Ji2027,
author = {Ji, Yuyang and Xu, Wenjian and Qian, Xiaohong and Zhang, Lei and Wan, Jian},
title = {Towards Optimal Speculative Decoding: A Comprehensive Survey on Optimizations and Future Directions},
journal = {ACM Computing Surveys},
year = {2027},
volume = {59},
number = {1},
pages = {1--35},
doi = {10.1145/3846171}
}
EUVIMED