EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis

Jiayi Mao, Liqun Li, Yanjie Gao, Zegang Peng, Shilin He, Chaoyun Zhang, Si Qin, Samia Khalid, Qingwei Lin, Saravan Rajmohan, Sitaram Lanka, Dongmei Zhang

Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 3070-3092

DOI: 10.1145/3808143 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Jiayi Mao und 11 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Effective incident management in large-scale IT systems relies on troubleshooting guides (TSGs), but their manual execution is slow and error-prone. While recent advances in LLMs offer promise for automating incident management tasks, existing LLM-based solutions lack specialized support for several key challenges, including managing TSG quality issues, interpreting complex control flow, handling data-intensive queries, and exploiting execution parallelism. We first conducted an empirical study on 92 real-world TSGs, and, guided by our findings, we present , a novel end-to-end agentic framework for troubleshooting guide automation. Our approach features a three-stage workflow: the first stage provides a comprehensive guide together with a tool, TSG Mentor, to assist site reliability engineers (SREs) in improving TSG quality; the second stage performs offline preprocessing using LLMs to extract structured execution directed acyclic graphs (DAGs) from unstructured TSGs and to create dedicated Query Preparation Plugins (QPPs); and the third stage executes online using a DAG-guided scheduler-executor framework with a memory system to ensure correct workflow and support parallel execution of independent steps. Our empirical evaluation on a collection of real-world TSGs and incidents demonstrates that achieves a ∼94% success rate on GPT-4.1, outperforming baselines with less time and token consumption. Furthermore, it achieves a remarkable execution time reduction of 32.9% to 70.4% for parallelizable TSGs. Our code and sample data are publicly available at https://github.com/microsoft/StepFly.

Zitieren

APA

Mao, J., Li, L., Gao, Y., Peng, Z., He, S., Zhang, C., Qin, S., Khalid, S., Lin, Q., Rajmohan, S., Lanka, S., & Zhang, D. (2026). StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis. Proceedings of the ACM on Software Engineering, 3(FSE), 3070-3092. https://doi.org/10.1145/3808143

Vancouver

Mao J, Li L, Gao Y, Peng Z, He S, Zhang C, et al. StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis. Proceedings of the ACM on Software Engineering. 2026;3(FSE):3070-3092. doi:10.1145/3808143

BibTeX
@article{Mao2026,
  author = {Mao, Jiayi and Li, Liqun and Gao, Yanjie and Peng, Zegang and He, Shilin and Zhang, Chaoyun and Qin, Si and Khalid, Samia and Lin, Qingwei and Rajmohan, Saravan and Lanka, Sitaram and Zhang, Dongmei},
  title = {StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2026},
  volume = {3},
  number = {FSE},
  pages = {3070--3092},
  doi = {10.1145/3808143}
}
Kostenlos starten →