Package Hallucinations as Phantoms in Open Source Software Supply Chains: An Empirical Security Analysis
ACM Transactions on Software Engineering and Methodology · 30.11.2026 · 35 (11) · 1-38
DOI: 10.1145/3830237 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Software Engineering and Methodology, online erschienen am 04.11.2026. Verfasst von Zhe Huang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Zhongguancun Laboratory and the National Cyber Security–National Science and Technology.
Abstract (Originalsprache)
In code generation tasks, large language models (LLMs) frequently recommend nonexistent software packages, referred to as package hallucinations. This may pose a security risk, as attackers could register these fake packages and exploit them in supply chain attacks. To investigate the real-world feasibility, this article conducts a systematic empirical study of this potential threat. We extracted 107 hallucinated packages from eight prominent LLMs and uploaded them to the npm registry over 6 months. The results demonstrate a tangible risk: 35.2% of package installations are attributed to developers, with most initial interactions occurring within a 2-week window post-registration. A name similarity analysis further reveals that 25.2% of hallucinated names fall within an absolute edit distance of \(\leq\) 1 from existing npm packages, posing a direct typosquatting risk. In our evaluated setting, tool-augmented generation, specifically using web search, fails to mitigate this threat. Instead, it significantly amplifies hallucination persistence from 11.2% to 17.3% through context contamination. In the npm registry, the entire attack campaign costs only an estimated $496, making it far more accessible than traditional supply chain attacks yet capable of enabling poisoning effects. To mitigate this emergent threat, we design an autonomous defense agent: NymGuard. It provides a proactive defense by automating the real-time detection of hallucinated packages, their preemptive registration to block malicious use, and the maintenance of a public warning registry. The full implementation has been open sourced and is available on GitHub ( https://github.com/xiaoye798/Package-Hallucination-Research ).
Förderung
- Zhongguancun Laboratory and the National Cyber Security–National Science and Technology
Zitieren
APA
Huang, Z., Liu, C., Wang, Z., Cui, X., Deng, X., & Xu, M. (2026). Package Hallucinations as Phantoms in Open Source Software Supply Chains: An Empirical Security Analysis. ACM Transactions on Software Engineering and Methodology, 35(11), 1-38. https://doi.org/10.1145/3830237
Vancouver
Huang Z, Liu C, Wang Z, Cui X, Deng X, Xu M. Package Hallucinations as Phantoms in Open Source Software Supply Chains: An Empirical Security Analysis. ACM Transactions on Software Engineering and Methodology. 2026;35(11):1-38. doi:10.1145/3830237
BibTeX
@article{Huang2026,
author = {Huang, Zhe and Liu, Chaoge and Wang, Zhi and Cui, Xiang and Deng, Xiao and Xu, Mengyang},
title = {Package Hallucinations as Phantoms in Open Source Software Supply Chains: An Empirical Security Analysis},
journal = {ACM Transactions on Software Engineering and Methodology},
year = {2026},
volume = {35},
number = {11},
pages = {1--38},
doi = {10.1145/3830237}
}
EUVIMED