Continual Learning of Large Language Models: A Comprehensive Survey
ACM Computing Surveys · 30.04.2026 · 58 (5) · 1-42
DOI: 10.1145/3735633 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 20.11.2025. Verfasst von Haizhou Shi und 8 weiteren Autor:innen. Die Arbeit wurde bisher 84-mal zitiert (Crossref) und nennt 304 Literaturangaben.
Abstract (Originalsprache)
The challenge of effectively and efficiently adapting statically pre-trained Large Language Models (LLMs) to ever-evolving data distributions remains predominant. When tailored for specific needs, pre-trained LLMs often suffer from significant performance degradation in previous knowledge domains—a phenomenon known as “catastrophic forgetting” . While extensively studied in the Continual Learning (CL) community, this problem presents new challenges in the context of LLMs. In this survey, we provide a comprehensive overview and detailed discussion of the current research progress on LLMs within the context of CL. Besides the introduction of the preliminary knowledge, this survey is structured into four main sections: we first describe an overview of continually learning LLMs, consisting of two directions of continuity: vertical continuity (or vertical continual learning) , i.e., continual adaptation from general to specific capabilities, and horizontal continuity (or horizontal continual learning) , i.e., continual adaptation across time and domains (Section 3 ). Following vertical continuity, we summarize three stages of learning LLMs in the context of modern CL: Continual Pre-Training (CPT), Domain-Adaptive Pre-training (DAP), and Continual Fine-Tuning (CFT) (Section 4 ). We then provide an overview of evaluation protocols for continual learning with LLMs, along with currently available data sources (Section 5 ). Finally, we discuss intriguing questions related to continual learning for LLMs (Section 6 ). This survey sheds light on the relatively understudied domain of continually pre-training, adapting, and fine-tuning large language models, suggesting the necessity for greater attention from the community. Key areas requiring immediate focus include the development of practical and accessible evaluation benchmarks, along with methodologies specifically designed to counter forgetting and enable knowledge transfer within the evolving landscape of LLM learning paradigms. The full list of articles examined in this survey is available at https://github.com/Wang-ML-Lab/llm-continual-learning-survey.
Zitieren
APA
Shi, H., Xu, Z., Wang, H., Qin, W., Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., & Wang, H. (2026). Continual Learning of Large Language Models: A Comprehensive Survey. ACM Computing Surveys, 58(5), 1-42. https://doi.org/10.1145/3735633
Vancouver
Shi H, Xu Z, Wang H, Qin W, Wang W, Wang Y, et al. Continual Learning of Large Language Models: A Comprehensive Survey. ACM Computing Surveys. 2026;58(5):1-42. doi:10.1145/3735633
BibTeX
@article{Shi2026,
author = {Shi, Haizhou and Xu, Zihao and Wang, Hengyi and Qin, Weiyi and Wang, Wenyuan and Wang, Yibin and Wang, Zifeng and Ebrahimi, Sayna and Wang, Hao},
title = {Continual Learning of Large Language Models: A Comprehensive Survey},
journal = {ACM Computing Surveys},
year = {2026},
volume = {58},
number = {5},
pages = {1--42},
doi = {10.1145/3735633}
}
EUVIMED