Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
ACM Computing Surveys · 29.02.2024 · 56 (2) · 1-40
DOI: 10.1145/3605943 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Computing Surveys, online erschienen am 14.09.2023. Verfasst von Bonan Min und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1225-mal zitiert (Crossref) und nennt 227 Literaturangaben.
Abstract (Originalsprache)
Large, pre-trained language models (PLMs) such as BERT and GPT have drastically changed the Natural Language Processing (NLP) field. For numerous NLP tasks, approaches leveraging PLMs have achieved state-of-the-art performance. The key idea is to learn a generic, latent representation of language from a generic task once, then share it across disparate NLP tasks. Language modeling serves as the generic task, one with abundant self-supervised text available for extensive training. This article presents the key fundamental concepts of PLM architectures and a comprehensive view of the shift to PLM-driven NLP techniques. It surveys work applying the pre-training then fine-tuning, prompting, and text generation approaches. In addition, it discusses PLM limitations and suggested directions for future research.
Zitieren
APA
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024). Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Computing Surveys, 56(2), 1-40. https://doi.org/10.1145/3605943
Vancouver
Min B, Ross H, Sulem E, Veyseh APB, Nguyen TH, Sainz O, et al. Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey. ACM Computing Surveys. 2024;56(2):1-40. doi:10.1145/3605943
BibTeX
@article{Min2024,
author = {Min, Bonan and Ross, Hayley and Sulem, Elior and Veyseh, Amir Pouran Ben and Nguyen, Thien Huu and Sainz, Oscar and Agirre, Eneko and Heintz, Ilana and Roth, Dan},
title = {Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey},
journal = {ACM Computing Surveys},
year = {2024},
volume = {56},
number = {2},
pages = {1--40},
doi = {10.1145/3605943}
}
EUVIMED