Transformer technology in molecular science
WIREs Computational Molecular Science · 2024-07 · 14 (4)
DOI: 10.1002/wcms.1725 ↗Kurz gefasst
Fachartikel in der Zeitschrift WIREs Computational Molecular Science, online erschienen am 04.08.2024. Verfasst von Jian Jiang und 8 weiteren Autor:innen. Beteiligte Einrichtung: Research Center of Nonlinear Science, School of Mathematical and Physical Sciences, Wuhan Textile University, Wuhan, China. Die Arbeit wurde bisher 71-mal zitiert (Crossref) und nennt 201 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: NIAID NIH HHS, NIGMS NIH HHS, National Aeronautics and Space Administration.
Abstract (Originalsprache)
A transformer is the foundational architecture behind large language models designed to handle sequential data by using mechanisms of self‐attention to weigh the importance of different elements, enabling efficient processing and understanding of complex patterns. Recently, transformer‐based models have become some of the most popular and powerful deep learning (DL) algorithms in molecular science, owing to their distinctive architectural characteristics and proficiency in handling intricate data. These models leverage the capacity of transformer architectures to capture complex hierarchical dependencies within sequential data. As the applications of transformers in molecular science are very widespread, in this review, we only focus on the technical aspects of transformer technology in molecule domain. Specifically, we will provide an in‐depth investigation into the algorithms of transformer‐based machine learning techniques in molecular science. The models under consideration include generative pre‐trained transformer (GPT), bidirectional and auto‐regressive transformers (BART), bidirectional encoder representations from transformers (BERT), graph transformer, transformer‐XL, text‐to‐text transfer transformer, vision transformers (ViT), detection transformer (DETR), conformer, contrastive language‐image pre‐training (CLIP), sparse transformers, and mobile and efficient transformers. By examining the inner workings of these models, we aim to elucidate how their architectural innovations contribute to their effectiveness in processing complex molecular data. We will also discuss promising trends in transformer models within the context of molecular science, emphasizing their technical capabilities and potential for interdisciplinary research. This review seeks to provide a comprehensive understanding of the transformer‐based machine learning techniques that are driving advancements in molecular science. This article is categorized under: Data Science > Chemoinformatics Data Science > Artificial Intelligence/Machine Learning
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Research Center of Nonlinear Science, School of Mathematical and Physical Sciences, Wuhan Textile University, Wuhan, China.
- Department of Mathematics, Michigan State University, East Lansing, Michigan, USA.
- Key Laboratory of Computational Mathematics, Guangdong Province, and School of Mathematics, Sun Yat-sen University, Guangzhou, China.
- Department of Electrical and Computer Engineering, Michigan State University, East Lansing, Michigan, USA.
- Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, Michigan, USA.
ORCID-Profile: Jie Liu, Guo-Wei Wei
Förderung
- NIAID NIH HHS
- NIGMS NIH HHS
- National Aeronautics and Space Administration
- National Science Foundation
- National Institutes of Health
- Michigan State University Foundation
- Pfizer
Zitieren
APA
Jiang, J., Ke, L., Chen, L., Dou, B., Zhu, Y., Liu, J., Zhang, B., Zhou, T., & Wei, G. W. (2024). Transformer technology in molecular science. WIREs Computational Molecular Science, 14(4). https://doi.org/10.1002/wcms.1725
Vancouver
Jiang J, Ke L, Chen L, Dou B, Zhu Y, Liu J, et al. Transformer technology in molecular science. WIREs Computational Molecular Science. 2024;14(4). doi:10.1002/wcms.1725
BibTeX
@article{Jiang2024,
author = {Jiang, Jian and Ke, Lu and Chen, Long and Dou, Bozheng and Zhu, Yueying and Liu, Jie and Zhang, Bengong and Zhou, Tianshou and Wei, Guo-Wei},
title = {Transformer technology in molecular science},
journal = {WIREs Computational Molecular Science},
year = {2024},
volume = {14},
number = {4},
doi = {10.1002/wcms.1725}
}
EUVIMED