PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords
ACM Transactions on Asian and Low-Resource Language Information Processing · 30.11.2025 · 24 (11) · 1-17
DOI: 10.1145/3765962 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Asian and Low-Resource Language Information Processing, online erschienen am 25.10.2025. Verfasst von Panyut Sriwirote und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 36 Literaturangaben.
Abstract (Originalsprache)
Although WangchanBERTa has become the de facto standard in transformer-based Thai language modeling, it still has shortcomings in regard to the understanding of foreign words, most notably English words, which are often borrowed without orthographic assimilation into Thai in many contexts. We identify the lack of foreign vocabulary in WangchanBERTa’s tokenizer as the main source of these shortcomings. We then expand WangchanBERTa’s vocabulary via vocabulary transfer from XLM-R’s pretrained tokenizer and pretrain a new model using the expanded tokenizer, starting from WangchanBERTa’s checkpoint, on a new dataset that is larger than the one used to train WangchanBERTa. Our results show that our new pretrained model, PhayaThaiBERT, outperforms WangchanBERTa in many downstream tasks and datasets.
Zitieren
APA
Sriwirote, P., Rutherford, A. T., Thapiang, J., & Timtong, V. (2025). PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords. ACM Transactions on Asian and Low-Resource Language Information Processing, 24(11), 1-17. https://doi.org/10.1145/3765962
Vancouver
Sriwirote P, Rutherford AT, Thapiang J, Timtong V. PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords. ACM Transactions on Asian and Low-Resource Language Information Processing. 2025;24(11):1-17. doi:10.1145/3765962
BibTeX
@article{Sriwirote2025,
author = {Sriwirote, Panyut and Rutherford, Attapol T. and Thapiang, Jalinee and Timtong, Vasan},
title = {PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords},
journal = {ACM Transactions on Asian and Low-Resource Language Information Processing},
year = {2025},
volume = {24},
number = {11},
pages = {1--17},
doi = {10.1145/3765962}
}
EUVIMED