Dual-Scale Transformer with Variable Bitrate Synchronization for Neural Video Compression
ACM Transactions on Multimedia Computing, Communications, and Applications · 31.05.2026 · 22 (5) · 1-24
DOI: 10.1145/3801159 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 21.04.2026. Verfasst von Yiming Wang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China, Natural Science Foundation of Jiangsu Province.
Abstract (Originalsprache)
Neural video compression (NVC) has emerged as a promising paradigm for improving rate-distortion performance. However, existing neural video codecs predominantly rely on convolutional neural networks (CNNs) with limited local receptive fields to generate the latent representations, often neglecting global–local spatial correlations. This leads to suboptimal feature modeling and redundancy in the latent space. To address this limitation, we propose a novel Dual-Scale Transformer (DST) block specifically tailored for NVC, which effectively enhances coding efficiency. The DST block incorporates a Global–Local (Shifted) Window-based Self-Attention (GL(S)WSA) mechanism to jointly capture global structure information and local texture details. Moreover, we design a Cross-Gated Feed-Forward Network (CGFFN) to adaptively modulate complementary components, producing more compact and expressive latent representations. Furthermore, to overcome the drawbacks of traditional asynchronous training and further boost rate-distortion performance, we introduce a Variable Bitrate Synchronization (VBRS) strategy that leverages multi-GPU parallel training, with each GPU dedicated to a specific bitrate and synchronized via gradient backpropagation for joint optimization. Experimental results demonstrate that our proposed method achieves the higher coding performance compared to the previous state-of-the-art (SOTA) methods and significantly outperforms H.266/VVC (VTM-13.2) under various low delay B (LDB) coding configurations.
Förderung
- National Key R&D Program of China
- National Natural Science Foundation of China
- Natural Science Foundation of Jiangsu Province
Zitieren
APA
Wang, Y., Wu, Y., Zhang, Z., Huang, Q., Tang, B., Yang, Z., Zhang, K., & Zhang, L. (2026). Dual-Scale Transformer with Variable Bitrate Synchronization for Neural Video Compression. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(5), 1-24. https://doi.org/10.1145/3801159
Vancouver
Wang Y, Wu Y, Zhang Z, Huang Q, Tang B, Yang Z, et al. Dual-Scale Transformer with Variable Bitrate Synchronization for Neural Video Compression. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(5):1-24. doi:10.1145/3801159
BibTeX
@article{Wang2026,
author = {Wang, Yiming and Wu, Yaojun and Zhang, Zhaobin and Huang, Qian and Tang, Bin and Yang, Zhangjing and Zhang, Kai and Zhang, Li},
title = {Dual-Scale Transformer with Variable Bitrate Synchronization for Neural Video Compression},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2026},
volume = {22},
number = {5},
pages = {1--24},
doi = {10.1145/3801159}
}
EUVIMED