EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Accurate Models of NVIDIA Tensor Cores

Faizan Khattak, Mantas Mikaitis

ACM Transactions on Architecture and Code Optimization · 30.09.2026 · 23 (3) · 1-25

DOI: 10.1145/3830409 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Architecture and Code Optimization, online erschienen am 07.08.2026. Verfasst von Faizan Khattak und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: EPSRC.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Matrix multiplication is a fundamental operation in both training of neural networks and inference. To accelerate matrix multiplication, Graphical Processing Units (GPUs) provide it implemented in hardware. Due to the increased throughput over the software-based matrix multiplication, the multipliers are increasingly used outside AI, to accelerate various applications in scientific computing. However, matrix multipliers targeted at AI are at present not compliant with IEEE 754 floating-point arithmetic behaviour, with different vendors offering different numerical features. This leads to non-reproducible results across different generations of GPU architectures, at the matrix multiply-accumulate instruction level. To study numerical characteristics of matrix multipliers—such as rounding behaviour, accumulator width, normalization points, extra carry bits, and others—test vectors are typically constructed. Yet, these vectors may or may not distinguish between different hardware models, and due to limited hardware availability, their reliability across many different platforms remains largely untested. We present software models for emulating the inner product behavior of low-precision and mixed-precision matrix multipliers in the V100, A100, H100, and B200 data center GPUs in most supported input formats of interest to mixed-precision algorithm developers: 8-bit, 16-bit, and 19-bit floating point. These matrix multiplier models are first approximated by determining the numerical features via test vectors designed to trigger outputs sensitive to bit level differences in the implementation, followed by semi-exhaustive comparison (randomised input vectors of 10 7 values) between the models and the actual GPU matrix multipliers—this process is repeated until the model is bit-accurate. These models enable verification of test vectors before applying them to real hardware and also support computational scientists and mixed-precision algorithm developers with easy-to-use accurate models available in MATLAB—we demonstrate their use in multi-word emulation algorithms for matrix multiplication.

Förderung

  • EPSRC

Zitieren

APA

Khattak, F., & Mikaitis, M. (2026). Accurate Models of NVIDIA Tensor Cores. ACM Transactions on Architecture and Code Optimization, 23(3), 1-25. https://doi.org/10.1145/3830409

Vancouver

Khattak F, Mikaitis M. Accurate Models of NVIDIA Tensor Cores. ACM Transactions on Architecture and Code Optimization. 2026;23(3):1-25. doi:10.1145/3830409

BibTeX
@article{Khattak2026,
  author = {Khattak, Faizan and Mikaitis, Mantas},
  title = {Accurate Models of NVIDIA Tensor Cores},
  journal = {ACM Transactions on Architecture and Code Optimization},
  year = {2026},
  volume = {23},
  number = {3},
  pages = {1--25},
  doi = {10.1145/3830409}
}
Kostenlos starten →