New Preprint: Accurate Models of NVIDIA Tensor Cores

Category: Publications
Date: Monday 12 January 2026

Author: Faizan A. Khattak

Introduction

Recent TOP500 supercomputers show a clear architectural trend: over half of the systems in the TOP500 support mixed-precision matrix multiplication in hardware. This shift is driven by the need for extreme throughput and energy efficiency, especially for workloads dominated by dense linear algebra for instance in AI, and scientific computing applications. To illustrate this trend, consider JUPITER (see Figure 1), currently the 4th fastest computer in the world on the TOP500 (November 2025) list.
Jupiter features Grace-Hopper superchip (GH200) which is optimized for AI by, amongst other things, the inclusion of low and mixed precision matrix multipliers known as tensor cores on NVIDIA GPUs.

Almost all recent GPUs have such dedicated matrix multipliers designed to accelerate matrix operations. These matrix multipliers do not conform to the IEEE 754 floating-point standard, and their numerical features are determined through numerical feature testing methods.

The MATLAB Tensor Core v0.2, designed by members of Scientific Computation group Faizan Khattak and Mantas Mikaitis and documented in their new preprint paper [1], provides MATLAB-based models that simulate the numerical behavior of matrix multipliers (tensor cores) found in NVIDIA V100, A100, A2, A30, Ada 1000 RTX, L40S, H100, GH200, and B200 GPUs. Rather than executing computations on actual GPU hardware, the toolbox provides bit-equivalent numerical features as that of tensor cores on the respective GPUs and allows customisation.

Motivation

General matrix-matrix multiplication (GEMM) is defined as

$D = \alpha A B + \beta C,$

where $A$ and $B$ are input matrices with $\alpha,\beta$ as scalar inputs and $C$ is the pre-existing or accumulation matrix. For simple matrix multiplication i.e., $AB$ , GEMM with $\alpha=1$ and $\beta=0$ can be taken. While GEMM is extremely fast when implemented with tensor cores, the numerical behavior can differ significantly from standard implementation that uses basic floating-point arithmetic.

On GPUs, a GEMM operation may employ multiple tensor cores over different threads. The current version of the toolbox does not simulate the numerical features of the GEMM but instead of a single tensor core called recursively. Therefore, the GEMM implementation in the toolbox is kept recursive, i.e. (using MATLAB's indexing notation)

$\begin{align*} D(i,j)=\alpha\sum_{n=1}^{K/k}\mathrm{TC}(A[i,(n-1)k+1:nk],B[(n-1)k+1:nk,j])+\beta C(i,j) \end{align*}$

where $k$ is tensor core's input vector size, and $K$ is the inner dimension of $AB$ . We assume that $K$ is a multiple of $k$ for the purposes of demonstration. By explicitly modeling these tensor core operations in MATLAB, the proposed toolbox enables a systematic investigation of the effects of mixed-precision arithmetic on numerical accuracy, stability, and error propagation.

Existing approaches typically rely on CPFloat [2] or chop [3] to simulate rounding operations of the IEEE 754 standard. To emulate tensor-core-like behaviour, these methods apply CPFloat or chop repeatedly across a sequence of operations. While useful, such approximations are insufficient to accurately capture tensor core arithmetic, particularly the multi-operand accumulation performed without immediate normalization for a chain of addition operations.

Overview of the Toolbox

The toolbox is available on GitHub and is organized into a small number of core components.

Tensor Core Models

The models directory contains MATLAB functions emulating the tensor cores of the above mentioned GPUs with identical features. For instance, models/V100TC.m simulates the model of V100 tensor cores, and requires user to supply $\alpha$ , $A$ , $B$ , $\beta$ , $C$ and the output format - the input format of the tensor core in the V100 is by default binary16. The toolbox provides support for all fp8, fp16, bfloat16, TensorFloat32 input formats present on the latest GPUs.

Example Usage and Comparison With Current Practice

In the example case below we compare the current practice, that is with CPFloat or chop function, against one of the tensor core models (of NVIDIA A100) for binary16 input and binary32 as output for an example matrices $A\in\mathbb{R}^{M\times K},~B\in\mathbb{R}^{K\times N},~ C\in\mathbb{R}^{M\times N}$ with $M=K=N=5$
drawn from the normal distribution. The example assumes $\alpha=\beta=1$ , and the implementation and an example run is shown in Figure 2. We can see that the computed $D$ from CPFloat and from A100TC are different.

Figure 2: A100 tensor core model comparison to a basic use of CPFloat for simulating mixed-precision tensor core in MATLAB.

Figure 3 illustrates the V100 example case which only supports binary16 (half precision) as input precision format, and therefore, only requires the user to input the output format. The test is performed on the same data as in the previous example. Again, we get different results compared with the tensor core implemented via the CPFloat rounding operator. The difference in results is due to the different numerical features of V100 compared to that of A100 and later generation GPU tensor cores.

Figure 3: V100 tensor core model comparison to only CPFloat implementation for $AB+C$ in Matlab.

Based on the above example cases, it can be observed that CPFloat or chop alone cannot easily provide a bit-equivalent behaviour to tensor cores. It is important to note, however, that although the proposed package implements the functionality of a single tensor core, full GEMM capabilities are not yet supported and will be considered in a future release.

Validation

The proposed package has been extensively validated against GPU-based tensor core results. Specifically, it has been tested on $10^5$ randomized input cases, as well as on an additional $5\times10^4$ inputs targeting a special subset of floating-point subnormal numbers, which are less precise than most of other representable numbers in a particular format. In all tested scenarios, the results produced by the package match the corresponding GPU outputs exactly.

Example validation data for 5,000 randomly generated inputs is publicly available in the accompanying GitHub repository. The complete validation dataset is available from the authors upon reasonable request.

Conclusion

The MATLAB Tensor Core v0.2 bridges the gap between GPU hardware behavior and software numerical analysis. By providing accurate MATLAB models of tensor cores, it allows researchers and practitioners to easily explore how modern hardware influences matrix computations, compare different tensor cores, and compare each of them against the standard IEEE 754 arithmetic.

References

F. A. Khattak and M. Mikaitis. Accurate Models of NVIDIA Tensor Cores. arXiv:2512.07004 [cs.MS]. Dec. 2025.
M. Fasi and M. Mikaitis. CPFloat: A C library for emulating low-precision arithmetic. ACM Trans. Math. Software, 49:2. Jun. 2023.
N. J. Higham and S Pranesh. Simulating low precision floating-point arithmetic. SIAM Journal on Scientific Computing, 41 (5), C585-C602. Jan. 2019.
B. Hickmann and D. Bradford. Experimental analysis of matrix multiplication functional units. 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH). Oct. 2019.
M.Fasi, N. J. Higham, M. Mikaitis and S. Pranesh. Numerical Behavior of NVIDIA Tensor Cores. PeerJ Comput. Sci., 7:e330. Feb. 2021.
X. Li, A. Li, B. Fang, K. Swirydowicz, I. Laguna, G. Gopalakrishnan. FTTN: Feature-Targeted Testing for Numerical Properties of NVIDIA & AMD Matrix Accelerators. arXiv:2403.00232 [cs.AR]. Mar. 2024.

About the author

Faizan A. Khattak is a Research Fellow in the School of Computer Science.