Members of the Scientific Computation group organise a minisymposium within SIAM PP 2026
Author: Mantas Mikaitis
SIAM Conference on Parallel Processing for Scientific Computing (PP26) took place on March 3-6, 2026, in Berlin, Germany, at the Zuse Institute Berlin and Free University of Berlin. As part of the conference members of Scientific Computation group, Faizan Khattak and Mantas Mikaitis, have co-organised a minisymposium titled Matrix Multiplier, a Hardware Kernel of the 2020s: Numerical Features, Error Analysis & Standardisation with the following abstract.
Abstract:
With the growing demand for high performance, low-precision arithmetic emerged, offering efficiency and sufficient accuracy for AI applications. Acceptance of lower precision has led to floating-point data formats as low as 4 bits in the data center and consumer graphical processing units. Among computational kernels, the General Matrix Multiply—being the backbone of neural network training and inference—is accelerated by hardware matrix multipliers; traditionally basic 
operations sufficed. However, the operations within these units are neither conforming with IEEE 754 nor well documented. A need for standardization of low-precision arithmetic in the machine learning domain has been recognized by IEEE, forming the "IEEE P3109 Standard for Arithmetic Formats for Machine Learning" working group. The lack of documentation of matrix multipliers, and the variation of rounding, normalisation, and precision across architectures, raises challenge of reproducibility, code portability, and the interpretation of results across platforms-both in AI and scientific computing applications which increasingly utilize low precision alongside the traditional double precision. An effort to determine the numerical features of such matrix multipliers and the associated error analysis of algorithms is therefore essential. Lastly, within the context of the P3109 standard, it remains to be seen how these specialized low and mixed-precision units will adapt.
The symposium contained the following three talks.
Custom Hardware Accelerators for Matrix Multiplication
Speaker: George A. Constantinides (Imperial College London, UK).
Abstract: We will review results in the design of, and automated design of, custom hardware accelerators for matrix multiplication. We will look at the design of fixed-point, floating-point and MX hardware for this purpose in Field-Programmable Gate Arrays (FPGAs), review novel techniques developed for automating the design process, and comment on the limits of acceleration.
Compiler-Assisted Relative Error Analysis for Floating-Point: Tools and Reproducibility Challenges
Speaker: Ignacio Laguna (Lawrence Livermore National Laboratory, U.S.)
Abstract: Achieving high performance on modern computing architectures increasingly relies on lower-precision floating-point arithmetic. While this approach accelerates computation, it also introduces critical numerical challenges, including reduced dynamic range, increased susceptibility to exceptions (e.g., overflows, NaNs), amplified rounding errors, and reproducibility concerns across platforms. In this work, we introduce a comprehensive framework for multi-precision relative error analysis, leveraging Clang/LLVM-based instrumentation and dynamic analysis. Our methodology enables real-time detection and mitigation of floating-point exceptions at various precision levels, as well as runtime assessment of a program’s dynamic range, demonstrated through applications to linear solvers. We further explore reproducibility challenges, presenting case studies that reveal compiler-induced numerical discrepancies on emerging GPU platforms. Our tools and findings equip developers with actionable insights for understanding and managing floating-point behavior when porting scientific codes to mixed-precision hardware.
Error Estimates on Compensated Sum Algorithms
Speaker: Longfei Gao (Argonne National Laboratory, U.S.)
Abstract: Compensated sum is an old technique devised in the 60s to achieve higher precision than what the hardware natively supports. It was a particularly useful technique during a period when double precision was not prevalently available. It also found successes in niche scientific areas where higher than double precision is desired. However, over the years, it has become largely forgotten in the field of numerical analysis and floating point arithmetics.
In this talk, we will revisit some classic forms of compensated sum algorithms as well as their more nuanced variations. Error estimates associated with these algorithms will be presented, which can be very sharp. These tight error bounds are then used to guide the design of linear algebraic kernels for the purpose of detecting errors in large scale systems, including hardware defects, random bitflips, bugs in the software stack, etc.
About the author
Mantas Mikaitis is a Lecturer in the School of Computer Science.