ENUMATH 2025 Minisymposium on Probabilistic Rounding
Author: Mantas Mikaitis
The European Conference on Numerical Mathematics and Advanced Applications (ENUMATH) took place on September 1-5, 2025, in Heidelberg, Germany, on the Heidelberg University campus, and attracted more than 700 participants. As part of the conference Silviu-Ioan Filip (INRIA Rennes) and Mantas Mikaitis (University of Leeds) have co-organised a minisymposium titled Latest Advancements in Floating-Point Arithmetic with Probabilistic Rounding with the following abstract.
Abstract:
The default rounding mode in computer arithmetic traditionally is round-to-nearest ties-to-even specified in the IEEE 754 floating-point standard. Given a number, this rounding mode will return a number that can be represented by the underlying floating-point representation, with a certain precision, and minimize the rounding error. Other rounding modes can also be used. For instance, the IEEE 754 standard also specifies directed rounding modes: round-toward-zero, and round towards positive or negative infinity. Depending on the hardware device and the software used to program it, multiple rounding modes that adhere, or not, to the IEEE 754 standard may be provided.
In the last few years a probabilistic rounding mode called stochastic rounding has started to appear in hardware and is looking to gain more traction in the years to come. Stochastic rounding applied to a given number outputs one of the two neighbouring floating-point values with probabilities proportional to how far away they are from that number. It can therefore round the same number to two different floating-point values. This probabilistic feature of stochastic rounding means that in expectation it yields the original number that is
being rounded and indeed in practice has the effect of reducing error accumulation over a sequence of many rounding operations.
Stochastic rounding has been applied in mathematical operations within machine learning, numerical linear algebra, differential equation solvers, and quantum computing. It has been implemented in hardware devices from Graphcore, AMD, Tesla, and Amazon. From a research standpoint, most work has focused on rounding error models for stochastic rounding-based numerical algorithms, experimental research for discovering new applications which can benefit from stochastic rounding, and new algorithms for efficient stochastic rounding hardware implementation. This minisymposium will bring together speakers that work in these areas in order to present the latest advancements in the field and discuss open questions.
The symposium contained the following three talks.
Stochastic rounding: implementation, error analysis and applications (an update to the 2022 survey paper)
Speaker: Mantas Mikaitis (University of Leeds, UK).
Abstract: Stochastic rounding (SR) randomly maps a real number 
to one of the two nearest values in a finite precision number system. The probability of choosing either of these two numbers is 1 minus their relative distance to . This rounding mode was first proposed for use in computer arithmetic in the 1950s and it is currently experiencing a resurgence of interest. If used to compute the inner product of two vectors of length n in floating-point arithmetic, it yields an error bound with constant 
with high probability, where u is the unit round off. This is not necessarily the case for round to nearest (RN), for which the worst-case error bound has constant 
. A particular attraction of SR is that, unlike RN, it is immune to the phenomenon of stagnation, whereby a sequence of tiny updates to a relatively large quantity is lost. This talk will provide an update to the 2022 survey paper that reviewed 70 years worth of literature using, analysing, or designing hardware for SR.
Probabilistic error analysis of limited-precision stochastic rounding
Speaker: Silviu-Ioan Filip (INRIA Rennes - Bretagne Atlantique, France)
Abstract: Classical probabilistic rounding error analysis is particularly well suited to stochastic rounding (SR), and it yields strong results when dealing with floating-point algorithms that rely heavily on summation. For many numerical linear algebra algorithms, one can prove probabilistic error bounds that grow as 
, where n is the problem size and u is the unit roundoff. These probabilistic bounds are asymptotically tighter than the worst-case ones, which grow as 
. For certain classes of algorithms, SR has been shown to be unbiased. However, all these results were derived under the assumption that SR is implemented exactly, which typically requires a number of random bits that is too large to be suitable for practical implementations. We investigate the effect of the number of random bits on the probabilistic rounding error analysis of SR. To this end, we introduce a new rounding mode, limited-precision SR. By taking into account the number 
of random bits used, this new rounding mode matches hardware implementations accurately, unlike the ideal SR operator generally used in the literature. We show that this new rounding mode is biased and that the bias is a function of . As approaches infinity, however, the bias disappears, and limited-precision SR converges to the ideal, unbiased SR operator. We develop a novel model for probabilistic error analysis of algorithms employing SR. Several numerical examples corroborate our theoretical findings.
On the practical implementation of few-bit stochastic rounding, with applications to large language models
Speaker: Andrew Fitzgibbon (Graphcore, UK)
Abstract: Large-scale numerical computations make increasing use of low-precision (LP) floating point formats and mixed precision arithmetic, which can be enhanced by the technique of stochastic rounding (SR). SR requires, in addition to the high-precision input value, a source of random bits. As the provision of high-quality random bits is an additional computational cost, it is of interest to require as few bits as possible without loss of effectiveness. This paper examines a number of possible implementations of few-bit stochastic rounding (FBSR), and shows how several natural implementations can introduce sometimes significant bias into the rounding process, which are not present in the case of infinite-bit, infinite-precision implementations. We explore the impact of these biases in machine learning examples, and hence open another class of configuration parameters of which practitioners should be aware when developing or adopting low-precision floating point.
About the author
Mantas Mikaitis is a Lecturer in the School of Computer Science.