Taking another look at stochastic rounding to prevent stagnation in computer systems

Taking another look at stochastic rounding to prevent stagnation in computer systems
Alignment of bits in algorithms for stochastic rounding based on sums. The random bits are added to the significand mt followed by the truncation of it. How the bits are generated and added depends on the implementation—we may only add the k bits to the top k bits of the bottom part of the significand and then use a carry-out bit to control the rounding of mr after the truncation, or we may pack the k random bits in a word as long as mt and add it to mt using integer arithmetic: the propagating carry will cause rounding in the top p bits. Credit: Royal Society Open Science (2022). DOI: 10.1098/rsos.211631

A small team of researchers from the U.S., the U.K. and France has taken a fresh look at the possibility of using stochastic rounding (SR) in certain computer applications to reduce stagnation. In their paper published in the journal Royal Society Open Science, the group describes their approach to surveying the use of SR in applications such as machine learning.

The most common approach is rounding numbers down that are less than a certain value, while those that are more than that value are rounded up. Rounding 3.4, for example, down to 3 and 3.7 up to 4. Here, the rounding value is .5. In such cases, numbers that fall on the rounding value are chosen at random. In this example, 3.5 could be rounded up or down, depending on the whim of the person or computer doing the rounding. Computers also have to perform rounding operations when handling infinite expressions such as π. For humans, rounding up or down can be a useful tool for quick estimations—adding up all the prices of goods in a shopping cart, for example, to ensure there is enough cash on hand when heading to checkout. Rounding is useful for computers, as well, because it allows for making calculations with mathematical constants such as π, but it also introduces a problem—stagnation.

With computers, stagnation occurs when long sums of small quantities, such as 0.1, are lost to rounding. There are a number of approaches to the problem in general applications, but stagnation is a major issue with applications. In this new effort, the researchers are looking at the possibility of using SR in such applications.

SR is a rounding approach for computers that has been around for more than a half-century, but has seen limited use. Programs using SR round a given number using probabilities that are based on its distance from that number. As an example, the number 2.6 has a 60% chance of being rounded to 3 and a 40% chance of being rounded to 2. Such can be used, the researchers note, because they are that percentage "along the way" to the target number. 2.6, for example, is 60% along the way to 3. In SR, the midpoint is considered to be equally likely to be rounded up or down. In such scenarios, the direction is still considered random. Using this approach, the researchers suggest, could prevent rounding from going in the same direction too often and help avoid stagnation. The catch, of course, is the lack of true random number generators on most systems. To overcome this problem, the researchers suggest the use of multiple types of rounding.

More information: Matteo Croci et al, Stochastic rounding: implementation, error analysis and applications, Royal Society Open Science (2022). DOI: 10.1098/rsos.211631

Journal information: Royal Society Open Science

© 2022 Science X Network

Citation: Taking another look at stochastic rounding to prevent stagnation in computer systems (2022, March 23) retrieved 1 October 2023 from https://techxplore.com/news/2022-03-stochastic-rounding-stagnation.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Postpartum women prefer delayed physician rounding


Feedback to editors