binary32

A binary32 value is a sign, a biased exponent and a fraction, packed as s · e · f from the most significant bit down. When the exponent field is neither all zeros nor all ones the value is (−1)s × 1.f × 2e − 127: the leading 1 is implied, so 23 stored bits give 24 bits of precision. All zeros in the exponent means a subnormal, (−1)s × 0.f × 2-126, which lets the format lose precision gradually rather than dropping to zero. All ones means infinity when the fraction is zero and a NaN otherwise; the fraction's top bit says whether the NaN is quiet or signaling.

Precision is 24 bits, which is 7.22 decimal digits: any decimal with 7 significant digits survives a round trip through binary32, and 9 digits are enough to write any binary32 value down so that it reads back to the same bits. Every integer up to 2²⁴ = 16,777,216 is exact; above it the spacing between representable numbers is 2, then 4, then 8, and an integer that lands between them is rounded to a neighbour. The largest finite value is about 3.40e38, the smallest normal about 1.18e-38, and the subnormals reach down to 2^-149, about 1.40e-45.

The spacing is what matters in practice. Between two consecutive powers of two the representable values are evenly spaced, 2²³ of them per binade, and the spacing doubles with every binade. Relative to the value it is always between 2⁻²⁴ and 2⁻²³; absolute, it depends entirely on how big the value is.

binary32 is the format hardware likes. Every CPU since the 1980s computes it natively, every GPU is built around it, every shader language makes it the default, and the vector units of a modern processor do eight or sixteen of them per instruction. It is the format of graphics, of audio, of physics in games, of sensor data on microcontrollers, and, until the narrower formats arrived, of machine learning. Its case is bandwidth: four bytes per number is half of binary64, and for anything that streams through memory faster than it computes, half the bytes is twice the speed.

It is also the format that looks like it has more precision than it does. Seven digits is enough for a screen coordinate and not enough for a bank balance, and the 2²⁴ integer cliff at 16,777,216 is low enough that ordinary counts walk off it.

The failures of binary32 are the failures of binary64 arriving sooner. The integer cliff is at sixteen million instead of nine quadrillion; a millimetre is lost at ten kilometres instead of at the orbit of Neptune; a sum of a hundred thousand terms is wrong in the third digit instead of the twelfth. And one failure is its own: because a compiler is allowed to fuse a multiply and an add into one rounding, the same source line can give different bits with different compilers, and two GPU vendors can disagree in the last bit of the same shader and then diverge. Every case below runs here, and where your browser can do binary32 arithmetic itself, through Math.fround, its answer is compared bit for bit.

Every arithmetic operation computes the exact result and then rounds it once, under one of five attributes: to nearest with ties to even (the default everywhere), toward zero, toward −∞, toward +∞, and to nearest with ties away from zero. The directed attributes are how interval arithmetic gets rigorous bounds; ties-to-away is what some decimal conventions expect. Alongside the result, five flags record what happened: inexact when rounding changed the value, underflow when a tiny result was also inexact, overflow when the exact result was too large, divideByZero for a finite divided by zero, and invalid for an operation with no meaningful answer, such as 0/0 or ∞ − ∞, which delivers a quiet NaN.

At this format the tools are mostly switches: the arithmetic is in the hardware, and the question is whether the compiler and the driver are computing what the source says.

binary32 is where cross-vendor divergence is famous. A shader compiled by three GPU drivers is three different sequences of roundings, because each driver may fuse, reorder and approximate differently, all within the rules. For a renderer that iterates, a geometry library, a physics step, the last bit at frame one is a different picture at frame three thousand. The proven case that cft-fp256 exists to serve is atlas-engine's deterministic geometry library, which pins every operation and produces one hash across NVIDIA, AMD and Intel GPUs. The contract this page runs is the same bar reached from the other side: one exact result per operation, defined once, scored everywhere.

"Everywhere" is the point at this format. The library has no floating-point dependence of its own, it is integer arithmetic over 32-bit limbs, so it computes the same bits on a machine with no FPU at all. The project's embedded record has it replaying the published binary32 and binary64 vector sets on an ESP32, 508,000 cases with no disagreement, and its CFT_TINY profile fits the binary32 and binary64 arithmetic into the 32 KB of flash and 2 KB of RAM of an ATmega328P: the same source, on an Arduino Uno, giving the bits this tab gives. The replay below is that check, run here.

The move to binary64 is the cheapest upgrade in numerical computing: on a CPU it costs nothing but memory, and it moves the integer cliff from sixteen million to nine quadrillion and the digits from seven to sixteen. Do it when a count can pass 2²⁴, when positions are measured far from their origin, when a sum runs long, and whenever a binary32 result is going to be fed back into the next step of anything. Keep binary32 for the data itself when the data was never more than seven digits to begin with: an image, a waveform, a model's weights.

The exactness census below is the cleanest picture of the cliff: exact integer arithmetic, and the fraction of it that binary32 can no longer do exactly, watched as the numbers climb.