liveBook · Manning

Precision

As explained in GPU Programming with Triton

The fineness with which a numerical representation distinguishes nearby values.

Floating-point precision describes how finely a format distinguishes nearby values. Finite precision causes rounding: a small value can disappear when added to a much larger one. FP32 offers finer spacing and a wider range than FP16; FP16’s shorter mantissa rounds more aggressively. section 4.1.1 section 4.2.1

Because floating-point addition is not truly associative, regrouping a reduction can change its result. Parallel reduction trees, tile shapes, and warp counts can therefore produce numerically close but not bitwise-identical results. Small differences can matter when a reduction feeds a sensitive operation such as softmax. section 4.1.1

Regrouping changes the result

With the same values, (0.1 + 1e20) − 1e20 evaluates to 0.0, while 0.1 + (1e20 − 1e20) evaluates to 0.1. The first grouping rounds away 0.1. section 4.1.1

Figure 4.7 (Left) Dynamic range and precision comparison between FP32 and FP16. FP32 covers a far wider range with finer spacing between representable values. (Right) Bit-level breakdown showing how both formats encode the same number, with FP16 producing a less precise result due to its shorter mantissa.
CH04 F07 Fartale

Compare FP32 and FP16 spacing, dynamic range, and bit-level representation. section 4.2.1

How GPU Programming with Triton covers it

How deep it goes
Explains rounding, non-associativity, and format differences, then demonstrates max-shifting to stabilize softmax.
Examples
Uses regrouped sums involving 0.1 and 1e20, plus logits [1000, 1001, 1002] for stable softmax.
What you can do after
Evaluate how reduction order and reduced precision affect numerical results, and apply max-shifting to stabilize softmax.
How it connects
Connects floating-point precision to reductions, atomic updates, and softmax.

Also explained in

Compare how 13 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.