liveBook · Manning

Precision

As explained in Build a DeepSeek Model (From Scratch)

Precision is the fineness with which a numerical format represents significant digits and distinguishes nearby values.

Precision is the fineness with which a numerical format preserves significant digits and separates nearby values. In floating-point formats, mantissa bits determine precision: more mantissa bits produce more significant digits and smaller gaps between representable numbers. Exponent bits instead determine dynamic range, or how large and small a value the format can represent. section 5.5.3

Reducing bit-width reduces the palette of representable numbers. FP32 uses 23 mantissa bits, whereas FP16 uses 10, so FP16 saves memory but provides lower precision and a smaller range. BF16 keeps FP32's 8 exponent bits and therefore its broad range, but uses only 7 mantissa bits and has lower precision than FP16. section 5.5.1 section 5.5.3

Precision participates in practical format and training trade-offs:

  • FP16 can overflow or underflow because reducing exponent bits limits its dynamic range as well as its precision. section 5.5.3
  • FP8 E5M2 favors range with 5 exponent bits and 2 mantissa bits, while E4M3 favors precision with 4 exponent bits and 3 mantissa bits. section 5.6.4
  • Fine-grained quantization scales small independent blocks, reducing outlier effects and preventing values from being excessively squashed into a narrow scale. section 5.6.4

Uniform E4M3 in training

A conventional mixed-precision design uses E4M3 for the forward pass and E5M2 for backward gradients, where outliers may overflow E4M3. With fine-grained scaling mitigating local outliers, a training design can use the higher-precision E4M3 format uniformly for forward and backward operations. section 5.6.4

Figure 5.15 A visual comparison of a number represented in 32-bit floating-point (FP32) versus 16-bit floating-point (FP16). The diagram illustrates the reduction in the number of bits allocated for the exponent and the mantissa, which results in lower precision and a smaller memory footprint.

Compare FP32 and FP16 visually by inspecting how fewer exponent and mantissa bits reduce precision and memory footprint. section 5.5.1

How Build a DeepSeek Model (From Scratch) covers it

How deep it goes
Explains precision through mantissa allocation, contrasts FP32, FP16, BF16, and FP8, and connects local scaling to training stability.
Examples
Uses FP32, FP16, BF16, E5M2, E4M3, color-palette analogy, and uniform E4M3 training as concrete comparisons.
What you can do after
Evaluate numerical formats by comparing mantissa precision, exponent range, memory footprint, and susceptibility to overflow or underflow.
How it connects
Connects precision to quantization, mantissa, exponent, dynamic range, overflow, underflow, and mixed-precision training.

Also explained in

Compare how 13 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a DeepSeek Model (From Scratch). The section links point to the parts of the book it draws on.