Floating-point precision measures how finely a computer can represent numerical values. Unlike the continuous real-number line, a floating-point format provides only finitely many representable points between values. section 18.3.1
Common floating-point formats trade numerical detail, storage, and computation:
- float16, or half precision, stores numbers in 16 bits. section 18.3.1
- float32, or single precision, stores numbers in 32 bits and can represent very small values useful for stable training gradients. section 18.3.1 section 16.4.1
- float64, or double precision, stores numbers in 64 bits; float8 provides an even lower-precision option. section 18.3.1
Floating-point values are encoded using a sign, a significant value called the mantissa, and an exponent. Representable values are not evenly spaced: larger numbers generally have lower precision, so conversion error depends strongly on the value’s magnitude. section 18.3.1
Encoding π in float32
A float32 representation approximates π using a sign bit, an integer exponent, and an integer mantissa. The result is close to π rather than exactly equal to it because only a finite set of values is representable. section 18.3.1
The number pi encoded in single precision via a sign bit, an integer exponent, and an integer mantissa
Inspect the figure to see how π is encoded through its sign bit, exponent, and mantissa. section 18.3.1
