Precision sets how many bits represent model values. In language model operations, floating-point precision affects memory use and computational speed. section 6.5
Common format tradeoffs include:
- A 32-bit float uses 1 sign bit, 8 exponent bits, and 23 mantissa bits. section 6.5
- 16-bit floating point halves memory requirements compared with 32-bit, but can reduce performance. Mixed-precision training switches between 32-bit and 16-bit operations. section 6.5
- BF16 offers a wider dynamic range than FP16, with less precision. section 6.5
Quantization is a related but distinct approach: it converts floating-point weights to low-bit integers to reduce model size and support faster inference. section 6.0 section 6.5









