Mean squared error (MSE) is the average squared difference between corresponding numerical values. For quantized weights, reconstruction MSE measures how closely the quantized values match the originals; it does not alone determine end-to-end perplexity. section 8.2.3
Figure 8.5 Weight reconstruction MSE by format (log scale). INT8 per-channel achieves the lowest MSE (≈3.5×10⁻⁸ average), followed by FP8 E4M3 (≈2.1×10⁻⁷, roughly 6× worse), then FP8 E5M2 (≈8.5×10⁻⁷, roughly 4× worse than E4M3). INT8 wins here because per-channel scaling gives each output channel its own scale factor—fine-grained enough to neutralize FP8's density advantage.

The comparison shows lower reconstruction MSE for INT8 per-channel than for FP8 E4M3 or E5M2, illustrating how scaling granularity affects the measurement. section 8.2.3
















