An FP32 register is a hardware register on a CUDA Core that stores a 32-bit floating-point value. In low-precision matrix multiplication, it receives partial sums periodically transferred from a Tensor Core's low-precision accumulator. Continuing accumulation in FP32 reduces underflow risk while preserving the speed of burst-oriented FP8 computation. section 5.6.3
The accumulation path separates throughput from numerical stability:
- Tensor Cores execute WGMMA steps that multiply FP8 inputs and maintain a running low-precision partial sum. section 5.6.3
- At a fixed interval Nc, a CUDA Core promotes the partial sum into an FP32 register. section 5.6.3
- The FP32 register performs the final high-precision accumulation, limiting numerical errors from low-precision storage. section 5.6.3
Figure 5.31 The Increasing Accumulation Precision mechanism in detail. Low-precision accumulation happens in bursts on the Tensor Core, and results are periodically promoted to a high-precision FP32 register on the CUDA Core.

Trace how low-precision Tensor Core bursts periodically promote their results to an FP32 register on a CUDA Core. section 5.6.3
