Dequantization restores the magnitude of a low-precision matrix result by multiplying it with the corresponding activation and weight scaling factors. In fine-grained FP8 computation, Tensor Cores produce the intermediate result, while CUDA Cores perform this conversion before the next framework stage. section 5.6.2
A fine-grained dequantization path proceeds as follows:
- Partition activations into chunks and weights into blocks, assigning each region its own scaling factor. section 5.6.2
- Feed the quantized FP8 values to Tensor Cores for the bulk matrix multiplication. section 5.6.2
- Accumulate partial results in an FP32 CUDA Core register, periodically moving them out of the low-precision Tensor Core environment. section 5.6.3
- Multiply the accumulated result by the relevant scaling factors and store the restored output as BF16 for the next layer. section 5.6.3
Fine-grained matrix multiplication
For an output term Yᵢⱼ = Wᵢ₁X₁ⱼ + Wᵢ₂X₂ⱼ + ... + WᵢₖXₖⱼ, Tensor Cores calculate low-precision products and partial sums. CUDA Cores then combine those partial sums in FP32 and apply the activation and weight scales, producing a de-quantized value with restored magnitude and improved numerical stability. section 5.6.2 section 5.6.3
Figure 5.29 The complete Fine-Grained Quantization workflow.

Use the complete workflow diagram to trace independent activation and weight scaling through Tensor Core computation and CUDA Core de-quantization. section 5.6.2

