Two-pass computation separates a reduction from the transformation that uses its result. In RMSNorm, the first pass reduces squared inputs to an inverse RMS value; the second rereads the inputs and applies that value and a learned scale. section 4.1.4
The two-pass structure trades extra input reads for simpler resource use:
- Keeping a large row live while waiting for its statistic can exceed register capacity; caching it in shared memory is fast but capacity-limited. section 4.1.4
- Rereading inputs costs bandwidth, but can preserve occupancy and keep the kernel simple for large rows. section 4.1.4
Figure 4.6 The reduce-then-map pattern in RMSNorm. Pass 1 reads the input, squares each element, and reduces to compute the inverse RMS. Pass 2 re-reads the input and applies the normalization with the learned scale γ. Two passes are necessary because the statistic must be fully computed before any output can be written
The diagram shows the reduction pass followed by the reread-and-transform pass. section 4.1.4
