In RMSNorm, the learned scale vector γ supplies one trainable multiplicative factor per input element. After scaling the input by its shared inverse RMS, the kernel multiplies each normalized element by its corresponding γᵢ. RMSNorm does not subtract the mean. section 4.1.4
\[y_i = \frac{x_i}{\sqrt{\frac{1}{n} \sum_{j=1}^{n} x_j^2 + \epsilon}} \cdot \gamma_i\]
The equation shows γᵢ applied elementwise after the shared RMS normalization. section 4.1.4
