Equivalent transformation lets AWQ protect activation-salient weight channels before quantization without changing the layer’s output beforehand. Scale each weight column by s and apply the inverse scale to the matching activation channel; then quantize the scaled weights. At inference, fold the inverse scaling into the preceding operator, typically LayerNorm. section 7.4
Equation 7.3
The identity shows how weight and activation scaling cancel in the matrix product. section 7.4
Quantization error tradeoff
For OPT-6.7B layer 0 fc1 at 4-bit group size 128, AWQ reduces reconstruction error 17.1× versus round-to-nearest quantization. The inverse scale is folded into LayerNorm, avoiding runtime overhead. section 7.4
