Zero-point correction removes the activation offset from a quantized matrix product. With affine activations and symmetric weights, the corrected accumulator for each output is Σ(qₓqᵥ) − ZₓΣqᵥ, where Zₓ is the activation zero-point. Multiply by the activation and weight scales, then add the bias. section 2.5
The correction follows by expanding Σ((qₓ − Zₓ)qᵥ). Symmetric weight quantization fixes the weight zero-point at zero, so no weight-offset term is needed. If Zₓ is zero, the correction vanishes. section 2.5
Shifted activations
For shifted inputs, Zₓ is 48. Corrected output has mean absolute error 0.0302 against FP32; dropping the correction raises it to 1.4174, or 46.9 times as much. section 2.5
Listing 2.9 Linear8bit layer with hybrid quantization
class Linear8bit(nn.Module):
def __init__(self, in_features, out_features, bias=True):
super().__init__()
self.weight = nn.Parameter(torch.randn(out_features, in_features))
self.bias = nn.Parameter(torch.randn(out_features)) if bias else None
self.weight_quantizer = SymmetricQuantizer(bits=8) #A
self.input_quantizer = AffineQuantizer(bits=8) #A
def forward(self, x):
self.input_quantizer.calibrate(x)
x_q = self.input_quantizer.quantize(x) #B
self.weight_quantizer.calibrate(self.weight)
w_q = self.weight_quantizer.quantize(self.weight) #C
term1 = torch.mm(x_q.float(), w_q.float().t()) #D
sum_w_q = torch.sum(w_q.float(), dim=1)
term2 = self.input_quantizer.zero_point * sum_w_q #E
y_int = term1 - term2 #F
scale = self.weight_quantizer.scale * self.input_quantizer.scale
y_out = y_int * scale
if self.bias is not None:
y_out += self.bias
return y_outInspect the linear layer’s matrix product and per-output weight sums. The listing casts quantized values to float for the matrix multiplication; it does not demonstrate an integer hardware kernel. section 2.5
