liveBook · Manning

Zero-point correction

As explained in Quantization and Fast Inference

A zero-point correction is the arithmetic adjustment that compensates for an activation zero-point in quantized matrix multiplication.

Zero-point correction removes the activation offset from a quantized matrix product. With affine activations and symmetric weights, the corrected accumulator for each output is Σ(qₓqᵥ) − ZₓΣqᵥ, where Zₓ is the activation zero-point. Multiply by the activation and weight scales, then add the bias. section 2.5

The correction follows by expanding Σ((qₓ − Zₓ)qᵥ). Symmetric weight quantization fixes the weight zero-point at zero, so no weight-offset term is needed. If Zₓ is zero, the correction vanishes. section 2.5

Shifted activations

For shifted inputs, Zₓ is 48. Corrected output has mean absolute error 0.0302 against FP32; dropping the correction raises it to 1.4174, or 46.9 times as much. section 2.5

Listing 2.9 Linear8bit layer with hybrid quantization
class Linear8bit(nn.Module):
    def __init__(self, in_features, out_features, bias=True):
        super().__init__()
        self.weight = nn.Parameter(torch.randn(out_features, in_features))
        self.bias = nn.Parameter(torch.randn(out_features)) if bias else None
        self.weight_quantizer = SymmetricQuantizer(bits=8)         #A
        self.input_quantizer = AffineQuantizer(bits=8)             #A
 
    def forward(self, x):
        self.input_quantizer.calibrate(x)
        x_q = self.input_quantizer.quantize(x)                     #B
 
        self.weight_quantizer.calibrate(self.weight)
        w_q = self.weight_quantizer.quantize(self.weight)          #C
 
        term1 = torch.mm(x_q.float(), w_q.float().t())             #D
        sum_w_q = torch.sum(w_q.float(), dim=1)
        term2 = self.input_quantizer.zero_point * sum_w_q          #E
 
        y_int = term1 - term2                                      #F
        scale = self.weight_quantizer.scale * self.input_quantizer.scale
        y_out = y_int * scale
 
        if self.bias is not None:
            y_out += self.bias
        return y_out

Inspect the linear layer’s matrix product and per-output weight sums. The listing casts quantized values to float for the matrix multiplication; it does not demonstrate an integer hardware kernel. section 2.5

How Quantization and Fast Inference covers it

How deep it goes
Derives the correction from affine activations and symmetric weights, then implements it in a linear layer.
Examples
Compares corrected and uncorrected outputs for shifted activations against an FP32 baseline.
What it assumes
Assumes familiarity with scale, zero-point, affine mappings, and quantization error.
What you can do after
Build it from scratch in a PyTorch linear layer and compare its output error with an uncorrected calculation.
How it connects
Connects affine quantization, symmetric quantization, matrix multiplication, and mean absolute error.

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.