Affine quantization maps floating-point tensor values to a discrete integer grid. Its scale sets the spacing between levels, and its zero point identifies the integer representing real zero. A grid tailored to each tensor can use limited integer codes more effectively than a rigid fixed-point format, but rounding and clipping can cost accuracy. section 2.0 section 2.2
For scale S, integer zero point Z, and integer range [qmin, qmax], quantize r as q = clip(round(r / S + Z), qmin, qmax). Reconstruct it as r̂ = S × (q − Z). Values between grid points are rounded; values outside the representable range are clipped. section 2.2
To calibrate from a tensor, extend its observed range [rmin, rmax] to include zero. For a nonzero range, calculate S = (rmax − rmin) / (qmax − qmin), then Z = round(qmin − rmin / S), clamped to the integer range. If the range is zero, the example implementation uses S = 1 and Z = 0. Including zero and using an integer zero point makes real zero exactly representable, preserving ReLU sparsity and zero-padding. section 2.2 section 2.5
Unsigned 8-bit ReLU activations
For [0, 0, 0.25, 1.5, 6], calibration gives S = 6 / 255 ≈ 0.023529 and Z = 0. The quantized values are [0, 0, 11, 64, 255]; reconstruction gives approximately [0, 0, 0.258824, 1.505882, 6]. The intermediate values incur rounding error, while zero remains exact. section 2.2
Zero points also affect computation. An affine dot product uses SwSx × Σ((qw − Zw)(qx − Zx)). Expanding it introduces correction terms: the weight sum can be precomputed, but an input sum depends on the current input. Setting the weight zero point to zero removes the input-sum term, motivating symmetric weights alongside affine activations. section 2.2 section 2.5
Table 2.3 Affine dot product expansion terms
|
Term
|
Formula
|
Description
|
Computational Characteristic
|
|
1. Core Product
|
∑(qwqx)
|
Standard integer dot product
|
Expensive. Main O(N) MAC operation.
|
|
2. Weight Sum
|
−Zx∑qw
|
Weights scaled by input zero-point
|
Pre-computable.
∑qw is constant for trained model.
|
|
3. Input Sum
|
−Zw∑qx
|
Inputs scaled by weight zero-point
|
Runtime
Cost. ∑qx depends on input.
|
|
4. Offset Constant
|
+N⋅Zw⋅Zx
|
Constant offset
|
Pre-computable. Can fold into bias.
|
Inspect the expanded dot product to distinguish runtime corrections from terms that can be precomputed. section 2.2
Do not omit the zero-point correction
For shifted activations with input zero point 48, the corrected output has mean absolute error 0.0302 against FP32; omitting the correction raises it to 1.4174, or 46.9 times as much in this experiment. section 2.5
