Symmetric quantization centers the integer grid on zero and fixes the zero-point at 0. It suits roughly zero-centered weight distributions and simplifies inference arithmetic. For one-sided activations, it can waste much of the grid; asymmetric quantization often fits them better. section 2.4 section 2.6
For a symmetric INT8 weight grid of [−127, 127], calibrate the scale as S = max(|w|)/127, using S = 1 for an all-zero tensor. Quantize with q = clamp(round(w/S), −127, 127), and dequantize with ŵ = S × q. The code −128 is excluded because it has no positive counterpart, preserving a perfectly symmetric integer range. section 2.5
Quantizing a weight
If the largest absolute weight is 0.5, the scale is 0.5/127 ≈ 0.003937. A weight of 0.1 maps to round(0.1/0.003937) = 25 and dequantizes to approximately 0.0984. The difference is rounding distortion. section 2.5
Listing 2.7 SymmetricQuantizer for weights
class SymmetricQuantizer:
def __init__(self, bits=8):
self.q_max = (1 << (bits - 1)) - 1 #A
self.q_min = -self.q_max #A
self.scale = 1.0
self.zero_point = 0 #B
def calibrate(self, tensor):
abs_max = tensor.abs().max()
self.scale = abs_max / self.q_max if abs_max > 0 else 1.0 #C
def quantize(self, tensor):
scaled = tensor / self.scale
rounded = torch.round(scaled)
return torch.clamp(rounded, self.q_min, self.q_max).to(torch.int8)
def dequantize(self, q_tensor):
return q_tensor.float() * self.scale #D
Inspect the PyTorch calibration, quantization, and dequantization operations, including the all-zero fallback. section 2.5
An affine dot product has zero-point correction terms. Setting the weight zero-point to zero removes the terms that depend on it, but an asymmetric activation zero-point can still require correction. With quantized activations qx and weights qw, the remaining integer expression is ∑(qx·qw) − Zx∑qw, before applying the scales. section 2.4 section 2.5
Correction matters for shifted activations
In one shifted-activation test, Zx = 48. Including the correction produced a mean absolute error of 0.0302 versus FP32; omitting it produced 1.4174, or 46.9 times as much. section 2.5
Use symmetric weights with asymmetric activations when their distributions justify it. In a ReLU activation comparison, the symmetric grid used codes [0, 127] at scale 0.139, while the asymmetric grid used [0, 255] at scale 0.069—about twice the resolution. Inspect tensor distributions and validate error before choosing a configuration; activation ranges can shift between batches. section 2.5 section 2.6

