A zero-point nudge adjusts an asymmetric quantizer so real zero has an exact integer representation. It preserves ReLU sparsity and correct zero-padding while letting the integer grid accommodate a shifted activation distribution. section 2.5 section 2.6
In affine quantization, r = S(q − Z), where S is the scale and Z is the integer zero-point. Calibration first extends the observed minimum and maximum to include zero. It then computes S = (rmax − rmin)/(qmax − qmin) and Z = round(qmin − rmin/S), clamping Z to the integer range. If the real range is zero, the implementation uses S = 1 and Z = 0. section 2.5 section 2.2
Rounding Z is another sense of the nudge: it shifts the represented real range slightly because an arbitrary range cannot generally align perfectly with an integer zero-point. This shift is distinct from rounding individual values. Including zero can also expand the calibrated range, trading finer resolution for exact representation of zero. section 2.2 section 2.6
ReLU calibration
For activations [0, 0, 0.25, 1.5, 6] and unsigned 8-bit codes [0, 255], calibration gives S = 6/255 ≈ 0.023529 and Z = 0. The quantized values are [0, 0, 11, 64, 255]: zeros remain exact, while 0.25 reconstructs as approximately 0.258824. section 2.2
A nonzero activation zero-point must be accounted for during inference. With symmetric weights, the dot product becomes SwSx(Σqwqx − ZxΣqw); the weight sum can be precomputed because weights are fixed. In one shifted-activation experiment, omitting this correction increased mean absolute error versus FP32 from 0.0302 to 1.4174. section 2.5 section 2.2
Listing 2.8 AffineQuantizer for activations
class AffineQuantizer:
def __init__(self, bits=8):
self.q_min = 0
self.q_max = (1 << bits) - 1 #A
self.scale = 1.0
self.zero_point = 0
def calibrate(self, tensor):
r_min = tensor.min().item()
r_max = tensor.max().item()
r_min = min(r_min, 0.0) #B
r_max = max(r_max, 0.0) #B
real_range = r_max - r_min
int_range = self.q_max - self.q_min
if real_range == 0:
self.scale, self.zero_point = 1.0, 0
else:
self.scale = real_range / int_range
initial_z = self.q_min - (r_min / self.scale)
self.zero_point = int(round(initial_z)) #C
self.zero_point = max(self.q_min, min(self.q_max, self.zero_point))
def quantize(self, tensor):
scaled = (tensor / self.scale) + self.zero_point
rounded = torch.round(scaled)
return torch.clamp(rounded, self.q_min, self.q_max).to(torch.uint8)
def dequantize(self, q_tensor):
return self.scale * (q_tensor.float() - self.zero_point) #D
Inspect the unsigned activation quantizer’s zero-inclusion and zero-point calibration. section 2.5
