liveBook · Manning

Zero-point nudge

As explained in Quantization and Fast Inference

A zero-point nudge is the book's term for the adjustment of an asymmetric quantizer's zero-point to fit a shifted value distribution.

A zero-point nudge adjusts an asymmetric quantizer so real zero has an exact integer representation. It preserves ReLU sparsity and correct zero-padding while letting the integer grid accommodate a shifted activation distribution. section 2.5 section 2.6

In affine quantization, r = S(q − Z), where S is the scale and Z is the integer zero-point. Calibration first extends the observed minimum and maximum to include zero. It then computes S = (rmax − rmin)/(qmax − qmin) and Z = round(qmin − rmin/S), clamping Z to the integer range. If the real range is zero, the implementation uses S = 1 and Z = 0. section 2.5 section 2.2

Rounding Z is another sense of the nudge: it shifts the represented real range slightly because an arbitrary range cannot generally align perfectly with an integer zero-point. This shift is distinct from rounding individual values. Including zero can also expand the calibrated range, trading finer resolution for exact representation of zero. section 2.2 section 2.6

ReLU calibration

For activations [0, 0, 0.25, 1.5, 6] and unsigned 8-bit codes [0, 255], calibration gives S = 6/255 ≈ 0.023529 and Z = 0. The quantized values are [0, 0, 11, 64, 255]: zeros remain exact, while 0.25 reconstructs as approximately 0.258824. section 2.2

A nonzero activation zero-point must be accounted for during inference. With symmetric weights, the dot product becomes SwSx(Σqwqx − ZxΣqw); the weight sum can be precomputed because weights are fixed. In one shifted-activation experiment, omitting this correction increased mean absolute error versus FP32 from 0.0302 to 1.4174. section 2.5 section 2.2

Listing 2.8 AffineQuantizer for activations
class AffineQuantizer:
    def __init__(self, bits=8):
        self.q_min = 0
        self.q_max = (1 << bits) - 1                              #A
        self.scale = 1.0
        self.zero_point = 0
 
    def calibrate(self, tensor):
        r_min = tensor.min().item()
        r_max = tensor.max().item()
        r_min = min(r_min, 0.0)                                    #B
        r_max = max(r_max, 0.0)                                    #B
 
        real_range = r_max - r_min
        int_range = self.q_max - self.q_min
 
        if real_range == 0:
            self.scale, self.zero_point = 1.0, 0
        else:
            self.scale = real_range / int_range
            initial_z = self.q_min - (r_min / self.scale)
            self.zero_point = int(round(initial_z))                #C
            self.zero_point = max(self.q_min, min(self.q_max, self.zero_point))
 
    def quantize(self, tensor):
        scaled = (tensor / self.scale) + self.zero_point
        rounded = torch.round(scaled)
        return torch.clamp(rounded, self.q_min, self.q_max).to(torch.uint8)
 
    def dequantize(self, q_tensor):
        return self.scale * (q_tensor.float() - self.zero_point)  #D
 

Inspect the unsigned activation quantizer’s zero-inclusion and zero-point calibration. section 2.5

How Quantization and Fast Inference covers it

How deep it goes
Derives the affine parameters and integer rounding, then implements calibration and zero-point correction in a linear layer.
Examples
Uses a ReLU activation array and a shifted-activation experiment comparing corrected and uncorrected inference errors.
What you can do after
Build it from scratch by calibrating an affine quantizer and applying zero-point correction in a linear layer.
How it connects
Connects affine quantization to ReLU sparsity, zero-padding, symmetric weight quantization, and dot-product correction.

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.