liveBook · Manning

Symmetric quantization

As explained in Quantization and Fast Inference

Symmetric quantization uses a zero-centered representation, typically forcing the weight zero-point to zero.

Symmetric quantization centers the integer grid on zero and fixes the zero-point at 0. It suits roughly zero-centered weight distributions and simplifies inference arithmetic. For one-sided activations, it can waste much of the grid; asymmetric quantization often fits them better. section 2.4 section 2.6

For a symmetric INT8 weight grid of [−127, 127], calibrate the scale as S = max(|w|)/127, using S = 1 for an all-zero tensor. Quantize with q = clamp(round(w/S), −127, 127), and dequantize with ŵ = S × q. The code −128 is excluded because it has no positive counterpart, preserving a perfectly symmetric integer range. section 2.5

Quantizing a weight

If the largest absolute weight is 0.5, the scale is 0.5/127 ≈ 0.003937. A weight of 0.1 maps to round(0.1/0.003937) = 25 and dequantizes to approximately 0.0984. The difference is rounding distortion. section 2.5

Listing 2.7 SymmetricQuantizer for weights
class SymmetricQuantizer:
    def __init__(self, bits=8):
        self.q_max = (1 << (bits - 1)) - 1                        #A
        self.q_min = -self.q_max                                   #A
        self.scale = 1.0
        self.zero_point = 0                                        #B
 
    def calibrate(self, tensor):
        abs_max = tensor.abs().max()
        self.scale = abs_max / self.q_max if abs_max > 0 else 1.0 #C
 
    def quantize(self, tensor):
        scaled = tensor / self.scale
        rounded = torch.round(scaled)
        return torch.clamp(rounded, self.q_min, self.q_max).to(torch.int8)
 
    def dequantize(self, q_tensor):
        return q_tensor.float() * self.scale                       #D
 

Inspect the PyTorch calibration, quantization, and dequantization operations, including the all-zero fallback. section 2.5

An affine dot product has zero-point correction terms. Setting the weight zero-point to zero removes the terms that depend on it, but an asymmetric activation zero-point can still require correction. With quantized activations qx and weights qw, the remaining integer expression is ∑(qx·qw) − Zx∑qw, before applying the scales. section 2.4 section 2.5

Correction matters for shifted activations

In one shifted-activation test, Zx = 48. Including the correction produced a mean absolute error of 0.0302 versus FP32; omitting it produced 1.4174, or 46.9 times as much. section 2.5

Use symmetric weights with asymmetric activations when their distributions justify it. In a ReLU activation comparison, the symmetric grid used codes [0, 127] at scale 0.139, while the asymmetric grid used [0, 255] at scale 0.069—about twice the resolution. Inspect tensor distributions and validate error before choosing a configuration; activation ranges can shift between batches. section 2.5 section 2.6

How Quantization and Fast Inference covers it

How deep it goes
Derives scale and zero-point behavior and implements calibration, quantization, dequantization, and corrected linear inference.
Examples
Compares symmetric and asymmetric ReLU grids and tests zero-point correction on shifted activations.
What you can do after
Builds a symmetric weight quantizer from scratch and evaluates its use in a hybrid quantized linear layer.
How it connects
Connects affine quantization, zero-point correction, ReLU, calibration, rounding error, and clipping.

Also explained in

Compare how 2 books explain symmetric quantization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.