liveBook · Manning

Quantization scale

As explained in Quantization and Fast Inference

A scale factor is the multiplicative parameter that maps between quantized or fixed-point integers and their represented numerical values.

Quantization scale sets the spacing between representable values. A smaller scale gives finer resolution but, for a fixed integer range, covers a narrower real range. Choosing it is an accuracy decision: a grid that is too wide wastes precision, while one that is too narrow clips values. section 2.6 section 2.1

In fixed-point representation, a stored integer q represents q × S. For Q3.4, S = 2⁻⁴ = 1/16: 1.5 is stored as 24, and the signed 8-bit pattern 1111 1111 represents −1 × 1/16 = −0.0625. The scale is the multiplier used to decode the integer; its reciprocal, 16 here, is used to encode it. section 2.1

Affine quantization allows the scale to adapt to a tensor and adds a zero-point Z: reconstructed value r = S(q − Z). To quantize, compute q = clip(round(r/S + Z), qmin, qmax). Given calibrated endpoints, S = (rmax − rmin)/(qmax − qmin), and Z is obtained by rounding qmin − rmin/S. Include zero in the calibrated real range so an integer code can represent zero exactly. section 2.2 section 2.6

Calibrating unsigned 8-bit activations

For ReLU values from 0 to 6, the integer range is 0 to 255, so S = 6/255 ≈ 0.023529 and Z = 0. The value 0.25 rounds to integer 11 and reconstructs as approximately 0.258824; 6 maps to 255 and reconstructs as 6. section 2.2

For values inside the grid, nearest-level rounding error is bounded by S/2; clipping outside the grid can produce unbounded distortion. Activation ranges can change with inputs, so raw min/max calibration may need robust boundaries or recalibration when distributions drift. section 2.6

Choose scale granularity alongside the quantization scheme. A single scale per tensor is simple but crude; per-channel scales can improve accuracy, while group-wise schemes split the difference. Symmetric weights avoid a runtime input-sum correction caused by a nonzero weight zero-point; asymmetric activations can use the grid more effectively for one-sided distributions. Check these choices against tensor statistics, accuracy requirements, and target hardware. section 3.0 section 2.2 section 2.6

Figure 2.3 The range-versus-precision tradeoff in Q-format. For a fixed 8-bit container, every bit allocated to the integer part doubles the representable range but halves the precision (step size). Moving the binary point left (more fractional bits) gives finer resolution at the cost of a narrower range; moving it right does the reverse. There is no free lunch: the total information capacity is fixed by the bit-width.

Visualize how moving the fixed-point binary point trades representable range for step size at a fixed bit width. section 2.1

How Quantization and Fast Inference covers it

How deep it goes
Derives fixed-point and affine mappings, calculates scale and zero-point, and analyzes rounding, clipping, and dot-product correction costs.
Examples
Works through Q3.4 encoding of 1.5 and unsigned 8-bit calibration of ReLU values from 0 to 6.
What you can do after
Evaluate a quantization configuration for likely accuracy loss and calculate scale and zero-point from calibrated tensor bounds.
How it connects
Links scale to zero-point, calibration, quantization error, symmetric and asymmetric quantization, and quantization granularity.

Also explained in

Compare how 2 books explain quantization scale →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.