liveBook · Manning

Number rounding

As explained in Quantization and Fast Inference

Rounding maps a numerical value to a nearby representable value according to a specified tie-breaking or directional rule.

Rounding maps a value to a nearby representable value. In quantization, it selects an integer grid point; the grid spacing determines the resulting precision. The rounding rule matters, especially at halfway values, where different framework tie-breaking rules can cause drift. section 2.6 section 2.2

For affine quantization, reconstruct a real value with r = S(q − Z), where S is the step size and Z is the zero-point. To quantize, compute q = clip(round(r/S + Z), qmin, qmax). Rounding selects the nearest integer; clipping separately enforces the integer range. section 2.2

Rounding ReLU values to uint8

For values in [0, 6], use S = 6/255 ≈ 0.023529 and Z = 0. The value 0.25 maps to round(0.25/S) = 11, which reconstructs as approximately 0.258824. The rounding error, original minus reconstructed, is approximately −0.008824; 6.0 maps exactly to 255. section 2.2

For nearest rounding on a uniform grid, an in-range value has granular error of at most S/2. That bound does not cover clipping: a value outside the grid can have arbitrarily large overload error. At fixed bit width, widening the range reduces clipping but increases grid spacing and rounding error; tightening it does the reverse. section 2.3

Figure 2.5 The granular-versus-overload error tradeoff. Widening the quantization grid (increasing the clipping range) reduces overload error—fewer values are clipped—but coarsens the step size, increasing granular error for values inside the range. Tightening the grid does the reverse. The optimal clipping boundary minimizes total distortion and depends on the shape of the data distribution: heavy-tailed distributions push the boundary outward; tightly concentrated distributions pull it inward.

Compare how grid width changes rounding and clipping error. section 2.3

Fixed-point multiplication also needs rounding. In Q3.4, each input is scaled by 16, so the wide integer product is scaled by 256. Returning to Q3.4 requires a four-bit right shift. Shifting alone drops the remainder and rounds down, introducing systematic negative error; adding half a step before shifting rounds to nearest instead. section 2.1

Check rounding and tie-breaking behavior when comparing implementations, and choose calibration boundaries with both error types in mind. Floating-point arithmetic also rounds results back into its format, alongside normalization and other hardware steps. section 2.2 section 2.6 section 1.3.1

How Quantization and Fast Inference covers it

How deep it goes
Works through grid error and shift-based rounding, while identifying framework tie-breaking differences without specifying a universal rule.
Examples
Walks through uint8 ReLU quantization and Q3.4 multiplication, including reconstruction and rounding bias.
What you can do after
Diagnose/debug rounding error and distinguish it from clipping when comparing quantized results.
How it connects
Connects rounding to affine quantization, fixed-point arithmetic, clipping, calibration, and floating-point normalization.

Also explained in

Compare how 7 books explain number rounding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.