liveBook · Manning

Hybrid consensus

As explained in Quantization and Fast Inference

Hybrid Consensus is a quantization strategy that applies symmetric quantization to weights and asymmetric quantization to activations.

Hybrid consensus uses symmetric quantization for weights and asymmetric quantization for activations. It keeps weight arithmetic efficient while fitting the activation grid to the data, particularly for one-sided ReLU outputs. section 2.4 section 2.6

Weights typically cluster around zero, so a centered grid fits them with little wasted range. Activations can be one-sided or shifted; an asymmetric grid uses a calculated zero-point to cover their range more effectively. Its zero-point can still be zero, as with a nonnegative ReLU range. section 2.4 section 2.5

Table 2.4 Quantization scheme comparison
Feature
Symmetric
Asymmetric
Hybrid Consensus
Zero Point (Z)
Fixed at 0
Calculated from min/max
Zw=0, Zx calculated
Grid Constraint
Centralized ([−α,α])
Flexible ([α,β]
Weights central, activations flexible
Compute Overhead
Lowest. Raw dot product.
Highest. Requires ∑qx and ∑qw.
Low. Removes ∑qx overhead.
ReLU Efficiency
Poor. Wastes ~50% of bits.
High. Uses full range.
High. Activations use full range.
Weight Fit
Excellent. Weights are bell-shaped.
Good, but marginal gain.
Excellent.
Primary Use Case
Mobile/DSP (ARM SDOT)
Legacy (early TFLite)
Standard INT8 Inference

Compare grid constraints, correction overhead, and fit across symmetric, asymmetric, and hybrid schemes. section 2.4

For weights, the demonstrated INT8 quantizer uses codes from −127 to 127, zero-point Z_w = 0, and scale S_w = max(|w|)/127. It divides by the scale, rounds, and clamps. Excluding −128 preserves a symmetric integer range. section 2.5

For activations, the demonstrated affine quantizer extends the observed minimum and maximum to include zero, then uses codes from 0 to 255. Its scale is the real range divided by 255; the zero-point is calculated from the minimum, rounded, and clamped to the integer range. Including zero supports zero-padding and preserves ReLU sparsity. section 2.5 section 2.6

For an activation vector and a weight vector, the scaled dot product is S_x S_w [∑(q_x q_w) − Z_x∑q_w]. Setting Z_w to zero removes the correction involving ∑q_x, but the activation zero-point correction remains. It must not be omitted when Z_x is nonzero. section 2.4 section 2.5

Resolution and correction

In one ReLU comparison, symmetric quantization used codes 0–127 with scale 0.139; asymmetric quantization used 0–255 with scale 0.069, a 2.01× resolution gain. In a separate shifted-activation test, Z_x was 48: mean absolute error against FP32 was 0.0302 with correction and 1.4174 without it. section 2.5

Inspect tensor distributions and output error when choosing a grid. Activation ranges can shift between batches, so raw min/max calibration may need recalibration; percentile clipping and entropy minimization are alternatives for selecting boundaries. section 2.6

How Quantization and Fast Inference covers it

How deep it goes
Derives the grid and correction tradeoffs and implements calibrated quantizers and a linear-layer inference experiment.
Examples
Compares ReLU grid resolution and measures corrected versus naive error on shifted activations.
What you can do after
Evaluate quantization choices and diagnose missing zero-point corrections by comparing quantized linear-layer outputs with FP32.
How it connects
Connects symmetric and asymmetric quantization with zero-point correction, ReLU, and calibration.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.