liveBook · Manning

Per-channel quantization

As explained in Quantization and Fast Inference

Per-channel quantization assigns separate quantization parameters to each tensor channel.

Per-channel quantization assigns separate quantization parameters to each tensor channel, preventing a large value in one channel from setting a coarse quantization grid for all others. For weights, channels usually correspond to outputs; affine quantization can use a separate scale and zero point for each channel. section 3.2 section 2.2

For symmetric INT8 weights W shaped [out_features, in_features], calculate S_c = max(|W_c|)/127 for each output channel c. Quantize with q_c = clamp(round(W_c/S_c), -127, 127) and reconstruct with Ŵ_c = q_c × S_c. Broadcast the scale vector across the remaining dimensions; use a nonzero fallback scale for an all-zero channel. section 3.2 section 5.4.1

Resolution and reconstruction error

For ResNet-18’s first convolution, a single tensor-wide scale leaves channels using 39.89% of the integer range on average. Per-channel INT8 reduces weight reconstruction MSE from 0.00000466 to 0.00000129, a 3.61× improvement. This is a weight-error comparison, not a guarantee of the same improvement in model accuracy. section 3.2

The extra scales cost memory and must be supported efficiently by the target runtime. With FP32 scales and INT8 weights, scale storage is 2.72% of weight storage for ResNet-18 conv1 but 0.10% for a Llama-7B query projection. Check kernel support and measure latency rather than assuming improved accuracy implies faster inference. section 3.2 section 2.2

In quantization-aware training, fake quantization applies each channel’s grid in the forward pass and uses a straight-through estimator for gradients. Recompute scales as weights evolve, but detach scale computation from gradients. In one INT4 ResNet-18 conv1 experiment, the estimator masked 67.4% of gradients per channel on average with a tensor-wide scale, versus 31.5% with per-channel scales; the mask also excludes quantized boundary values. section 5.4.1

Listing 5.11 Per-channel fake quantization for QAT
class FakeQuantizePerChannel(torch.autograd.Function):
    """Per-channel fake quantization with STE."""
    @staticmethod
    def forward(ctx, weight, scales, q_min, q_max, axis):
        shape = [1] * weight.dim()
        shape[axis] = -1                                    #A
        scales_bc = scales.view(shape)
 
        w_q = torch.clamp(torch.round(weight / scales_bc),
                          q_min, q_max)
        w_dq = w_q * scales_bc
        mask = (w_q > q_min) & (w_q < q_max)               #B
        ctx.save_for_backward(mask)
        return w_dq
 
    @staticmethod
    def backward(ctx, grad_output):
        mask, = ctx.saved_tensors
        return grad_output * mask.float(), None, None, None, None  #C

Inspect scale broadcasting and the element-wise straight-through gradient mask in a per-channel fake quantizer. section 5.4.1

Choose granularity against the deployment target:

  • Use output-channel axis 0 for convolution and linear weights; test accuracy and runtime compatibility on the intended model and hardware. section 5.4.2 section 3.7
  • Consider group-wise quantization for INT4 weights when per-channel error remains too high; smaller groups add scale metadata and impose layout and kernel constraints. section 3.7 section 2.2
  • Treat activation and KV-cache granularity separately: activation statistics vary with input, while cache keys and values can favor different granularities. section 3.0 section 3.7

How Quantization and Fast Inference covers it

How deep it goes
Derives symmetric scales and broadcasting, implements fake quantization, and measures reconstruction, gradient masking, metadata, and training effects.
Examples
Compares ResNet-18 convolution weights and INT4 gradient survival, and measures scale overhead for transformer projections.
What you can do after
Build a symmetric per-channel quantizer and fake quantizer, and evaluate their error and deployment tradeoffs.
How it connects
Connects affine quantization, group-wise quantization, fake quantization, straight-through estimators, and quantization-aware training.

Also explained in

Compare how 2 books explain per-channel quantization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.