liveBook · Manning

Quantization-aware training

As explained in Quantization and Fast Inference

Quantization-aware training incorporates simulated quantization effects while optimizing a model so it can retain accuracy at deployment precision.

Quantization-aware training (QAT) adapts a model to deployment precision by simulating quantization during optimization. Try post-training quantization (PTQ) first: it avoids retraining and often suffices at INT8. Escalate when validation misses the accuracy target, especially at lower bit-widths, in small models, or under tight accuracy budgets. QAT can recover accuracy, but requires training compute, tuning, and engineering work. section 4.1 section 5.0

Fake quantization maps values to an integer grid, clips them to the supported range, then converts them back to floating point for the forward pass. Full-precision weights remain available for updates. Because rounding has zero gradient almost everywhere, a straight-through estimator (STE) substitutes a gradient of 1 through rounding. A clipped STE additionally zeros gradients for values outside the quantizable range. section 5.2 section 5.2.1

Rounding blocks the learning signal

With scale 0.5, quantize-dequantize maps inputs [0.3, 0.7, 1.2, 1.8] to [0.5, 0.5, 1.0, 2.0]. Backpropagating through ordinary rounding gives four zero gradients. An STE preserves the same simulated forward values while passing a surrogate gradient backward. section 5.2

Listing 5.4 Fake quantization with straight-through estimator
import torch
import torch.nn as nn
 
class FakeQuantizeSTE(torch.autograd.Function):
    """Fake quantization with straight-through estimator for gradients."""
    
    @staticmethod
    def forward(ctx, x, scale, zero_point, q_min, q_max):
        # Quantize: scale to integer range, round, clip
        x_int = torch.round(x / scale + zero_point)
        x_int = torch.clamp(x_int, q_min, q_max)
        
        # Dequantize: back to floating point
        x_q = (x_int - zero_point) * scale
        
        # Save for backward (needed for clipping gradient)
        ctx.save_for_backward(x, scale)
        ctx.q_min = q_min
        ctx.q_max = q_max
        ctx.zero_point = zero_point
        
        return x_q
    
    @staticmethod
    def backward(ctx, grad_output):
        x, scale = ctx.saved_tensors
        
        # STE: pass gradient through unchanged
        # But zero out gradients for values that were clipped
        x_int = x / scale + ctx.zero_point
        mask = (x_int >= ctx.q_min) & (x_int <= ctx.q_max)
        
        grad_input = grad_output * mask.float()
        
        # No gradient for scale, zero_point, q_min, q_max
        return grad_input, None, None, None, None
 
def fake_quantize(x, scale, zero_point, q_min, q_max):
    """Convenience wrapper for fake quantization."""
    return FakeQuantizeSTE.apply(x, scale, zero_point, q_min, q_max)

Inspect the fake-quantization forward pass and its STE-based backward pass. section 5.2

A practical QAT workflow:

  • Validate PTQ against an FP32 baseline and profile layer sensitivity. Diffuse degradation may warrant QAT across layers; concentrated sensitivity can favor mixed precision instead. section 4.5 section 5.6
  • Insert fake quantizers into selected convolutional or linear layers. A common configuration uses per-channel symmetric weights and per-tensor asymmetric activations; observers track ranges for quantization parameters. section 5.2.1
  • Fine-tune with a lower learning rate, typically 1e-4 to 1e-5, for 5–20 epochs, with gradient clipping. Warm up the learning rate, freeze observer statistics after scales stabilize, and consider progressively lowering precision. section 5.2.2 section 5.6
  • Monitor validation accuracy and scale changes together; stop when scales and accuracy have flattened. Verify that the deployment runtime supports the chosen precision and execution path: quantized storage alone does not guarantee faster inference. section 5.6 section 4.5

Per-channel weight quantization is especially useful when channel magnitudes vary widely: separate scales can prevent small channels from losing their gradients. Use output-channel axis 0 for convolutional and linear weights where the target supports it. The benefit is model-dependent: a small CNN with similarly sized channels showed nearly identical INT4 per-tensor and per-channel training curves. section 5.4.2 section 5.6

Selective transformer QAT

On BERT-base and SST-2, INT4 PTQ reached 50.9% accuracy. After three QAT epochs, attention-only quantization was within 0.11 percentage points of FP32, while quantizing all 72 linear layers was 1.61 points below it. Profile weight sensitivity before selecting layers; in this experiment, feed-forward layers dominated weight-quantization output error. Keep LayerNorm, softmax, and GELU in FP32. section 5.5 section 5.6

Figure 5.16 Selective QAT fine-tuning on SST-2 (INT4, 3 epochs). Left: accuracy per epoch. Right: best accuracy by strategy. QAT recovers BERT from the INT4 PTQ catastrophe (50.9%) to within 1.6% of FP32. Attention-only (48 layers at INT4) nearly matches FP32 at just −0.11%. FFN-only (24 layers) costs −0.57%. Quantizing all 72 layers incurs the largest gap (−1.61%).

Compare accuracy across selective INT4 QAT strategies and the FP32 baseline. section 5.5

QAT is commonly used for fine-tuning, but can also train a model from scratch under a low-precision constraint. In a toy regression experiment with ternary weights, a BitLinear model trained with QAT reached test MSE 0.0070, versus 0.0199 for post-hoc ternarization of the FP32 model. section 5.2.2 section 8.5.2

How Quantization and Fast Inference covers it

How deep it goes
Works through fake quantization and surrogate gradients, then specifies layer selection, fine-tuning, observer scheduling, and validation practices.
Examples
Compares INT4 strategies on FashionMNIST and BERT SST-2, and ternary training against post-hoc quantization on toy regression.
What it assumes
Assumes familiarity with trained models, quantization scales, and backpropagation while deriving the rounding-gradient obstacle and STE workaround.
What you can do after
Build fake-quantized layers and fine-tuning loops, then evaluate selective precision choices against accuracy and runtime requirements.
How it connects
Connects post-training quantization, straight-through estimators, per-channel quantization, mixed precision, and transformer sensitivity.

Also explained in

Compare how 2 books explain quantization-aware training →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.