liveBook · Manning

Fake quantization

As explained in Quantization and Fast Inference

FakeQuantize is a module that applies simulated quantization and dequantization using configured quantization parameters during computation.

Fake quantization simulates low-precision deployment during training: the forward pass quantizes and dequantizes values, while full-precision weights remain available for gradient updates. This lets a model adapt to quantization noise instead of merely tolerating it. section 5.0 section 5.2.1

q = round(xS) . S

The scale maps values to a quantization grid and back; rounding creates the gradient problem addressed by the straight-through estimator. section 5.2

In a QAT forward pass:

  • Round and clip values to the configured integer range, then dequantize them for computation. section 5.2
  • Use a straight-through estimator (STE) to pass gradients through rounding; the clipped variant zeros gradients for saturated values. section 5.2
  • Per-channel scales can preserve gradient flow across channels; per-tensor scales may leave small-magnitude channels clipped. section 5.4

Quantizing with a scale of 0.5

The values [0.3, 0.7, 1.2, 1.8] become [0.5, 0.5, 1.0, 2.0]. Ordinary rounding produces zero gradients; an STE supplies a learning signal during backpropagation. section 5.2

For linear and convolution layers, fake-quantize weights and activations in the forward pass while retaining full-precision weights for training. Observers can track value ranges separately from the quantization operation. section 5.2.1 section 5.2.1

How Quantization and Fast Inference covers it

How deep it goes
Implements quantize-dequantize forward computation and STE backpropagation, including clipping and layer-level application.
Examples
Uses a scale-0.5 tensor example and fake-quantized linear and convolution layers.
What it assumes
Assumes a trainable neural network and quantization parameters; derives the rounding-gradient problem and STE workaround.
What you can do after
Apply fake quantization in QAT fine-tuning with an STE and clipped gradients.
How it connects
Connects quantization-aware training with straight-through estimators, observers, and per-channel quantization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.