liveBook · Manning

Progressive quantization

As explained in Quantization and Fast Inference

Progressive quantization introduces successively lower numerical precision over stages rather than applying the final precision immediately.

Progressive quantization reduces the training shock of quantization-aware training (QAT): instead of moving directly from FP32 to INT4, the model first trains with a finer grid, then steps down to the target precision. Weights can adapt to each grid before it becomes coarser. section 5.3.1

A typical schedule moves from INT8 to INT6 to INT4. Coordinate each bit-width transition with the learning-rate schedule and observer state; loss may spike when precision drops, then recover as training continues. Fold batch normalization before QAT so quantization parameters match the weights used at deployment. section 5.3.2 section 5.3.1

Twenty-epoch schedule

On SmallCNN, a schedule stepped from INT8 to INT6 at epoch 6 and INT4 at epoch 11. Loss rose from 0.0498 to 0.0844 at the INT4 transition, then recovered; validation accuracy reached 90.30% against a 90.00% FP32 baseline. section 5.3.2

Figure 5.10 The complete QAT schedule coordinates four components. Bit-width decreases progressively (INT8→INT6→INT4). The learning rate warms up during INT8 calibration, then cosine-decays through the remaining phases. Observers freeze at epoch 8, after they've seen the INT6 distribution. Batch normalization is folded before training begins.

Use the schedule diagram to see how bit-width, learning rate, observer freezing, and batch-normalization folding fit together. section 5.3.2

How Quantization and Fast Inference covers it

How deep it goes
Implements configurable bit-width phases and integrates them with observer freezing, learning-rate scheduling, and batch-normalization folding.
Examples
Uses an INT8→INT6→INT4, 20-epoch schedule and reports SmallCNN accuracy through the INT4 transition.
What you can do after
Configure QAT to lower bit-width in stages while coordinating training schedules and observer state.
How it connects
Connects quantization-aware training with observer freezing, learning-rate scheduling, and batch-normalization folding.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.