liveBook · Manning

Quantization-aware training

Also called QAT

Quantization-aware training optimizes a model while simulating or incorporating quantization effects so it can retain performance with reduced-precision representations.

How 2 Manning books explain it

Quantization and Fast Inference

Quantization-aware training incorporates simulated quantization effects while optimizing a model so it can retain accuracy at deployment precision.

How it is used
Recovers accuracy when post-training quantization misses deployment targets, and supports training models under ternary constraints.
How deep it goes
Works through fake quantization and surrogate gradients, then specifies layer selection, fine-tuning, observer scheduling, and validation practices.
Examples
Compares INT4 strategies on FashionMNIST and BERT SST-2, and ternary training against post-hoc quantization on toy regression.
What you can do after
Build fake-quantized layers and fine-tuning loops, then evaluate selective precision choices against accuracy and runtime requirements.

Read how Quantization and Fast Inference explains quantization-aware training →

Transformers in Action

Quantization-aware training retrains a model while simulating or applying quantized weights and activations so the model adapts to quantization effects.

How it is used
Improves model retention of performance when deploying extremely low-precision weights and activations.
How deep it goes
Defines retraining with quantized weights and activations, notes computational cost, and stops short of implementation procedure.
Examples
Uses 2-bit quantization as a low-precision target and compares float32, float16, int8, and int4 formats.
What you can do after
Evaluate the training-cost and precision trade-off between quantization-aware and posttraining quantization.

Read how Transformers in Action explains quantization-aware training →

Also covered in

What to know first

Related concepts

Questions about quantization-aware training

What is quantization-aware training?

Quantization-aware training optimizes a model while simulating or incorporating quantization effects so it can retain performance with reduced-precision representations.

Which Manning books explain quantization-aware training?

Quantization and Fast Inference and Transformers in Action.

What should I know before learning quantization-aware training?

Manning books that teach quantization-aware training usually introduce bfloat16, accuracy, language model and int4 quantization first.

What is the difference between quantization-aware training and post-training quantization?

Quantization-aware training optimizes a model while simulating or incorporating quantization effects so it can retain performance with reduced-precision representations. Post-training quantization converts a trained model’s parameters or computations to lower-precision representations without further model training.

What is quantization-aware training also called?

Quantization-aware training is also called QAT.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.