How deeply does Transformers in Action cover quantization-aware training?

Defines retraining with quantized weights and activations, notes computational cost, and stops short of implementation procedure.

Quantization-aware training

As explained in Transformers in Action

Quantization-aware training retrains a model while simulating or applying quantized weights and activations so the model adapts to quantization effects.

Quantization-aware training retrains a model with quantized weights and activations, allowing it to adapt to quantization-induced changes and retain performance at very low precision. section 8.3.3

Use quantization-aware training when:

Targeting extremely low precision, including 2-bit quantization, while preserving model performance. section 8.3.3
Accepting additional training cost in exchange for adaptation to quantization effects. section 8.3.3
Choosing it over posttraining quantization, which requires no further training but typically remains limited to 8-bit or 6-bit precision. section 8.3.3

Table 8.1 Comparison of data types

Data type	Bit width	Range	Memory reduction from FP32
float32	32 bits	≈ ± 3.4 × 10^{38}	–
float16	16 bits	≈ ± 6.5 × 10^{4}	50%
int8	8 bits	-128 to 127	75%
int4	4 bits	-8 to 7	87.5%

Compare bit widths, numeric ranges, and memory reductions for common lower-precision data types. section 8.3.3

How Transformers in Action covers it

liveBook manning.com

Read Transformers in Action on liveBook Get the book at manning.com

Related concepts

quantizationkind of · Quantization maps numerical values to a finite set of discrete representations, often using reduced precision to lower model storage or computation requirements.
post-training quantizationalternative to · Post-training quantization is the conversion of a trained model’s parameters or computations to lower-precision representations without additional training.
trainingkind of · Training is a process of developing or improving a person’s or model’s capabilities through instruction, experience, or adjustment using data.
gradient clippinguses · Gradient clipping is an optimization technique that limits gradient values or their norm before parameter updates to reduce excessively large or unstable updates.

This explainer was generated by Manning's knowledge graph from the text of Transformers in Action. The section links point to the parts of the book it draws on.