Post-training quantization (PTQ) reduces a trained language model’s numeric precision to shrink memory use and potentially accelerate inference. It trades precision for deployment efficiency, unlike quantization-aware training, which incorporates quantization during training. section 6.2
PTQ involves practical tradeoffs and several implementation choices:
- Converts floating-point weights, commonly FP32, into lower-bit integers such as INT8; lower precision reduces memory but introduces quantization error. section 6.2
- Uses zero-point quantization for asymmetric distributions or absmax quantization, which scales values by a tensor’s absolute maximum. section 6.2
- Evaluates whether the compressed model preserves useful behavior through weight comparisons, generated text, and metrics such as perplexity. section 6.2.1
- Extends to mixed-precision, vector-wise, dynamic, and 4-bit approaches when INT8 quantization does not meet quality, speed, or hardware constraints. section 6.5 section 9.0
8-bit GPT-2 experiment
Load GPT-2 small in FP32 with PyTorch and Transformers, copy its weights, apply absmax quantization, and dequantize them for inference. Compare weight histograms, generated continuations, and perplexity to assess the resulting error and behavior. section 6.2.1
import torch
def absmax_quantize(X):
scale = 127 / torch.max(torch.abs(X))
X_quant = (scale * X).round()
X_dequant = X_quant / scale
return X_quant.to(torch.int8), X_dequant
The absmax_quantize listing shows the operational sequence: calculate a scale, round values into INT8, and dequantize them. section 6.2.1
