In GPTQ, error compensation reduces reconstruction error during column-wise weight quantization. After rounding a column, it redistributes the rounding error across columns still to be processed, weighted by their inverse-Hessian relationships. section 7.3
This preserves information that independent rounding discards: strongly related dimensions receive larger corrections, while weakly related dimensions receive smaller ones. Calibration activations provide the Hessian used to guide the corrections. section 7.3
Budget-rounding analogy
If rounding one department’s budget down creates a shortfall, add a weighted correction to later budgets before rounding them. The analogy illustrates how GPTQ spreads error instead of accepting each rounding loss independently. section 7.3
Figure 7.7 GPTQ algorithm flow. Calibration activations X produce the Hessian H = XXT, which is Cholesky-factored for numerical stability. The weight matrix W is processed in blocks of B=128 columns. Within each block, each column is quantized to INT4, and the rounding error is distributed across remaining columns using the inverse Hessian. The output is the quantized weight matrix Q plus per-group scales.

Shows the GPTQ workflow from calibration activations and Hessian construction through blocked INT4 quantization and inverse-Hessian error redistribution. section 7.3
