For layer quantization, the Hessian measures how reconstruction loss changes with the weights. Its diagonal indicates which input dimensions are most sensitive to rounding; GPTQ uses the inverse Hessian to distribute rounding error across weights. section 7.3
Equation 7.2
For calibration activations X, the layer’s Hessian is computed as H = XXᵀ. section 7.3
Sensitivity in OPT-6.7B
In one layer, the top 10 of 4,096 input dimensions have Hessian diagonal values 100–1,000 times the baseline. GPTQ reduces that layer’s reconstruction error 112× at 4-bit and 216× at 3-bit versus round-to-nearest quantization. section 7.3


