liveBook · Manning

Hessian matrix

As explained in Quantization and Fast Inference

A Hessian is a matrix of second-order partial derivatives describing the local curvature of a scalar-valued function.

For layer quantization, the Hessian measures how reconstruction loss changes with the weights. Its diagonal indicates which input dimensions are most sensitive to rounding; GPTQ uses the inverse Hessian to distribute rounding error across weights. section 7.3

Equation 7.2
H=XXT

For calibration activations X, the layer’s Hessian is computed as H = XXᵀ. section 7.3

Sensitivity in OPT-6.7B

In one layer, the top 10 of 4,096 input dimensions have Hessian diagonal values 100–1,000 times the baseline. GPTQ reduces that layer’s reconstruction error 112× at 4-bit and 216× at 3-bit versus round-to-nearest quantization. section 7.3

How Quantization and Fast Inference covers it

How deep it goes
Defines the Hessian through layer reconstruction loss, gives H = XXᵀ, and connects its inverse to error compensation.
Examples
Uses OPT-6.7B sensitivity outliers and compares GPTQ with round-to-nearest quantization at 3-bit and 4-bit.
What it assumes
Uses calibration activations from a layer and derives their role in the Hessian from the reconstruction objective.
What you can do after
Apply it via GPTQ to guide weight quantization using calibration activations and compensate rounding errors.
How it connects
Connects second derivatives to GPTQ, calibration activations, and layer reconstruction error.

Also explained in

Compare how 3 books explain Hessian matrix →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.