liveBook · Manning

Quantization calibration

Quantization calibration uses representative data to determine ranges and parameters for mapping model values to a lower-precision representation.

How 2 Manning books explain it

Quantization and Fast Inference

Calibration is the process of measuring representative model values to determine their actual numerical range for configuring quantization.

How it is used
Uses representative activation data to configure post-training quantization for deployment without retraining.
How deep it goes
Derives symmetric and affine parameters, implements a running min-max observer, and contrasts clipping with rounding error.
Examples
Uses an activation observer, an outlier-driven scale comparison, and a quantized linear layer compared with FP32.
What you can do after
Build it from scratch by collecting activation ranges and deriving symmetric or affine quantization parameters.

Read how Quantization and Fast Inference explains quantization calibration →

CUDA for LLMs

Quantization calibration determines quantization parameters from representative data to map values into a lower-precision representation.

How it is used
Determines reusable quantization parameters from representative data for static inference.
How deep it goes
Explains range collection, min-max and percentile selection, granularity, validation, and runtime tradeoffs.
Examples
Uses an activation outlier, transformer attention heads, and 4096 × 4096 feed-forward weights to illustrate range and granularity choices.
What you can do after
Evaluate calibration choices using separate validation data, downstream task metrics, and performance profiling.

Read how CUDA for LLMs explains quantization calibration →

Also covered in

What to know first

Related concepts

Questions about quantization calibration

What is quantization calibration?

Quantization calibration uses representative data to determine ranges and parameters for mapping model values to a lower-precision representation.

Which Manning books explain quantization calibration?

Quantization and Fast Inference and CUDA for LLMs.

What should I know before learning quantization calibration?

Manning books that teach quantization calibration usually introduce activation, attention head, accuracy and dequantization first.

What kinds of quantization calibration are there?

Kinds of quantization calibration include min-max calibration and percentile-based calibration.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.