liveBook · Manning

Min-max calibration

Min-max calibration determines quantization parameters from the minimum and maximum values observed in representative data.

How 2 Manning books explain it

Quantization and Fast Inference

Min-max calibration uses the minimum and maximum observed values to determine a quantization range and its parameters.

How it is used
Selects a histogram-free calibration method for static INT8 quantization when transformer exports make histogram calibration fail.
How deep it goes
Defines the range through observed extrema and explains a histogram-related failure mode, without detailing the range calculation.
Examples
Uses a BERT ONNX export with attention-mask constants near 1e4 to illustrate histogram binning failure.
What you can do after
Configure static INT8 calibration with CalibrationMethod.MinMax to avoid the observed histogram-bin failure on BERT exports.

Read how Quantization and Fast Inference explains min-max calibration →

CUDA for LLMs

Min-max calibration sets the quantization range using the observed minimum and maximum values.

How it is used
Sets quantization scales from observed activation and weight ranges in representative calibration data.
How deep it goes
Defines full-range scaling and works a UINT8 example, but gives no broader procedure for choosing calibration methods.
Examples
Uses values [0.1, 0.5, 0.8, 100.0] to show how one outlier compresses ordinary values into nearly identical UINT8 integers.
What you can do after
Apply it by measuring representative data ranges and mapping their endpoints to quantization limits.

Read how CUDA for LLMs explains min-max calibration →

Related concepts

Questions about min-max calibration

What is min-max calibration?

Min-max calibration determines quantization parameters from the minimum and maximum values observed in representative data.

Which Manning books explain min-max calibration?

Quantization and Fast Inference and CUDA for LLMs.

What is the difference between min-max calibration and percentile-based calibration?

Min-max calibration determines quantization parameters from the minimum and maximum values observed in representative data. Percentile-based calibration sets quantization ranges from selected percentiles of observed values, potentially clipping values outside those ranges.

What is min-max calibration a kind of?

Min-max calibration is a kind of quantization calibration. Quantization calibration uses representative data to determine ranges and parameters for mapping model values to a lower-precision representation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.