liveBook · Manning

Min-max calibration

As explained in Quantization and Fast Inference

Min-max calibration uses the minimum and maximum observed values to determine a quantization range and its parameters.

For static INT8 quantization, min-max calibration is a histogram-free option. In transformer exports, it can avoid a build failure caused by histogram binning across extreme attention-mask values. section 9.2

The method uses observed minimum and maximum values to set the quantization range. In a BERT ONNX export, attention-mask constants near 1e4 can make histogram bin edges NaN; ONNX Runtime then fails its nonnegative-scale assertion. MinMax avoids that histogram path, allowing the build to finish. section 9.2

How Quantization and Fast Inference covers it

How deep it goes
Defines the range through observed extrema and explains a histogram-related failure mode, without detailing the range calculation.
Examples
Uses a BERT ONNX export with attention-mask constants near 1e4 to illustrate histogram binning failure.
What you can do after
Configure static INT8 calibration with CalibrationMethod.MinMax to avoid the observed histogram-bin failure on BERT exports.
How it connects
Connects static INT8 quantization with ONNX Runtime and Optimum quantizers.

Also explained in

Compare how 2 books explain min-max calibration →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.