liveBook · Manning

Cross-runtime calibration drift

As explained in Quantization and Fast Inference

Cross-runtime calibration drift is a difference in quantization calibration results produced by distinct runtimes or compilers for the same model and input data.

Cross-runtime calibration drift occurs when different runtimes or compilers produce different quantization calibration results for the same model and input data. Separate INT8 calibration passes can assign different per-operation activation-range brackets even when they use the same representative dataset. section 11.4

The differences can shift accuracy by 1–3 percentage points across vendors. Calibration discipline is required for each vendor compilation, so build and validate a conversion pipeline for every target. section 11.4

One model, multiple edge runtimes

A team converts EfficientNet for Jetson, OpenVINO, Coral, and Hailo; Edge TPU and Hailo calibration passes can yield different activation ranges on the same data. section 11.4

How Quantization and Fast Inference covers it

How deep it goes
Traces differing INT8 calibration passes to per-operation activation ranges and reports accuracy shifts of 1–3 percentage points.
Examples
Compares EfficientNet conversion across Jetson, OpenVINO, Coral, and Hailo, including Edge TPU and Hailo calibration.
What it assumes
Assumes a source model and developer-supplied representative dataset for vendor calibration.
What you can do after
Evaluate deployment accuracy and performance by building and validating a separate conversion pipeline for each vendor.
How it connects
Links INT8 quantization and calibration to representative datasets, model conversion, and inference runtimes.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.