Use quantize_static to quantize an ONNX model’s weights and activations using representative calibration data. Unlike dynamic quantization, it can cover operators such as Conv layers that weight-only approaches may leave in FP32. section 6.4
The workflow has three steps:
- Preprocess the FP32 graph with shape inference and graph optimization. section 6.4
- Calibrate with representative data to collect activation ranges. section 6.4
- Quantize the graph by inserting QuantizeLinear/DequantizeLinear (QDQ) pairs. section 6.4
ResNet-18
On ImageNette, static quantization produces an 11.28 MB model, a 3.95× reduction, with 0.10 percentage points of accuracy loss. Dynamic quantization reaches only 1.03× because it targets the single Gemm layer rather than the model’s Conv layers. section 6.4
quantize_static(
model_input=preprocessed_path,
model_output="resnet18_static_int8.onnx",
calibration_data_reader=calib_reader,
quant_format=QuantFormat.QDQ, #A
activation_type=QuantType.QUInt8, #B
weight_type=QuantType.QInt8, #C
per_channel=True, #D
)The call configures QDQ format, unsigned 8-bit activations, signed 8-bit weights, and per-channel quantization. section 6.4
