ONNX is an open format for portable machine-learning models. It represents each model as a directed acyclic graph of named tensor inputs and outputs, with operators defined by standardized opsets and graph data serialized using Protocol Buffers. ONNX is the format; ONNX Runtime and other engines execute it. section 6.4
Use ONNX to carry a model from its training framework to a different deployment runtime, avoiding a rewrite of the model code. The same graph format can feed ONNX Runtime, TensorRT, OpenVINO, or CoreML converters. section 6.4
ResNet-18 quantization
ONNX Runtime dynamic quantization covers MatMul only and reaches 1.03× compression. Static quantization with calibration covers all operators, reaches 3.95× compression, and loses 0.10 percentage points of accuracy. section 6.4
Listing 6.6 Export ResNet-18 to ONNX
torch.onnx.export(
model,
dummy_input, #A
"resnet18_fp32.onnx",
opset_version=17, #B
input_names=["input"],
output_names=["output"],
dynamic_axes={"input": {0: "batch"},
"output": {0: "batch"}}, #C
dynamo=False, #D
)Inspect a PyTorch ResNet-18 export to an ONNX file using opset 17 and a dynamic batch axis. section 6.4
quantize_static(
model_input=preprocessed_path,
model_output="resnet18_static_int8.onnx",
calibration_data_reader=calib_reader,
quant_format=QuantFormat.QDQ, #A
activation_type=QuantType.QUInt8, #B
weight_type=QuantType.QInt8, #C
per_channel=True, #D
)Inspect the ONNX Runtime static quantization configuration, including calibration, QDQ format, and per-channel weights. section 6.4








