ONNX is a portable model format; ONNX Runtime (ORT) is an inference engine that executes ONNX files. This separation lets a model trained in one framework run in a different deployment environment. section 6.4
ORT also quantizes ONNX models. Dynamic quantization covers MatMul in the ResNet-18 comparison, while calibrated static quantization reaches convolution layers too: it achieves 3.95× compression with 0.10 percentage-point accuracy loss, versus 1.03× for dynamic quantization. section 6.4
Listing 6.6 Export ResNet-18 to ONNX
torch.onnx.export(
model,
dummy_input, #A
"resnet18_fp32.onnx",
opset_version=17, #B
input_names=["input"],
output_names=["output"],
dynamic_axes={"input": {0: "batch"},
"output": {0: "batch"}}, #C
dynamo=False, #D
)Exporting a PyTorch ResNet-18 model with opset 17, named inputs and outputs, and a dynamic batch axis creates an ONNX artifact for runtime use. section 6.4




