ONNX Runtime’s `quantize_dynamic` applies weight-only quantization to an ONNX graph, scaling activations on the fly. It works directly on valid FP32 ONNX files, without PyTorch, TensorFlow, or a GPU. section 6.4
By default, it targets MatMul and Gemm operators, so its impact depends on the model’s operator mix. On ResNet-18, that means quantizing only the classifier Gemm: the model shrinks from 43.12 MB by just 1.03×. For this convolution-heavy model, static quantization is the more effective path. section 6.4
