QOperator is an ONNX quantization format that replaces targeted operators with explicit integer operators; for example, MatMul becomes QLinearMatMul. Its graph declaration affects which kernels an execution provider can dispatch. section 9.2
Choose the format with the execution provider in mind:
- On CPU EP, the BERT benchmark was slower with QOperator than QDQ: 292.7 ms versus 223.4 ms. ORT flags S8S8-QOperator as potentially slow on x86-64. section 9.2
- On CUDA EP, the same benchmark measured 13.8 ms with QOperator and 14.4 ms with QDQ. Limited INT8 kernel coverage and reference-path fallbacks can make an INT8 artifact a poor choice for CUDA deployment. section 9.2
