OpenVINOExecutionProvider lets ONNX Runtime load FP32 or QDQ ONNX models while delegating execution to the OpenVINO CPU plugin. This route preserves ONNX Runtime serving code, which can simplify maintenance across mixed CPU and GPU fleets. The execution artifact matters: the ORT route and standalone OpenVINO route can perform differently, so benchmark both for the model and deployment. section 9.4
Model-dependent performance
On Sapphire Rapids, via-ORT INT8 reached 96.28 ms for BERT-base, versus 110.17 ms for standalone OpenVINO INT8. For ResNet-18, standalone INT8 reached 41.79 ms, while via-ORT INT8 was 38% slower. These results favor measurement over a blanket route recommendation. section 9.4
