Keep ONNX Runtime serving code and load an FP32 or QDQ ONNX model through OpenVINOExecutionProvider. Both this route and standalone OpenVINO use the OpenVINO CPU plugin, but supply it different artifacts; performance depends on the model, so benchmark both routes. section 9.4
Model-specific latency
On Sapphire Rapids, BERT-base INT8 via ORT ran in 96.28 ms versus 110.17 ms for standalone OpenVINO INT8. For ResNet-18, standalone INT8 was faster; via-ORT INT8 cost 38% relative to native. These results favor measurement over a universal route recommendation. section 9.4
