liveBook · Manning

OpenVINO

As explained in Quantization and Fast Inference

OpenVINO is an Intel toolkit for optimizing and deploying machine-learning models across diverse hardware targets.

OpenVINO is a runtime option for deploying quantized models on Intel silicon. Select it as part of matching the target hardware to the runtime and production artifact. section 9.0

How Quantization and Fast Inference covers it

How deep it goes
Places OpenVINO in runtime selection for Intel CPUs but gives no implementation or quantization procedure.
Examples
Uses quantizing models for Intel CPUs with OpenVINO as a deployment example.
What it assumes
Assumes a quantized model artifact and a target hardware choice.
What you can do after
Apply OpenVINO when selecting a runtime for quantized inference on Intel silicon.
How it connects
Connects runtime selection with quantization, ONNX Runtime, and TensorRT.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.