OpenVINO is a runtime option for deploying quantized models on Intel silicon. Select it as part of matching the target hardware to the runtime and production artifact. section 9.0
OpenVINO
OpenVINO is an Intel toolkit for optimizing and deploying machine-learning models across diverse hardware targets.
How Quantization and Fast Inference covers it
- How deep it goes
- Places OpenVINO in runtime selection for Intel CPUs but gives no implementation or quantization procedure.
- Examples
- Uses quantizing models for Intel CPUs with OpenVINO as a deployment example.
- What it assumes
- Assumes a quantized model artifact and a target hardware choice.
- What you can do after
- Apply OpenVINO when selecting a runtime for quantized inference on Intel silicon.
- How it connects
- Connects runtime selection with quantization, ONNX Runtime, and TensorRT.
Related concepts
- CPUuses · A central processing unit (CPU) is a computer processor that executes program instructions and performs general-purpose computational operations.
- generationuses · Generation is the production of an output by a computational model from supplied input or context, often using patterns learned from data.
- ONNXalternative to · ONNX (Open Neural Network Exchange) is an open standard and format for representing and exchanging machine-learning models across frameworks, tools, and runtimes.
- ONNX Runtimeused by · ONNX Runtime is an open-source, cross-platform runtime for executing and optimizing machine-learning models in the ONNX format.
- environmentuses · An environment is the external context, system, or conditions within which an agent or application operates and interacts.
This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.
