A model artifact is a packaged, deployable representation of a machine-learning model and the information needed to execute it.
A model artifact is the deployable output consumed by a runtime. Its graph carries the precision; serving configuration alone does not prove that the intended dtype executes. section 9.6
Common deployment choices include:
A TensorRT .plan, an OpenVINO IR pair, or quantized ONNX served through ONNX Runtime. section 9.6
A manifest sidecar and content hash recording the quantizer, format, calibration fingerprint, required hardware, and runtime compatibility. section 9.6
Verify the executing dtype on the deployment host: unsupported kernels can silently fall back to FP32. section 9.6
How Quantization and Fast Inference covers it
How deep it goes
Compares runtime-specific artifact forms, metadata, and dtype verification, but does not detail artifact serialization internals.
Examples
Uses a TensorRT .plan, an OpenVINO IR pair, and quantized ONNX served through ONNX Runtime.
What it assumes
Assumes familiarity with quantized checkpoints, inference runtimes, execution providers, and target hardware.
What you can do after
Operate deployment by packaging precision metadata and verifying the executing dtype on the target host.
How it connects
Connects quantization and precision metadata with ONNX Runtime, TensorRT, OpenVINO, and execution providers.
Machine Learning Platform EngineeringA model artifact type is a KFP artifact type representing a trained machine-learning model and its framework-specific metadata.
inference platformpart of · A system surrounding a machine-learning model that serves it through an API and provides operational functions such as routing, protection, and monitoring.
runtime familyused by · A runtime family is a class of model-execution software sharing a deployment backend or hardware-oriented execution model.
GPT4Allused by · GPT4All is an open-source software ecosystem for running and interacting with language models locally on consumer or organizational hardware.
local aiused by · Local AI is artificial-intelligence computation performed by a model on a user's device or other locally controlled infrastructure rather than through a remote service.
This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.