liveBook · Manning

Model artifact

As explained in Quantization and Fast Inference

A model artifact is a packaged, deployable representation of a machine-learning model and the information needed to execute it.

A model artifact is the deployable output consumed by a runtime. Its graph carries the precision; serving configuration alone does not prove that the intended dtype executes. section 9.6

Common deployment choices include:

  • A TensorRT .plan, an OpenVINO IR pair, or quantized ONNX served through ONNX Runtime. section 9.6
  • A manifest sidecar and content hash recording the quantizer, format, calibration fingerprint, required hardware, and runtime compatibility. section 9.6
  • Verify the executing dtype on the deployment host: unsupported kernels can silently fall back to FP32. section 9.6

How Quantization and Fast Inference covers it

How deep it goes
Compares runtime-specific artifact forms, metadata, and dtype verification, but does not detail artifact serialization internals.
Examples
Uses a TensorRT .plan, an OpenVINO IR pair, and quantized ONNX served through ONNX Runtime.
What it assumes
Assumes familiarity with quantized checkpoints, inference runtimes, execution providers, and target hardware.
What you can do after
Operate deployment by packaging precision metadata and verifying the executing dtype on the target host.
How it connects
Connects quantization and precision metadata with ONNX Runtime, TensorRT, OpenVINO, and execution providers.

Also explained in

Compare how 3 books explain model artifact →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.