A runtime contract states which model-artifact and runtime pairs are expected to load and execute, and under what conditions. The artifact graph may encode precision, scales, and zero points, but not the calibration fingerprint, quantization recipe, or minimum compute capability; a manifest sidecar records these alongside hardware and runtime requirements. section 9.5
Make the contract operational by:
- Keeping a Triton model-repository directory for each artifact-runtime pair. Configuration usually describes inputs, outputs, batching, and instances; ORT's OpenVINO execution-provider selection is configured there. section 9.5
- Smoke-testing each applicable pair on the target host, recording whether it loads, which providers resolve, and whether execution uses the claimed precision. section 9.5
Loading is not sufficient
On an NVIDIA L4, BERT INT8 QDQ ONNX loaded through ORT's CUDA execution provider but was demoted: execution was partly INT8 and partly FP32. It measured 14.4 ms, versus 8.96 ms for optimized FP32, because CUDA EP coverage was incomplete. section 9.5
Figure 9.10 Packaging verification matrix on an NVIDIA L4 host. Rows are artifact kinds; columns are runtimes. Solid cells executed at the claimed precision (the OpenVINO Core CPU column reports per-op precision fractions; ORT cells verify load and run, since ORT's Python API does not expose per-op precision). Hatched cells loaded but fell back. ResNet INT8 × ORT TRT EP routed 19 nodes to CUDA EP because TensorRT's parser rejects ORT's INT32 zero-point format on Conv DQ; BERT INT8 × ORT CUDA EP is the 9.2 anti-pattern resurfacing. Filled cells are runtime, hardware, or artifact absent; information about the deploy target, not bugs.

Compare artifact-runtime pairs by whether they execute at the claimed precision, fall back, or are unavailable on the host. section 9.5
