A TensorRT engine is a production artifact compiled for serving silicon. It differs from portable ONNX, which can be loaded by multiple runtimes. section 9.1
Choose the deployment route to match the fleet:
- For a single-SKU deployment at scale, serve a standalone engine through the vendor API to favor peak latency. section 9.1
- For a heterogeneous fleet, keep ONNX Runtime serving and use its TensorRT execution provider for compatible subgraphs. section 9.1
Figure 9.1 Hardware selects the runtime family, and the execution provider selects which kernels dispatch. The leaf node is the production artifact. The two routes to TensorRT and OpenVINO, standalone versus via ORT, trade peak latency against a unified serving surface across heterogeneous fleets. Branches marked Chapter 10 and Chapter 11 point to regimes covered there.

Trace how hardware and runtime choice lead to a production artifact, and compare standalone serving with the ONNX Runtime route. section 9.1
