Tactic selection chooses operation implementations from NVIDIA’s kernel library while TensorRT builds an engine for a target GPU. The build selects tactics ahead of time, can fuse Q/DQ pairs into INT8 supernodes, and produces a serialized plan. section 9.3
The resulting engine depends on the quantization path as well as the target and model. TensorRT supports implicit quantization, where it chooses INT8 layers, and explicit Q/DQ quantization, where it must honor placements encoded in ONNX. section 9.3
Quantization choices produce model-dependent results
On an L4, implicit INT8 runs ResNet-18 at 1.04 ms versus 1.73 ms for explicit INT8. For BERT-base, explicit INT8 leads at 2.73 ms, while implicit INT8 performs near FP16. section 9.3
