TensorRT is an NVIDIA SDK for optimizing and executing neural-network inference on GPUs. It is positioned for peak throughput on NVIDIA hardware and for building INT8 and FP8 engines. In deployment, verify that quantized kernels actually run: a quantized artifact can still serve through a silent FP32 fallback. section 9.0
Quantized artifact, unquantized execution
An INT8 endpoint shows p99 latency at the FP32 baseline, while GPU dispatch counters show no quantized kernels. The artifact alone does not prove the serving path uses INT8. section 9.0

