Operator coverage describes how much of a model is affected by an optimization or quantization method. Check it alongside latency: an INT8 label does not guarantee that every operator executes in INT8, and low operator-count coverage can still be effective when the covered operators dominate compute. section 6.5 section 6.6
Interpret a coverage percentage by its counting unit. An ONNX graph can show which operators were quantized or wrapped in quantize/dequantize (QDQ) pairs. A TensorRT engine inspector instead reports the execution precision of compiled compute layers; fusion can change the layer count. For the reported engine fractions, Reformat, Constant, and Shuffle layers are excluded. These percentages are not necessarily directly comparable. section 6.5 section 9.3
Figure 6.8 Operator quantization coverage by ONNX variant. Left: ResNet-18 ORT Dynamic — 4% coverage, only 1 MatMulInteger node (the classification head's Gemm) out of 51 compute ops. All 20 Conv and 17 Relu stay in FP32. Center: ResNet-18 ORT Static — 100% coverage, all 20 Conv, 8 Add, and the Gemm wrapped in QDQ pairs. Right: BERT-base ORT Dynamic — 28% coverage, 74 MatMulInteger nodes (the linear projections) quantized, but LayerNormalization, Softmax, Div, and Sqrt remain in FP32.

Compare graph-level coverage across dynamic and static ONNX Runtime quantization, including operators left in FP32. section 6.5
Coverage depends on the graph and quantization path
For ResNet-18, ONNX Runtime dynamic quantization reports 4% coverage: the classification head becomes a MatMulInteger node, while all 20 Conv and 17 Relu nodes remain FP32. Static QDQ quantization reports 100% coverage, with all 20 Conv, eight Add, and the Gemm wrapped in QDQ pairs. Dynamic quantization is more effective on BERT, whose linear projections provide more MatMul targets. section 6.5 section 6.6
Inspect the compiled engine
On an NVIDIA L4, TensorRT implicit INT8 reaches a 95.7% INT8 fraction and 1.04 ms latency for ResNet-18, but 0.0% and 3.77 ms for BERT-base. Explicit INT8 reaches 83.3% and 1.73 ms for ResNet-18, versus 97.8% and 2.73 ms for BERT. Fusion and precision placement affect both the counts and latency, so measure the built engine rather than inferring execution from the build flag. section 9.3
Table 9.2 Engine sizes and execution-precision distribution, L4. Compute layers exclude Reformat/Constant/Shuffle.
|
Model
|
Engine
|
Compute layers
|
INT8 fraction
|
Size (MB)
|
Latency (ms)
|
|
ResNet-18
|
FP32
|
25
|
0.0%
|
47.0
|
6.66
|
|
ResNet-18
|
FP16
|
23
|
0.0%
|
23.8
|
2.67
|
|
ResNet-18
|
INT8 implicit
|
23
|
95.7%
|
12.1
|
1.04
|
|
ResNet-18
|
INT8 explicit
|
24
|
83.3%
|
13.9
|
1.73
|
|
BERT-base
|
FP32
|
124
|
0.0%
|
438.6
|
8.77
|
|
BERT-base
|
FP16
|
86
|
0.0%
|
221.1
|
3.79
|
|
BERT-base
|
INT8 implicit
|
86
|
0.0%
|
221.1
|
3.77
|
|
BERT-base
|
INT8 explicit
|
183
|
97.8%
|
184.8
|
2.73
|
Inspect engine sizes, compute-layer counts, INT8 fractions, and latency together for both models. section 9.3
Coverage also does not establish numerical equivalence. Run identical inputs through the FP32 baseline and each variant, compare raw logits and prediction agreement, and check the FP32 export first to separate export error from quantization error. Then inspect coverage and benchmark the deployed runtime. section 6.5 section 6.6
