liveBook · Manning

Validation protocol

As explained in Quantization and Fast Inference

A validation protocol is a repeatable procedure for evaluating whether a model or transformation satisfies specified performance requirements.

Compare a quantized model with its FP32 baseline to measure accuracy loss and size reduction, then compare calibration methods. This protocol validates accuracy and size, not latency. section 4.4

Run the comparison in order:

  • If the model has batch-normalization layers, fold them into preceding linear or convolutional layers before calibration. section 4.4
  • Measure the FP32 baseline, quantize with each selected calibration method, then compare accuracy and model size. section 4.4

ResNet-18

On 500 ImageNet validation samples, the FP32 baseline reaches 72.20% accuracy at 44.59 MB. INT8 with MSE calibration reaches 72.00% at 11.18 MB, a 4.0× reduction. section 4.4

Table 4.3 PTQ validation results
Model
Config
Accuracy
Δ Accuracy
Size (MB)
Compression
ResNet-18
FP32
72.20%
—
44.59
—
 
INT8 (absmax)
71.60%
-0.60%
11.18
4.0×
 
INT8 (percentile)
70.60%
-1.60%
11.18
4.0×
 
INT8 (mse)
72.00%
-0.20%
11.18
4.0×
BERT-base
FP32
93.40%
—
326.72
—
 
INT8 (mse)
93.40%
+0.00%
82.03
4.0×

Compare accuracy and compression results for ResNet-18 and BERT-base across FP32 and INT8 configurations. section 4.4

How Quantization and Fast Inference covers it

How deep it goes
Defines a baseline-and-compare experiment for accuracy and size, and leaves latency measurement to runtime-specific validation.
Examples
Uses ResNet-18 on ImageNet and BERT-base on SST-2 to compare FP32 baselines with INT8 results.
What it assumes
Assumes a calibration set, selected range-estimation method, and quantized model; requires folding batch-normalization layers before calibration when present.
What you can do after
Evaluate accuracy loss, compression, and calibration choices by comparing quantized models against FP32 baselines.
How it connects
Connects post-training quantization with calibration data, range-estimation methods, batch-normalization folding, and latency measurement.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.