Choose an intervention by first checking whether PTQ meets accuracy requirements. If it does, deploy PTQ; if not, use bit width and layer sensitivity to guide the next step. section 5.1.2
Use this decision sequence when PTQ misses its accuracy target:
- For INT8, revisit calibration first: failures almost always trace to calibration problems. section 5.1.2
- For INT4, run layer sensitivity analysis. If sensitivity is concentrated in a few layers, keep those layers at INT8 and quantize the rest to INT4; if it is diffuse, use QAT. section 5.1.2
Listing 5.2 Layer sensitivity analysis
def run_layer_sensitivity_analysis(model, data_loader, device, eval_fn, bits=4):
"""Quantize one layer at a time and measure impact."""
baseline_acc = eval_fn(model, data_loader, device)
sensitivities = {}
for name, module in model.named_modules():
if isinstance(module, (nn.Linear, nn.Conv2d)):
# Quantize only this layer (weights + activations)
q_model = quantize_single_layer(model, name, bits=bits)
acc = eval_fn(q_model, data_loader, device)
sensitivities[name] = baseline_acc - acc #A
return sorted(sensitivities.items(), key=lambda x: x[1], reverse=True)
The analysis quantizes one layer at a time and measures the resulting accuracy drop, producing a ranked sensitivity list. section 5.1.2
Concentrated versus diffuse sensitivity
At INT4, one BERT layer causes a 26.4% accuracy drop when quantized alone, suggesting a target for mixed precision. ResNet-18 sensitivity is diffuse, with no single layer dominating its degradation. section 5.1.2
