Use perturbation testing to find layers where quantization causes disproportionate accuracy loss. Quantize one layer at a time, evaluate the model, and rank layers by impact. section 3.1
Measure layer impact
Record baseline accuracy, copy the model for each layer, quantize that layer’s weights using max-absolute scaling divided by 127, then calculate baseline accuracy minus quantized accuracy. Larger drops indicate greater sensitivity. section 3.1
Common sensitivity patterns include:
- First and last layers are often more sensitive than middle layers. section 3.1
- Attention layers tend to be more sensitive than feedforward layers; small weight ranges can make quantization noise relatively larger. section 3.1

