TorchAO’s quantize_() walks a model’s module tree and replaces eligible nn.Linear weights with AffineQuantizedTensor representations storing INT8 data and per-channel scales. The modules remain nn.Linear; weight-only configurations need no calibration data. section 6.2
Compression depends on how many parameters belong to nn.Linear. Conv2d weights remain untouched: the stable configurations reshape weights to two dimensions, which does not fit Conv2d’s four-dimensional weights. section 6.2
Table 6.1 Architecture determines compression ceiling
|
Model
|
Linear layers
|
Linear params (%)
|
Theoretical INT8 ceiling
|
Measured
|
|
ResNet-18
|
1
|
513,000 (4.4%)
|
1.03×
|
1.03×
|
|
BERT-base
|
74
|
85,609,730 (78.2%)
|
2.42×
|
2.40×
|
|
TinyLlama-1.1B
|
155
|
1,034,420,224 (94.0%)
|
3.57×
|
3.38×
|
Compare parameter distribution with theoretical and measured INT8 compression across ResNet-18, BERT-base, and TinyLlama. section 6.2
