Compression ratio measures how many times smaller a model’s stored representation is than its original. For INT8 quantization of a fraction p of FP32 parameters, the theoretical ceiling is 1 / (0.25p + 1 − p); unquantized parameters remain full size. section 6.2
Architecture changes the result
TorchAO INT8 weight-only quantization measured 1.03× compression for ResNet-18, 2.40× for BERT-base, and 3.38× for TinyLlama-1.1B. The results track how many parameters are in nn.Linear layers; per-channel scales also add storage overhead. section 6.2
Table 6.1 Architecture determines compression ceiling
|
Model
|
Linear layers
|
Linear params (%)
|
Theoretical INT8 ceiling
|
Measured
|
|
ResNet-18
|
1
|
513,000 (4.4%)
|
1.03×
|
1.03×
|
|
BERT-base
|
74
|
85,609,730 (78.2%)
|
2.42×
|
2.40×
|
|
TinyLlama-1.1B
|
155
|
1,034,420,224 (94.0%)
|
3.57×
|
3.38×
|
Compare each model’s nn.Linear parameter share with its theoretical INT8 ceiling and measured compression. section 6.2

