AffineQuantizedTensor stores INT8 weight data with per-channel FP32 scales. TorchAO’s quantize_() replaces nn.Linear weights with this subclass while leaving the module structure unchanged; the scales add storage overhead. section 6.2
BERT weight representation
A BERT query weight is represented with shape [768, 768], block_size=(1, 768), and int_dtype=torch.int8.
