A state_dict is the portable serialization path for quantized PyTorch models: save the mapping, reconstruct the quantized model skeleton, then load the mapping. This avoids relying on full-model pickling, which can fail across PyTorch versions. section 6.2
Listing 6.3 Save/load round-trip for TorchAO models
# Save
torch.save(model.state_dict(), "bert_int8wo.pt") #A
# Load
loaded = AutoModelForSequenceClassification.from_pretrained(...)
quantize_(loaded, Int8WeightOnlyConfig()) #B
loaded.load_state_dict(
torch.load("bert_int8wo.pt", weights_only=False) #C
)Follow the save/load round-trip, including reconstruction of the quantized model before loading its state. section 6.2
