liveBook · Manning

Architectural sensitivity

As explained in Quantization and Fast Inference

Architectural sensitivity is the degree to which a model's output changes in response to perturbations of its parameters or activations.

Architectural sensitivity measures how much a model’s output shifts when its parameters are perturbed. For quantization, injected Gaussian weight noise approximates rounding perturbations; the relative output change estimates sensitivity. Architecture family alone does not reliably predict the result. section 4.1

Measured sensitivity

With Gaussian noise scaled to 1% of each weight tensor’s mean absolute value, averaged over ten trials, MobileNetV2 shifts by about 11.6%, versus 7.6% for ResNet-18 and 7.4% for BERT-base. section 4.1

Listing 4.1 Measuring architecture noise sensitivity
def measure_noise_sensitivity(model, input_tensor, noise_scale=0.01, num_trials=10):
    model.eval()
    
    with torch.no_grad():
        baseline = model(input_tensor)
        if hasattr(baseline, 'last_hidden_state'):
            baseline = baseline.last_hidden_state                 #A
        
        output_changes = []
        for _ in range(num_trials):
            original_weights = {}
            for name, param in model.named_parameters():
                original_weights[name] = param.data.clone()
                noise = torch.randn_like(param) * noise_scale * param.abs().mean()
                param.data.add_(noise)                            #B
            
            noisy_output = model(input_tensor)
            if hasattr(noisy_output, 'last_hidden_state'):
                noisy_output = noisy_output.last_hidden_state
            
            relative_change = (noisy_output - baseline).abs().mean() / (baseline.abs().mean() + 1e-8)
            output_changes.append(relative_change.item())
            
            for name, param in model.named_parameters():
                param.data.copy_(original_weights[name])          #C
        
        return sum(output_changes) / len(output_changes)
 
torch.manual_seed(42)
resnet = resnet18(weights="IMAGENET1K_V1")
mobilenet = mobilenet_v2(weights="IMAGENET1K_V1")
bert = BertModel.from_pretrained('bert-base-uncased')
print(f"ResNet-18:   {measure_noise_sensitivity(resnet, torch.randn(1, 3, 224, 224)):.4f}")
print(f"MobileNetV2: {measure_noise_sensitivity(mobilenet, torch.randn(1, 3, 224, 224)):.4f}")
print(f"BERT-base:   {measure_noise_sensitivity(bert, torch.randint(0, 30000, (1, 32))):.4f}")

Inspect the listing to see how parameter noise is applied and output changes are measured. section 4.1

How Quantization and Fast Inference covers it

How deep it goes
Measures output changes after scaled Gaussian weight perturbations, treating them as a proxy rather than direct quantization results.
Examples
Compares measured output shifts for ResNet-18, MobileNetV2, and BERT-base under injected Gaussian weight noise.
What you can do after
Evaluate sensitivity by injecting scaled Gaussian noise into weights and measuring relative output change.
How it connects
Connects quantization noise and post-training quantization with residual connections and cross-layer equalization.

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.