The outlier ratio compares the largest activation magnitude with the 99th percentile of magnitudes: max(|x|) / p99(|x|). It shows how much the distribution’s tail can affect range estimation for quantization. A high ratio can make absmax calibration spend precision on outliers rather than typical values. section 4.3
BERT activation tail
For BERT-base layer 6 FFN output, the maximum is 11.52 and the 99th percentile is 1.92, giving a ratio of about 6×. section 4.3
Listing 4.8 Outlier ratio diagnostic
def compute_outlier_ratio(activations):
"""The single most predictive diagnostic for range estimation."""
abs_vals = activations.abs()
return abs_vals.max().item() / torch.quantile(abs_vals, 0.99).item()Compute the ratio directly from activation magnitudes with this diagnostic. section 4.3
