liveBook · Manning

Outlier ratio

As explained in Quantization and Fast Inference

Outlier ratio is the ratio of the largest observed magnitude to a representative high percentile of value magnitudes, indicating the influence of the distribution tail.

The outlier ratio compares the largest activation magnitude with the 99th percentile of magnitudes: max(|x|) / p99(|x|). It shows how much the distribution’s tail can affect range estimation for quantization. A high ratio can make absmax calibration spend precision on outliers rather than typical values. section 4.3

BERT activation tail

For BERT-base layer 6 FFN output, the maximum is 11.52 and the 99th percentile is 1.92, giving a ratio of about 6×. section 4.3

Listing 4.8 Outlier ratio diagnostic
def compute_outlier_ratio(activations):
    """The single most predictive diagnostic for range estimation."""
    abs_vals = activations.abs()
    return abs_vals.max().item() / torch.quantile(abs_vals, 0.99).item()

Compute the ratio directly from activation magnitudes with this diagnostic. section 4.3

How Quantization and Fast Inference covers it

How deep it goes
Defines the max-to-99th-percentile calculation and relates a high ratio to precision loss under absmax calibration.
Examples
Compares a BERT-base layer 6 FFN maximum of 11.52 with a 99th percentile of 1.92, yielding about 6×.
What it assumes
Assumes access to calibration activations whose absolute magnitudes can be measured.
What you can do after
Diagnose activation tails by computing the maximum-to-99th-percentile ratio on calibration activations.
How it connects
Connects activation outliers to absmax calibration, percentile clipping, and MSE-optimal range estimation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.