liveBook · Manning

Outlier channel

As explained in Quantization and Fast Inference

An outlier channel or dimension is a feature coordinate whose activation magnitudes are substantially larger than those of most other coordinates.

An outlier channel is a feature dimension whose activations are much larger than those of most other dimensions. In activation quantization, a few such channels can set a shared scale so coarse that ordinary channels lose precision; clipping the outliers instead can damage model quality. section 3.3 section 7.1

With symmetric per-tensor INT8, the scale is typically the largest absolute activation divided by 127. A large outlier therefore widens the spacing between representable values for every channel. Per-channel scales reduce this shared-scale penalty, but the outlier dimensions can still dominate residual error. section 7.1

Quantization error in OPT-6.7B

In a layer-wise comparison, per-channel INT8 reduces total activation MSE by 1,872× in layer 0 and 442× in layer 31 relative to per-tensor INT8. Yet outlier dimensions still account for 82% of residual error in early layers. These are model- and measurement-specific results, not universal speed or accuracy guarantees. section 7.1

Outliers can be structured rather than random. In BERT, hot channels vary by layer but can recur across inputs. In OPT-6.7B, five dimensions appear in the top 1% by magnitude in every layer; the peak layer max-to-median ratio reaches 106×. This cross-layer coordination makes targeted handling practical. section 3.3 section 7.1

Figure 7.1 Per-channel activation magnitude heatmap for OPT-6.7B (32 layers × 4,096 hidden dimensions, log scale). Bright vertical stripes mark the same hidden dimensions spiking across all layers. Dashed lines mark the five universal outlier channels—dimensions 639, 1778, 2394, 2972, and 3136—that appear in the top 1% of every layer.

The activation heatmap shows the repeated high-magnitude dimensions as vertical stripes across OPT-6.7B layers. section 7.1

A practical profiling and mitigation workflow:

  • Hook the relevant layer inputs or outputs, flatten token positions, and record absolute maxima per feature dimension over calibration inputs. Compare channels within each layer and check whether the same dimensions recur across layers. section 3.3 section 7.1
  • Compare per-tensor and per-channel reconstruction error before deciding whether a shared scale is adequate. A channel that sets the scale may itself quantize cleanly while ordinary channels absorb most of the error. section 7.1
  • For severe outliers, split the activation and weight columns into normal and outlier paths. LLM.int8() detects outliers dynamically, computes the normal path in INT8 and the outlier path at higher precision, then sums the results. section 7.2

Deployment tradeoff

For OPT-6.7B on WikiText-2, LLM.int8() matches the FP16 baseline perplexity of 16.31 while using 6.4 GB versus 12.4 GB. With a 6.0 threshold, the measured fraction routed through the higher-precision path averages 0.23% of dimensions; it varies by layer. Validate the threshold and overhead on the target model and hardware. section 7.2

Listing 7.4 Mixed-precision decomposition (the LLM.int8() algorithm)
def mixed_precision_matmul(X, W, threshold=6.0):
    # Step 1: Identify outlier feature dimensions
    outlier_mask = X.abs().max(dim=0).values > threshold          #A
    outlier_dims = outlier_mask.nonzero(as_tuple=True)[0]
    normal_dims = (~outlier_mask).nonzero(as_tuple=True)[0]
 
    # Step 2: Split into two paths
    X_outlier = X[:, outlier_dims].float()                        #B
    W_outlier = W[:, outlier_dims].float()
    X_normal = X[:, normal_dims].float()
    W_normal = W[:, normal_dims]
 
    # Step 3a: Full-precision matmul for outlier dimensions
    out_outlier = X_outlier @ W_outlier.T                         #C
 
    # Step 3b: INT8 matmul for normal dimensions
    W_q, W_s = quantize_absmax_int8(W_normal)                     #D
    X_scales = X_normal.abs().amax(dim=1, keepdim=True) / 127.0
    X_q = torch.round(X_normal / X_scales).clamp(-128, 127)
    out_normal = (X_q.float() @ W_q.float().T) * (X_scales * W_s.T)
 
    # Step 4: Sum the two paths
    return out_outlier + out_normal                                #E
 

The decomposition listing makes the feature-dimension split and the two matrix-multiplication paths explicit. section 7.2

How Quantization and Fast Inference covers it

How deep it goes
Profiles channel magnitudes and quantization error, then implements mixed-precision decomposition and examines threshold-dependent results.
Examples
Profiles BERT and OPT channel magnitudes and compares OPT-6.7B reconstruction error and WikiText-2 perplexity.
What you can do after
Builds mixed-precision decomposition from scratch and applies LLM.int8() through bitsandbytes configuration.
How it connects
Connects activation quantization, per-channel scaling, mixed precision, LLM.int8(), and activation-aware weight quantization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.