Activation outliers are a small number of unusually large values among mostly small activations. In large language models, they can dominate quantization ranges: clipping them can damage model quality, while preserving them can coarsen the grid for most values. section 7.0 section 2.3
Their quantization impact motivates methods that handle outlier dimensions separately or protect sensitive channels:
- LLM.int8() routes outlier dimensions through an FP16 path alongside a dense INT8 path. section 7.7
- AWQ rescales channels associated with high-magnitude activations before quantizing weights. section 7.7
Listing 2.6 Visualizing quantization risk
plt.hist(activations, bins=100) plt.axvline(-alpha, color='r') #A plt.axvline(alpha, color='r') #A plt.show()
A histogram with clipping boundaries makes it possible to inspect which values fall outside the quantization range. section 2.3



















