Scalar quantization converts a high-precision tensor into a lower-precision numerical range using one shared scale. It finds the maximum absolute value α, computes a scale such as s = 127/α, multiplies each value by s, and rounds. Dequantization divides the integer values by s to approximate the originals. section 5.6.2
The shared scale makes the method simple but leaves it highly sensitive to outliers. A single large activation consumes most of the target range, reducing the precision available to ordinary values across the tensor. section 5.6.2
Outlier-dominated vector
For [2.0, 3.0, 500.0] quantized to signed 8-bit integers, α = 500 and s = 0.254. Rounding produces [1, 1, 127], erasing the distinction between 2.0 and 3.0. Dequantization returns approximately [3.94, 3.94, 500.0], preserving the outlier while badly distorting the smaller values. section 5.6.2
Figure 5.27 Fine-Grained Quantization for an activation vector. The vector is partitioned into smaller groups. Each group is scaled independently based on the maximum value within that group, preserving precision for values in groups that do not contain large outliers.

Compare shared tensor scaling with grouped scaling: partitioning an activation vector allows groups without large outliers to retain more precision. section 5.6.2

