An activation is an intermediate tensor produced by a neural-network operation and passed to subsequent computation. Unlike weights, activations are recomputed for every input, so their ranges can shift; transformer hidden states can also develop extreme outliers. These properties make activation quantization a separate precision decision from weight quantization. section 3.0
Affine quantization adapts the integer grid to an activation tensor’s observed range. Calibration includes zero in that range, then calculates a scale and integer zero-point. Quantization divides each value by the scale, rounds, adds the zero-point, and clamps to the integer range. This can use more of the grid than symmetric quantization when activations are predominantly positive. section 2.5
Listing 2.8 AffineQuantizer for activations
class AffineQuantizer:
def __init__(self, bits=8):
self.q_min = 0
self.q_max = (1 << bits) - 1 #A
self.scale = 1.0
self.zero_point = 0
def calibrate(self, tensor):
r_min = tensor.min().item()
r_max = tensor.max().item()
r_min = min(r_min, 0.0) #B
r_max = max(r_max, 0.0) #B
real_range = r_max - r_min
int_range = self.q_max - self.q_min
if real_range == 0:
self.scale, self.zero_point = 1.0, 0
else:
self.scale = real_range / int_range
initial_z = self.q_min - (r_min / self.scale)
self.zero_point = int(round(initial_z)) #C
self.zero_point = max(self.q_min, min(self.q_max, self.zero_point))
def quantize(self, tensor):
scaled = (tensor / self.scale) + self.zero_point
rounded = torch.round(scaled)
return torch.clamp(rounded, self.q_min, self.q_max).to(torch.uint8)
def dequantize(self, q_tensor):
return self.scale * (q_tensor.float() - self.zero_point) #D
Inspect calibration of an 8-bit affine activation quantizer, including the zero-range fallback and zero-point calculation. section 2.5
ReLU activations
For one ReLU activation distribution, symmetric quantization uses codes 0–127 with scale 0.139; asymmetric quantization uses 0–255 with scale 0.069. The latter has approximately 2.01 times finer resolution in this comparison. section 2.5
In a linear layer with symmetric weights and affine activations, the activation zero-point must be corrected in the matrix-product arithmetic. Omitting the correction can introduce substantial error when the zero-point is nonzero. The demonstration multiplies quantized values after converting them to floating point, so it illustrates the arithmetic rather than establishing integer-kernel performance. section 2.5
Shifted activation inputs
With an activation zero-point of 48, the corrected output has mean absolute error 0.0302 against FP32; dropping the correction raises it to 1.4174, or 46.9 times as much. section 2.5
Choose quantization granularity with activation statistics, acceptable accuracy loss, and target hardware in mind. A single scale per tensor is simple but crude; finer schemes trade additional granularity against implementation and kernel constraints. section 3.0





