An observer determines static activation-quantization parameters during calibration: it records tensor extrema, then computes a scale and zero point to freeze for inference. This avoids runtime range-measurement overhead, but accuracy can degrade when deployed inputs differ from calibration data. section 3.3
Listing 3.18 Activation observer for static quantization
class ActivationObserver:
def __init__(self):
self.min_val = float('inf')
self.max_val = float('-inf')
def observe(self, tensor: torch.Tensor):
self.min_val = min(self.min_val, tensor.min().item())
self.max_val = max(self.max_val, tensor.max().item())
def get_scale_zeropoint(self, bits=8, symmetric=False):
if symmetric:
abs_max = max(abs(self.min_val), abs(self.max_val))
q_max = (1 << (bits - 1)) - 1 #A
scale = abs_max / q_max if abs_max > 0 else 1.0
zero_point = 0
else:
q_min, q_max = 0, (1 << bits) - 1 #B
scale = (self.max_val - self.min_val) / (q_max - q_min)
zero_point = int(round(q_min - self.min_val / scale))
return scale, zero_point
The implementation accumulates minimum and maximum values across observed tensors and derives parameters for symmetric or asymmetric quantization. section 3.3
