liveBook · Manning

Observer

As explained in Quantization and Fast Inference

An observer is a software component that records representative tensor statistics during calibration to determine quantization parameters.

An observer determines static activation-quantization parameters during calibration: it records tensor extrema, then computes a scale and zero point to freeze for inference. This avoids runtime range-measurement overhead, but accuracy can degrade when deployed inputs differ from calibration data. section 3.3

Listing 3.18 Activation observer for static quantization
class ActivationObserver:
    def __init__(self):
        self.min_val = float('inf')
        self.max_val = float('-inf')
        
    def observe(self, tensor: torch.Tensor):
        self.min_val = min(self.min_val, tensor.min().item())
        self.max_val = max(self.max_val, tensor.max().item())
    
    def get_scale_zeropoint(self, bits=8, symmetric=False):
        if symmetric:
            abs_max = max(abs(self.min_val), abs(self.max_val))
            q_max = (1 << (bits - 1)) - 1                         #A
            scale = abs_max / q_max if abs_max > 0 else 1.0
            zero_point = 0
        else:
            q_min, q_max = 0, (1 << bits) - 1                     #B
            scale = (self.max_val - self.min_val) / (q_max - q_min)
            zero_point = int(round(q_min - self.min_val / scale))
        return scale, zero_point
 

The implementation accumulates minimum and maximum values across observed tensors and derives parameters for symmetric or asymmetric quantization. section 3.3

How Quantization and Fast Inference covers it

How deep it goes
Implements min/max accumulation and scale/zero-point calculation, while leaving runtime ranges fixed after calibration.
Examples
Uses a Python activation-observer implementation with symmetric and asymmetric parameter calculations.
What it assumes
Uses representative calibration inputs to estimate ranges before deployment.
What you can do after
Build it from scratch by accumulating activation extrema and calculating quantization scale and zero point.
How it connects
Connects calibration, static quantization, dynamic quantization, and activation ranges.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.