liveBook · Manning

Activation range

As explained in Quantization and Fast Inference

An activation range is the interval of numerical values produced by a neural-network activation across inputs.

An activation range is the span of values a neural network produces across inputs. In post-training quantization, calibration data estimates the ranges used to set quantization grids. Ranges that are too wide waste precision; ranges that are too narrow clip values. section 4.2

Input conditions change layer ranges

For ResNet-18, Layer 1 activation maxima are 3.07 for low-contrast images and 5.91 for high-contrast images. Calibrating on just one condition can misestimate production ranges. section 4.2

Figure 4.4 Left: Sample images under four conditions (original, low-light, high-contrast, noisy) showing the visual transformations applied to STL-10 images. Right: Layer-wise activation maximums through ResNet-18, where each line corresponds to one condition. Notice how low-light images produce consistently lower activation ranges while high-contrast images push activation maximums higher in deeper layers—calibrating on only one condition would systematically mis-estimate the ranges encountered in production.

Compare activation maxima across image conditions and layers to see how input variation shifts the estimated ranges. section 4.2

How Quantization and Fast Inference covers it

How deep it goes
Explains range mismatch and layer-wise maxima, then focuses on representative sampling and validation.
Examples
Uses sentiment, vision, and language-model distribution mismatches, plus layer maxima across image conditions.
What you can do after
Build a representative calibration set and validate its layer-wise activation coverage against production.
How it connects
Connects activation ranges to calibration data, quantization grids, scale factors, and zero-points.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.