Asymmetric quantization
Asymmetric quantization maps values to discrete levels using a scale and a zero point, allowing the represented range to be shifted relative to zero.
Quantization and Fast Inference
Asymmetric quantization maps values to a grid whose range and zero-point can be shifted to fit a non-centered distribution.
- How it is used
- Matches shifted integer grids to non-centered activations while retaining symmetric quantization for centered weights.
- How deep it goes
- Derives affine calibration and zero-point correction, then implements and measures a hybrid linear layer.
- Examples
- Compares ReLU grid resolution and measures the effect of omitting correction for shifted activations.
- What you can do after
- Evaluate quantization configurations for wasted codes, accuracy loss, and zero-point correction overhead.
Read how Quantization and Fast Inference explains asymmetric quantization →
CUDA for LLMs
Asymmetric quantization maps a numerical range to quantization levels without requiring the range to be symmetric around zero.
- How it is used
- Fits integer levels to skewed activation or embedding distributions to avoid wasting precision on unused values.
- How deep it goes
- Explains scale and zero-point arithmetic with CUDA quantization and dequantization kernels, calibration choices, and implementation tradeoffs.
- Examples
- Uses UINT8 ReLU activations from 0 to 10 and a calibration dataset with a large outlier.
- What you can do after
- Build quantization and dequantization kernels from scratch and evaluate their model-level accuracy and speed.
What to know first
- activationAn activation is a numerical value or tensor produced by a neural-network unit, layer, or operation during a forward computation.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- mean absolute errorMean absolute error is the arithmetic mean of the absolute differences between corresponding predicted and observed numerical values.
- mean squared errorMean squared error is the arithmetic mean of the squared differences between corresponding predicted and reference numerical values.
Related concepts
- symmetric quantizationcontrasts with · Symmetric quantization maps values to integers using a zero-centered representation and a scale without an offset.
- hybrid consensusused by · Hybrid Consensus is a quantization strategy that applies symmetric quantization to weights and asymmetric quantization to activations.
- zero-pointuses · A quantization zero-point is the integer value that represents real zero in an affine mapping between real values and quantized integer values.
- quantization scaleuses · A quantization scale is a multiplicative factor relating quantized values or steps to the numerical values they represent.
- quantizationkind of · Model quantization represents model parameters or computations with reduced-precision numerical values, often to lower storage or computational requirements.
Questions about asymmetric quantization
What is asymmetric quantization?
Asymmetric quantization maps values to discrete levels using a scale and a zero point, allowing the represented range to be shifted relative to zero.
Which Manning books explain asymmetric quantization?
Quantization and Fast Inference and CUDA for LLMs.
What should I know before learning asymmetric quantization?
Manning books that teach asymmetric quantization usually introduce activation, accuracy, mean absolute error and mean squared error first.
What is the difference between asymmetric quantization and symmetric quantization?
Asymmetric quantization maps values to discrete levels using a scale and a zero point, allowing the represented range to be shifted relative to zero. Symmetric quantization maps values to integers using a zero-centered representation and a scale without an offset.
What is asymmetric quantization a kind of?
Asymmetric quantization is a kind of quantization. Model quantization represents model parameters or computations with reduced-precision numerical values, often to lower storage or computational requirements.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

