liveBook · Manning

Asymmetric quantization

Asymmetric quantization maps values to discrete levels using a scale and a zero point, allowing the represented range to be shifted relative to zero.

How 2 Manning books explain it

Quantization and Fast Inference

Asymmetric quantization maps values to a grid whose range and zero-point can be shifted to fit a non-centered distribution.

How it is used
Matches shifted integer grids to non-centered activations while retaining symmetric quantization for centered weights.
How deep it goes
Derives affine calibration and zero-point correction, then implements and measures a hybrid linear layer.
Examples
Compares ReLU grid resolution and measures the effect of omitting correction for shifted activations.
What you can do after
Evaluate quantization configurations for wasted codes, accuracy loss, and zero-point correction overhead.

Read how Quantization and Fast Inference explains asymmetric quantization →

CUDA for LLMs

Asymmetric quantization maps a numerical range to quantization levels without requiring the range to be symmetric around zero.

How it is used
Fits integer levels to skewed activation or embedding distributions to avoid wasting precision on unused values.
How deep it goes
Explains scale and zero-point arithmetic with CUDA quantization and dequantization kernels, calibration choices, and implementation tradeoffs.
Examples
Uses UINT8 ReLU activations from 0 to 10 and a calibration dataset with a large outlier.
What you can do after
Build quantization and dequantization kernels from scratch and evaluate their model-level accuracy and speed.

Read how CUDA for LLMs explains asymmetric quantization →

What to know first

Related concepts

Questions about asymmetric quantization

What is asymmetric quantization?

Asymmetric quantization maps values to discrete levels using a scale and a zero point, allowing the represented range to be shifted relative to zero.

Which Manning books explain asymmetric quantization?

Quantization and Fast Inference and CUDA for LLMs.

What should I know before learning asymmetric quantization?

Manning books that teach asymmetric quantization usually introduce activation, accuracy, mean absolute error and mean squared error first.

What is the difference between asymmetric quantization and symmetric quantization?

Asymmetric quantization maps values to discrete levels using a scale and a zero point, allowing the represented range to be shifted relative to zero. Symmetric quantization maps values to integers using a zero-centered representation and a scale without an offset.

What is asymmetric quantization a kind of?

Asymmetric quantization is a kind of quantization. Model quantization represents model parameters or computations with reduced-precision numerical values, often to lower storage or computational requirements.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.