liveBook · Manning

Chapter 3: Choosing What to Quantize and at What Granularity

What you will learn in chapter 3 of Quantization and Fast Inference

This introduction explains why weights, activations, and the KV cache need different quantization choices and why scale granularity must balance accuracy with practical constraints.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains why weights, activations, and the KV cache need different quantization choices and why scale granularity must balance accuracy with practical constraints.

Key terms: weights, activations, KV cache, outliers, per-tensor, per-channel, group-wise, scale

3.1 Identify where precision loss matters

This section compares the precision risks and benefits of quantizing weights, activations, and transformer KV caches, then shows how to identify particularly sensitive layers.

Key terms: weights, symmetric quantization, asymmetric quantization, zero-point, weight quantization, model size, embedding tables, classification heads, bias vectors, activations

3.2 Quantize weights with per-tensor and per-channel choices

This section compares per-tensor and per-channel weight quantization, showing why channel-specific scales usually preserve more precision at little memory cost.

Key terms: per-tensor quantization, per-channel quantization, scale, output channel, INT8, outlier penalty, integer range utilization, quantization error, scale metadata, weight-only quantization

3.3 Quantize activations under outliers and dynamic ranges

This section explains why input-dependent ranges and outliers make activation quantization difficult, then compares static, dynamic, per-token, and mixed-precision responses.

Key terms: activation statistics, range ratio, outliers, scale, quantization levels, static quantization, dynamic quantization, calibration, observer, scale factors

3.4 Quantize transformer key-value cache with mixed precision

This section explains why transformer keys and values benefit from different KV-cache quantization granularities, tests that choice, and shows its scale layout and storage cost.

Key terms: KV cache, query, key, value, attention weights, per-channel, per-token, outlier channels, outlier tokens, MSE

3.5 Use group and block schemes and understand memory layout

The section explains how group and block quantization trade scale granularity against accuracy and metadata cost, and why memory layout and INT4 packing matter for efficient inference.

Key terms: granularity, per-channel, per-group, INT4, MSE, scale factor, group quantization, group size, input dimension, scale factors

3.6 Follow a decision checklist for granularity

This section turns quantization choices into a checklist for selecting targets, granularity, and bit-width, then checking architecture needs, runtime support, and common failure signs.

Key terms: target selection, granularity, scale factor scope, bit-width, hardware compatibility gate, weights, activations, calibration data, outlier channels, KV cache

3.7 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 2: Building Quantization from First Principles Chapter 4: Applying Post-Training Quantization and Calibration →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.