liveBook · Manning

Chapter 2: Building Quantization from First Principles

What you will learn in chapter 2 of Quantization and Fast Inference

This chapter introduction frames quantization as the translation from floating-point neural-network values to the small integers used for inference.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This chapter introduction frames quantization as the translation from floating-point neural-network values to the small integers used for inference.

Key terms: floating-point, quantization, integer levels

2.1 The fixed-point world

This section explains how fixed-point numbers are encoded and computed, why their fixed grid creates range and rounding tradeoffs, and how floating-point uses a movable grid instead.

Key terms: fixed-point, Q-format, scale factor, Q3.4, two's complement, sign bit, Overflow Cliff, -1 Bridge, wide_product, right shift

2.2 Define the affine mapping with scale and zero point

This section derives affine quantization from scale and zero-point, then explains its accuracy benefits, exact-zero constraint, granularity choices, and arithmetic costs.

Key terms: fixed-point, affine quantization, scale, zero-point, ReLU, dequantization, quantization, scale factor S, zero-point Z, rmin

2.3 Characterize granular and overload errors

This section explains the two kinds of quantization error and how clipping boundaries trade rounding precision against clipping risk.

Key terms: granular error, overload error, step size S, clipping, clipping boundary, percentile-based calibration, outliers, LLMs, naive INT8, scale

2.4 Compare symmetric and asymmetric choices

This section explains why the shape of weight and activation distributions leads to a practical choice of symmetric weights and asymmetric activations.

Key terms: weight distributions, bell-shaped distribution, Symmetric Quantization, activations, ReLU, asymmetric quantization, bit-depth, Hybrid Consensus, symmetric weights, asymmetric activations

2.5 End-to-end INT8 inference in PyTorch

The section implements hybrid INT8 quantization in a PyTorch linear layer and uses grid visualizations and FP32 comparisons to explain code allocation, output error, and zero-point correction.

Key terms: symmetric quantization, absolute maximum, scale, zero-point, [-127, 127], affine quantizer, asymmetric quantization, zero-point nudge, ReLU Trap, granular error

2.6 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 1: Facing the Efficiency Wall Chapter 3: Choosing What to Quantize and at What Granularity →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.