liveBook · Manning

Static quantization

Static quantization determines quantization parameters from representative data before inference and keeps them fixed during inference.

How 2 Manning books explain it

CUDA for LLMs

Static quantization computes activation quantization parameters before inference using calibration data and reuses them for subsequent inputs.

How it is used
Uses precomputed activation parameters to avoid calibration reductions during inference.
How deep it goes
Details calibration, parameter storage, granularity, validation, and profiling, with guidance for choosing alternatives when accuracy suffers.
Examples
Uses a customer-support language model and a transformer with different quantization granularities across layers.
What you can do after
Evaluate and configure a static quantization workflow using representative calibration data, separate accuracy tests, and profiling.

Read how CUDA for LLMs explains static quantization →

Quantization and Fast Inference

Static quantization determines quantization ranges during calibration before deployment and keeps the resulting parameters fixed at runtime.

How it is used
Sets activation quantization ranges before deployment to avoid computing them during inference.
How deep it goes
Explains calibration, observer-based range collection, fixed scales, and accuracy risks under distribution shift.
Examples
Uses BERT activation percentiles and a static-versus-dynamic MSE comparison across typical, narrow, wide, and outlier-heavy inputs.
What you can do after
Configure it by calibrating representative inputs, freezing scales, and checking quantization scope and runtime precision on the deployment host.

Read how Quantization and Fast Inference explains static quantization →

What to know first

Related concepts

Questions about static quantization

What is static quantization?

Static quantization determines quantization parameters from representative data before inference and keeps them fixed during inference.

Which Manning books explain static quantization?

CUDA for LLMs and Quantization and Fast Inference.

What should I know before learning static quantization?

Manning books that teach static quantization usually introduce mean squared error first.

What is the difference between static quantization and dynamic quantization?

Static quantization determines quantization parameters from representative data before inference and keeps them fixed during inference. Dynamic quantization is a method that determines or adjusts quantization parameters at runtime from the values being processed, often quantizing activations during inference.

What is static quantization a kind of?

Static quantization is a kind of quantization. Model quantization represents model parameters or computations with reduced-precision numerical values, often to lower storage or computational requirements.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.