liveBook · Manning

Quantization

Model quantization represents model parameters or computations with reduced-precision numerical values, often to lower storage or computational requirements.

How 14 Manning books explain it

LLMs in Production

Quantization is the conversion of model parameters or computations from higher-precision numerical representations to lower-precision representations.

How it is used
Reduces model memory requirements to make LLM serving feasible on constrained hardware.
How deep it goes
Explains precision and deployment trade-offs, then demonstrates post-training quantization and format selection without deriving floating-point arithmetic.
Examples
Uses an INT8 PyTorch model and a Q4_K_M 7B GGUF model on a Raspberry Pi.
What you can do after
Apply it via PyTorch or llama.cpp and compare quantization formats against memory and quality constraints.

Read how LLMs in Production explains quantization →

Domain-Specific Small Language Models

Methods that reduce numeric precision or representational size of model values to improve efficiency, especially for inference; here the absmax method is a specific quantization procedure.

How it is used
Reduces LLM inference memory requirements while targeting practical latency, throughput, and deployment constraints.
How deep it goes
Works through 8-bit scaling and reconstruction, then applies 4-bit libraries; defers GPTQ’s low-level mathematics to its implementation sources.
Examples
Uses GPT-2 absmax weight comparisons, GPT-2 XL AutoGPTQ configuration, and an 8-bit antibody-model latency benchmark.
What you can do after
Apply it via AutoGPTQ, ggml, or Bitsandbytes, then evaluate model size, inference latency, and output quality.

Read how Domain-Specific Small Language Models explains quantization →

CUDA for LLMs

Quantization represents numerical values using a limited set of discrete levels, often to reduce model storage and computation costs.

How it is used
Reduces model storage and memory traffic to make inference practical on constrained GPUs.
How deep it goes
Works through scaling, rounding, clamping, CUDA kernels, granularity, calibration, and deployment tradeoffs.
Examples
Uses a four-weight INT8 conversion and packed INT4 kernels to illustrate numerical mapping and storage.
What you can do after
Build quantization and dequantization kernels from scratch, then evaluate accuracy and profile inference overhead.

Read how CUDA for LLMs explains quantization →

AI-Powered Search

Quantization is the transformation of numerical values into a smaller set of discrete values to reduce representation precision and resource requirements.

How it is used
Reduces embedding storage and similarity-search costs in dense vector retrieval.
How deep it goes
Explains scalar range mapping and benchmarks quantized indexes, including a combined optimization pipeline.
Examples
Uses outdoors post embeddings and queries such as “tent poles” to compare full-precision and Int8 indexes.
What you can do after
Apply it via FAISS and sentence-transformers to index embeddings and evaluate size, speed, and recall.

Read how AI-Powered Search explains quantization →

Deep Learning with Python, Third Edition

The technique of reducing numeric precision of model weights or activations to speed up computation while preserving output behavior as much as possible.

How it is used
Speeds up and shrinks inference models for deployment under memory, power, or latency constraints.
How deep it goes
Works through abs-max scaling, int8 matrix multiplication, output rescaling, and rounding error before recommending the Keras API.
Examples
Uses a small matrix multiplication to compare quantized and float32 outputs, then calls model.quantize("int8").
What you can do after
Apply it via Keras to an already trained model for inference and assess the accuracy tradeoff.

Read how Deep Learning with Python, Third Edition explains quantization →

Deep Learning with R, Third Edition

Model quantization is the conversion of model computations or parameters to lower-precision numerical representations to reduce resource requirements.

How it is used
Uses quantization to reduce deployment resource requirements and speed up inference.
How deep it goes
Works through scaling, rounding, int8 matrix multiplication, and unscaling before recommending the built-in API.
Examples
Uses a small matrix multiplication to compare quantized and float32 results, plus a Keras prediction workflow.
What you can do after
Apply it via Keras’s quantize_weights() to a trained model before prediction.

Read how Deep Learning with R, Third Edition explains quantization →

Math and Architectures of Deep Learning

The conversion of floating-point region boundaries or ratios into integer-aligned indices for feature extraction.

How it is used
Uses discretization to represent continuous values or align variable-sized RoIs with fixed pooling grids.
How deep it goes
Defines bucketization and integer rounding, explains their effects on probability distributions and RoI pooling, and defers RoIAlign details.
Examples
Uses height and weight buckets, fractional RoI dimensions, and fixed-size pooled feature maps as concrete examples.
What you can do after
Apply it via RoI pooling to turn variable-sized regions into fixed-sized feature representations.

Read how Math and Architectures of Deep Learning explains quantization →

Generative AI in Action

Quantization is the reduction of numerical precision used to represent model parameters, typically to reduce memory use and computational cost.

How it is used
Supports efficient LLM deployment and inference when memory, computation, energy, or GPU capacity constrain operations.
How deep it goes
Defines reduced-precision representation and explains benefits and risks, but stops short of a quantization workflow or hardware-specific implementation.
Examples
Uses Code Llama on a consumer GPU, QLoRA low-rank matrices, and DistilBERT compared with BERT.
What you can do after
Evaluate precision, memory, inference, energy, accuracy, and hardware-efficiency trade-offs when selecting quantized models.

Read how Generative AI in Action explains quantization →

Deep Learning with PyTorch, Second Edition

A model optimization technique that reduces numerical precision of parameters and operations to lower memory use and often speed up inference.

How it is used
Optimizes deployed models by reducing parameter and operation footprints for inference.
How deep it goes
Defines 8-bit conversion and demonstrates both PyTorch API usage and custom linear-layer implementation.
Examples
Uses serialized model-size measurement, custom int8 linear layers, and PyTorch dynamic quantization.
What you can do after
Apply it via PyTorch’s dynamic quantization API or a custom int8 implementation.

Read how Deep Learning with PyTorch, Second Edition explains quantization →

Transformers in Action

Quantization is the transformation of numerical model parameters or activations into lower-precision representations to reduce memory consumption or computational cost.

How it is used
Reduces large-language-model inference and fine-tuning resource demands through lower-precision weights and activations.
How deep it goes
Defines precision changes, quantifies memory savings, identifies range and accuracy risks, and demonstrates configuration without detailed quantization-error analysis.
Examples
Uses a data-type comparison table and a bitsandbytes configuration for loading Falcon 7B while measuring GPU memory.
What you can do after
Operate/configure bitsandbytes to load a 4-bit Falcon 7B model and measure its GPU memory impact.

Read how Transformers in Action explains quantization →

The Complete Obsolete Guide to Generative AI

A quantized model is a machine-learning model whose numerical parameters or activations are represented at reduced precision to lower resource requirements.

How it is used
Enables local LLM deployment with less hardware by compressing models and preparing quantized weights.
How deep it goes
Demonstrates building, downloading, launching, and experimenting with quantized weights through command-line tooling.
Examples
Uses a RedPajama chat build, quantized weight download, launch command, and q4_1 output-type experiment.
What you can do after
Operate/configure it: build a compressed chat environment, obtain quantized weights, and test quantization-script arguments.

Read how The Complete Obsolete Guide to Generative AI explains quantization →

Time Series Forecasting Using Foundation Models

Quantization is the conversion of continuous numerical values into a finite set of discrete levels or tokens.

How it is used
Converts continuous time-series observations into language-model-compatible tokens for probabilistic forecasting.
How deep it goes
Explains uniform binning, clipping, precision loss, and decoding, while deferring the rationale for the clipping range.
Examples
Uses scaled time-series plots and contrasts 100-bin and 5-bin vocabularies to show retained shape and lost precision.
What you can do after
Apply uniform binning to represent scaled time-series values as tokens and recognize the resulting precision trade-off.

Read how Time Series Forecasting Using Foundation Models explains quantization →

Natural Language Processing in Action, Second Edition

Quantization is the transformation of numerical values into lower-precision discrete representations, often to reduce memory use and accelerate computation.

How it is used
Accelerates semantic-search indexing by shrinking vectors and replacing floating-point range work with faster integer comparisons.
How deep it goes
Defines scaling, rounding, integer storage, and product quantization, but stops short of detailed clustering or index implementation.
Examples
Demonstrates NumPy vectors, integer scaling, precision levels, and approximate-nearest-neighbor results on the New York Times dataset.
What you can do after
Apply it via NumPy-style scaling and integer conversion, and recognize product-quantization patterns in semantic-search indexes.

Read how Natural Language Processing in Action, Second Edition explains quantization →

Building Reliable AI Systems

Quantization reduces the numerical precision used to represent model values, often lowering storage or computation requirements.

How it is used
Reduces model-weight storage requirements in QLoRA fine-tuning and supports production-scale deployment.
How deep it goes
Explains 4-bit weights, NF4, double quantization, and higher-precision calculations, but does not detail implementation steps.
Examples
Uses an image-compression analogy and describes fine-tuning 7-billion- and 70-billion-parameter models on single GPUs.
What you can do after
Apply 4-bit quantization through QLoRA to reduce memory demands when fine-tuning large models.

Read how Building Reliable AI Systems explains quantization →

Related concepts

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.