liveBook · Manning

Quantization

As explained in AI Model Evaluation

A model optimization technique that represents model values with lower-precision numerical formats to reduce computation, memory use, or latency.

Quantization represents model values in lower-precision numerical formats. It is one option for reducing model inference latency: the time a model takes to process inputs and produce predictions. section 4.2.2 section 4.4.2

Before optimizing, measure end-to-end response time and break it down by stage, including feature retrieval, model inference, tool calls, network transfer, and rendering. If inference accounts for most of the delay, quantization may help. If feature retrieval or backend calls dominate, database optimization, caching, or fewer service calls may have more impact. section 4.4.2

Quantization is one of several inference optimizations, alongside pruning, distillation, and hardware selection. Batching can improve throughput but reduce responsiveness, so the choice of optimizations should reflect the product’s latency and engineering requirements. section 4.2.2

How AI Model Evaluation covers it

How deep it goes
Identifies quantization as an optimization candidate but does not explain its implementation.
Examples
Uses a latency breakdown to distinguish inference bottlenecks from feature retrieval and backend delays.
What you can do after
Evaluate whether quantization is a candidate when model inference dominates response time.
How it connects
Connects quantization to inference latency, pruning, distillation, hardware selection, and batching.

Also explained in

Compare how 22 books explain quantization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.