liveBook · Manning

Quantized model

A quantized model is a machine-learning model whose parameters or computations use reduced-precision numerical representations.

How 2 Manning books explain it

Rearchitecting LLMs

A quantized model is a neural model whose parameters or computational values are represented using reduced-precision numerical formats.

How it is used
Uses a 4-bit model as the base for QLoRA fine-tuning on clinical data.
How deep it goes
Explains gradient preparation and frozen base weights, but not the quantization mechanism itself.
Examples
Trains on clinical notes paired with validated JSON extraction targets.
What you can do after
Apply Hugging Face PEFT’s `prepare_model_for_kbit_training` before configuring and training adapters.

Read how Rearchitecting LLMs explains quantized model →

Local Image Generation

A quantized model is a machine-learning model whose parameters or computations use reduced numerical precision.

How it is used
Enables large Flux models to run on constrained hardware by reducing the numerical storage required during inference.
How deep it goes
Explains reduced-bit storage through rounding and applies it to FP8 model loading, without detailing quantization algorithms.
Examples
Uses decimal multiplication, an fp8 safetensors file, and an output-quality comparison.
What you can do after
Operate ComfyUI by selecting a quantized UNET when standard Flux model loading exceeds available hardware.

Read how Local Image Generation explains quantized model →

Related concepts

Questions about quantized model

What is quantized model?

A quantized model is a machine-learning model whose parameters or computations use reduced-precision numerical representations.

Which Manning books explain quantized model?

Rearchitecting LLMs and Local Image Generation.

What is the difference between quantized model and 16-bit precision?

A quantized model is a machine-learning model whose parameters or computations use reduced-precision numerical representations. 16-bit precision represents numerical values using 16 bits per value.

What is quantized model a kind of?

Quantized model is a kind of model. A model is an abstract representation of a system, phenomenon, application, or domain that captures relevant structure, behavior, data, or beliefs.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.