liveBook · Manning

Quantized model

As explained in Local Image Generation

A quantized model is a machine-learning model whose parameters or computations use reduced numerical precision.

A quantized model stores numerical values with fewer bits. Think of it as writing numbers with fewer decimal places: the result may change slightly, but the files use less space and VRAM. section 10.1.3

In a Flux workflow, quantization helps when the model is too large for available hardware:

  • Choose a quantized UNET through the Load Diffusion Model node instead of loading an entire checkpoint. section 10.1.3
  • Select a file such as flux1-dev-fp8.safetensors; “fp8” identifies its reduced numerical precision. section 10.1.3
  • Expect similar output with a possible small quality difference, while reducing the storage burden on constrained hardware. section 10.1.3

A smaller numerical footprint

Rounding 3.14159 and 2.71828 to 3.14 and 2.72 produces 8.54 instead of a more precise product. Quantization applies the same space-saving idea to model numbers. section 10.1.3

Figure 10.5 Using the fp8 model only minorly impacts the quality of our output.

Compare the fp8 model output with the regular version to see the practical quality trade-off. section 10.1.3

How Local Image Generation covers it

How deep it goes
Explains reduced-bit storage through rounding and applies it to FP8 model loading, without detailing quantization algorithms.
Examples
Uses decimal multiplication, an fp8 safetensors file, and an output-quality comparison.
What it assumes
Assumes familiarity with loading model components and introduces numerical precision through a decimal-rounding analogy.
What you can do after
Operate ComfyUI by selecting a quantized UNET when standard Flux model loading exceeds available hardware.
How it connects
Connects quantization with Flux, UNET loading, VRAM, model checkpoints, and output quality.

Also explained in

Compare how 2 books explain quantized model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Local Image Generation. The section links point to the parts of the book it draws on.