liveBook · Manning

Quantization

As explained in Vibe Engineering

Quantization is the representation of model parameters with fewer numerical bits to reduce model storage and memory requirements.

Quantization reduces model storage and memory pressure by rounding parameters from higher precision to fewer bits. Q8 uses eight bits, Q4 uses four; smaller files fit on constrained machines but sacrifice some accuracy. section 9.3.1

The engineering dilemma is not simply quality versus size. Memory saved by quantization can keep an IDE, browser, containers, and test runner usable, while excessive reduction can damage reasoning and coherent code generation. section 9.3.1 section 9.3.5

Choose a level according to workload and available RAM:

  • Start with Q4_K_M for most developer workloads; it balances model size, quality, and memory headroom. section 9.3.5
  • Prefer Q6_K when the machine has ample RAM and quality matters more than the additional model size. section 9.3.5
  • Avoid going below Q4 for function-length generation or test-suite creation, where variable consistency and control-flow correctness degrade noticeably. section 9.3.5
  • Treat Q2 and Q3 as constrained-memory options rather than general defaults; short tab completion may tolerate their losses better than multi-step generation. section 9.3.5

A practical download decision

A 16GB MacBook Pro running Qwen 2.5 Coder 7B at Q4_K_M can enter swap because the model shares unified memory with the development environment. On a 32GB machine, the same small-model setup is more comfortable; with 64GB or more, Q6_K becomes a stronger quality-first option. section 9.3.1 section 9.3.5

Table 9.5 Quantization levels for local coding - what to download
Level
Size (14B model)
RAM Needed
Quality*
Speed
Recommendation
FP16
28 GB
32+ GB
100%
1.0x
Only for eval
Q8
15 GB
20+ GB
~99%
1.3x
If RAM allows
Q6_K
12 GB
16+ GB
~97%
1.5x
Quality-first
Q4_K_M
8.5 GB
12+ GB
~94%
1.8x
Sweet spot ←
Q4_K_S
8.0 GB
11+ GB
~93%
1.9x
Slightly smaller
Q3_K_M
6.5 GB
9+ GB
~88%
2.0x
RAM-constrained
Q2_K
5.0 GB
7+ GB
~78%
2.2x
Last resort

Compare quantization levels by model size, RAM requirement, estimated quality, speed, and recommended use. section 9.3.5

How Vibe Engineering covers it

How deep it goes
Explains bit reduction, memory effects, quality trade-offs, and workload thresholds, then stops short of quantization internals.
Examples
Uses Q4, Q4_K_M, Q6_K, Qwen 2.5 Coder 7B, Mac memory limits, and a quantization comparison table.
What it assumes
Assumes basic familiarity with model downloads and RAM, while defining parameter precision and quantization labels from first principles.
What you can do after
Operate local models through Ollama or Hugging Face downloads by matching quantization levels to RAM and generation workloads.
How it connects
Connects model parameters, local inference, RAM, tab completion, classification, and multi-step code generation.

Also explained in

Compare how 22 books explain quantization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.