Bits per weight (bpw) expresses a quantized format’s average storage budget per model weight. It includes block scales and metadata, so a format’s effective bpw can differ from its nominal bit width. section 10.2.1
GGUF formats span a wide range of effective bit budgets:
- Legacy Q4_0 uses 4.50 bpw; hierarchical Q2_K uses 2.62 bpw and Q6_K uses 6.56 bpw. section 10.2.1
- IQ1_S reaches 1.56 bpw; recipe variants such as Q4_K_M can use 0.2–0.4 bpw more than their base format because selected tensors are promoted. section 10.2.1
Lower bpw reduces model storage and bytes read during inference, but format choice also affects quality and runtime performance. Compare measured quality and hardware behavior rather than treating bpw alone as a ranking. section 12.4 section 10.3.4
Figure 10.2 GGUF quantization formats by bits per weight, including per-block scale and metadata overhead. Legacy quants (dotted) span Q4_0 (4.50 bpw) through Q8_0 (8.50). K-quants (diagonal) span Q2_K (2.62) through Q6_K (6.56). Bars show each base format's bpw; the recipe-level M-variants (Q3_K_M, Q4_K_M, Q5_K_M) sit 0.2–0.4 bpw above their base because of bumped tensors, as 10.2.4 makes explicit. IQ-quants (cross) span IQ1_S (1.56) through IQ4_XS (4.25). Importance-matrix calibration pushes below the k-quant floor. Reference dashes mark INT4 and INT8.

Compare effective bpw ranges across legacy, K-quant, and IQ formats, including overhead and recipe-level variants. section 10.2.1
