Q4_K_M is a llama.cpp quantization format used in GGUF model files. Its canonical file-type code is 15; measured at 3.89 GB with perplexity 5.43, it provides a compact option whose quality and runtime behavior should be checked on the target hardware. section 10.3.4 section 10.4.1
Compare the format against deployment constraints:
- In one Wikitext-2 comparison, IQ4_XS measured 3.45 GB and PPL 5.36, versus Q4_K_M at 3.89 GB and PPL 5.43. The test used a short slice with a ±0.47 standard-error band, so it does not establish a statistically reliable difference. section 10.3.4
- On measured x86 kernel families, IQ4_XS decoded 6–13% faster. Q4_K_M remains a safer fallback when runtimes cannot read 16-entry codebooks, including some older GPU kernels, pre-Skylake CPUs, and embedded targets without efficient byte-permute. section 10.4.5
Figure 10.5 Wikitext-2 perplexity (4 chunks × 512 tokens, CPU-only) versus on-disk size. F16 (12.85 GB, PPL 5.27) anchors the lossless baseline. F16 → Q8_0 → Q5_K_M is a clean staircase down. Q4_K_M sits to the upper side of Q5_K_M as expected. IQ4_XS sits to the lower-left of Q4_K_M—smaller and lower perplexity at the same bit tier.

Compare on-disk size and Wikitext-2 perplexity across formats; the figure places Q4_K_M above and to the right of IQ4_XS. section 10.3.4

