In K-quants, a superblock groups 256 weights under one FP16 super-scale. Sixteen INT8 sub-scales capture local variation, one for each 16-weight chunk. section 10.2.1
This hierarchy lets K-quants represent weight variation with fewer raw bits than a flat block using one scale. Q6_K stores each 256-weight superblock in 210 bytes. section 10.2.1 section 10.2.5
Reconstructing a Q6_K weight
For weight 0, low bits 1101 and high bits 01 form code 29. Subtracting 32 gives −3; applying sub-scale 94 and super-scale −1.14 × 10⁻⁵ reconstructs approximately +3.21 × 10⁻³. section 10.2.5
Listing 10.4 Q6_K superblock dump for blk.0.ffn_down.weight in llama-2-7b.Q4_K_M.gguf
─── Q6_K superblock layout (210 bytes / 256 weights) ─────────────────
0x00 ql (128 B: low 4 bits of 256 weights) dd fe e8 99 ... #A
0x80 qh (64 B: high 2 bits of 256 weights) e9 aa 79 ae ... #B
0xc0 scales (16 B: 16 INT8 sub-scales) 5e 80 64 a0 ...
0xd0 d (FP16 super-scale) bf 80 #C
Decoded super-scale d = 0x80bf -> approx -1.14e-5
scales[0] = 94
Reconstructed weight[0] = +0.003210
(formula: w = d * scales[i // 16] * ((low4 | (high2 << 4)) - 32))
Inspect the Q6_K byte regions and decoded values used in the reconstruction example. section 10.2.5
