A sub-scale adjusts a smaller chunk of weights within hierarchical quantization. In Q6_K, each INT8 sub-scale covers 16 weights and combines with a shared FP16 super-scale and each weight’s signed code. section 10.2.5
Reconstructing a Q6_K weight
For weight 0, the packed bits produce code 29, which becomes −3 after subtracting 32. With sub-scale 94 and super-scale −1.14 × 10⁻⁵, the reconstructed weight is approximately +3.21 × 10⁻³. section 10.2.5
Listing 10.4 Q6_K superblock dump for blk.0.ffn_down.weight in llama-2-7b.Q4_K_M.gguf
─── Q6_K superblock layout (210 bytes / 256 weights) ─────────────────
0x00 ql (128 B: low 4 bits of 256 weights) dd fe e8 99 ... #A
0x80 qh (64 B: high 2 bits of 256 weights) e9 aa 79 ae ... #B
0xc0 scales (16 B: 16 INT8 sub-scales) 5e 80 64 a0 ...
0xd0 d (FP16 super-scale) bf 80 #C
Decoded super-scale d = 0x80bf -> approx -1.14e-5
scales[0] = 94
Reconstructed weight[0] = +0.003210
(formula: w = d * scales[i // 16] * ((low4 | (high2 << 4)) - 32))
Inspect a Q6_K superblock’s packed weight bits, sub-scales, super-scale, and reconstructed weight. section 10.2.5
