liveBook · Manning

Superblock

As explained in Quantization and Fast Inference

A superblock is a group of weights in a K-quant scheme that has an overall scale and contains smaller groups with their own scales.

In K-quants, a superblock groups 256 weights under one FP16 super-scale. Sixteen INT8 sub-scales capture local variation, one for each 16-weight chunk. section 10.2.1

This hierarchy lets K-quants represent weight variation with fewer raw bits than a flat block using one scale. Q6_K stores each 256-weight superblock in 210 bytes. section 10.2.1 section 10.2.5

Reconstructing a Q6_K weight

For weight 0, low bits 1101 and high bits 01 form code 29. Subtracting 32 gives −3; applying sub-scale 94 and super-scale −1.14 × 10⁻⁵ reconstructs approximately +3.21 × 10⁻³. section 10.2.5

Listing 10.4 Q6_K superblock dump for blk.0.ffn_down.weight in llama-2-7b.Q4_K_M.gguf
─── Q6_K superblock layout (210 bytes / 256 weights) ─────────────────
0x00   ql        (128 B: low 4 bits of 256 weights)    dd fe e8 99 ...               #A
0x80   qh        (64 B: high 2 bits of 256 weights)    e9 aa 79 ae ...               #B
0xc0   scales    (16 B: 16 INT8 sub-scales)            5e 80 64 a0 ...
0xd0   d         (FP16 super-scale)                    bf 80                         #C
 
  Decoded super-scale d        = 0x80bf -> approx -1.14e-5
  scales[0]                    = 94
  Reconstructed weight[0]      = +0.003210
    (formula: w = d * scales[i // 16] * ((low4 | (high2 << 4)) - 32))
 

Inspect the Q6_K byte regions and decoded values used in the reconstruction example. section 10.2.5

How Quantization and Fast Inference covers it

How deep it goes
Explains the two-level scale structure and traces Q6_K bytes through signed-code and scale reconstruction.
Examples
Uses a Q6_K byte dump to reconstruct one weight from its packed bits and scales.
What you can do after
Understand how Q6_K packed bytes and hierarchical scales reconstruct a weight value.
How it connects
Connects K-quants to quantization, dequantization, and GGUF weight storage.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.