liveBook · Manning

Codebook

As explained in Quantization and Fast Inference

A quantization codebook is a collection of representative values used to encode and reconstruct quantized inputs.

A quantization codebook is a lookup table of representative values. In TurboQuant, it replaces each rotated vector coordinate with the representative value for its bin. section 7.5

TurboQuant uses the codebook as part of a three-step quantization pipeline:

  • Normalize each vector by its L2 norm and store the norm in FP16. section 7.5
  • Rotate the normalized vector so its coordinates follow a known Beta distribution. section 7.5
  • Find each coordinate’s bin, store its index, and use the representative value to reconstruct it. section 7.5

A 3-bit codebook

Three bits select one of eight representative values across [−1, 1]. The values are spaced unevenly, with more entries near zero, where rotated coordinates are concentrated. section 7.5

Listing 7.13 TurboQuant vector quantization (Algorithm 1)
class TurboQuantizer:
    def __init__(self, head_dim, codebooks, seed=42, device="cuda"):
        self.head_dim = head_dim
        gen = torch.Generator()
        gen.manual_seed(seed)
        G = torch.randn(head_dim, head_dim, generator=gen)
        Pi, _ = torch.linalg.qr(G)                          #A
        self.Pi = Pi.to(device)
        self.centroids = {}
        self.boundaries = {}
        for bits, cb in codebooks.items():
            self.centroids[bits] = torch.tensor(
                cb["centroids"], dtype=torch.float32, device=device)
            self.boundaries[bits] = torch.tensor(
                cb["boundaries"], dtype=torch.float32, device=device)
 
    def quantize(self, vectors, bits):
        shape = vectors.shape
        flat = vectors.reshape(-1, self.head_dim).float()
        norms = torch.norm(flat, dim=1, keepdim=True)        #B
        normalized = flat / torch.clamp(norms, min=1e-8)     #C
        rotated = normalized @ self.Pi.T                     #D
        interior_b = self.boundaries[bits][1:-1]
        indices = torch.searchsorted(interior_b, rotated)    #E
        return {"indices": indices, "norms": norms.half(),
                "bits": bits, "shape": shape}
 
    def dequantize(self, compressed):
        c = self.centroids[compressed["bits"]]
        recon_rot = c[compressed["indices"]]                 #F
        recon_norm = recon_rot @ self.Pi                     #G
        recon = recon_norm * compressed["norms"].float()     #H
        return recon.reshape(compressed["shape"])
 

Inspect the TurboQuant implementation, including its seeded rotation and codebook centroid and boundary setup. section 7.5

How Quantization and Fast Inference covers it

How deep it goes
Details normalization, rotation, centroid lookup, inverse reconstruction, and the Lloyd–Max principle for deriving representatives.
Examples
Uses a 3-bit, eight-entry codebook over [−1, 1], with representatives concentrated near zero for rotated coordinates.
What you can do after
Apply TurboQuant to quantize and reconstruct KV vectors through centroid lookup and inverse rotation.
How it connects
Links codebooks to vector quantization, KV caches, random rotations, and the Lloyd–Max algorithm.

Also explained in

Compare how 3 books explain codebook →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.