A quantization codebook is a lookup table of representative values. In TurboQuant, it replaces each rotated vector coordinate with the representative value for its bin. section 7.5
TurboQuant uses the codebook as part of a three-step quantization pipeline:
- Normalize each vector by its L2 norm and store the norm in FP16. section 7.5
- Rotate the normalized vector so its coordinates follow a known Beta distribution. section 7.5
- Find each coordinate’s bin, store its index, and use the representative value to reconstruct it. section 7.5
A 3-bit codebook
Three bits select one of eight representative values across [−1, 1]. The values are spaced unevenly, with more entries near zero, where rotated coordinates are concentrated. section 7.5
Listing 7.13 TurboQuant vector quantization (Algorithm 1)
class TurboQuantizer:
def __init__(self, head_dim, codebooks, seed=42, device="cuda"):
self.head_dim = head_dim
gen = torch.Generator()
gen.manual_seed(seed)
G = torch.randn(head_dim, head_dim, generator=gen)
Pi, _ = torch.linalg.qr(G) #A
self.Pi = Pi.to(device)
self.centroids = {}
self.boundaries = {}
for bits, cb in codebooks.items():
self.centroids[bits] = torch.tensor(
cb["centroids"], dtype=torch.float32, device=device)
self.boundaries[bits] = torch.tensor(
cb["boundaries"], dtype=torch.float32, device=device)
def quantize(self, vectors, bits):
shape = vectors.shape
flat = vectors.reshape(-1, self.head_dim).float()
norms = torch.norm(flat, dim=1, keepdim=True) #B
normalized = flat / torch.clamp(norms, min=1e-8) #C
rotated = normalized @ self.Pi.T #D
interior_b = self.boundaries[bits][1:-1]
indices = torch.searchsorted(interior_b, rotated) #E
return {"indices": indices, "norms": norms.half(),
"bits": bits, "shape": shape}
def dequantize(self, compressed):
c = self.centroids[compressed["bits"]]
recon_rot = c[compressed["indices"]] #F
recon_norm = recon_rot @ self.Pi #G
recon = recon_norm * compressed["norms"].float() #H
return recon.reshape(compressed["shape"])
Inspect the TurboQuant implementation, including its seeded rotation and codebook centroid and boundary setup. section 7.5


