Product quantization compresses vectors by splitting them into subvectors and representing them with entries from learned codebooks. Its compact representations reduce storage while retaining information for approximate similarity search. section 4.6.2
The benefit is most relevant when high-dimensional vectors and growing corpora make index memory a latency, storage, or cost bottleneck. Compression is not always necessary for small applications. section 4.6.2
Choose an index strategy around the workload:
- Consider IVF-PQ when memory is constrained or documents are added frequently; favor HNSW when query latency is paramount and memory overhead is acceptable. Benchmark both on the target workload. section 4.5.2
- Use compressed vectors for first-pass candidate retrieval, then rerank candidates with a cross-encoder when precision matters. section 4.6.2
