A Quantized Johnson–Lindenstrauss (QJL) correction reserves one bit to produce unbiased inner-product estimates, useful when ranking fidelity matters in vector search. For KV-cache attention, those estimates pass through softmax: its exponential amplifies variance, and QJL consistently worsens perplexity. Use TurboQuant’s MSE-only algorithm for KV caches. section 7.5
Figure 7.19 Per-layer reconstruction MSE on OPT-6.7B with and without QJL correction. MSE-only (solid lines) beats QJL-corrected (dashed lines) across all 32 layers, both bit-widths, and both keys and values. QJL TQ4 is worse than MSE-only TQ3 in most key cache layers. Log-scale Y-axis.

Compare per-layer reconstruction error with and without QJL correction across keys, values, and bit-widths. section 7.5
