Kullback–Leibler divergence measures the mismatch between two probability distributions. In retrieval training, a reader converts each passage’s answer log-probability into a target distribution, and the retriever learns to match it. section 6.2.2 section 6.2.3
The feedback loop uses KL divergence to transfer usefulness from the reader to the retriever:
- Backpropagate through the retriever while keeping the reader’s supervision signal fixed; the reader and retriever losses can also be combined during training. section 6.2.3
A lexical trap becomes a retrieval signal
For the question “Why did AWS operating margins decline in Q3 2024?”, the passage explaining increased generative-AI infrastructure investment receives reader log-probability −1.3, while three distractors score roughly −35 to −38. Softmax therefore assigns nearly all target weight to the explanatory passage. After optimization, the retriever shifts from [0.989, 0.007, 0.004, 0.000] to [0.000, 0.000, 0.999, 0.000], while KL falls from 5.599 to 0.001.


















