liveBook · Manning

Kullback–Leibler divergence

As explained in Retrieval Augmented Generation, The Foundational Ideas

Kullback-Leibler divergence is a measure of how one probability distribution differs from a reference distribution.

Kullback–Leibler divergence measures the mismatch between two probability distributions. In retrieval training, a reader converts each passage’s answer log-probability into a target distribution, and the retriever learns to match it. section 6.2.2 section 6.2.3

The feedback loop uses KL divergence to transfer usefulness from the reader to the retriever:

  • Backpropagate through the retriever while keeping the reader’s supervision signal fixed; the reader and retriever losses can also be combined during training. section 6.2.3

A lexical trap becomes a retrieval signal

For the question “Why did AWS operating margins decline in Q3 2024?”, the passage explaining increased generative-AI infrastructure investment receives reader log-probability −1.3, while three distractors score roughly −35 to −38. Softmax therefore assigns nearly all target weight to the explanatory passage. After optimization, the retriever shifts from [0.989, 0.007, 0.004, 0.000] to [0.000, 0.000, 0.999, 0.000], while KL falls from 5.599 to 0.001.

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains distribution construction, PyTorch implementation, optimization, and loss combination, but does not derive KL mathematically.
Examples
Demonstrates AWS operating-margin passages, reader log-probabilities, retriever distributions, and falling KL during optimization.
What it assumes
Assumes probability distributions, retriever scores, reader log-probabilities, softmax, and gradient-based optimization.
What you can do after
Apply it via PyTorch to train a retriever against reader-derived probability targets.
How it connects
Connects KL divergence to retrievers, readers, language models, Perplexity Distillation, and retrieval augmentation.

Also explained in

Compare how 19 books explain Kullback–Leibler divergence →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.