A retriever can rank a passage that repeats a question above one that answers it. Perplexity distillation (PDist) uses the reader to correct that ranking: documents receive higher training weight when they make the target answer more likely. It needs question–answer examples, but no document-relevance labels. section 6.2.2 section 6.7
For each training question and target answer:
- Retrieve the top-k documents. Give each document to the reader separately and measure the log-probability it assigns to the target answer. section 6.2.2 section 6.2.3
- Apply softmax to those scores to form a target distribution. Compare it with the retriever’s distribution over the same documents using KL divergence, then update the retriever. section 6.2.3
Moving an explanation above a lexical match
Asked why AWS operating margins declined, a similarity-based retriever initially puts 0.989 probability on a passage quoting margin figures and just 0.004 on the passage explaining the decline. The reader assigns the explanatory passage a target-answer log-probability of −1.3, versus roughly −35 to −38 for the others. In a laptop demonstration, 20 retriever updates shift its probability to 0.999 on the explanation. The reader is frozen in this demonstration; its scores supply the supervision. section 6.2.3 section 6.8
Listing 6.3 One step of Perplexity Distillation.
retriever = SentenceTransformer("all-MiniLM-L6-v2")
opt = torch.optim.Adam(retriever.parameters(), lr=2e-5)
def retriever_dist() -> torch.Tensor:
"""Question-to-passage similarity, as logits."""
feats = retriever.tokenize([QUESTION] + PASSAGES)
emb = F.normalize( #A
retriever(feats)["sentence_embedding"], dim=-1)
return emb[0] @ emb[1:].T / 0.1 #B
for step in range(20):
loss = F.kl_div( #C
F.log_softmax(retriever_dist(), dim=0),
target, reduction="sum")
opt.zero_grad()
loss.backward() #D
opt.step()
The training step shows the retriever’s softmax scores being matched to the reader-derived target with KL divergence. section 6.2.3
PDist is a relatively cheap feedback signal, not a substitute for a useful starting retriever: a random retriever may fetch no helpful document, leaving the reader unable to distinguish candidates. In Atlas’s 64-shot ablation, PDist scores 45.7 against 42.2 for a fixed retriever; omitting joint pre-training drops the score to 30.0. Other tested losses perform similarly, while leave-one-out evaluation more closely reflects multi-document inference at higher computational cost. For a full pipeline, start from a jointly pre-trained retriever–reader checkpoint rather than treating the small demonstration as a deployment recipe. section 6.2.2 section 6.2.3 section 6.7
