Performance-based training optimizes a component according to its effect on downstream task performance rather than surface similarity alone. In retrieval-augmented generation, a retriever should fetch documents that help the reader answer correctly, not merely documents that resemble the query. section 3.0
REALM connects retrieval and reading through a shared training objective:
- The dual-encoder scores candidate documents against a masked query using vector inner products. section 3.3
- The model assigns probabilities across candidate documents and predicts masked tokens from retrieved evidence. section 3.3
- When several documents could support a correct prediction, their probabilities are summed before applying the negative logarithm. section 3.3
- The resulting loss sends feedback from prediction quality back to retrieval, teaching the retriever to prefer useful evidence. section 3.3 section 3.5
Making the signal visible
A three-document calculation starts with retrieval probabilities of 0.391, 0.233, and 0.376 and a loss of 0.7783. Increasing the first document's retrieval score raises its probability to 0.826 and lowers the loss to 0.4413, showing how retrieval preference follows downstream prediction value.
# [batch_size, num_candidates]
candidate_score = retrieval_score
candidate_log_probs = tf.math.log_softmax(candidate_score)
…
# [batch_size, num_candidates, num_masks]
joint_gold_log_probs = (
tf.expand_dims(candidate_log_probs, -1) + gold_log_probs
)
# [batch_size, num_masks]
marginal_gold_log_probs = tf.reduce_logsumexp(joint_gold_log_probs, 1)Inspect the tensor operations that combine candidate log-probabilities with answer probabilities and marginalize over sufficient documents. section 3.3
The approach is self-supervised: masked tokens provide targets without external labels, while the retriever learns from the reader's prediction objective. Its cost is substantial end-to-end pretraining, including asynchronous index refreshes, which can make the method impractical for many applications. section 3.5
