Marginal probability sums the joint probabilities of all alternatives that produce the desired outcome. In REALM training, it combines probability across multiple documents that could support the answer, then applies a negative logarithm as the loss. section 3.3
Three-document retrieval
A query assigns probabilities 0.391, 0.233, and 0.376 to three documents. Combining their document probabilities with answer probabilities produces marginal values of 0.356 and 0.593, with a resulting loss of 0.7783.




