liveBook · Manning

Probability theory

As explained in Retrieval Augmented Generation, The Foundational Ideas

Probability is a numerical measure representing the likelihood of an event or outcome.

Probability measures how likely an event or outcome is. In retrieval-augmented generation, it turns raw scores into normalized weights and represents how likely a model is to produce or support an answer. section 4.2.1 section 14.3.1

Two practical uses illustrate the distinction between scores, probabilities, and probability-weighted results:

  • Normalize retrieval scores so p(z|x) sums to 1; this preserves ranking while making the values usable as document weights. section 4.2.1
  • Combine retrieval relevance with generation likelihood; a lower-ranked document can dominate when it supports the answer much better. section 4.2.1
  • Represent rating uncertainty with token probabilities; weighting possible scores produces a continuous result instead of a potentially misleading integer. section 14.3.1

A lower-ranked document becomes decisive

Five documents receive normalized retrieval weights. Document C ranks third at 0.199, but its generation score is 0.92 because it directly explains the answer's root causes. Multiplication gives 0.183, exceeding Document A's 0.034 and Document B's 0.043; summing the products yields p(y|x) = 0.500 for the answer string. section 4.2.1

score = ∑i=1npisi

Use the probability-weighted score equation to see how likely rating tokens combine into a fine-grained evaluation. section 14.3.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains normalization, multiplication, and probability-weighted scoring with worked values, but stops short of general probability theory.
Examples
Demonstrates document-score normalization, a subprime-mortgage retrieval example, and continuous G-Eval ratings.
What you can do after
Evaluate RAG outputs with probability-weighted ratings and combine retrieval relevance with answer-generation likelihood.
How it connects
Connects probability with marginalization, retrieval-augmented generation, token probabilities, and chain-of-thought prompting.

Also explained in

Compare how 22 books explain probability theory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.