liveBook · Manning

Prior probability

As explained in Grokking Bayes

A prior probability is the probability assigned to a hypothesis before incorporating observed data.

Suppose it is not rainy season. Before looking outside, you give rain today a low probability. That is a prior probability: how plausible a hypothesis seems before incorporating the data you are about to observe. A dark cloud is new evidence; the probability you assign after seeing it is a posterior probability. The prior is a starting belief, not the final answer. section 1.5

A prior distribution assigns a probability to every hypothesis under consideration. For a discrete set, those probabilities must sum to 1. The choices can express existing knowledge or uncertainty: equal probabilities form a uniform prior, while a more concentrated prior favors particular possibilities. Choosing a prior involves judgment, so it is worth checking how that choice affects the result. section 2.9 section 2.14

To update the probabilities after observing data:

  • For each hypothesis, find the likelihood: the probability of seeing the data if that hypothesis were true. section 2.2
  • Multiply its prior probability by its likelihood to get an unnormalized posterior. section 2.2
  • Divide each product by the sum of all the products. The resulting posterior probabilities sum to 1. section 2.2

A positive test is not a diagnosis

Suppose a disease affects about 1 in 150 people, and assume no related health conditions. Its prior probability is about 0.007. A test is positive for 98% of people with the disease, but also for 5% without it. After a positive result, multiplying each prior by its corresponding likelihood and normalizing gives roughly a 12% probability of having the disease. Ignoring the low starting probability would make the result sound more conclusive than it is. section 2.5

For a survey about tea versus coffee, one simple model considers eleven possible proportions of tea drinkers: 0.0, 0.1, …, 1.0. A uniform prior gives each possibility probability 1/11. With 3 tea drinkers among 5 respondents, the likelihoods shift that flat starting distribution toward proportions near 0.6. NumPy arrays make the same multiply-and-normalize calculation manageable across all eleven possibilities. section 2.9 section 2.11

ps = np.linspace(0, 1, 11)         #A
prior = np.ones_like(ps) / len(ps) #B
 
k = 3                              #C
n = 5                              #D

The code sets up eleven candidate proportions, gives each an equal prior probability, and records the survey counts before calculating likelihoods. section 2.11

Priors matter most visibly when data are limited. Different starting distributions can yield different posteriors from the same five survey responses. With much more data, many reasonable priors produce similar results—but a hypothesis assigned prior probability zero cannot gain posterior probability, however favorable its likelihood. For a continuously varying quantity, use a prior density rather than assigning a positive probability to every exact value. section 2.14 section 3.2

How Grokking Bayes covers it

How deep it goes
Works through assignment, multiplication, normalization, and prior sensitivity; distinguishes discrete probabilities from continuous densities.
Examples
Uses rain prediction, a rare-disease test, and tea-versus-coffee survey proportions to show starting beliefs and updates.
What it assumes
Builds Bayesian updating from hypotheses and familiar examples; introduces probability densities when moving beyond discrete values.
What you can do after
Apply it via NumPy to assign discrete prior probabilities and calculate normalized posteriors from observed counts.
How it connects
Connects likelihoods and posterior distributions through Bayes’ theorem, and connects prior choice to regularization.

Also explained in

Compare how 12 books explain prior probability →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.