liveBook · Manning

Bayesian model

As explained in Grokking Bayes

A fully Bayesian model represents uncertainty over its relevant unknown quantities with probability distributions and computes the corresponding posterior distributions.

Suppose it is not rainy season, so you think rain is unlikely. Then you see a dark cloud and reconsider. A Bayesian model makes this kind of change in belief explicit: it uses probability distributions to represent uncertainty about an unknown quantity and updates them when data arrives. section 1.5

The model has three main parts:

  • A prior distribution represents what you believe about the unknown quantity before observing the data. section 1.5
  • Data provides the evidence used to update that belief. A likelihood measures how well each possible hypothesis explains the data. section 2.2
  • A posterior distribution represents the updated belief. It is the result the model seeks. section 1.5

Bayes’ theorem connects these parts. For each hypothesis, multiply its prior probability by the likelihood of the observed data. Then divide by the combined total across hypotheses so the posterior probabilities sum to one. The posterior describes uncertainty after the update rather than giving only a single guess. section 2.2 section 1.5

Do coworkers prefer tea or coffee?

Let p be the unknown proportion of coworkers who prefer tea. In a survey of five coworkers, three choose tea. The simple estimate is 3/5, or 0.6; a Bayesian model instead treats p as uncertain and uses the survey data to update a prior distribution over its possible values. section 2.8 section 2.0

This approach is useful when prior knowledge matters, data is limited, or a decision must account for uncertainty. It also takes work to choose and refine a prior, and Bayesian methods can carry computational costs. Not every system that uses conditional probabilities is fully Bayesian: the discussion of large language models distinguishes their efficient prediction shortcuts from calculating a full posterior. section 1.6 section 1.7

Pr(hypothesis | 𝐷) ∝ Pr(𝐷 | hypothesis) × Pr(hypothesis),

The proportional form of Bayes’ theorem highlights the central update: posterior probability follows likelihood multiplied by prior probability. section 2.2

How Grokking Bayes covers it

How deep it goes
Works through prior–likelihood updating and normalization, then sets up a small survey example without supplying its computed posterior here.
Examples
Uses a rain prediction and a five-person tea-preference survey, contrasting uncertain inference with the point estimate 3/5.
What it assumes
Starts with everyday uncertainty and does not require advanced mathematics or prior statistics knowledge.
What you can do after
Build it from scratch by assigning hypothesis priors, evaluating data likelihoods, and normalizing the resulting posterior probabilities.
How it connects
Connects prior distributions, likelihoods, posterior distributions, Bayes’ theorem, and decision-making under uncertainty.

Also explained in

Compare how 3 books explain Bayesian model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.