liveBook · Manning

Bayesian statistics

As explained in Bayesian Optimization in Action

A Bayesian update is the revision of a prior belief into a posterior belief using observed data and a likelihood.

Bayesian statistics revises a belief about an unknown quantity by combining a prior belief with the likelihood of observed data, producing a posterior belief. Relevant evidence can sharply change the posterior; irrelevant evidence leaves the prior unchanged. section 2.1

The update starts with a probability distribution representing what is believed before observing data. Bayes’ theorem then combines that prior with likelihood terms describing how probable the observations are under competing explanations. section 2.1 section 3.1

Estimating a house price

Suppose a house initially has a broad estimated price range of 150k–300k. A nearby, similar house selling for a known price provides relevant evidence and narrows the posterior; a distant, dissimilar house in California does not change the estimate. section 2.1

Testing a claimed psychic

After 100 correct guesses of numbers from 0 to 9, observers can reach different conclusions because they began with different priors. A true psychic has likelihood 1 for a correct guess, while a non-psychic has likelihood 1/10. section 3.1

Figure 2.2 Bayes' theorem, which gives a method of updating a belief about a quantity of interest, represented as a probability distribution of a random variable. Before observing any data, we have the prior belief about X. After being updated with data, we obtain the posterior belief about X.

Visualizes the transition from a prior distribution before observation to a posterior distribution after incorporating data. section 2.1

How Bayesian Optimization in Action covers it

How deep it goes
Defines priors, likelihoods, and posteriors with toy examples but treats Bayes’ theorem only at a high level.
Examples
Illustrates updates through house-price comparisons and repeated number guesses by a supposed psychic.
What it assumes
Introduces the update intuition from zero while assuming only basic probability language such as prior, likelihood, and posterior.
What you can do after
Apply Bayesian reasoning to combine an initial belief with evidence and interpret the resulting posterior.
How it connects
Connects Bayesian updates with probability distributions, likelihood, priors, posteriors, and Gaussian processes.

Also explained in

Compare how 8 books explain bayesian statistics →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.