liveBook · Manning

Likelihood

As explained in Grokking Bayes

A likelihood is a function that measures how compatible observed data are with candidate values of a model's unknown quantities.

Suppose you see a dark cloud and wonder whether it will rain. A likelihood asks how probable that observation would be under each possibility. It does not, by itself, tell you the probability of rain after seeing the cloud. section 2.4 section 2.2

For observed data D and a hypothesis H, the likelihood is Pr(D | H): the probability of the data assuming H. A hypothesis that makes the observation more probable has a higher likelihood. Keep the direction straight: Pr(D | H) is not Pr(H | D). section 2.2 section 11.7

Three tea drinkers out of five

Let p be a proposed proportion of people who prefer tea. If three of five surveyed people prefer tea, the likelihood is 10 × p³ × (1 − p)². The tea preferences contribute p³, the coffee preferences contribute (1 − p)², and 10 counts the possible arrangements of three tea preferences among five people. At p = 0.6, the likelihood is 0.3456; at p = 0.2, it is 0.0512. The observed count fits the first proposal better. section 2.10 section 3.4

binom_coeff = comb(n, k)    #A
likelihood = (
    binom_coeff
    * (ps ** k)
    * ((1 - ps) ** (n - k))
)

The NumPy calculation evaluates the same likelihood for every proposed value in ps; comb supplies the arrangement count. section 2.11

To update beliefs, multiply each hypothesis’s likelihood by its prior probability, then divide each product by the sum of the products. The resulting posterior probabilities sum to 1. Thus, a likelihood compares how well hypotheses explain the data; it is not already an updated probability for each hypothesis. A zero likelihood rules a hypothesis out in this update. section 2.2 section 2.6

A positive test is not a diagnosis

For a rare disease affecting about 1 in 150 people, the likelihood of a positive test is 0.98 if the person has the disease and 0.05 if they do not. Accounting for those likelihoods and the disease’s rarity gives about a 12% posterior probability of disease after a positive result—not 98%. section 2.5

The form of a likelihood depends on the model. Tea-preference counts use a binomial likelihood; a house-price regression with normally distributed noise uses a normal likelihood. Likelihoods also support model comparison through a Bayes factor, a ratio of how likely the observed data are under competing models. Very small likelihoods can be more stable to calculate on a logarithmic scale. section 3.4 section 3.13 section 11.5

How Grokking Bayes covers it

How deep it goes
Works from conditional probability through a counted binomial example, vectorized calculation, and posterior normalization; also identifies normal likelihoods.
Examples
Uses tea preferences, dark clouds, a rare-disease test, and house-price regression to distinguish observations under competing assumptions.
What it assumes
Builds from conditional probability and simple multiplication, explaining the arrangement count before scaling the calculation to code.
What you can do after
Apply it via NumPy and SciPy to calculate binomial likelihoods and update probabilities across proposed tea-preference rates.
How it connects
Links likelihood to prior and posterior probabilities, binomial and normal distributions, conjugate priors, and Bayes factors.

Also explained in

Compare how 7 books explain likelihood →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.