liveBook · Manning

Bayes factor

As explained in Grokking Bayes

A Bayes factor is a ratio of the marginal likelihoods of observed data under two competing models or hypotheses.

Suppose a website records 63 conversions from 500 visits. Does that result better fit a fixed conversion rate of 0.1, or a model that allows the rate to vary? A Bayes factor compares how well two specified hypotheses predict the observed data. section 11.5

For models M₁ and M₂ and data D, BF(M₁, M₂) = p(D | M₁) / p(D | M₂). Each term is a marginal likelihood: it accounts for the possible parameter values within that model. A factor above 1 favors M₁; below 1 favors M₂; exactly 1 favors neither. The order matters—reversing the models reverses the ratio. section 4.9 section 11.5

Tea or coffee?

When 3 of 5 people prefer tea, comparing Beta(4, 3) with Beta(1, 1) gives a Bayes factor of about 1.558: the first distribution assigns the data about 1.5 times the marginal likelihood. Comparing Beta(2, 4) with Beta(1, 1) instead gives 0.952, slightly favoring the second. These are relative comparisons, not probabilities that a model is true. section 4.10

from scipy.special import comb, beta
 
def marginal_likelihood(k, n, a, b):
    return (
        comb(n, k)
        * beta(a + k, b + n - k)
        / beta(a, b)
    )

Use the SciPy function to calculate a Beta-model marginal likelihood, then divide the results for two models to obtain their Bayes factor. section 4.9

Table 4.1 Interpretations of Bayes factors
Bayes factor range
Strength of evidence
1 to 3.2
Not worth more than a bare mention
3.2 to 10
Substantial
10 to 100
Strong
More than 100
Decisive.

Use the ranges as guidance for describing evidence, not as a replacement for checking whether either model is suitable. section 4.10 section 4.12

Because marginal likelihood considers a model's range of predictions, a model that can explain many datasets need not beat a more focused model that explains these data well. This is the Bayesian Occam’s razor effect. A Bayes factor helps choose between explanations; predictive checks address the separate question of whether a model makes reasonable predictions. section 4.11 section 4.0

How Grokking Bayes covers it

How deep it goes
Defines the ratio, works through numerical Beta comparisons, and gives interpretive ranges and a marginal-likelihood calculation.
Examples
Uses tea preferences and a website conversion-rate test to compare Beta models and a fixed-rate hypothesis.
What it assumes
Builds on priors, likelihoods, and binomial and Beta distributions while explaining how marginal likelihood enables comparison.
What you can do after
Apply it via a SciPy marginal-likelihood calculation to quantify which of two specified models the data favor.
How it connects
Links marginal likelihood, Bayesian hypothesis testing, Occam’s razor, and predictive model checking.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.