Suppose a website records 63 conversions from 500 visits. Does that result better fit a fixed conversion rate of 0.1, or a model that allows the rate to vary? A Bayes factor compares how well two specified hypotheses predict the observed data. section 11.5
For models M₁ and M₂ and data D, BF(M₁, M₂) = p(D | M₁) / p(D | M₂). Each term is a marginal likelihood: it accounts for the possible parameter values within that model. A factor above 1 favors M₁; below 1 favors M₂; exactly 1 favors neither. The order matters—reversing the models reverses the ratio. section 4.9 section 11.5
Tea or coffee?
When 3 of 5 people prefer tea, comparing Beta(4, 3) with Beta(1, 1) gives a Bayes factor of about 1.558: the first distribution assigns the data about 1.5 times the marginal likelihood. Comparing Beta(2, 4) with Beta(1, 1) instead gives 0.952, slightly favoring the second. These are relative comparisons, not probabilities that a model is true. section 4.10
from scipy.special import comb, beta
def marginal_likelihood(k, n, a, b):
return (
comb(n, k)
* beta(a + k, b + n - k)
/ beta(a, b)
)
Use the SciPy function to calculate a Beta-model marginal likelihood, then divide the results for two models to obtain their Bayes factor. section 4.9
Table 4.1 Interpretations of Bayes factors
|
Bayes factor range
|
Strength of evidence
|
|
1 to 3.2
|
Not worth more than a bare mention
|
|
3.2 to 10
|
Substantial
|
|
10 to 100
|
Strong
|
|
More than 100
|
Decisive.
|
Use the ranges as guidance for describing evidence, not as a replacement for checking whether either model is suitable. section 4.10 section 4.12
Because marginal likelihood considers a model's range of predictions, a model that can explain many datasets need not beat a more focused model that explains these data well. This is the Bayesian Occam’s razor effect. A Bayes factor helps choose between explanations; predictive checks address the separate question of whether a model makes reasonable predictions. section 4.11 section 4.0
