liveBook · Manning

BIC

As explained in Timeless Algorithms, The Foundational Ideas

The Bayesian information criterion is a likelihood-based model-selection score that penalizes model complexity, with a penalty that increases with sample size.

When several models fit the same data, BIC balances goodness of fit against complexity. It starts from maximized likelihood and penalizes additional parameters; lower values indicate a more favorable balance and help guard against overfitting. section 3.7.2

BIC penalizes complexity more strongly than AIC, particularly as sample size increases, and tends to favor simpler models. It is not a full Bayesian inference procedure: its basis is a Bayesian approximation to marginal likelihood under suitable assumptions, rather than direct application of Bayes’ theorem. section 2.4.1 section 3.7.2

Information criteria offer an alternative when candidate models are not nested and likelihood ratio tests are inappropriate. BIC supports model comparison, not predictions or belief updates. section 2.4.1 section 3.7.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains the maximized-likelihood basis, complexity penalty, and Bayesian approximation, without applying Bayes’ theorem directly.
What it assumes
Assumes candidate models have been fitted to the same data and their maximized likelihoods can be compared.
What you can do after
Evaluate competing fitted models by preferring the candidate with the lower BIC.
How it connects
Connects maximum likelihood, marginal likelihood, AIC, and overfitting.

Also explained in

Compare how 5 books explain BIC →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.