liveBook · Manning

Hyperparameter

As explained in Grokking Bayes

A hyperparameter is a quantity that controls the form or behavior of a prior or model distribution rather than being directly inferred as an ordinary model parameter.

Suppose you want to estimate the proportion of people who prefer tea. Before surveying anyone, you need a way to express what proportions you consider plausible. A hyperparameter controls the shape of that prior distribution; it is not the unknown proportion you are trying to estimate. section 3.7

For a beta prior, two hyperparameters, α and β, can express different beliefs about a proportion between 0 and 1. Beta(1, 1) is flat: it gives no part of the range a higher density than another. Choosing Beta(0.5, 0.5) instead favors proportions near the extremes. The choice matters because the posterior reflects both the prior and the observed data. section 3.7 section 3.9

Update a tea-preference prior

Start with Beta(1, 1). If 3 of 5 people surveyed prefer tea, add the 3 successes to α and the 2 other responses to β. The result is Beta(4, 3). The numbers have changed, but the distribution remains in the beta family—a convenient property of the beta–binomial conjugate pair. section 3.7 section 3.18

Figure 3.3 Comparison between the posterior distributions of 𝑝 under different prior distributions.

Compare how different beta hyperparameters change the prior and, after the same data, the posterior. section 3.9

Hyperparameters also describe priors for real-valued quantities. A normal prior for the slope relating house area to price uses a mean for its central estimate and a standard deviation for its spread. A mean of 100 and standard deviation of 100 express a $100-per-square-foot estimate with substantial uncertainty. After observing prices and areas, updated values describe a posterior that is more concentrated and centered near 200. section 3.12 section 3.14

The term can also name a user-chosen control outside a prior distribution. In an upper confidence bound decision rule, β sets how strongly uncertainty counts alongside mean utility: a larger β favors exploration more. Check what a hyperparameter controls before interpreting it. section 12.10

How Grokking Bayes covers it

How deep it goes
Works through beta updates and prior-choice comparisons, then shows normal-prior updates and a decision-rule setting.
Examples
Uses a five-person tea survey, house-price regression, and action utilities to show distinct hyperparameter choices.
What it assumes
Introduces normal-distribution mean and spread while building on the ideas of priors, likelihoods, and observed data.
What you can do after
Apply it via NumPy and SciPy to update beta or normal distribution settings and compare prior and posterior densities.
How it connects
Connects hyperparameters to beta–binomial conjugacy, normal priors, Bayesian linear regression, and upper confidence bound decision rules.

Also explained in

Compare how 26 books explain hyperparameter →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.