Uncertainty quantification

As explained in Bayesian Optimization in Action

Uncertainty quantification is the assignment or estimation of numerical uncertainty associated with predictions or inferred quantities.

Uncertainty quantification assigns a numerical measure to how uncertain a prediction or inferred quantity is. In high-risk decisions such as Bayesian optimization, uncertainty complements the predicted average: two choices with the same expected value may differ greatly in reliability. section 1.2.1

Gaussian processes quantify predictive uncertainty by:

• Representing each prediction as a probability distribution rather than a single point estimate. section 2.0
• Using covariance functions to encode similarity and model how observations influence uncertainty. section 2.0
• Reporting confidence intervals that indicate the predictive level of uncertainty. section 1.2.1
• Retaining specified noise at observed data points when labels or observations are noisy, rather than forcing uncertainty to zero. section 1.2.1

Choosing a neural-network depth

Suppose a Gaussian process predicts 85% accuracy for both 15 and 25 layers. The 15-layer model has a 95% confidence interval of 84–86%, while the 25-layer model has an interval of 81–89%. Preferring 15 layers reflects greater certainty, not a higher predicted mean. section 1.2.1

Figure 1.3 Non-Bayesian models, such as ridge regressors, make pointwise estimates, while GPs produce probability distributions as predictions. GPs thus offer a calibrated quantification of uncertainty, which is an important factor when making high-risk decisions.

Compare pointwise ridge-regression estimates with Gaussian-process distributions and their calibrated confidence intervals. section 1.2.1

How Bayesian Optimization in Action covers it

How deep it goes

Explains distributions, confidence intervals, covariance, noisy observations, and conditioning, while stopping short of a general uncertainty framework beyond Gaussian processes.

Examples

Uses neural-network layer selection, noisy labels, house prices, ridge regression, and conditioned Gaussian variables to illustrate uncertainty.

What it assumes

Assumes basic machine-learning prediction concepts while developing Gaussian distributions, covariance, conditioning, and predictive uncertainty.

What you can do after

Apply confidence intervals and Gaussian-process predictions to compare options when expected outcomes alone are insufficient.

How it connects

Connects Gaussian processes with covariance functions, Gaussian distributions, confidence intervals, Bayesian optimization, and decision-making under uncertainty.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

• probability distributionuses · A probability distribution is a mathematical specification assigning probabilities to the possible values or outcomes of a random variable, with total probability one.
• credible intervalmeasured by · A credible interval is a Bayesian interval containing a specified posterior probability of an uncertain quantity.
• Gaussian processproduced by · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.