liveBook · Manning

Neural network-Gaussian process model

As explained in Bayesian Optimization in Action

A neural network-Gaussian process model combines a neural network transformation of inputs with Gaussian process prediction on the transformed representation.

A neural network–Gaussian process model sends structured input through a neural network before applying a Gaussian process mean function and kernel. The network acts as a feature extractor, learning representations that make similarity calculations more useful while preserving uncertainty-calibrated predictions. section 13.3.1

The combined model works by:

  • Transforming structured inputs into numerical features with layered computations. section 13.3.1
  • Feeding the transformed features to the GP mean and covariance modules, which still produce a multivariate normal distribution. section 13.3.2
  • Optimizing neural-network weights alongside GP mean, kernel, likelihood, and other hyperparameters by minimizing negative marginal log likelihood. section 13.3.1

MNIST feature extraction

A four-layer ReLU network maps each 784-dimensional MNIST input to two features. The learned two-dimensional representation clusters equal labels and forms a label gradient, giving the GP a smoother space to model. section 13.3.2

How Bayesian Optimization in Action covers it

How deep it goes
Explains the feature-extraction intuition, GP data flow, joint optimization, and GPyTorch implementation, stopping short of deeper theoretical analysis.
Examples
Uses a 784-dimensional MNIST input, a four-layer 2D feature extractor, clustered labels, and smooth-versus-rough feature comparisons.
What it assumes
Assumes familiarity with regular Gaussian-process training and introduces neural-network feature extraction and its integration with GP components.
What you can do after
Build it from scratch in PyTorch and GPyTorch, train the extractor and GP jointly, inspect learned features, and evaluate predictions.
How it connects
Connects neural networks, feature extraction, Gaussian processes, kernels, covariance functions, multivariate normal distributions, and gradient descent.
Cover of Bayesian Optimization in Action
liveBookmanning.com
Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.