Neural network

As explained in Bayesian Optimization in Action

A neural network is a machine-learning model composed of interconnected computational layers that transform inputs into outputs.

A neural network is a layered model that transforms inputs through chained computations. Each layer performs matrix multiplication followed by a nonlinear activation, allowing the network to model complex functions. section 13.3.1

Neural networks are especially useful for structured data whose important relationships a simple covariance function cannot capture. In a combined model, the network processes the input before a Gaussian process applies its mean function and kernel. section 13.3 section 13.3.1

A forward pass works by:

• Passing the input through multiple computational layers. section 13.3.1
• Applying a nonlinear activation after each layer’s matrix multiplication. section 13.3.1
• Producing a flexible numerical representation that can expose important features of structured data. section 13.3.1

Housing-price inputs

A kernel that compares houses only by front-door color can assign inappropriate similarities. A neural network can instead process structured housing information, such as relevant features, into numerical values that a Gaussian-process kernel can use. section 13.0 section 13.3.1

Figure 13.9 A neural network is a collection of layered computations. By chaining multiple layers of computations together, a neural network can model complex functions well.

Visualizes a neural network as chained layered computations for modeling complex functions. section 13.3.1

Figure 13.10 Combining a neural network with a GP. The neural network processes the structured data input x first and then feeds the output to the mean function and the kernel of the GP.

Shows the neural network transforming input before the Gaussian process mean function and kernel receive it. section 13.3.1

How Bayesian Optimization in Action covers it

How deep it goes

Explains layered matrix computations, nonlinear activations, and feature extraction, but defers thorough neural-network treatment.

Examples

Uses housing-price similarity, front-door color, structured inputs, and layered-computation diagrams as teaching examples.

What you can do after

Evaluate neural networks as flexible feature extractors for Gaussian-process models of structured data.

How it connects

Connects neural networks to Gaussian processes, covariance functions, kernels, mean functions, and feature extraction.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Designing Deep Learning Systems: A guide for software engineersA computational model composed of interconnected parameterized units that learns mappings from inputs to outputs.
• 

  liveBookmanning.com
  Demand Forecasting Best PracticesA neural network is a computational model composed of interconnected parameterized units organized in layers that transform inputs into outputs.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningA neural network is a parameterized computational model composed of connected units that transform inputs through learned weights and nonlinear functions.

Compare how 4 books explain neural network →

Related concepts

• deep learningused by · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
• autoencoderincludes · An autoencoder is a neural network that encodes input data into an internal representation and decodes it to reconstruct the input or an approximation.
• recurrent neural networkincludes · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
• U-Netincludes · U-Net is an encoder-decoder neural-network architecture with downsampling and upsampling paths connected by skip connections for combining features at multiple spatial scales.
• gradient descentuses · Gradient descent is an iterative optimization method that updates parameters opposite the gradient of an objective function to reduce its value.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.