Boltzmann learning rule

As explained in Data Without Labels

The Boltzmann learning rule is a learning procedure that updates connection weights to reduce the energy of a stochastic neural network while matching data and model distributions.

The Boltzmann learning rule is an unsupervised, probabilistic procedure for adjusting neural-network weights so lower-energy states become more likely and the model distribution matches the data distribution. section 8.8 section 8.8.2

The training process combines several ideas:

• Represent the network as a probabilistic recurrent system with stochastic units connected in a fully connected graph. section 8.8.1
• Assign each visible-hidden state an energy E(v,h), then model its probability with a Boltzmann distribution normalized by the partition function. section 8.8.1
• Update weights using the gradient-derived rule: increase visible-hidden co-activation from the data distribution and contrast it with co-activation from the model distribution. section 8.8.1
• Use contrastive divergence when the basic rule is too computationally expensive for practical training. section 8.13 section 8.8.2

Typical uses

Apply the rule to unsupervised tasks such as dimensionality reduction, feature extraction, pattern recognition, optimization, and generative modeling. Lower-energy configurations represent more likely patterns, while higher-energy configurations represent less likely ones. section 8.8.1 section 8.8.2

(8.7)

Inspect the energy-based probability equation alongside the weight-update equation to connect state likelihood, energy minimization, and distribution matching. section 8.8.1

How Data Without Labels covers it

How deep it goes

Explains stochastic units, energy, Boltzmann probabilities, correlations, and weight updates, then defers computational acceleration to contrastive divergence.

Examples

Uses visible-hidden co-activation, energy states, dimensionality reduction, feature extraction, pattern recognition, optimization, and generative modeling as concrete teaching examples.

What it assumes

Builds on neural-network weights, visible and hidden units, probability distributions, and energy functions within foundational deep learning.

What you can do after

Evaluate how weight updates reduce energy and align model probabilities with observed data, including why approximations may be necessary.

How it connects

Connects energy-based models, Boltzmann machines, stochastic neural networks, dimensionality reduction, feature extraction, and contrastive divergence.

liveBookmanning.com

Read Data Without Labels on liveBook Get the book at manning.com

Related concepts

• deep learninguses · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• unsupervised learninguses · Unsupervised learning is a machine-learning approach that discovers patterns, structure, or representations in data without labeled target outputs.
• contrastive divergenceuses · Contrastive divergence is a stochastic approximation technique that estimates the gradient of a generative model's log-likelihood using short Gibbs-sampling chains.

This explainer was generated by Manning's knowledge graph from the text of Data Without Labels. The section links point to the parts of the book it draws on.