A restricted Boltzmann machine (RBM) is a generative stochastic neural-network model with visible and hidden layers of binary latent units. The visible layer represents observed data, while the hidden layer captures underlying features. section 8.9 section 8.9.1
Use an RBM as a building block for unsupervised representation learning:
- Train its weights to model the data distribution by optimizing generative-model log-likelihood. section 8.9.1
- Approximate the gradient with contrastive divergence, which alternates hidden- and visible-layer updates through Gibbs sampling. section 8.9.1
- Stack independently pretrained RBMs in a deep belief network so successive layers learn increasingly abstract features. section 8.9.1
Layer-wise feature learning
An initial RBM can learn low-level patterns from data; later RBMs can transform those representations into increasingly complex features for applications such as image or speech recognition. section 8.9.1
