Fine-tuning adapts a pretrained model’s parameters for a target task. In a deep belief network (DBN), it follows unsupervised layer-wise pretraining and uses labeled data with backpropagation. section 8.9.1
A DBN fine-tuning workflow proceeds as follows:
- Pretrain each layer as a restricted Boltzmann machine (RBM), using contrastive divergence to learn increasingly abstract representations without labels. section 8.9.1
- Fine-tune the entire network jointly with backpropagation and a labeled dataset. section 8.9.1
- Adjust the weights to minimize prediction error for classification or regression. section 8.9.1
Unsupervised pretraining provides a more informative starting point and may reduce overfitting during supervised optimization. Full-network fine-tuning can still be computationally expensive, time-consuming, sensitive to hyperparameters, and difficult in very deep networks because of vanishing gradients. section 8.9.1
DBN-based recognition
A DBN can learn increasingly abstract sound or image features during pretraining, then fine-tune its complete network for supervised speech or image recognition. section 8.9.1
