7 Bayesian learning
This chapter covers
- Identifying extrapolation as the Achilles heel of DL
- A gentle introduction to Bayesian modeling
- The concept of model uncertainty, which is called epistemic uncertainty
- The Bayesian approach as state-of-the-art method to dealing with parameter uncertainty
The aim of this chapter is to introduce Bayesian models. Besides the likelihood approach, the Bayesian approach is the most important method to fit the parameters of a probabilistic model and to estimate the associated parameter uncertainty. The Bayesian modeling approach incorporates an additional kind of uncertainty which is called epistemic uncertainty. You will see that incorporating the epistemic uncertainty results in a better prediction performance and more appropriate quantification of the uncertainty of the predicted outcome distribution. The epistemic uncertainty becomes especially important when applying prediction models to situations not seen during the training. In regression, this is known as extrapolation.