6 Probabilistic deep learning models in the wild

Many real-world data like sound samples or images come from complex and high-dimensional distributions. In this chapter, you’ll learn how to define complex probability distributions that can be used to model real-world data. In the last two chapters, you learned to set up models that work with easy-to-handle distributions. You worked with linear regression models with a Gaussian conditional probability distribution (CPD) or a Poisson model with its distribution as a CPD. (Maybe you find yourself in the figure at the top of this chapter, where the ranger stands in a protected area with some domestic animals, but the animals out in the world are more wilder than the ones you’ve worked with up to now.) You also learned enough about different kinds of domestic probabilistic models to join us and journey into the wild to state-of-the-art models that handle complex CPDs.

One way to model complex distributions are mixtures of simple distributions such as Normal, Poisson, or logistic distributions, which you know from the previous chapters. Mixture models are used in state-of-the-art networks like Google’s parallel WaveNet or OpenAI’s PixelCNN++ to model the output.

6.1 Flexible probability distributions in state-of-the-art DL models

6.1.1 Multinomial distribution as a flexible distribution

6.1.2 Making sense of discretized logistic mixture

6.2 Case study: Bavarian roadkills

6.3 Go with the flow: Introduction to normalizing flows (NFs)

6.3.1 The principle idea of NFs

6.3.2 The change of variable technique for probabilities

6.3.3 Fitting an NF to data

6.3.4 Going deeper by chaining flows

6.3.5 Transformation between higher dimensional spaces*

6.3.6 Using networks to control flows

6.3.7 Fun with flows: Sampling faces

Summary