6 Probabilistic deep learning models in the wild
This chapter covers
- Probabilistic deep learning in state-of-the-art models
- Flexible distributions in modern architectures
- Mixtures of probability distributions for flexible CPDs
- Normalizing flows to generate complex data-like facial images
Many real-world data like sound samples or images come from complex and high dimensional distributions. In this chapter you’ll learn how to define complex probability distributions that can be used to model real-world data. In the last two chapters you have learned to set up models that work with easy to handle distributions. You worked with linear regression models with a Gaussian CPD or a Poisson model with a Poisson distribution as CPD. Maybe you find yourself in the figure above, where the ranger stands in a protected area with some domestic animals, but the animals out in the world are more wild than the ones you worked with up to now. You also have learned enough about different kinds of domestic probabilistic models to join us into the wild to state-of-the art models that can handle complex CPDs.
One way to model complex distributions are mixtures of simple distributions, such as Normal, Poisson, or Logistic distributions, which you know from the previous chapters. Mixture models are used in state-of-the-art networks like Google's parallel WaveNet or OpenAI’s PixelCNN++ to model the output.