4 Diffusion Models: Forward Diffusion

 

This chapter covers

  • Introduction to Diffusion models and how their work
  • Intuition behind Diffusion-based approaches to generative modeling
  • Forward Diffusion process and its role in Diffusion models

Diffusion models offer a unique approach to synthesizing images that is both intriguing in theory and effective in practice. This class of models stands in contrast to other prominent generative paradigms, including VAEs and GANs, which we explored in previous chapters. While VAEs and GANs typically generate images directly from a latent representation, Diffusion models adopt a more gradual approach with many intermediate, iterative steps. This methodological shift offers a compelling alternative in the field of AI image synthesis, achieving unprecedented realism, training stability, output diversity, and other desirable characteristics.

4.1 Introduction to Diffusion Models

4.1.1 Forward and Reverse Diffusion Phases

4.2 Foundations and Core Concepts

4.2.1 Forward Diffusion

4.2.2 Reverse Diffusion

4.3 Images, Probability Distributions, and the Visual World

4.3.1 The Infinite Possibilities of Pixel Arrangements

4.3.2 Diffusion Models and the Quest for Coherence

4.4 Forward Diffusion In-Depth

4.4.1 Reducing Data Distribution Complexity by Adding Noise

4.5 Mathematical Foundations of the Forward Diffusion Process

4.5.1 Forward Diffusion Process

4.5.2 Closed-Form Formula for Skipping Steps in Forward Diffusion

4.6 Visualizing Forward Diffusion in 1D

4.6.1 Probability Density of an Example 1D Dataset

4.6.2 Applying Forward Diffusion to the 1D Distribution

4.7 Conclusion

4.8 Summary