liveBook · Manning

Diffusion model

Also called diffusion-based models

A diffusion model is a generative model that learns to reverse a gradual noising process, generating samples through iterative denoising.

How 11 Manning books explain it

Build a Text-to-Image Generator (from Scratch)

A denoising diffusion probabilistic model is a diffusion generative model that samples through a probabilistic sequence of noise-removal steps.

How it is used
Generates images from noise, first without prompts and then under class-label or text-prompt guidance.
How deep it goes
Works through noise scheduling, U-Net noise prediction, training loss, and sampling; uses a pretrained model for computationally costly latent diffusion.
Examples
Uses Fashion MNIST clothing generation, staged image-noising visualizations, and text-guided generation in latent space.
What you can do after
Build it from scratch as a class-conditional Fashion MNIST DDPM, then apply a pretrained latent model to text-to-image generation.

Read how Build a Text-to-Image Generator (from Scratch) explains diffusion model →

Deep Learning with R, Third Edition

A diffusion model is a generative model that iteratively transforms noise into structured data by reversing a process that progressively adds noise.

How it is used
Uses diffusion for unconditioned image generation and text-guided image generation.
How deep it goes
Explains repeated denoising and a cosine schedule, and decomposes pretrained generation while deferring text-to-image training from scratch.
Examples
Uses flower denoising, an astronaut riding an origami elephant, negative prompts, and interpolation between text prompts.
What you can do after
Apply a pretrained Stable Diffusion model via KerasHub to generate images and explore prompt embeddings.

Read how Deep Learning with R, Third Edition explains diffusion model →

Deep Learning with PyTorch, Second Edition

Generative models that create outputs by reversing a diffusion process, mentioned here as part of the Hugging Face model collection.

How it is used
Introduces image synthesis and editing through iterative denoising, including text-guided generation and masked inpainting.
How deep it goes
Develops forward noise schedules, neural-network noise prediction, and iterative sampling in a basic implementation.
Examples
Uses a PyTorch logo outline as 2D points and a masked horse-to-zebra edit to illustrate diffusion.
What you can do after
Builds a simple diffusion model from scratch to denoise and sample a 2D point representation.

Read how Deep Learning with PyTorch, Second Edition explains diffusion model →

Learn Generative AI with PyTorch

A generative model that learns to produce samples by progressively reversing a noising process over discrete time steps.

How it is used
Uses diffusion to generate flower images and as the image-refinement mechanism in text-to-image systems.
How deep it goes
Works through forward noising, scheduler-based visualization, denoising U-Net architecture, training, and image generation.
Examples
Uses standardized flower images and a 1,000-step noising visualization to illustrate diffusion.
What you can do after
Build and train a denoising model to generate flower-like images from random noise.

Read how Learn Generative AI with PyTorch explains diffusion model →

Grokking AI Algorithms, Second Edition

A diffusion model is a generative model that learns to create structured data by reversing a gradual noise-addition process.

How it is used
Introduces diffusion as a way to generate images from noise, including text-guided generation.
How deep it goes
Walks through data preparation, noise schedules, U-Net training, loss calculation, inference, and training monitoring.
Examples
Traces a tree emerging from noise and a photograph becoming noisier across timesteps.
What you can do after
Build a small diffusion image model from components and use it for prompt-guided generation.

Read how Grokking AI Algorithms, Second Edition explains diffusion model →

Deep Learning with Python, Third Edition

A diffusion-based generative model that starts from noise and iteratively denoises it into an image.

How it is used
Uses diffusion for unconditioned image generation and extends it to text-guided generation.
How deep it goes
Explains iterative denoising and the diffusion schedule, then trains a flower model and uses pretrained text-to-image generation rather than training it from scratch.
Examples
Generates Oxford Flowers images and uses a pretrained model for an astronaut riding an origami elephant.
What you can do after
Builds an unconditioned diffusion model in Keras and applies pretrained Stable Diffusion through KerasHub.

Read how Deep Learning with Python, Third Edition explains diffusion model →

Local Image Generation

A generative model that creates images by progressively transforming random noise into structured visual content.

How it is used
Frames text-guided image generation as iterative removal of noise from a seeded random starting point.
How deep it goes
Explains high-level iterative denoising and sampling steps, while deferring deeper treatment of the model’s component parts.
Examples
Uses cloud-shape recognition, a tree-at-night prompt, seeded random-number sequences, and coffee-image generation.
What you can do after
Operate/configure Stable Diffusion generation by preserving seeds and weighing sampling speed against output quality.

Read how Local Image Generation explains diffusion model →

Generative AI in Action

A diffusion model is a generative model that learns to create data by reversing a process that progressively adds noise to examples.

How it is used
Supports high-quality generation of complex, high-dimensional, continuous data when generation speed is unimportant.
How deep it goes
Explains latent-space learning through forward noise addition, reverse neural denoising, and Markov-chain steps without mathematical derivation.
Examples
Uses panda images, optimized chairs, speech, video, and enterprise image-generation tools as examples.
What you can do after
Evaluate diffusion models for image, speech, video, and generative-design use cases against speed and output-diversity needs.

Read how Generative AI in Action explains diffusion model →

Introduction to Generative AI, Second Edition

A diffusion model is a generative model that learns to create data by reversing a process that progressively adds noise to examples.

How it is used
Creates or alters synthetic images and videos through learned gradual denoising.
How deep it goes
Explains the reverse-noising intuition and comparative role, but stops short of implementation or training details.
Examples
Uses gradual image denoising as the concrete example of diffusion-based generation.
What you can do after
Evaluate diffusion models as a leading synthetic-media technique while recognizing associated trust, privacy, and intellectual-property concerns.

Read how Introduction to Generative AI, Second Edition explains diffusion model →

Deep Learning with JAX

A generative model that iteratively denoises noise to synthesize outputs such as images, audio, or 3D structures.

How it is used
Deploys pretrained pipelines for generating images, audio, video, and 3D structures from conditioned inputs.
How deep it goes
Sketches encoders, denoising, schedulers, and pretrained models but does not derive diffusion internals.
Examples
Uses Stable Diffusion, FlaxStableDiffusionPipeline, JAX, bfloat16 parameters, and accelerator-backed image generation.
What you can do after
Operate pretrained Flax diffusion pipelines and begin building custom image-generation systems with diffusers.

Read how Deep Learning with JAX explains diffusion model →

Machine Learning Algorithms in Depth

A diffusion model is a generative model that learns to reverse a gradual noise-adding process to generate samples from a data distribution.

How it is used
Frames diffusion models as likelihood-based generative models for producing high-resolution images.
How deep it goes
Defines Markov-chain structure and forward and reverse passes, but does not derive or implement training.
Examples
Uses progressively noised images and denoising diffusion probabilistic models as concrete reference points.
What you can do after
Evaluate/understand how noisy images become samples matching a target distribution.

Read how Machine Learning Algorithms in Depth explains diffusion model →

What to know first

Related concepts

Questions about diffusion model

What is diffusion model?

A diffusion model is a generative model that learns to reverse a gradual noising process, generating samples through iterative denoising.

Which Manning books explain diffusion model?

Build a Text-to-Image Generator (from Scratch), Deep Learning with R, Third Edition, Deep Learning with PyTorch, Second Edition, Learn Generative AI with PyTorch, Grokking AI Algorithms, Second Edition, Deep Learning with Python, Third Edition, Local Image Generation, Generative AI in Action and 3 more.

What should I know before learning diffusion model?

Manning books that teach diffusion model usually introduce Stable Diffusion, prompt, generative adversarial network and language model first.

What is the difference between diffusion model and denoising?

A diffusion model is a generative model that learns to reverse a gradual noising process, generating samples through iterative denoising. Denoising is the process of removing unwanted noise or corruption from data while preserving meaningful structure.

What kinds of diffusion model are there?

Kinds of diffusion model include Flux.1-dev, pretrained diffusion model and Stable Diffusion.

What is diffusion model also called?

Diffusion model is also called diffusion-based models.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.