Diffusion model
A diffusion model is a generative model that learns to reverse a gradual noising process, generating samples through iterative denoising.
Build a Text-to-Image Generator (from Scratch)
A denoising diffusion probabilistic model is a diffusion generative model that samples through a probabilistic sequence of noise-removal steps.
- How it is used
- Generates images from noise, first without prompts and then under class-label or text-prompt guidance.
- How deep it goes
- Works through noise scheduling, U-Net noise prediction, training loss, and sampling; uses a pretrained model for computationally costly latent diffusion.
- Examples
- Uses Fashion MNIST clothing generation, staged image-noising visualizations, and text-guided generation in latent space.
- What you can do after
- Build it from scratch as a class-conditional Fashion MNIST DDPM, then apply a pretrained latent model to text-to-image generation.
Read how Build a Text-to-Image Generator (from Scratch) explains diffusion model →
Deep Learning with R, Third Edition
A diffusion model is a generative model that iteratively transforms noise into structured data by reversing a process that progressively adds noise.
- How it is used
- Uses diffusion for unconditioned image generation and text-guided image generation.
- How deep it goes
- Explains repeated denoising and a cosine schedule, and decomposes pretrained generation while deferring text-to-image training from scratch.
- Examples
- Uses flower denoising, an astronaut riding an origami elephant, negative prompts, and interpolation between text prompts.
- What you can do after
- Apply a pretrained Stable Diffusion model via KerasHub to generate images and explore prompt embeddings.
Read how Deep Learning with R, Third Edition explains diffusion model →
Deep Learning with PyTorch, Second Edition
Generative models that create outputs by reversing a diffusion process, mentioned here as part of the Hugging Face model collection.
- How it is used
- Introduces image synthesis and editing through iterative denoising, including text-guided generation and masked inpainting.
- How deep it goes
- Develops forward noise schedules, neural-network noise prediction, and iterative sampling in a basic implementation.
- Examples
- Uses a PyTorch logo outline as 2D points and a masked horse-to-zebra edit to illustrate diffusion.
- What you can do after
- Builds a simple diffusion model from scratch to denoise and sample a 2D point representation.
Read how Deep Learning with PyTorch, Second Edition explains diffusion model →
Learn Generative AI with PyTorch
A generative model that learns to produce samples by progressively reversing a noising process over discrete time steps.
- How it is used
- Uses diffusion to generate flower images and as the image-refinement mechanism in text-to-image systems.
- How deep it goes
- Works through forward noising, scheduler-based visualization, denoising U-Net architecture, training, and image generation.
- Examples
- Uses standardized flower images and a 1,000-step noising visualization to illustrate diffusion.
- What you can do after
- Build and train a denoising model to generate flower-like images from random noise.
Read how Learn Generative AI with PyTorch explains diffusion model →
Grokking AI Algorithms, Second Edition
A diffusion model is a generative model that learns to create structured data by reversing a gradual noise-addition process.
- How it is used
- Introduces diffusion as a way to generate images from noise, including text-guided generation.
- How deep it goes
- Walks through data preparation, noise schedules, U-Net training, loss calculation, inference, and training monitoring.
- Examples
- Traces a tree emerging from noise and a photograph becoming noisier across timesteps.
- What you can do after
- Build a small diffusion image model from components and use it for prompt-guided generation.
Read how Grokking AI Algorithms, Second Edition explains diffusion model →
Deep Learning with Python, Third Edition
A diffusion-based generative model that starts from noise and iteratively denoises it into an image.
- How it is used
- Uses diffusion for unconditioned image generation and extends it to text-guided generation.
- How deep it goes
- Explains iterative denoising and the diffusion schedule, then trains a flower model and uses pretrained text-to-image generation rather than training it from scratch.
- Examples
- Generates Oxford Flowers images and uses a pretrained model for an astronaut riding an origami elephant.
- What you can do after
- Builds an unconditioned diffusion model in Keras and applies pretrained Stable Diffusion through KerasHub.
Read how Deep Learning with Python, Third Edition explains diffusion model →
Local Image Generation
A generative model that creates images by progressively transforming random noise into structured visual content.
- How it is used
- Frames text-guided image generation as iterative removal of noise from a seeded random starting point.
- How deep it goes
- Explains high-level iterative denoising and sampling steps, while deferring deeper treatment of the model’s component parts.
- Examples
- Uses cloud-shape recognition, a tree-at-night prompt, seeded random-number sequences, and coffee-image generation.
- What you can do after
- Operate/configure Stable Diffusion generation by preserving seeds and weighing sampling speed against output quality.
Generative AI in Action
A diffusion model is a generative model that learns to create data by reversing a process that progressively adds noise to examples.
- How it is used
- Supports high-quality generation of complex, high-dimensional, continuous data when generation speed is unimportant.
- How deep it goes
- Explains latent-space learning through forward noise addition, reverse neural denoising, and Markov-chain steps without mathematical derivation.
- Examples
- Uses panda images, optimized chairs, speech, video, and enterprise image-generation tools as examples.
- What you can do after
- Evaluate diffusion models for image, speech, video, and generative-design use cases against speed and output-diversity needs.
Introduction to Generative AI, Second Edition
A diffusion model is a generative model that learns to create data by reversing a process that progressively adds noise to examples.
- How it is used
- Creates or alters synthetic images and videos through learned gradual denoising.
- How deep it goes
- Explains the reverse-noising intuition and comparative role, but stops short of implementation or training details.
- Examples
- Uses gradual image denoising as the concrete example of diffusion-based generation.
- What you can do after
- Evaluate diffusion models as a leading synthetic-media technique while recognizing associated trust, privacy, and intellectual-property concerns.
Read how Introduction to Generative AI, Second Edition explains diffusion model →
Deep Learning with JAX
A generative model that iteratively denoises noise to synthesize outputs such as images, audio, or 3D structures.
- How it is used
- Deploys pretrained pipelines for generating images, audio, video, and 3D structures from conditioned inputs.
- How deep it goes
- Sketches encoders, denoising, schedulers, and pretrained models but does not derive diffusion internals.
- Examples
- Uses Stable Diffusion, FlaxStableDiffusionPipeline, JAX, bfloat16 parameters, and accelerator-backed image generation.
- What you can do after
- Operate pretrained Flax diffusion pipelines and begin building custom image-generation systems with diffusers.
Machine Learning Algorithms in Depth
A diffusion model is a generative model that learns to reverse a gradual noise-adding process to generate samples from a data distribution.
- How it is used
- Frames diffusion models as likelihood-based generative models for producing high-resolution images.
- How deep it goes
- Defines Markov-chain structure and forward and reverse passes, but does not derive or implement training.
- Examples
- Uses progressively noised images and denoising diffusion probabilistic models as concrete reference points.
- What you can do after
- Evaluate/understand how noisy images become samples matching a target distribution.
Read how Machine Learning Algorithms in Depth explains diffusion model →
What to know first
- Stable DiffusionStable Diffusion is a family of latent diffusion models that generates and edits images from text and other conditioning inputs.
- promptA prompt is text or other input supplied to an AI model to provide instructions, context, or other information that guides its response or generated output.
- generative adversarial networkA generative adversarial network is a generative model in which a generator and a discriminator are trained adversarially, with the generator learning to produce samples resembling data from a target distribution.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
Related concepts
- denoisingcontrasts with · Denoising is the process of removing unwanted noise or corruption from data while preserving meaningful structure.
- Flux.1-devincludes · Flux.1-dev is an open Flux image-generation model version intended for general image generation.
- noiseuses · Noise is variation, information, or content that is random, irrelevant, or unwanted relative to a signal, pattern, measurement, or task.
- noise scheduleconfigured by · A noise schedule is a prescribed sequence of noise levels or coefficients governing noise addition and removal across diffusion timesteps.
- Stable Diffusionincludes · Stable Diffusion is a family of latent diffusion models that generates and edits images from text and other conditioning inputs.
Questions about diffusion model
What is diffusion model?
A diffusion model is a generative model that learns to reverse a gradual noising process, generating samples through iterative denoising.
Which Manning books explain diffusion model?
Build a Text-to-Image Generator (from Scratch), Deep Learning with R, Third Edition, Deep Learning with PyTorch, Second Edition, Learn Generative AI with PyTorch, Grokking AI Algorithms, Second Edition, Deep Learning with Python, Third Edition, Local Image Generation, Generative AI in Action and 3 more.
What should I know before learning diffusion model?
Manning books that teach diffusion model usually introduce Stable Diffusion, prompt, generative adversarial network and language model first.
What is the difference between diffusion model and denoising?
A diffusion model is a generative model that learns to reverse a gradual noising process, generating samples through iterative denoising. Denoising is the process of removing unwanted noise or corruption from data while preserving meaningful structure.
What kinds of diffusion model are there?
Kinds of diffusion model include Flux.1-dev, pretrained diffusion model and Stable Diffusion.
What is diffusion model also called?
Diffusion model is also called diffusion-based models.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.










