Diffusion model
A diffusion model is a generative model that learns to produce data by reversing a process that progressively adds noise to data.
Build a Text-to-Image Generator (from Scratch)
A denoising diffusion probabilistic model is a diffusion generative model that samples through a probabilistic sequence of noise-removal steps.
- How it is used
- Generates images from noise, first without prompts and then under class-label or text-prompt guidance.
- How deep it goes
- Works through noise scheduling, U-Net noise prediction, training loss, and sampling; uses a pretrained model for computationally costly latent diffusion.
- Examples
- Uses Fashion MNIST clothing generation, staged image-noising visualizations, and text-guided generation in latent space.
- What you can do after
- Build it from scratch as a class-conditional Fashion MNIST DDPM, then apply a pretrained latent model to text-to-image generation.
Read how Build a Text-to-Image Generator (from Scratch) explains diffusion model →
Deep Learning with R, Third Edition
A diffusion model is a generative model that iteratively transforms noise into structured data by reversing a process that progressively adds noise.
- How it is used
- Generates images from noise, either without conditioning or guided by a text prompt.
- How deep it goes
- Explains iterative denoising, diffusion time, a cosine schedule, and pretrained generation; defers training a capable text-to-image model from scratch.
- Examples
- Uses noise-to-image denoising, a cosine schedule, Stable Diffusion 3 prompts, and interpolation between prompt embeddings.
- What you can do after
- Apply it via KerasHub’s pretrained Stable Diffusion 3 task to generate and steer images with prompts and diffusion steps.
Read how Deep Learning with R, Third Edition explains diffusion model →
Deep Learning with PyTorch, Second Edition
Generative models that create outputs by reversing a diffusion process, mentioned here as part of the Hugging Face model collection.
- How it is used
- Generates new images and edits existing ones by reversing gradual noising.
- How deep it goes
- Builds a small diffusion model through scheduled noising, noise-prediction training, and reverse sampling using 2D points rather than full images.
- Examples
- Uses a logo outline made of 2D points and a masked horse-to-zebra image edit.
- What you can do after
- Build it from scratch with PyTorch to noise 2D points, train a noise predictor, and sample a reconstructed shape.
Read how Deep Learning with PyTorch, Second Edition explains diffusion model →
Learn Generative AI with PyTorch
A generative model that learns to produce samples by progressively reversing a noising process over discrete time steps.
- How it is used
- Uses progressive noising and denoising to generate flower images and establish the image-generation mechanism behind text-to-image systems.
- How deep it goes
- Works through time steps, noise weighting, scheduler visualization, and U-Net denoising before moving to conceptual text conditioning.
- Examples
- Visualizes four Oxford 102 Flower images becoming noise across selected steps and uses a DDIM scheduler to produce the noisy images.
- What you can do after
- Apply it via DDIMScheduler to visualize forward diffusion and use a trained denoising U-Net to generate flower images from noise.
Read how Learn Generative AI with PyTorch explains diffusion model →
Grokking AI Algorithms, Second Edition
A diffusion model is a generative model that learns to create structured data by reversing a gradual noise-addition process.
- How it is used
- Generates new images from random noise, with text prompts guiding the result.
- How deep it goes
- Traces forward noising, U-Net noise prediction, training updates, and iterative generation; also outlines schedules and training-quality checks.
- Examples
- Follows a noisy image becoming a detailed tree and gives a timestep table for a noise schedule.
- What you can do after
- Build it from scratch as a small image-generation workflow, then assess training with validation loss, FID, and checkpoints.
Read how Grokking AI Algorithms, Second Edition explains diffusion model →
Deep Learning with Python, Third Edition
A diffusion-based generative model that starts from noise and iteratively denoises it into an image.
- How it is used
- Generates new images by repeatedly denoising pure noise, either without a prompt or guided by text.
- How deep it goes
- Works through diffusion time, a cosine schedule, and flower-model training, but defers training a strong text-to-image model from scratch.
- Examples
- Uses generated Oxford Flowers images and a pretrained Stable Diffusion 3 prompt, with negative prompts and step counts as controls.
- What you can do after
- Build it from scratch for unconditioned flowers, or apply it via KerasHub’s pretrained TextToImage task for prompt-guided images.
Read how Deep Learning with Python, Third Edition explains diffusion model →
Generative AI in Action
A diffusion model is a generative model that learns to create data by reversing a process that progressively adds noise to examples.
- How it is used
- Supports high-quality generation of complex, high-dimensional, continuous data when generation speed is unimportant.
- How deep it goes
- Explains latent-space learning through forward noise addition, reverse neural denoising, and Markov-chain steps without mathematical derivation.
- Examples
- Uses panda images, optimized chairs, speech, video, and enterprise image-generation tools as examples.
- What you can do after
- Evaluate diffusion models for image, speech, video, and generative-design use cases against speed and output-diversity needs.
Introduction to Generative AI, Second Edition
A diffusion model is a generative model that learns to create data by reversing a process that progressively adds noise to examples.
- How it is used
- Creates or alters synthetic images and videos through learned gradual denoising.
- How deep it goes
- Explains the reverse-noising intuition and comparative role, but stops short of implementation or training details.
- Examples
- Uses gradual image denoising as the concrete example of diffusion-based generation.
- What you can do after
- Evaluate diffusion models as a leading synthetic-media technique while recognizing associated trust, privacy, and intellectual-property concerns.
Read how Introduction to Generative AI, Second Edition explains diffusion model →
Deep Learning with JAX
A generative model that iteratively denoises noise to synthesize outputs such as images, audio, or 3D structures.
- How it is used
- Deploys pretrained pipelines for generating images, audio, video, and 3D structures from conditioned inputs.
- How deep it goes
- Sketches encoders, denoising, schedulers, and pretrained models but does not derive diffusion internals.
- Examples
- Uses Stable Diffusion, FlaxStableDiffusionPipeline, JAX, bfloat16 parameters, and accelerator-backed image generation.
- What you can do after
- Operate pretrained Flax diffusion pipelines and begin building custom image-generation systems with diffusers.
Machine Learning Algorithms in Depth
A diffusion model is a generative model that learns to reverse a gradual noise-adding process to generate samples from a data distribution.
- How it is used
- Frames diffusion models as likelihood-based generative models for producing high-resolution images.
- How deep it goes
- Defines Markov-chain structure and forward and reverse passes, but does not derive or implement training.
- Examples
- Uses progressively noised images and denoising diffusion probabilistic models as concrete reference points.
- What you can do after
- Evaluate/understand how noisy images become samples matching a target distribution.
Read how Machine Learning Algorithms in Depth explains diffusion model →
Related concepts
- denoisingcontrasts with · Denoising is the process of removing unwanted noise or corruption from data while preserving meaningful structure.
- noiseuses · Noise is variation, content, or data treated as unwanted, irrelevant, or distinct from the signal, pattern, or structure of interest.
- noise scheduleconfigured by · A noise schedule is a prescribed sequence of noise levels or coefficients governing noise addition and removal across diffusion timesteps.
- Stable Diffusionincludes · Stable Diffusion is a family of latent diffusion models that generates and edits images from text and other conditioning inputs.
- U-Netuses · U-Net is an encoder-decoder neural-network architecture with downsampling and upsampling paths connected by skip connections for combining features at multiple spatial scales.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.