liveBook · Manning

Diffusion model

As explained in Local Image Generation

A generative model that creates images by progressively transforming random noise into structured visual content.

A diffusion model makes an image by starting with random noise and repeatedly removing what it judges to be noise until structured content emerges. A text prompt guides those guesses, nudging the result toward the requested image rather than toward random visual mush. section 4.2.1

Think of it as finding pictures in clouds: noise supplies the cloud, the prompt suggests what to look for, and successive iterations reveal broad shapes before finer details. The number of sampling steps controls how long that cleanup process runs. section 4.2.1 section 6.1

Generation is reproducible when its starting conditions are retained:

  • A seed initializes the pseudo-random sequence used to create the starting noise. section 1.2.3
  • Using the same prompt, arguments, and seed produces the same Stable Diffusion output; a seed of -1 requests a new random seed in A1111. section 1.2.3
  • Changing samplers or schedules changes how noise is removed across steps, balancing speed against image quality. section 6.1

From pure noise to a prompted image

After training to remove noise from images, a model can receive pure noise plus a description such as a tree at night. At each step it removes a little more noise in a way that better fits the description, allowing an image to emerge. section 4.2.1

Figure 4.9 Guessing what the original image was by removing noise

This visualizes the core move: estimating an original image by removing added noise. section 4.2.1

How Local Image Generation covers it

How deep it goes
Explains high-level iterative denoising and sampling steps, while deferring deeper treatment of the model’s component parts.
Examples
Uses cloud-shape recognition, a tree-at-night prompt, seeded random-number sequences, and coffee-image generation.
What it assumes
Introduces AI, machine learning, and neural networks before using deeper Stable Diffusion customization concepts.
What you can do after
Operate/configure Stable Diffusion generation by preserving seeds and weighing sampling speed against output quality.
How it connects
Connects Stable Diffusion with random seeds, prompts, sampling steps, samplers, schedulers, CLIP, and variational autoencoders.

Also explained in

Compare how 11 books explain diffusion model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Local Image Generation. The section links point to the parts of the book it draws on.