liveBook · Manning

Diffusers

As explained in Local Image Generation

Diffusers is an open-source Python library from Hugging Face that provides pretrained diffusion models and pipelines for generating and manipulating images, audio, and other data.

Diffusers is a Python library that makes running complex diffusion models feel less like assembling a spaceship from spare parts. Load a pretrained model into a pipeline, set image options, and generate from code; a notebook keeps the experiment readable while leaving room for serious customization and automation. section 3.0 section 3.1

A typical workflow creates a `DiffusionPipeline` from a model path, chooses a precision and CUDA device, makes a seeded Torch generator, and calls the pipeline with a prompt, dimensions, guidance scale, and optional negative prompt. The returned result exposes generated images for saving or further use. section 3.4

Listing 3.1 Using the same configuration in Diffusers as last chapter
from diffusers import DiffusionPipeline
import torch
 
pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0",  #A
                                         torch_dtype=torch.float16,
                                         use_safetensors=True, variant="fp16")
pipe.to("cuda")  #B
 
 
prng = torch.Generator("cuda").manual_seed(1433094685)  #C
 
prompt = "A scene from a scifi movie,  a sandworm with a mouth, teeth”  #D
negative_prompt = "black and white, eyes"
width = 680
height = 520
cfg_scale = 13
 
pipe(  #E
    prompt=prompt,
    negative_prompt=negative_prompt,
    width=width,
    height=height,
    guidance_scale=cfg_scale,
    generator=prng
    ).images[0]

This minimal pipeline shows the useful moving parts in one place: load Stable Diffusion XL, move it to CUDA, fix a seed, configure the prompt, then generate an image. section 3.4

For repeatable batches, separate what changes from what stays fixed:

  • Store each image or panel's prompt and dimensions in configuration dictionaries. section 3.5
  • Loop over configurations, create an output folder, and generate a chosen number of candidates with a seeded generator. section 3.5
  • Keep setup cells separate from work cells so prompts and settings are easy to revise and rerun. section 3.5

Generate a graphic-novel page

Define five panel configurations, generate ten candidate images for each panel, save them into separate folders, then choose favorites for layout. This turns one-off prompting into a repeatable production pass; if the results miss, adjust prompts or settings and run it again. section 3.5

Changing models can be surprisingly small-scale: replace the repository path passed to `from_pretrained`. The same pipeline setup used for Stable Diffusion XL can instead load a Pixel Art Diffusion XL checkpoint, changing the visual result without rebuilding the workflow. section 5.3.2

How Local Image Generation covers it

How deep it goes
Demonstrates pipeline loading, CUDA placement, seeded generation, configuration dictionaries, loops, saving, and checkpoint switching; defers custom-model training.
Examples
Uses a sandworm prompt, five graphic-novel panels with ten candidates each, and a Pixel Art Diffusion XL rendering of a poem.
What it assumes
Assumes a configured notebook environment and installed Hugging Face libraries while allowing readers to copy and modify the Python.
What you can do after
Apply it via DiffusionPipeline to generate seeded images, automate saved batches, and swap model checkpoints by changing a path.
How it connects
Connects DiffusionPipeline with Stable Diffusion XL, Python notebooks, CUDA, Torch generators, prompts, guidance scale, and custom checkpoints.

Also explained in

Compare how 3 books explain diffusers →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Local Image Generation. The section links point to the parts of the book it draws on.