liveBook · Manning

Model inference code

As explained in Local Image Generation

Inference is the execution of a trained model to compute outputs from supplied inputs.

Model inference code runs a trained model to turn supplied inputs into outputs. For image generation, inference is the generation step itself—not the much heavier process of training or fine-tuning the model. section 9.2.1 section 11.1

When planning an inference setup, keep three practical facts in view:

  • Inference needs substantially less memory than training because it does not repeatedly update model weights or retain the same training information. section 11.1
  • Large models can still be demanding: Flux needs about 24GB of VRAM at full precision, while quantization can reduce that to roughly 8–12GB. section 11.1
  • Hardware requirements shape accessibility; older Stable Diffusion models remain easier to run locally than Flux. section 9.2.1

Inference versus training

Generating an image with Flux is inference. Creating a custom Flux checkpoint is continued training, which can require roughly 200–300GB of VRAM without optimizations and remains far more resource-intensive. section 9.2.1 section 11.1

How Local Image Generation covers it

How deep it goes
Defines inference operationally and contrasts its memory demands with training, but does not detail implementation code or execution internals.
Examples
Uses Flux, Stable Diffusion 1.5, Stable Diffusion XL, quantized models, GPUs, and M-series MacBooks as concrete cases.
What it assumes
Assumes familiarity with trained models and image generation, while clarifying the distinction between inference and training.
What you can do after
Evaluate hardware and quantization choices for running image-generation models locally.
How it connects
Connects inference with diffusion models, image generation, quantization, VRAM, training, fine-tuning, and checkpoints.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Local Image Generation. The section links point to the parts of the book it draws on.