liveBook · Manning

CLIP

As explained in Local Image Generation

CLIP is a multimodal model trained to align textual and visual representations in a shared embedding space.

CLIP turns text into a dense numerical representation that can guide image generation. It works like a translator: prompts become numbers arranged to relate meaningfully to image representations. section 4.3.2

A prompt passes through CLIP in a few practical stages:

  • Tokenization splits the prompt into learned pieces and assigns them numerical identifiers. section 4.3.2
  • The encoder combines those token values into a dense representation that captures the prompt's meaning. section 4.3.2
  • Stable Diffusion uses that representation to steer denoising toward an image matching the prompt. section 4.3.2
  • Flux uses two CLIP encoder files, helping explain its stronger prompt adherence. section 10.1.2 section 10.5

A working Flux setup

Place clip_l.safetensors and t5xxl_fp16.safetensors in ComfyUI/models/clip, then generate an image from a prompt such as “crocheted red dragon, similar style to the famous painting by William Blake only a knitted stuffed animal. Wings spread in the air.” section 10.1.2

Figure 4.17 Closer look at the CLIP encoding.

Use the closer-look diagram to follow how tokenized text becomes CLIP encoding before it guides image generation. section 4.3.2

How Local Image Generation covers it

How deep it goes
Explains learned tokenization and dense encoding, then stops short of low-level neural-network internals.
Examples
Uses tokenized prompts, CLIP model files, ComfyUI directories, and a crocheted red dragon generation.
What it assumes
Assumes a basic Stable Diffusion workflow while teaching tokenization and text encoding from the ground up.
What you can do after
Operate/configure CLIP encoders in ComfyUI and understand how prompts influence generated images.
How it connects
Connects tokenization, VAE, Stable Diffusion, Flux, denoising, and prompt adherence.

Also explained in

Compare how 5 books explain CLIP →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Local Image Generation. The section links point to the parts of the book it draws on.