An image caption is text that describes an image’s visual content, attributes, or relationships. In generative image modeling, the caption forms the language half of an image–text pair, helping the model connect words with pixels. section 12.4 section 12.1
Captions affect what a model can learn. A massive, diverse collection of accurately captioned images helps it associate language with subjects, visual styles, composition, textures, and other image attributes. Poor or irrelevant captions weaken that connection, so captions are cleaned and standardized alongside the images. section 12.5 section 12.3
Captions support several parts of an image-generation workflow:
- Pair language with images during dataset preparation. section 12.4
- Guide denoising while a model learns to predict the noise added to an image. section 12.3
- Provide the language signal used to generate a new image from random noise. section 12.3
From a weak caption to useful training data
A pet-blog image might be a high-resolution landscape JPG with a watermark and the caption “img_2207.” That caption says almost nothing about the visual content. Cleaning the image-text pair removes distracting overlays, standardizes the image, and replaces or filters the uninformative caption so the model can learn from the image itself rather than irrelevant artifacts. section 12.6
Figure 12.8 Preparing image data in the diffusion model training workflow
Use the workflow figure to see where image captions enter the preparation and training process. section 12.4
