A prompt is user-provided guidance that tells a model what to produce or which object or region to select. In image segmentation, prompts can be points, boxes, or masks; in inpainting, a text prompt describes the intended edit. section 15.3.1 section 2.2.1
Prompts clarify different parts of an image task:
- A text prompt says what the result should contain, such as turning a horse into a zebra. section 2.2.1
- A point, box, or mask identifies the object or region that a segmentation model should select. section 15.3.1
- In inpainting, the prompt describes the change, while the image supplies the starting scene and the mask limits where changes occur. section 2.2.1
Replacing a horse with a zebra
An inpainting pipeline can receive the prompt "a zebra replacing the original horse, same pose, same lighting, background unchanged" together with a horse image and mask. The prompt specifies the replacement; the mask confines the edit to the selected region. section 2.2.2
# In[5]:
mask_img = Image.open("../data/p1ch2/horse_mask.jpg")
prompt = ("a zebra replacing the original horse, same pose, same lighting, background unchanged")
negative = "distorted background, blurry, text, watermark" #1
out = pipe(
prompt=prompt,
image=img,
mask_image=mask_img,
negative_prompt=negative,
guidance_scale=7.5,
strength=0.8,
generator=torch.Generator(device).manual_seed(42)
)
Inspect a complete prompt passed alongside an input image, mask, negative prompt, and generation settings. section 2.2.2
