Classifier-free guidance (CFG) steers diffusion sampling without a separately trained classifier. Instead, one denoising model learns to predict with and without conditioning information. At generation time, its conditional and unconditional predictions are combined to control how strongly an image follows a label or text prompt. section 6.1.1 section 6.1.2
A class-conditioned implementation has three stages:
- Train a denoising U-Net on noisy images, time steps, and clothing-item labels. Randomly mask some labels so the same model also learns unconditional denoising. section 6.3 section 6.3.2
- During sampling, evaluate each noisy image with its requested label and with that label masked. The two outputs supply conditional and unconditional noise predictions. section 6.3.2
- Combine those predictions using a guidance-strength parameter, then continue the reverse diffusion process. Adjust the parameter to trade variety against adherence to the condition. section 6.1.2 section 9.5
Generating a requested clothing item
To request a sandal rather than a random Fashion MNIST item, sample with the sandal’s class label. A conditional branch uses that label; an unconditional branch masks it. The guide_w parameter controls the strength of label guidance. The same principle later applies to open-ended text prompts. section 6.3.2 section 6.1.1
Listing 6.6 Defining a sample() function to generate images
import numpy as np
@torch.no_grad()
def sample(ddpm, model, n_sample, size, device,
guide_w = 0.0, step_size=1):
x_i = torch.randn(n_sample, *size).to(device)
c_i = torch.arange(0,10).to(device) #1
c_i = c_i.repeat(int(n_sample/c_i.shape[0]))
context_mask = torch.zeros_like(c_i).to(device)
c_i = c_i.repeat(2)
context_mask = context_mask.repeat(2)
context_mask[n_sample:] = 1. #2
x_i_store = []
for i in range(ddpm.n_T, 0, -step_size):
t_is = torch.tensor([i / ddpm.n_T]).to(device)
t_is = t_is.repeat(n_sample,1,1,1)
x_i = x_i.repeat(2,1,1,1)
t_is = t_is.repeat(2,1,1,1)
z = torch.randn(n_sample,*size).to(device) if i>1 else 0
eps = model(x_i, c_i, t_is, context_mask)
eps1 = eps[:n_sample] #3
eps2 = eps[n_sample:] #4
eps = (1+guide_w)*eps1 - guide_w*eps2 #5
x_i = x_i[:n_sample]
x_i = (ddpm.oneover_sqrta[i] * (x_i -
eps * ddpm.mab_over_sqrtmab[i])
+ ddpm.sqrt_beta_t[i] * z)
if i%20==0 or i==ddpm.n_T or i<8:
x_i_store.append(x_i.detach().cpu().numpy())
x_i_store = np.array(x_i_store)
return x_i, x_i_store
The sample() listing duplicates labels and noisy images, masks the second half of the batch, and separates the resulting predictions into conditional and unconditional halves. section 6.3.2
Increasing guidance can make output more faithful to its condition; lower guidance permits more diversity. In latent text-to-image generation, cfg_scale exposes this choice for text prompts. CFG removes the external classifier, but it still depends on a diffusion model trained for both kinds of prediction. section 9.5 section 6.1
