A noise schedule controls how much noise a diffusion model adds at each forward step. Each step uses a beta value, βt, to gradually destroy the structure of the original data. As βt increases, the data contributes less and the accumulated noise contributes more. section 10.5 section 10.3
A schedule determines the behavior of the forward diffusion process:
- Assigns a beta value to every timestep from 0 through T. section 10.5
- Can increase noise gradually; the worked example uses a linear schedule from a small β0 to a larger βT. section 10.5
- Makes later data increasingly distorted while the relative effect of newly added noise becomes smaller as noise accumulates. section 10.5
- Affects training outcome and helps define the reverse process that learns to restore structure. section 10.5 section 10.3
A linear schedule in PyTorch
Use torch.linspace to create 1,000 beta values from 0.0001 to 0.02: betas = torch.linspace(0.0001, 0.02, 1000). The first value adds very little noise, while the final value adds substantially more. These values can then drive repeated calls to a diffusion step or a closed-form sampler.
# In[]:
def linear_beta_schedule(timesteps, start=0.0001, end=0.02):
return torch.linspace(start, end, timesteps)
T = 1000
betas = linear_beta_schedule(T)
print(betas[0], betas[10], betas[T-1])
# Out[]:
tensor(1.0000e-04) tensor(0.0003) tensor(0.0200)
Inspect the schedule values directly to see how beta grows from the first timestep to the last. section 10.5
Figure 10.7 PyTorch logo with all points diffused over 1,000 steps
Compare the visual effect of applying the schedule across 1,000 forward-diffusion steps. section 10.5
