liveBook · Manning

Linear noise schedule

As explained in Build a Text-to-Image Generator (from Scratch)

A linear diffusion schedule is a noise schedule in which the noise parameter changes linearly across diffusion time steps.

A linear noise schedule increases the variance parameter βₜ linearly across diffusion time steps. During forward diffusion, it gradually replaces a clean image with random noise; the resulting schedule is simple and easy to implement. section 5.1 section 5.5

With T = 1,000 steps, βₜ ranges from β₁ = 0.0001 to βT = 0.02. The schedule derives αₜ = 1 − βₜ, accumulates these values into ᾱₜ, and produces weights for the clean image and injected noise. section 5.1.2

Listing 5.1 The linear diffusion schedule
import torch

T=1000
t=torch.arange(0, T + 1,dtype=torch.float32)/T
def linear_scheduler():
    beta1, beta2 = 0.0001, 0.02
    beta_t = (beta2 - beta1) * t  + beta1  #1
    alpha_t = 1 - beta_t
    log_alpha_t = torch.log(alpha_t)
    alphabar_t = torch.cumsum(log_alpha_t, dim=0).exp()
    sqrtab = torch.sqrt(alphabar_t)  #2
    sqrtmab = torch.sqrt(1 - alphabar_t)  #3
    return {„sqrtab": sqrtab,"sqrtmab": sqrtmab}

Builds the schedule in PyTorch by interpolating βₜ, accumulating log αₜ, and returning √ᾱₜ and √(1 − ᾱₜ). section 5.1.2

Forward diffusion across image timesteps

Apply the schedule to bird, dog, mountain, and horse images at t = 0, 200, 400, 600, 800, and 1,000. Increasing t injects progressively more noise until the images become random-noise-like. section 5.1.2

Compared with a cosine schedule, the linear schedule changes the noise and clean-image weights less uniformly: the noise weight becomes flat after roughly t/T = 0.6, while the clean-image weight flattens after roughly 0.8. The cosine alternative produced better image quality in the cited comparison. section 5.1.3

How Build a Text-to-Image Generator (from Scratch) covers it

How deep it goes
Defines βₜ, αₜ, cumulative ᾱₜ, and image-noising code, then compares behavior with cosine scheduling.
Examples
Demonstrates staged noise addition on bird, dog, mountain, horse, and bunny images using plotted timestep comparisons.
What you can do after
Build it from scratch in PyTorch and visualize scheduled noise addition across image timesteps.
How it connects
Connects forward diffusion, reverse diffusion, denoising diffusion models, U-Nets, and cosine diffusion schedules.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Text-to-Image Generator (from Scratch). The section links point to the parts of the book it draws on.