Diffusion Reverse Process

The reverse process is where all the actual learning happens: a neural network is trained to undo the forward process's noising, one small step at a time, gradually transforming pure noise back into a realistic sample.

Formula

Given a noisy image \(\mathbf{x}_t\), the model predicts the parameters of a distribution over the slightly-less-noisy previous step \(\mathbf{x}_{t-1}\). \(\theta\) denotes this is computed by a trained neural network — this is the learnable half of the diffusion model, mirroring the forward process's fixed formula but running in reverse.

Generation — Starting From Pure Noise

1. Sample \(\mathbf{x}_T \sim \mathcal{N}(0,\mathbf{I})\) — start from pure random noise.
2. For \(t = T, T-1, \ldots, 1\): use the trained model to predict \(\mathbf{x}_{t-1}\) from \(\mathbf{x}_t\), gradually removing a small amount of noise at each step.
3. After all \(T\) steps, \(\mathbf{x}_0\) is the final generated sample — ideally a realistic, novel image.

This is exactly why diffusion generation is slow — unlike a GAN's single forward pass, this loop must execute sequentially, once per noise level, to produce one final sample.

Diagram

Code — The Generation Loop

import torch

@torch.no_grad()
def generate(model, T, betas, alphas, alpha_bar, image_shape):
    x_t = torch.randn(image_shape)   # start from PURE noise

    for t in reversed(range(T)):
        predicted_noise = model(x_t, t)   # the trained network predicts the noise present at this step
        alpha_t = alphas[t]
        alpha_bar_t = alpha_bar[t]
        beta_t = betas[t]

        # Use the predicted noise to estimate a slightly LESS noisy x_{t-1}
        noise_term = beta_t ** 0.5 * torch.randn_like(x_t) if t > 0 else 0
        x_t = (1 / alpha_t ** 0.5) * (x_t - (1 - alpha_t) / (1 - alpha_bar_t) ** 0.5 * predicted_noise) + noise_term

    return x_t   # after T steps, x_t IS x_0 -- the final generated sample

Common Mistakes

• Assuming the reverse process directly predicts the clean image \(\mathbf{x}_0\) in a single step from \(\mathbf{x}_t\) — it predicts only a small denoising step toward \(\mathbf{x}_{t-1}\), applied repeatedly; this incremental approach is exactly what makes the learning problem at each step tractable (as opposed to the much harder problem of denoising an arbitrarily noisy image in one shot).
• Forgetting the reverse process must run in exactly the opposite order of the forward process's noise schedule — starting from \(t=T\) (most noise) and working down to \(t=0\) (clean).

Interview Relevance

Q: "Why does diffusion generation require many sequential steps, rather than one direct denoising pass?" Directly predicting a clean image from pure noise in a single step is an extremely difficult, underdetermined problem. Breaking the reverse process into many small, incremental denoising steps — each only slightly reducing the noise level — makes each individual step's prediction task far more tractable for the network to learn well, at the cost of requiring many sequential passes to complete generation.

Practice Question

What happens to a diffusion model's generated image quality if you drastically reduce the number of reverse process steps (e.g. from 1000 to 10) without any other changes?

Related DL Notes

• Diffusion Forward Process
• Noise Prediction
• U-Net (Diffusion)