Denoising Diffusion Probabilistic Models

Inside the paper

The problem

Diffusion models had a tractable training setup, but convincing image samples had been difficult to demonstrate. The authors also wanted to understand how sample quality relates to likelihood and the amount of information preserved during generation.

The idea

The model learns to undo a fixed process that gradually corrupts images with Gaussian noise. Its practical training objective asks a neural network to predict the noise added at a chosen stage of that process.

How it works

• The forward process adds small amounts of Gaussian noise according to a prescribed schedule until the image approaches an unstructured noise sample.
• During training, choose an image and a noise level, construct the corresponding noisy image directly, and teach the network to estimate the noise.
• For generation, start with Gaussian noise and repeatedly apply learned reverse transitions, producing a less noisy image at each step.
• The authors connect this parameterization to denoising score matching and examine the intermediate steps as a form of progressive lossy decoding.

What it showed

The paper reports an FID of 3.17 on unconditional CIFAR10 and shows samples on several image datasets. It also finds that good sample quality does not by itself imply competitive likelihood, a distinction visible in its compression analysis.

Read it for

Read Sections 2 and 3.2 alongside Algorithms 1 and 2 to follow training and sampling, then look at Section 4 and its samples. The detailed variational derivation can be skimmed on a first pass and revisited for the objective’s weighting.

Written in our own words from the paper itself. A draft the editors have not yet reviewed.