Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

Inside the paper

The problem

A robot may have several valid responses to the same visual observation, and a single averaged action can be a poor choice. Predicting a sequence also requires balancing consistent movement with responsiveness to new observations.

The idea

The authors learn a distribution over action sequences by adapting a denoising diffusion model to robot demonstrations. Visual observations condition the process that turns noise into actions.

How it works

• Add noise to demonstrated action sequences during training and train a network to predict the added noise given the observations and noise level.
• At execution, start with random noise and repeatedly apply the learned denoising steps to generate a candidate action sequence.
• Execute only part of the predicted sequence, then observe again and generate a new plan to react to changes.
• The paper evaluates convolutional and transformer-based noise predictors, with visual features supplied as conditioning rather than generated as outputs.

What it showed

The reported benchmark covers simulated and physical manipulation tasks and compares the method with other imitation-learning policies. The paper also studies how execution horizon and architecture affect behavior, including tasks where actions have multiple plausible modes.

Read it for

Start with Figure 2 and Section 2.3 for the action-generation loop, then read the benchmark results and analysis in Sections 4–5. The architecture details in Section 3 can wait until you want to implement it.

Written in our own words from the paper itself. A draft the editors have not yet reviewed.