Towards demystifying the creativity of diffusion models

Diffusion models generate novel data because neural networks learn a smoothed approximation of the score function rather than perfectly memorizing it. This smoothing prevents denoising trajectories from collapsing directly onto training examples, allowing samples to interpolate between them. In high-dimensional data spaces, it helps recover the underlying data manifold while preserving realism and novelty.

How Diffusion Models Denoise

• Training corrupts real samples with noise, then teaches a model to reverse the corruption step by step.
• The score function acts like a force field, directing noisy points toward meaningful data.
• A perfectly learned score function would pull every generated sample onto one of the training examples, resulting in memorization.
• In practice, neural networks learn an approximation of this function and therefore can generalize beyond the training set.

Score Smoothing Creates Interpolation

• Neural network regularization, including weight decay, makes sharp changes in the ideal score function difficult to represent.
• In a one-dimensional example with training points at -1 and +1:
  • The perfect score sharply switches direction at zero.
  • Generated points eventually collapse onto either -1 or +1.
  • A smoothed score creates a gentler transition near zero.
  • Points in this transition region move more slowly and can settle between the training points.
• This interpolation produces novel but plausible samples.
• Smoothing can arise from explicit regularization or implicit regularization caused by gradient-based optimization.

Recovering the Hidden Data Manifold

• Real images occupy a small, structured manifold within a much larger high-dimensional pixel space.
• Generating new images requires recovering this manifold from finite training data.
• Score smoothing behaves differently depending on direction:
  • Along the manifold, it slows movement toward individual training examples.
  • Toward the manifold, the score is already relatively smooth, so smoothing has little effect.
• This directional behavior prevents samples from becoming blurry in empty regions while reducing memorization along the manifold.
• The result is a balance between fidelity and creativity: generated outputs remain realistic while differing from the training examples.

Conclusion

The paper argues that diffusion-model creativity is a predictable mathematical consequence of score smoothing. Neural networks’ regularized, approximate learning allows denoising trajectories to interpolate across the data manifold instead of merely retrieving memorized samples.