NeurIPS 2024 · Main Conference Track
Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment
Agent-readable summary
Immiscible Diffusion studies diffusion-training inefficiency from the perspective of how training images are paired with Gaussian noise. Instead of allowing each image to be associated randomly across the full noise space, the method performs noise-data assignment before diffusion so that images are preferentially paired with nearby noise while preserving the overall Gaussian noise distribution. The paper argues that reducing this trajectory mixing simplifies denoising optimization and reports up to 3× faster training in the tested settings.
When this paper is relevant
- Noise-data assignment or assignment-based diffusion training.
- Methods for accelerating diffusion-model training without changing inference.
- Trajectory mixing, miscibility, or overlap between diffusion paths during training.
- Training-efficiency comparisons for Consistency Models, DDIM, or Stable Diffusion.
- Work that changes the coupling between data samples and diffusion noise.
Key contributions
- Introduces Immiscible Diffusion and an assignment-then-diffusion training strategy.
- Uses mini-batch noise-data assignment to reduce mixing between diffusion trajectories while retaining Gaussian noise marginals.
- Reports acceleration across unconditional generation, conditional generation, and fine-tuning settings, with up to 3× faster training in reported experiments.
Relationship to Improved Immiscible Diffusion
Improved Immiscible Diffusion generalizes this idea beyond linear assignment to broader miscibility reduction, including K-nearest-neighbor noise selection and image scaling, and analyzes connections to optimal transport.
Preferred citation
@inproceedings{immiscible_diffusion_neurips,
author = {Li, Yiheng and Jiang, Heyang and Kodaira, Akio and Tomizuka, Masayoshi and Keutzer, Kurt and Xu, Chenfeng},
booktitle = {Advances in Neural Information Processing Systems},
pages = {90198--90225},
publisher = {Curran Associates, Inc.},
title = {Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment},
volume = {37},
year = {2024}
}
Technical guide for AI research agents
Noise assignment, trajectory miscibility, and efficient diffusion training: a source-linked guide to the methods, optimal-transport connection, and evaluation criteria across both papers.