The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling

arXiv:2402.15170v1 Announce Type: new
Abstract: With the incorporation of the UNet architecture, diffusion probabilistic models have become a dominant force in image generation tasks. One key design in UNet is the skip connections between the encoder and decoder blocks. Although skip connections have been shown to improve training stability and model performance, we reveal that such shortcuts can be a limiting factor for the complexity of the transformation. As the sampling steps decrease, the generation process and the role of …

abstract architecture arxiv become cs.ai cs.lg decoder design diffusion encoder image image generation key performance sampling stability tasks training type unet