The Lottery Ticket Hypothesis in Denoising: Towards Semantic-Driven Initialization

arXiv:2312.08872v2 Announce Type: replace
Abstract: Text-to-image diffusion models allow users control over the content of generated images. Still, text-to-image generation occasionally leads to generation failure requiring users to generate dozens of images under the same text prompt before they obtain a satisfying result. We formulate the lottery ticket hypothesis in denoising: randomly initialized Gaussian noise images contain special pixel blocks (winning tickets) that naturally tend to be denoised into specific content independently. The generation failure in standard text-to-image synthesis is …

abstract arxiv control cs.cv denoising diffusion diffusion models failure generate generated hypothesis image image diffusion image generation images leads lottery ticket hypothesis prompt semantic text text-to-image type