Diffscaler: Enhancing the Generative Prowess of Diffusion Transformers

arXiv:2404.09976v1 Announce Type: new
Abstract: Recently, diffusion transformers have gained wide attention with its excellent performance in text-to-image and text-to-vidoe models, emphasizing the need for transformers as backbone for diffusion models. Transformer-based models have shown better generalization capability compared to CNN-based models for general vision tasks. However, much less has been explored in the existing literature regarding the capabilities of transformer-based diffusion backbones and expanding their generative prowess to other datasets. This paper focuses on enabling a single pre-trained diffusion …

abstract arxiv attention capability cnn cs.cv diffusion diffusion models diffusion transformers general generative however image performance tasks text text-to-image transformer transformer-based models transformers type vision