All are Worth Words: A ViT Backbone for Diffusion Models. (arXiv:2209.12152v2 [cs.CV] UPDATED)

Vision transformers (ViT) have shown promise in various vision tasks while
the U-Net based on a convolutional neural network (CNN) remains dominant in
diffusion models. We design a simple and general ViT-based architecture (named
U-ViT) for image generation with diffusion models. U-ViT is characterized by
treating all inputs including the time, condition and noisy image patches as
tokens and employing long skip connections between shallow and deep layers. We
evaluate U-ViT in unconditional and class-conditional image generation, as well
as …

arxiv diffusion diffusion models words