Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

arXiv:2404.02905v1 Announce Type: new
Abstract: We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution prediction", diverging from the standard raster-scan "next-token prediction". This simple, intuitive methodology allows autoregressive (AR) transformers to learn visual distributions fast and generalize well: VAR, for the first time, makes AR models surpass diffusion transformers in image generation. On ImageNet 256x256 benchmark, VAR significantly improve AR baseline by improving Frechet inception distance …

abstract arxiv cs.ai cs.cv image image generation images learn methodology modeling next paradigm prediction resolution scalable scale simple standard token transformers type via visual