Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models

arXiv:2405.05252v1 Announce Type: cross
Abstract: Diffusion Models (DMs) have exhibited superior performance in generating high-quality and diverse images. However, this exceptional performance comes at the cost of expensive architectural design, particularly due to the attention module heavily used in leading models. Existing works mainly adopt a retraining process to enhance DM efficiency. This is computationally expensive and not very scalable. To this end, we introduce the Attention-driven Training-free Efficient Diffusion Model (AT-EDM) framework that leverages attention maps to perform run-time …

arxiv attention cs.ai cs.cv cs.lg diffusion diffusion models eess.iv eess.sp efficiency free training type