Huawei’s DiffFit Unlocks the Transferability of Large Diffusion Models to New Domains

In the new paper DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning, a Huawei Noah’s Ark Lab research team introduces DiffFit, a parameter-efficient fine-tuning technique that enables fast adaptation to new domains for diffusion image generation. Compared to full fine-tuning approaches, DiffFit achieves 2x training speed-ups while using only ~0.12 percent of trainable parameters.

ai artificial intelligence deep-neural-networks diffusion diffusion model diffusion models fine-tuning huawei image image generation lab machine learning machine learning & data science ml paper research research team speed team technology training transfer learning ups