Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks

arXiv:2403.00644v1 Announce Type: new
Abstract: Diffusion models trained on large-scale datasets have achieved remarkable progress in image synthesis. However, due to the randomness in the diffusion process, they often struggle with handling diverse low-level tasks that require details preservation. To overcome this limitation, we present a new Diff-Plugin framework to enable a single pre-trained diffusion model to generate high-fidelity results across a variety of low-level tasks. Specifically, we first propose a lightweight Task-Plugin module with a dual branch design to …

abstract arxiv cs.cv datasets diff diffusion diffusion models diverse framework image low plugin preservation process progress randomness scale struggle synthesis tasks type