Gradient Guidance for Diffusion Models: An Optimization Perspective

arXiv:2404.14743v1 Announce Type: cross
Abstract: Diffusion models have demonstrated empirical successes in various applications and can be adapted to task-specific needs via guidance. This paper introduces a form of gradient guidance for adapting or fine-tuning diffusion models towards user-specified optimization objectives. We study the theoretic aspects of a guided score-based sampling process, linking the gradient-guided diffusion model to first-order optimization. We show that adding gradient guidance to the sampling process of a pre-trained diffusion model is essentially equivalent to solving …

abstract applications arxiv cs.lg diffusion diffusion models fine-tuning form gradient guidance optimization paper perspective process sampling stat.ml study type via