Pixel-wise RL on Diffusion Models: Reinforcement Learning from Rich Feedback

arXiv:2404.04356v1 Announce Type: new
Abstract: Latent diffusion models are the state-of-the-art for synthetic image generation. To align these models with human preferences, training the models using reinforcement learning on human feedback is crucial. Black et. al 2024 introduced denoising diffusion policy optimisation (DDPO), which accounts for the iterative denoising nature of the generation by modelling it as a Markov chain with a final reward. As the reward is a single value that determines the model's performance on the entire image, …

abstract art arxiv cs.cv cs.lg denoising diffusion diffusion models feedback human human feedback image image generation iterative latent diffusion models nature optimisation pixel policy reinforcement reinforcement learning state synthetic training type wise