Regularized Conditional Diffusion Model for Multi-Task Preference Alignment

arXiv:2404.04920v1 Announce Type: new
Abstract: Sequential decision-making is desired to align with human intents and exhibit versatility across various tasks. Previous methods formulate it as a conditional generation process, utilizing return-conditioned diffusion models to directly model trajectory distributions. Nevertheless, the return-conditioned paradigm relies on pre-defined reward functions, facing challenges when applied in multi-task settings characterized by varying reward functions (versatility) and showing limited controllability concerning human preferences (alignment). In this work, we adopt multi-task preferences as a unified condition for …

abstract alignment arxiv challenges cs.lg decision diffusion diffusion model diffusion models functions human making paradigm process tasks trajectory type