Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement

arXiv:2402.09712v1 Announce Type: new
Abstract: Disentangled representation learning strives to extract the intrinsic factors within observed data. Factorizing these representations in an unsupervised manner is notably challenging and usually requires tailored loss functions or specific structural designs. In this paper, we introduce a new perspective and framework, demonstrating that diffusion models with cross-attention can serve as a powerful inductive bias to facilitate the learning of disentangled representations. We propose to encode an image to a set of concept tokens and …

abstract arxiv attention bias cs.ai cs.cv data designs diffusion diffusion model extract framework functions inductive intrinsic loss paper perspective representation representation learning type unsupervised