An Analysis of Human Alignment of Latent Diffusion Models

arXiv:2403.08469v1 Announce Type: new
Abstract: Diffusion models, trained on large amounts of data, showed remarkable performance for image synthesis. They have high error consistency with humans and low texture bias when used for classification. Furthermore, prior work demonstrated the decomposability of their bottleneck layer representations into semantic directions. In this work, we analyze how well such representations are aligned to human responses on a triplet odd-one-out task. We find that despite the aforementioned observations: I) The representational alignment with humans …

abstract alignment analysis arxiv bias classification cs.lg data diffusion diffusion models error human humans image latent diffusion models layer low performance prior semantic synthesis texture type work