Structure by Architecture: Structured Representations without Regularization

arXiv:2006.07796v4 Announce Type: replace
Abstract: We study the problem of self-supervised structured representation learning using autoencoders for downstream tasks such as generative modeling. Unlike most methods which rely on matching an arbitrary, relatively unstructured, prior distribution for sampling, we propose a sampling technique that relies solely on the independence of latent variables, thereby avoiding the trade-off between reconstruction quality and generative performance typically observed in VAEs. We design a novel autoencoder architecture capable of learning a structured representation without the …

abstract architecture arxiv autoencoders cs.cv cs.lg distribution generative generative modeling modeling prior regularization representation representation learning sampling stat.ml study tasks type unstructured variables