Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning

arXiv:2404.10282v1 Announce Type: new
Abstract: Inductive biases are crucial in disentangled representation learning for narrowing down an underspecified solution set. In this work, we consider endowing a neural network autoencoder with three select inductive biases from the literature: data compression into a grid-like latent space via quantization, collective independence amongst latents, and minimal functional influence of any latent on how other latents determine data generation. In principle, these inductive biases are deeply complementary: they most directly specify properties of the …

abstract arxiv autoencoder biases collective compression cs.cv cs.lg data data compression grid inductive literature network neural network quantization representation representation learning set solution space type via work