Deep Variational Multivariate Information Bottleneck -- A Framework for Variational Losses

Variational dimensionality reduction methods are known for their high accuracy, generative abilities, and robustness. We introduce a framework to unify many existing variational methods and design new ones. The framework is based on an interpretation of the multivariate information bottleneck, in which an encoder graph, specifying what information to compress, is traded-off against a decoder graph, specifying a generative model. Using this framework, we rederive existing dimensionality reduction methods including the deep variational information bottleneck and variational auto-encoders. The framework …

accuracy cond-mat.stat-mech cs.it cs.lg design dimensionality encoder framework generative graph information interpretation losses math.it multivariate physics.data-an robustness