Learning Sparse Codes with Entropy-Based ELBOs

arXiv:2311.01888v2 Announce Type: replace-cross
Abstract: Standard probabilistic sparse coding assumes a Laplace prior, a linear mapping from latents to observables, and Gaussian observable distributions. We here derive a solely entropy-based learning objective for the parameters of standard sparse coding. The novel variational objective has the following features: (A) unlike MAP approximations, it uses non-trivial posterior approximations for probabilistic inference; (B) unlike for previous non-trivial approximations, the novel objective is fully analytical; and (C) the objective allows for a novel principled …

abstract arxiv coding cs.lg entropy features linear map mapping novel observable parameters prior standard stat.ml type