Subtractive Mixture Models via Squaring: Representation and Learning

arXiv:2310.00724v2 Announce Type: replace
Abstract: Mixture models are traditionally represented and learned by adding several distributions as components. Allowing mixtures to subtract probability mass or density can drastically reduce the number of components needed to model complex distributions. However, learning such subtractive mixtures while ensuring they still encode a non-negative function is challenging. We investigate how to learn and perform inference on deep subtractive mixtures by squaring them. We do this in the framework of probabilistic circuits, which enable us …

abstract arxiv components cs.ai cs.lg encode function however negative probability reduce representation type via