A Geometric Explanation of the Likelihood OOD Detection Paradox

arXiv:2403.18910v1 Announce Type: new
Abstract: Likelihood-based deep generative models (DGMs) commonly exhibit a puzzling behaviour: when trained on a relatively complex dataset, they assign higher likelihood values to out-of-distribution (OOD) data from simpler sources. Adding to the mystery, OOD samples are never generated by these DGMs despite having higher likelihoods. This two-pronged paradox has yet to be conclusively explained, making likelihood-based OOD detection unreliable. Our primary observation is that high-likelihood regions will not be generated if they contain minimal probability …

arxiv cs.ai cs.cv cs.lg detection likelihood paradox stat.ml type