Web: http://arxiv.org/abs/2204.07172

June 24, 2022, 1:11 a.m. | Gabriel Loaiza-Ganem, Brendan Leigh Ross, Jesse C. Cresswell, Anthony L. Caterini

stat.ML updates on arXiv.org arxiv.org

Likelihood-based, or explicit, deep generative models use neural networks to
construct flexible high-dimensional densities. This formulation directly
contradicts the manifold hypothesis, which states that observed data lies on a
low-dimensional manifold embedded in high-dimensional ambient space. In this
paper we investigate the pathologies of maximum-likelihood training in the
presence of this dimensionality mismatch. We formally prove that degenerate
optima are achieved wherein the manifold itself is learned but not the
distribution on it, a phenomenon we call manifold overfitting. We …

arxiv deep deep generative models manifold ml models overfitting

More from arxiv.org / stat.ML updates on arXiv.org

Machine Learning Researcher - Saalfeld Lab

@ Howard Hughes Medical Institute - Chevy Chase, MD | Ashburn, Virginia

Project Director, Machine Learning in US Health

@ ideas42.org | Remote, US

Data Science Intern

@ NannyML | Remote

Machine Learning Engineer NLP/Speech

@ Play.ht | Remote

Research Scientist, 3D Reconstruction

@ Yembo | Remote, US

Clinical Assistant or Associate Professor of Management Science and Systems

@ University at Buffalo | Buffalo, NY