all AI news
On the connection between Noise-Contrastive Estimation and Contrastive Divergence
Feb. 27, 2024, 5:45 a.m. | Amanda Olmin, Jakob Lindqvist, Lennart Svensson, Fredrik Lindsten
stat.ML updates on arXiv.org arxiv.org
Abstract: Noise-contrastive estimation (NCE) is a popular method for estimating unnormalised probabilistic models, such as energy-based models, which are effective for modelling complex data distributions. Unlike classical maximum likelihood (ML) estimation that relies on importance sampling (resulting in ML-IS) or MCMC (resulting in contrastive divergence, CD), NCE uses a proxy criterion to avoid the need for evaluating an often intractable normalisation constant.
Despite apparent conceptual differences, we show that two NCE criteria, ranking NCE (RNCE) and …
abstract arxiv cs.lg data divergence energy importance likelihood mcmc modelling noise popular sampling stat.ml type
More from arxiv.org / stat.ML updates on arXiv.org
Learning linear dynamical systems under convex constraints
3 days, 3 hours ago |
arxiv.org
Inverse Unscented Kalman Filter
4 days, 3 hours ago |
arxiv.org
Jobs in AI, ML, Big Data
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Codec Avatars Research Engineer
@ Meta | Pittsburgh, PA