Web: http://arxiv.org/abs/2205.06226

May 13, 2022, 1:10 a.m. | Zixin Wen, Yuanzhi Li

stat.ML updates on arXiv.org arxiv.org

Recently the surprising discovery of the Bootstrap Your Own Latent (BYOL)
method by Grill et al. shows the negative term in contrastive loss can be
removed if we add the so-called prediction head to the network. This initiated
the research of non-contrastive self-supervised learning. It is mysterious why
even when there exist trivial collapsed global optimal solutions, neural
networks trained by (stochastic) gradient descent can still learn competitive
representations. This phenomenon is a typical example of implicit bias in deep …

arxiv head learning prediction self-supervised learning supervised learning

More from arxiv.org / stat.ML updates on arXiv.org

Data Analyst, Patagonia Action Works

@ Patagonia | Remote

Data & Insights Strategy & Innovation General Manager

@ Chevron Services Company, a division of Chevron U.S.A Inc. | Houston, TX

Faculty members in Research areas such as Bayesian and Spatial Statistics; Data Privacy and Security; AI/ML; NLP; Image and Video Data Analysis

@ Ahmedabad University | Ahmedabad, India

Director, Applied Mathematics & Computational Research Division

@ Lawrence Berkeley National Lab | Berkeley, Ca

Business Data Analyst

@ MainStreet Family Care | Birmingham, AL

Assistant/Associate Professor of the Practice in Business Analytics

@ Georgetown University McDonough School of Business | Washington DC