Feb. 15, 2024, 5:43 a.m. | Gr\'egoire Mialon, Randall Balestriero, Yann LeCun

cs.LG updates on arXiv.org arxiv.org

arXiv:2209.14905v2 Announce Type: replace
Abstract: Self-Supervised Learning (SSL) methods such as VICReg, Barlow Twins or W-MSE avoid collapse of their joint embedding architectures by constraining or regularizing the covariance matrix of their projector's output. This study highlights important properties of such strategy, which we coin Variance-Covariance regularization (VCReg). More precisely, we show that {\em VCReg combined to a MLP projector enforces pairwise independence between the features of the learned representation}. This result emerges by bridging VCReg applied on the projector's …

abstract architectures arxiv covariance cs.lg embedding highlights matrix regularization self-supervised learning ssl strategy study supervised learning twins type variance

Doctoral Researcher (m/f/div) in Automated Processing of Bioimages

@ Leibniz Institute for Natural Product Research and Infection Biology (Leibniz-HKI) | Jena

Seeking Developers and Engineers for AI T-Shirt Generator Project

@ Chevon Hicks | Remote

Software Engineer for AI Training Data (School Specific)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Python)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Tier 2)

@ G2i Inc | Remote

Principal Research Engineer - Materials

@ GKN Aerospace | Westlake, TX, US