Web: http://arxiv.org/abs/2206.10698

June 23, 2022, 1:12 a.m. | Jiachen Zhu, Rafael M. Moraes, Serkan Karakulak, Vlad Sobol, Alfredo Canziani, Yann LeCun

cs.CV updates on arXiv.org arxiv.org

We present Transformation Invariance and Covariance Contrast (TiCo) for
self-supervised visual representation learning. Similar to other recent
self-supervised learning methods, our method is based on maximizing the
agreement among embeddings of different distorted versions of the same image,
which pushes the encoder to produce transformation invariant representations.
To avoid the trivial solution where the encoder generates constant vectors, we
regularize the covariance matrix of the embeddings from different images by
penalizing low rank solutions. By jointly minimizing the transformation
invariance …

arxiv covariance cv learning representation representation learning transformation

More from arxiv.org / cs.CV updates on arXiv.org

Machine Learning Researcher - Saalfeld Lab

@ Howard Hughes Medical Institute - Chevy Chase, MD | Ashburn, Virginia

Project Director, Machine Learning in US Health

@ ideas42.org | Remote, US

Data Science Intern

@ NannyML | Remote

Machine Learning Engineer NLP/Speech

@ Play.ht | Remote

Research Scientist, 3D Reconstruction

@ Yembo | Remote, US

Clinical Assistant or Associate Professor of Management Science and Systems

@ University at Buffalo | Buffalo, NY