MVEB: Self-Supervised Learning with Multi-View Entropy Bottleneck

arXiv:2403.19078v1 Announce Type: new
Abstract: Self-supervised learning aims to learn representation that can be effectively generalized to downstream tasks. Many self-supervised approaches regard two views of an image as both the input and the self-supervised signals, assuming that either view contains the same task-relevant information and the shared information is (approximately) sufficient for predicting downstream tasks. Recent studies show that discarding superfluous information not shared between the views can improve generalization. Hence, the ideal representation is sufficient for downstream tasks …

abstract arxiv cs.ai cs.cv entropy generalized image information learn regard representation self-supervised learning supervised learning tasks type view