Enhancing Representations through Heterogeneous Self-Supervised Learning

arXiv:2310.05108v2 Announce Type: replace
Abstract: Incorporating heterogeneous representations from different architectures has facilitated various vision tasks, e.g., some hybrid networks combine transformers and convolutions. However, complementarity between such heterogeneous architectures has not been well exploited in self-supervised learning. Thus, we propose Heterogeneous Self-Supervised Learning (HSSL), which enforces a base model to learn from an auxiliary head whose architecture is heterogeneous from the base model. In this process, HSSL endows the base model with new characteristics in a representation learning way …

abstract architectures arxiv cs.cv however hybrid learn networks self-supervised learning supervised learning tasks through transformers type vision