AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One

arXiv:2312.06709v4 Announce Type: replace
Abstract: A handful of visual foundation models (VFMs) have recently emerged as the backbones for numerous downstream tasks. VFMs like CLIP, DINOv2, SAM are trained with distinct objectives, exhibiting unique characteristics for various downstream tasks. We find that despite their conceptual differences, these models can be effectively merged into a unified model through multi-teacher distillation. We name this approach AM-RADIO (Agglomerative Model -- Reduce All Domains Into One). This integrative approach not only surpasses the performance …

abstract arxiv clip cs.cv differences domains foundation foundation model radio reduce sam tasks type vision visual