ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions

arXiv:2403.07392v2 Announce Type: replace
Abstract: Although Vision Transformer (ViT) has achieved significant success in computer vision, it does not perform well in dense prediction tasks due to the lack of inner-patch information interaction and the limited diversity of feature scale. Most existing studies are devoted to designing vision-specific transformers to solve the above problems, which introduce additional pre-training costs. Therefore, we present a plain, pre-training-free, and feature-enhanced ViT backbone with Convolutional Multi-scale feature interaction, named ViT-CoMer, which facilitates bidirectional interaction …

arxiv cs.cv feature predictions scale transformer type vision vit