Mamba-R: Vision Mamba ALSO Needs Registers

arXiv:2405.14858v1 Announce Type: new
Abstract: Similar to Vision Transformers, this paper identifies artifacts also present within the feature maps of Vision Mamba. These artifacts, corresponding to high-norm tokens emerging in low-information background areas of images, appear much more severe in Vision Mamba -- they exist prevalently even with the tiny-sized model and activate extensively across background regions. To mitigate this issue, we follow the prior solution of introducing register tokens into Vision Mamba. To better cope with Mamba blocks' uni-directional …

abstract arxiv cs.cv feature images information low mamba maps norm paper tokens transformers type vision vision transformers