MCA: Moment Channel Attention Networks

arXiv:2403.01713v1 Announce Type: new
Abstract: Channel attention mechanisms endeavor to recalibrate channel weights to enhance representation abilities of networks. However, mainstream methods often rely solely on global average pooling as the feature squeezer, which significantly limits the overall potential of models. In this paper, we investigate the statistical moments of feature maps within a neural network. Our findings highlight the critical role of high-order moments in enhancing model capacity. Consequently, we introduce a flexible and comprehensive mechanism termed Extensive Moment …

abstract arxiv attention attention mechanisms cs.cv endeavor feature global maps moments networks paper pooling representation statistical type