Mix-Pooling Strategy for Attention Mechanism. (arXiv:2208.10322v2 [cs.LG] UPDATED)

Recently many effective attention modules are proposed to boot the model
performance by exploiting the internal information of convolutional neural
networks in computer vision. In general, many previous works ignore considering
the design of the pooling strategy of the attention mechanism since they adopt
the global average pooling for granted, which hinders the further improvement
of the performance of the attention mechanism. However, we empirically find and
verify a phenomenon that the simple linear combination of global max-pooling
and global …

arxiv attention pooling strategy