AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer

arXiv:2309.12689v3 Announce Type: replace
Abstract: Mixup is an effective data augmentation method that generates new augmented samples by aggregating linear combinations of different original samples. However, if there are noises or aberrant features in the original samples, Mixup may propagate them to the augmented samples, leading to over-sensitivity of the model to these outliers . To solve this problem, this paper proposes a new Mixup method called AMPLIFY. This method uses the Attention mechanism of Transformer itself to reduce the …

amplify arxiv attention cs.cl cs.lg improvement performance transformer type