Unveiling Vulnerability of Self-Attention

arXiv:2402.16470v1 Announce Type: new
Abstract: Pre-trained language models (PLMs) are shown to be vulnerable to minor word changes, which poses a big threat to real-world systems. While previous studies directly focus on manipulating word inputs, they are limited by their means of generating adversarial samples, lacking generalization to versatile real-world attack. This paper studies the basic structure of transformer-based PLMs, the self-attention (SA) mechanism. (1) We propose a powerful perturbation technique \textit{HackAttend}, which perturbs the attention scores within the SA …

abstract adversarial arxiv attention basic big cs.cl focus inputs language language models paper samples self-attention studies systems threat type vulnerability vulnerable word world