SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalization

Vision Transformers (ViTs) have gained prominence as a preferred choice for a wide range of computer vision tasks due to their exceptional performance. However, their widespread adoption has raised concerns about security in the face of malicious attacks. Most existing methods rely on empirical adjustments during the training process, lacking a clear theoretical foundation. In this study, we address this gap by introducing SpecFormer, specifically designed to enhance ViTs' resilience against adversarial attacks, with support from carefully derived theoretical guarantees. …

adoption attacks computer computer vision concerns cs.cv cs.lg face performance process robustness security singular tasks training transformer transformers value via vision vision transformers