The Impact of Quantization on the Robustness of Transformer-based Text Classifiers

arXiv:2403.05365v1 Announce Type: cross
Abstract: Transformer-based models have made remarkable advancements in various NLP areas. Nevertheless, these models often exhibit vulnerabilities when confronted with adversarial attacks. In this paper, we explore the effect of quantization on the robustness of Transformer-based models. Quantization usually involves mapping a high-precision real number to a lower-precision value, aiming at reducing the size of the model at hand. To the best of our knowledge, this work is the first application of quantization on the robustness …

abstract adversarial adversarial attacks arxiv attacks classifiers cs.cl cs.lg explore impact mapping nlp paper precision quantization robustness text transformer type vulnerabilities