Towards Building a Robust Toxicity Predictor

arXiv:2404.08690v1 Announce Type: cross
Abstract: Recent NLP literature pays little attention to the robustness of toxicity language predictors, while these systems are most likely to be used in adversarial contexts. This paper presents a novel adversarial attack, \texttt{ToxicTrap}, introducing small word-level perturbations to fool SOTA text classifiers to predict toxic text samples as benign. ToxicTrap exploits greedy based search strategies to enable fast and effective generation of toxic adversarial examples. Two novel goal function designs allow ToxicTrap to identify weaknesses …

abstract adversarial arxiv attention building classifiers cs.ai cs.cl cs.cr cs.lg language literature nlp novel paper robust robustness samples small sota systems text toxicity type word