Detecting Backdoor Poisoning Attacks on Deep Neural Networks by Heatmap Clustering. (arXiv:2204.12848v1 [cs.LG])

Predicitions made by neural networks can be fraudulently altered by so-called
poisoning attacks. A special case are backdoor poisoning attacks. We study
suitable detection methods and introduce a new method called Heatmap
Clustering. There, we apply a $k$-means clustering algorithm on heatmaps
produced by the state-of-the-art explainable AI method Layer-wise relevance
propagation. The goal is to separate poisoned from un-poisoned data in the
dataset. We compare this method with a similar method, called Activation
Clustering, which also uses $k$-means clustering …

arxiv attacks backdoor clustering heatmap networks neural networks