Kernelized Concept Erasure

arXiv:2201.12191v5 Announce Type: replace
Abstract: The representation space of neural models for textual data emerges in an unsupervised manner during training. Understanding how those representations encode human-interpretable concepts is a fundamental problem. One prominent approach for the identification of concepts in neural representations is searching for a linear subspace whose erasure prevents the prediction of the concept from the representations. However, while many linear erasure algorithms are tractable and interpretable, neural networks do not necessarily represent concepts in a linear …

abstract arxiv concept concepts cs.cl cs.lg data encode human identification linear prediction representation searching space textual training type understanding unsupervised