The Role of Syntactic Span Preferences in Post-Hoc Explanation Disagreement

arXiv:2403.19424v1 Announce Type: new
Abstract: Post-hoc explanation methods are an important tool for increasing model transparency for users. Unfortunately, the currently used methods for attributing token importance often yield diverging patterns. In this work, we study potential sources of disagreement across methods from a linguistic perspective. We find that different methods systematically select different classes of words and that methods that agree most with other methods and with humans display similar linguistic preferences. Token-level differences between methods are smoothed out …

abstract arxiv cs.ai cs.cl importance patterns perspective role study token tool transparency type work