Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retraining. (arXiv:2110.08412v2 [cs.CL] UPDATED)

To explain NLP models, importance measures such as attention inform which
inputs tokens are important for a prediction are popular. However, an open
question is how well these explanations accurately reflect a model's logic, a
property called faithfulness.

To answer this question, we propose an new faithfulness benchmark called
Recursive ROAR. This works by recursively masking allegedly important tokens
and then retrain the model. The principle is, that this should result in worse
model performance compared to masking random tokens. …

arxiv importance nlp tokens