Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models

While the Large Language Models (LLMs) dominate a majority of language understanding tasks, previous work shows that some of these results are supported by modelling spurious correlations of training datasets. Authors commonly assess model robustness by evaluating their models on out-of-distribution (OOD) datasets of the same task, but these datasets might share the bias of the training dataset.
We propose a simple method for measuring a scale of models' reliance on any identified spurious feature and assess the robustness towards …

authors correlations cs.ai cs.cl datasets distribution efficiency language language models language understanding large language large language models llms measuring modelling model robustness prediction question question answering robustness shows tasks think training understanding work