FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering

arXiv:2404.18585v1 Announce Type: new
Abstract: Table Question Answering (TQA) aims at composing an answer to a question based on tabular data. While prior research has shown that TQA models lack robustness, understanding the underlying cause and nature of this issue remains predominantly unclear, posing a significant obstacle to the development of robust TQA systems. In this paper, we formalize three major desiderata for a fine-grained evaluation of robustness of TQA systems. They should (i) answer questions regardless of alterations in …

abstract arxiv benchmark cs.cl data development evaluation fine-grained issue nature prior question question answering research robustness table tabular tabular data type understanding while