EEE-QA: Exploring Effective and Efficient Question-Answer Representations

arXiv:2403.02176v1 Announce Type: new
Abstract: Current approaches to question answering rely on pre-trained language models (PLMs) like RoBERTa. This work challenges the existing question-answer encoding convention and explores finer representations. We begin with testing various pooling methods compared to using the begin-of-sentence token as a question representation for better quality. Next, we explore opportunities to simultaneously embed all answer candidates with the question. This enables cross-reference between answer choices and improves inference throughput via reduced memory usage. Despite their simplicity …

abstract arxiv challenges convention cs.cl current encoding explore language language models next pooling quality question question answering representation roberta testing token type work