RealTime QA: What's the Answer Right Now?

arXiv:2207.13332v2 Announce Type: replace
Abstract: We introduce REALTIME QA, a dynamic question answering (QA) platform that announces questions and evaluates systems on a regular basis (weekly in this version). REALTIME QA inquires about the current world, and QA systems need to answer questions about novel events or information. It therefore challenges static, conventional assumptions in open-domain QA datasets and pursues instantaneous applications. We build strong baseline models upon large pretrained language models, including GPT-3 and T5. Our benchmark is an …

abstract arxiv challenges cs.cl current dynamic events information novel platform question question answering questions realtime systems type world