Boter: Bootstrapping Knowledge Selection and Question Answering for Knowledge-based VQA

arXiv:2404.13947v1 Announce Type: new
Abstract: Knowledge-based Visual Question Answering (VQA) requires models to incorporate external knowledge to respond to questions about visual content. Previous methods mostly follow the "retrieve and generate" paradigm. Initially, they utilize a pre-trained retriever to fetch relevant knowledge documents, subsequently employing them to generate answers. While these methods have demonstrated commendable performance in the task, they possess limitations: (1) they employ an independent retriever to acquire knowledge solely based on the similarity between the query and …

abstract arxiv bootstrapping cs.cv documents fetch generate knowledge paradigm question question answering questions retriever them type visual vqa