LibriSQA: A Novel Dataset and Framework for Spoken Question Answering with Large Language Models

arXiv:2308.10390v4 Announce Type: replace
Abstract: While Large Language Models (LLMs) have demonstrated commendable performance across a myriad of domains and tasks, existing LLMs still exhibit a palpable deficit in handling multimodal functionalities, especially for the Spoken Question Answering (SQA) task which necessitates precise alignment and deep interaction between speech and text features. To address the SQA challenge on LLMs, we initially curated the free-form and open-ended LibriSQA dataset from Librispeech, comprising Part I with natural conversational formats and Part II …

arxiv cs.cl dataset framework language language models large language large language models novel question question answering spoken type