An Embarrassingly Simple Approach for LLM with Strong ASR Capacity

arXiv:2402.08846v1 Announce Type: new
Abstract: In this paper, we focus on solving one of the most important tasks in the field of speech processing, i.e., automatic speech recognition (ASR), with speech foundation encoders and large language models (LLM). Recent works have complex designs such as compressing the output temporally for the speech encoder, tackling modal alignment for the projector, and utilizing parameter-efficient fine-tuning for the LLM. We found that delicate designs are not necessary, while an embarrassingly simple composition of …

abstract arxiv asr automatic speech recognition capacity cs.ai cs.cl cs.mm cs.sd designs eess.as focus foundation language language models large language large language models llm paper processing recognition simple speech speech processing speech recognition tasks type