A Large-Scale Evaluation of Speech Foundation Models

arXiv:2404.09385v1 Announce Type: cross
Abstract: The foundation model paradigm leverages a shared foundation model to achieve state-of-the-art (SOTA) performance for various tasks, requiring minimal downstream-specific modeling and data annotation. This approach has proven crucial in the field of Natural Language Processing (NLP). However, the speech processing community lacks a similar setup to explore the paradigm systematically. In this work, we establish the Speech processing Universal PERformance Benchmark (SUPERB) to study the effectiveness of the paradigm for speech. We propose a …

abstract annotation art arxiv community cs.cl data data annotation eess.as eess.sp evaluation explore foundation foundation model however language language processing modeling natural natural language natural language processing nlp paradigm performance processing scale setup sota speech speech foundation models speech processing state tasks type