Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech

arXiv:2309.09510v2 Announce Type: replace-cross
Abstract: Text language models have shown remarkable zero-shot capability in generalizing to unseen tasks when provided with well-formulated instructions. However, existing studies in speech processing primarily focus on limited or specific tasks. Moreover, the lack of standardized benchmarks hinders a fair comparison across different approaches. Thus, we present Dynamic-SUPERB, a benchmark designed for building universal speech models capable of leveraging instruction tuning to perform multiple tasks in a zero-shot fashion. To achieve comprehensive coverage of diverse …

abstract arxiv benchmark benchmarks capability collaborative comparison cs.lg cs.sd dynamic eess.as fair focus however language language models processing specific tasks speech speech processing studies tasks text type zero-shot