Investigating the Emergent Audio Classification Ability of ASR Foundation Models

arXiv:2311.09363v2 Announce Type: replace
Abstract: Text and vision foundation models can perform many tasks in a zero-shot setting, a desirable property that enables these systems to be applied in general and low-resource settings. There has been far less work, however, on the zero-shot abilities of ASR foundation models, with these systems typically fine-tuned to specific tasks or constrained to applications that match their training criterion and data annotation. In this work we investigate the ability of Whisper and MMS, ASR …

abstract arxiv asr audio classification cs.cl foundation general however low property systems tasks text type vision work zero-shot