Echotune: A Modular Extractor Leveraging the Variable-Length Nature of Speech in ASR Tasks

arXiv:2309.07765v2 Announce Type: replace-cross
Abstract: The Transformer architecture has proven to be highly effective for Automatic Speech Recognition (ASR) tasks, becoming a foundational component for a plethora of research in the domain. Historically, many approaches have leaned on fixed-length attention windows, which becomes problematic for varied speech samples in duration and complexity, leading to data over-smoothing and neglect of essential long-term connectivity. Addressing this limitation, we introduce Echo-MSA, a nimble module equipped with a variable-length attention mechanism that accommodates a …

abstract architecture arxiv asr attention automatic speech recognition cs.cl cs.sd domain eess.as foundational modular nature recognition research speech speech recognition tasks transformer transformer architecture type windows