Skipformer: A Skip-and-Recover Strategy for Efficient Speech Recognition

arXiv:2403.08258v1 Announce Type: cross
Abstract: Conformer-based attention models have become the de facto backbone model for Automatic Speech Recognition tasks. A blank symbol is usually introduced to align the input and output sequences for CTC or RNN-T models. Unfortunately, the long input length overloads computational budget and memory consumption quadratically by attention mechanism. In this work, we propose a "Skip-and-Recover" Conformer architecture, named Skipformer, to squeeze sequence input length dynamically and inhomogeneously. Skipformer uses an intermediate CTC output as criteria …

abstract arxiv attention automatic speech recognition become budget computational consumption cs.cl cs.lg memory memory consumption recognition rnn speech speech recognition strategy tasks type