Zipformer: A faster and better encoder for automatic speech recognition

arXiv:2310.11230v3 Announce Type: replace-cross
Abstract: The Conformer has become the most popular encoder model for automatic speech recognition (ASR). It adds convolution modules to a transformer to learn both local and global dependencies. In this work we describe a faster, more memory-efficient, and better-performing transformer, called Zipformer. Modeling changes include: 1) a U-Net-like encoder structure where middle stacks operate at lower frame rates; 2) reorganized block structure with more modules, within which we re-use attention weights for efficiency; 3) a …

arxiv automatic speech recognition cs.lg cs.sd eess.as encoder faster recognition speech speech recognition type