Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition

arXiv:2312.17279v3 Announce Type: replace
Abstract: In this paper, we propose an efficient and accurate streaming speech recognition model based on the FastConformer architecture. We adapted the FastConformer architecture for streaming applications through: (1) constraining both the look-ahead and past contexts in the encoder, and (2) introducing an activation caching mechanism to enable the non-autoregressive encoder to operate autoregressively during inference. The proposed model is thoughtfully designed in a way to eliminate the accuracy disparity between the train and inference time …

abstract applications architecture arxiv automatic speech recognition cache caching cs.cl eess.as encoder inference look paper recognition speech speech recognition streaming through type