Language-Universal Speech Attributes Modeling for Zero-Shot Multilingual Spoken Keyword Recognition

arXiv:2406.02488v1 Announce Type: cross
Abstract: We propose a novel language-universal approach to end-to-end automatic spoken keyword recognition (SKR) leveraging upon (i) a self-supervised pre-trained model, and (ii) a set of universal speech attributes (manner and place of articulation). Specifically, Wav2Vec2.0 is used to generate robust speech representations, followed by a linear output layer to produce attribute sequences. A non-trainable pronunciation model then maps sequences of attributes into spoken keywords in a multilingual setting. Experiments on the Multilingual Spoken Words Corpus …

abstract arxiv attributes cs.cl cs.sd eess.as generate language modeling multilingual novel pre-trained model recognition robust set speech spoken type universal wav2vec2 zero-shot