Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations

arXiv:2403.08738v1 Announce Type: new
Abstract: Acoustic word embeddings (AWEs) are vector representations of spoken words. An effective method for obtaining AWEs is the Correspondence Auto-Encoder (CAE). In the past, the CAE method has been associated with traditional MFCC features. Representations obtained from self-supervised learning (SSL)-based speech models such as HuBERT, Wav2vec2, etc., are outperforming MFCC in many downstream tasks. However, they have not been well studied in the context of learning AWEs. This work explores the effectiveness of CAE with …

abstract arxiv auto cae cs.cl cs.sd eess.as embeddings encoder features self-supervised learning speech spoken ssl supervised learning through training type vector word word embeddings words