Encoding of lexical tone in self-supervised models of spoken language

arXiv:2403.16865v1 Announce Type: new
Abstract: Interpretability research has shown that self-supervised Spoken Language Models (SLMs) encode a wide variety of features in human speech from the acoustic, phonetic, phonological, syntactic and semantic levels, to speaker characteristics. The bulk of prior research on representations of phonology has focused on segmental features such as phonemes; the encoding of suprasegmental phonology (such as tone and stress patterns) in SLMs is not yet well understood. Tone is a suprasegmental feature that is present in …

abstract arxiv bulk cs.cl eess.as encode encoding features human interpretability interpretability research language language models prior research semantic slms speaker speech spoken type