The Shape of Word Embeddings: Recognizing Language Phylogenies through Topological Data Analysis

arXiv:2404.00500v1 Announce Type: new
Abstract: Word embeddings represent language vocabularies as clouds of $d$-dimensional points. We investigate how information is conveyed by the general shape of these clouds, outside of representing the semantic meaning of each token. Specifically, we use the notion of persistent homology from topological data analysis (TDA) to measure the distances between language pairs from the shape of their unlabeled embeddings. We use these distance matrices to construct language phylogenetic trees over 81 Indo-European languages. Careful evaluation …

abstract analysis arxiv cs.cl data data analysis embeddings general information language math.at meaning notion semantic through token type word word embeddings