Axis Tour: Word Tour Determines the Order of Axes in ICA-transformed Embeddings

arXiv:2401.06112v2 Announce Type: replace
Abstract: Word embedding is one of the most important components in natural language processing, but interpreting high-dimensional embeddings remains a challenging problem. To address this problem, Independent Component Analysis (ICA) is identified as an effective solution. ICA-transformed word embeddings reveal interpretable semantic axes; however, the order of these axes are arbitrary. In this study, we focus on this property and propose a novel method, Axis Tour, which optimizes the order of the axes. Inspired by Word …

abstract analysis arxiv components cs.cl embedding embeddings however important independent language language processing natural natural language natural language processing problem processing replace semantic solution type word word embedding word embeddings