Extracting Lexical Features from Dialects via Interpretable Dialect Classifiers

arXiv:2402.17914v1 Announce Type: new
Abstract: Identifying linguistic differences between dialects of a language often requires expert knowledge and meticulous human analysis. This is largely due to the complexity and nuance involved in studying various dialects. We present a novel approach to extract distinguishing lexical features of dialects by utilizing interpretable dialect classifiers, even in the absence of human experts. We explore both post-hoc and intrinsic approaches to interpretability, conduct experiments on Mandarin, Italian, and Low Saxon, and experimentally demonstrate that …

abstract analysis arxiv classifiers complexity cs.ai cs.cl differences expert extract features human knowledge language novel nuance studying type via